Jian Huang

Jian Huang

Data Architect
United States of America

نبذة عني

I'm a data architect with over 15 years of experience in designing and optimizing large-scale data pipelines within the Fintech sector. I have successfully led global engineering teams to handle over 500 TBs of sensitive…

الخبرة

Data Architect

Intuit, Inc., Mountain View, CA
Oct 2022 - حتى الآن · 3 سنوات 9 أشهر

Enhanced and optimized data pipeline by adding automated routing feature, achieving a 20% increase in data processing efficiency.
Improved system monitoring by centralizing log management, simplifying data analysis and decreasing ownership costs by 15%.
Achieved $600K in annual savings for the organization by conducting thorough assessments of cloud infrastructure, streamline CICD pipeline using terraform and AWS SDK to consolidate monitoring jobs and enhanced efficiency with reduced costs.
Achieved 40% improvement in job execution times and a 33% reduction in operational overhead by Successfully migrating legacy Python batch jobs to AWS EKS, utilizing Argo framework for enhanced management and monitoring.
Developed comprehensive best practices for infrastructure deployment, resulting in streamlined processes and standardized configurations across the organization.
Spearheaded the implementation of a cutting-edge question and answer system using the RAG framework for an in-house monitoring application, significantly improving issue resolution for over 500 engineers.

Data Architect

Intuit, Inc
Oct 2022 - حتى الآن · 3 سنوات 9 أشهر

● Enhanced and optimized data pipeline by adding automated routing feature, achieving a 20% increase in data processing efficiency.
● Improved system monitoring by centralizing log management, simplifying data analysis and decreasing ownership costs by 15%.
● Achieved $600K in annual savings for the organization by conducting thorough assessments of cloud infrastructure, streamline CICD pipeline using terraform and AWS SDK to consolidate monitoring jobs and enhanced efficiency with reduced costs.
● Achieved 40% improvement in job execution times and a 33% reduction in operational overhead by Successfully migrating legacy Python batch jobs to AWS EKS, utilizing Argo framework for enhanced management and monitoring.
● Developed comprehensive best practices for infrastructure deployment, resulting in streamlined processes and standardized configurations across the organization.
● Spearheaded the implementation of a cutting-edge question and answer system using the RAG framework for an in-house

Senior Data Architect, Data Infrastructure

PayPal, Inc., San Jose, CA
Apr 2016 - Sep 2022 · 6 سنوات 5 أشهر

Implemented and matured a data pipeline capable of processing 500 TBs of highly sensitive data
By automating the dataset creation process, we eliminated the need for manual intervention, which previously required 5 development days, resulting in a cost savings of $10 million.
Saved >5 PBs by applying optimization technique and shrinking the data set to 2% of its original size.
Lead a team of 15 engineers in a global setting to increase the data quality of the Core dataset
Reduced the total cost of ownership by 18% by leading the migration of on-premises data processing jobs to Snowflake.
Proficient at designing data models, especially in regarding ATO (account takeover) and user session validations for offline fraud data marts which contribute Reduced fraud rate over 40 percent
Proficient at developing data strategy and aligning it with business objectives. Identifying the types of data that are relevant to organizations, determining how data should be collected, stored, and used, and establishing policies for data governance, security, and privacy
Familiar with Kafka and ETL tools such as Informatica and ab initio
Proficient in working with cloud services and developing solutions on AWS cloud
Strong background working with various SQL, NoSQL database technologies in a complex, highly available and scalable environment
Have extensive knowledge working with data security technologies both in transit and at rest

Manager, Data Infrastructure

PayPal, Inc., San Jose, CA
Jan 2010 - Mar 2016 · 6 سنوات 2 أشهر

Successfully built teams in the various parts of the company.
Led and managed many critical projects in merchant reporting, fraud detection, and compliance domains

Principal Software Engineer, Data Infrastructure

PayPal, Inc., San Jose, CA
Apr 2008 - Dec 2009 · 1 سنة 8 أشهر

Awarded as a top talent
Architected Paypal's first low latency data ingestion platform capable of handling hundreds of datasets with high availability capabilities.

Software Engineer 4, Data Infrastructure

PayPal, Inc., San Jose, CA
Apr 2006 - Mar 2008 · 1 سنة 11 أشهر

Designed and implemented a data processing platform for providing data insight for Paypal merchants.

Software Engineer 3, Data Infrastructure

PayPal, Inc., San Jose, CA
Nov 2004 - Mar 2006 · 1 سنة 4 أشهر

Developed data movement platform to replicate data from Paypal OLTP databases to multiple downstream data marts which have restartable components, an advanced concept at the time

Senior ETL Engineer

Siebel System, Inc., San Mateo, CA
Jun 2000 - Oct 2004 · 4 سنوات 4 أشهر

Designed and implemented a large-scale ETL system to transfer vast amounts of data from OLTP systems to an OLAP environment.
This system enhances business operations by streamlining the reconciliation process, improving fraud control, and enabling detailed transaction reporting.

المهارات

العمارة ذكاء الأعمال (BI) تقييم البيانات تنسيق إدارة قاعدة البيانات دمج وتحويل البيانات باستخدام ETL (ETL) البنية التحتية لينكس ماي إس كيو إل لغة الاستعلامات الهيكلية (SQL) نسخة احتياطية عملية التشغيل السحابة مستودع البيانات بي/إسكيوإل إجراءات RAG Nodejs Amazon Glue dbt Mysql Postgresql Amazon DynamoDB Amazon Redshift Avro Parquet Protobuf ORC Json Airflow RabbitMQ Collibra Informatica looker snowflake Python Node.js Teradata Oracle PostgreSQL MongoDB Redis JSON Kubernetes Kafka Hive Hadoop Terraform Jenkins Tableau Looker AWS Snowflake LangChain LlamaIndex Embedchain AWS SDK Argo CI/CD NoSQL Data governance Data security Data privacy Data modeling Data architecture Data pipelines Cloud infrastructure Monitoring Log management
الإبلاغ عن هذا الملف الشخصي؟