Harika Chandupatla

Harika Chandupatla

Data Engineer
India
Telugu, English, Hindi
Full-time Hybrid

نبذة عني

AWS Data Engineer with 4.9 years of professional experience in designing, building and maintaining scalable data pipelines. Skilled in extracting, transforming and loading data from diverse sources into cloud-based data …

الخبرة

Senior Associate Consultant

Infosys Limited · Hyderabad
Sep 2021 - حتى الآن · 5 سنوات 1 شهر

<p>Designed and implemented AWS ETL pipelines using AWS Glue, Step functions, Lambda, DynamoDB to extract, orchestrate, and transform data from multiple AWS accounts based on business rules.</p><p> Designed and implemented serverless ingestion pipelines using AWS Lambda to extract data from REST APIs.</p><p> Implemented incremental data loading for relational databases using AWS DMS, IAM, VPC, Lambda, Glue and automated the ingestion process using Airflow, reducing data latency and maintaining a sequential process.</p><p> Optimized AWS DMS replication infrastructure by right-sizing overprovisioned replication instances based on CPU, memory, and CDC latency metrics, reducing migration platform costs by up to 50% while maintaining performance targets.</p><p> Integrated data from databases, APIs and file-based sources into AWS S3 and Snowflake.</p><p> Developed reusable PySpark and Python frameworks for data processing.</p><p> Developed Airflow DAGs to orchestrate end-to-end pipelines, scheduling jobs at multiple frequencies including 30 min, 6-hour, daily and event-driven intervals.</p><p> Built monitoring and alerting using Amazon CloudWatch, SNS and PagerDuty to ensure proactive incident management.</p><p> Developed automated scripts using AWS boto3 module including metadata updating utility, data catalog updating utility using dynamic S3 paths, and dynamic schema generation tool.</p><p> Performed root-cause analysis and resolved production issues while meeting SLA commitments.</p><p> Participated in code reviews, deployment activities and release management using Git and CI/CD practices.</p><p> Configured lifecycle rules to archive infrequently accessed historical data, reducing long-term storage expenses.</p>

المهارات

Python SQL AWS AWS Glue Snowflake ELT Apache Spark PySpark AWS S3 AWS Lake Formation AWS DMS AWS DataSync Parquet CSV JSON Snowflake AWS CloudWatch PagerDuty IAM Terraform Git Jenkins Star Schema Snowflake Schema Dimensional Modeling Step Functions Lambda DynamoDB Airflow SNS Boto3 CI/CD
الإبلاغ عن هذا الملف الشخصي؟