swathi D

swathi D

Data Engineer
India
Contract Work from Home

About Me

Data Engineer with 3 years of experience in building scalable cloud-based data solutions. Strong experience in designing and developing end-to-end ETL and ELT pipelines. Hands-on experience with Google Cloud Platform inc…

Experience

Data Engineer

Epsilon Softech Pvt Ltd · Hyderabad , Telangana , India
Sep 2024 - Apr 2026 · 1 year 7 months

<p>Built automated ingestion pipelines using Apache Airflow to load data from Google Cloud Storage into BigQuery.</p><p> Processed and transformed datasets using SQL and PySpark for business reporting and analytics needs.</p><p> Implemented BigQuery partitioning and clustering to improve query performance and optimize processing efficiency.</p><p> Worked with dbt for modular data transformations and maintainable data model development.</p><p> Supported structured data flow across landing, staging, and curated layers for analytics consumption.</p><p> Designed and optimized ELT pipelines for scalable data processing in BigQuery.</p><p> Improved query performance using partitioning and clustering strategies.</p><p> Ensured data quality through validation and transformation logic.</p><p> Automated workflows using Airflow DAGs with dependency handling.</p><p> Monitored pipeline execution and handled failures for reliable data delivery.</p><p> Collaborated with stakeholders to refine data models for reporting needs.</p>

Data Engineer

Syntrons Technologies Pvt Ltd · Hyderabad , Telangana , India
May 2023 - Aug 2024 · 1 year 3 months

<p>Migrated telecom datasets from the Hadoop ecosystem to Google Cloud Platform.</p><p> Processed large-scale datasets using Apache Spark and PySpark on Dataproc.</p><p> Designed ETL pipelines across landing, staging, and curated data layers.</p><p> Used Apache Airflow for workflow scheduling, dependency management, and pipeline execution.</p><p> Explored the Databricks environment for distributed data processing and analytics workloads.</p><p> Supported scalable cloud-based processing aligned with telecom data migration requirements.</p><p> Optimized Spark jobs for better performance and reduced processing time.</p><p> Ensured data consistency and integrity during migration.</p><p> Worked on scalable batch processing architecture in GCP.</p><p> Standardized schemas and transformation logic across datasets.</p><p> Improved job performance through resource tuning and partitioning.</p><p> Validated migrated data to ensure accuracy and completeness.</p>

Skills

Google Cloud Platform BigQuery GCS Dataproc Cloud Composer Dataflow Pub/Sub Apache Spark PySpark Databricks dbt Apache Airflow Python SQL Hadoop Git GitHub ETL Pipelines ELT Pipelines Data Modeling Partitioning Clustering
Report this Profile?