About Me
Data Engineer with 3+ years designing and operating production-grade PySpark and Databricks pipelines on Google Cloud
Platform, backed by 12+ years of overall IT delivery experience. Built scalable ETL/ELT workflows usin…
Data Engineer with 3+ years designing and operating production-grade PySpark and Databricks pipelines on Google Cloud
Platform, backed by 12+ years of overall IT delivery experience. Built scalable ETL/ELT workflows using Delta Lake, Unity
Catalog, and Medallion Architecture (Bronze/Silver/Gold) to support 500,000+ autonomous vehicle simulation scenarios across 23 projects. Led a production infrastructure migration from Azure to GCP with zero data loss across 10+ production flows.
Experienced in Agile delivery, CI/CD pipeline automation, and technical leadership of a 7-engineer team
Experience
Data Engineer
<p>• Led a production infrastructure re-architecture from Azure to GCP (Pub/Sub, Cloud Storage, Cloud Logging), achieving zero</p><p>data loss across 10+ production flows</p><p>• Architected scalable PySpark ETL/ELT pipelines on Databricks, powering a GCP-based cloud data warehouse across 23</p><p>projects and 500,000+ simulation scenarios — improving processing efficiency by 30%</p><p>• Automated GCP Cloud Storage-to-Delta Lake ingestion using Databricks Auto Loader and Workflows, enforcing Medallion</p><p>Architecture (Bronze/Silver/Gold) and Unity Catalog governance controls</p><p>• Engineered cloud-native batch pipelines in Python, PySpark and Spark SQL, covering ingestion through downstream</p><p>warehouse processing</p><p>• Collaborated with data scientists and cross-functional engineers to validate PySpark transformation logic, lifting data</p><p>validation accuracy by 25%</p><p>• Monitored and stabilized production pipelines across GCP, Databricks and PostgreSQL, cutting downtime and incidents</p><p>by 40%</p><p>• Authored 10+ Software Design Documents formalizing ETL architecture standards across a 7-engineer team</p><p>• Standardized CI/CD deployment for PySpark pipelines via GitLab CI/CD and Docker with Pytest/Mock unit testing, raising</p><p>release quality by 30%</p><p>• Directed Agile delivery and technical coordination for a 7-engineer team across 23 concurrent projects</p><p></p>