Data Engineer with 2 years of experience at Tata Consultancy Services, supporting data engineering work for Standard Chartered Bank. Hands-on experience developing and supporting PySpark ETL pipelines, SQL data processin…
Data Engineer with 2 years of experience at Tata Consultancy Services, supporting data engineering work for Standard Chartered Bank. Hands-on experience developing and supporting PySpark ETL pipelines, SQL data processing, Teradata-to-PySpark migration, batch orchestration, and production data operations. Experienced with Azure Databricks, ADLS Gen2, Delta Lake, Delta Live Tables, Azure Data Factory, Control-M, Git, Azure DevOps and large-scale data processing. Experienced in handling 20–200+ GB regional data loads and production support for 400+ scheduled batch jobs.
<p>Develop PySpark ETL pipelines from business requirements and migrate legacy Teradata ETL workloads to PySpark while preserving functional logic, data integrity, and downstream compatibility.<br>
Process and validate 20–200+ GB of data per regional load across multiple upstream systems through multi-stage ingestion and transformation pipelines.<br>
Build and maintain downstream database views supporting regulatory reporting across APAC markets, including Hong Kong and Singapore.<br>
Create and maintain Control-M job definitions and Azure DevOps deployment pipelines, using Git-based workflows across development, testing, and production environments.<br>
Provide production support for 400+ batch jobs, performing monitoring, failure analysis, root-cause investigation, reruns, and pipeline recovery within SLA windows.<br>
Collaborate with project teams on ETL changes, production incidents, data validation, and deployment activities across the data pipeline lifecycle.</p>
The AI-first Recruitment Operating System that unifies and automates the entire hiring lifecycle for modern enterprises worldwide.
Dr Job FZ LLC. 2026 © All Rights Reserved
