Kunal Sachdeva

Kunal Sachdeva

Azure Data Engineer
India
Full-time Hybrid

نبذة عني

Azure Data Engineer with 4+ years of specialized data engineering experience and 6+ years of total technology experience across banking, e-commerce and enterprise environments. Hands-on expertise in Azure Databricks, Del…

الخبرة

Senior Data Engineer

Capgemini - National Australia Bank (NAB) · Remote
Aug 2025 - حتى الآن · 1 سنة 3 أشهر

<p>Develop and deliver production-grade Silver-layer SQL transformations in Azure Databricks, processing 5M+ banking records daily and converting raw Bronze-layer data into validated, analytics-ready datasets within a Bronze/Silver/Gold Medallion architecture.</p><p> Translate 10+ approved source-to-target mapping documents into complex SQL transformation logic, implementing 200+ attributes across 5+ upstream banking systems while ensuring business-rule accuracy, data consistency, and traceability.</p><p> Delivered 30+ Silver-layer tables and supported 30+ Delta Live Tables (DLT) pipelines, validating transformation outputs, data-quality rules, reconciliations, and downstream datasets prior to release.</p><p> Investigate and resolve 8–10 production incidents per month across ingestion and transformation layers, performing root-cause analysis on pipeline failures and data discrepancies while achieving 95%+ resolution within SLA targets.</p><p> Collaborate with business analysts, source-system owners, and QA teams to identify mapping gaps and transformation issues early, contributing to a 50% reduction in SIT, UAT defects, and downstream rework.</p><p> Perform end-to-end validation of source data, transformation rules, and Delta Lake outputs, including active participation in SIT and UAT testing cycles, ensuring production datasets meet defined data-quality, reconciliation, and business requirements for downstream analytics and reporting.</p>

Azure Data Engineer

Tata Consultancy Services - US E-Commerce Client · Remote
Jan 2022 - Aug 2025 · 3 سنوات 7 أشهر

<p>Modernized and migrated on-premises Informatica ETL workloads to Azure Databricks and Azure Data Factory, improving scalability, reliability and batch-processing performance.</p><p> Designed and maintained end-to-end ADF pipelines using Copy Activity, Data Flows, Databricks notebooks, Azure Functions, Lookup activities and stored procedures.</p><p> Created a reusable PySpark and Spark SQL framework for logging, control flags, schema definitions and error handling, reducing new pipeline build effort by approximately 40%.</p><p> Built HTTP-triggered Azure Functions to execute parameterized queries and stored procedures dynamically.</p><p> Implemented GitHub Actions CI/CD workflows to deploy pipeline versions to DBFS after pull-request merges, removing manual deployment steps.</p><p> Developed Power BI pricing-analytics dashboards covering price trends and sales performance for business decision-making.</p><p> Managed self-hosted and Azure Integration Runtimes, datasets and linked services across development, test and production environments.</p>

Site Reliability Engineer - Data Analytics

Tata Consultancy Services · Remote
Dec 2020 - Jan 2022 · 1 سنة 1 شهر

<p>Developed Python REST APIs to collect, transform and analyze operational data, enabling faster insight generation and process improvements.</p><p> Automated recurring reporting and recovery workflows, reducing manual effort and improving SLA adherence.</p><p> Analyzed incident patterns and performed root-cause analysis, contributing to USD 14 million in customer savings through hard and soft cost reductions.</p><p> Delivered executive reporting using Python, SQL, Excel and Power BI.</p>

DevOps Engineer

Tata Consultancy Services - State Bank of India · New Delhi, India
Jan 2020 - Dec 2020 · 10 أشهر

<p>Implemented Jenkins, Git and Maven CI/CD pipelines for a high-frequency banking web application.</p><p> Built and maintained Docker containers for consistent application deployment across environments.</p><p> Automated operational tasks using Python and provided on-call support for critical production incidents.</p>

المشاريع

NAB

Capgemini
المدة : 15-Sep-2025 - 01-Sep-2026

<p>Develop and deliver production-grade Silver-layer SQL transformations in Azure Databricks, processing 5M+</p><p>banking records daily and converting raw Bronze-layer data into validated, analytics-ready datasets within</p><p>a Bronze/Silver/Gold Medallion architecture.</p><p>• Translate 10+ approved source-to-target mapping documents into complex SQL transformation logic,</p><p>implementing 200+ attributes across 5+ upstream banking systems while ensuring business-rule accuracy, data</p><p>consistency, and traceability.</p><p>• Delivered 30+ Silver-layer tables and supported 30+ Delta Live Tables (DLT) pipelines, validating transformation</p><p>outputs, data-quality rules, reconciliations, and downstream datasets prior to release.</p><p>• Investigate and resolve 8–10 production incidents per month across ingestion and transformation layers,</p><p>performing root-cause analysis on pipeline failures and data discrepancies while achieving 95%+ resolution within</p><p>SLA targets.</p><p>• Collaborate with business analysts, source-system owners, and QA teams to identify mapping gaps and</p><p>transformation issues early, contributing to a 50% reduction in SIT, UAT defects, and downstream rework.</p><p>• Perform end-to-end validation of source data, transformation rules, and Delta Lake outputs, including active</p><p>participation in SIT and UAT testing cycles, ensuring production datasets meet defined data-quality, reconciliation,</p><p>and business requirements for downstream analytics and reporting.</p><p></p>

الشهادات

azure data engineer associate

microsoft · Delhi , Delhi , India · 2024

المهارات

Azure Databricks Delta Live Tables Delta Lake Unity Catalog Medallion Architecture Azure Data Factory Azure Data Lake Storage Azure Functions Azure Key Vault PySpark Spark SQL SQL Python REST APIs ETL/ELT source-to-target mapping data-quality rules schema management logging error handling root-cause analysis production support Git GitHub Actions Jenkins Docker CI/CD Linux JIRA ServiceNow Power BI Excel Informatica Copy Activity Data Flows Databricks notebooks Lookup activities stored procedures HTTP-triggered Azure Functions DBFS Azure Integration Runtimes datasets linked services Maven
الإبلاغ عن هذا الملف الشخصي؟