About Me
Azure Data Engineer with 6 years of experience in designing, developing, and maintaining data engineering solutions on Microsoft Azure.
Hands-on experience in building ETL/ELT pipelines using Azure Data Factory (ADF) and…
Azure Data Engineer with 6 years of experience in designing, developing, and maintaining data engineering solutions on Microsoft Azure.
Hands-on experience in building ETL/ELT pipelines using Azure Data Factory (ADF) and Azure Databricks to process data from multiple business systems.
Strong knowledge of PySpark, Python, SQL Server, Azure SQL, and ADLS Gen2 for data ingestion, transformation, and storage.
Worked on integrating data from APIs, SQL databases, flat files, and cloud storage, ensuring accurate and reliable data movement.
Experience in developing scalable data pipelines, implementing incremental loading, and optimizing workflows for better performance.
Good understanding of data warehousing concepts, dimensional modeling, and data quality validation throughout the ETL process.
Familiar with Azure Synapse Analytics, Microsoft Fabric, Power BI, and Delta Lake to support reporting and business analytics.
Experience in troubleshooting production issues, monitoring pipeline executions, and resolving data-related problems with minimal business impact.
Strong team player with experience working in Agile/Scrum environments, participating in sprint planning, code reviews, and production deployments.
Passionate about learning new Azure services and continuously improving data engineering skills while delivering reliable and scalable solutions.
Strong knowledge of CI/CD, Git, and Azure DevOps for version control, deployment, and release management.
Hands-on experience in pipeline monitoring, troubleshooting production issues, and root cause analysis (RCA) to ensure business continuity.
Good understanding of data governance, security best practices, and compliance in Azure cloud environments.
Skilled in developing reusable and efficient Python scripts for data transformation, workflow automation, and data quality validation across Azure data pipelines.
Experience
Azure Data Engineer
<p>Designed and developed end-to-end ETL pipelines in Azure Data Factory (ADF) to ingest data from APIs, flat files, and on-prem SQL servers into Azure Data Lake Storage (ADLS).</p><p> Built parameterized and metadata-driven ADF pipelines to support reusable, scalable, and automated data integration processes across multiple healthcare data sources.</p><p> Developed PySpark notebooks in Azure Databricks to perform data cleansing, transformation, and business rule implementation for large healthcare datasets.</p><p> Created and maintained Delta Lake tables in Azure Databricks to provide reliable, ACID-compliant storage for structured and semi-structured healthcare data.</p><p> Optimized Databricks jobs by implementing partitioning, caching, and performance tuning techniques, resulting in faster data processing and reduced execution time.</p><p> Integrated Azure Synapse Analytics with curated datasets to support analytical reporting, ad-hoc SQL queries, and downstream business intelligence requirements.</p><p> Developed robust data validation, logging, and exception handling mechanisms in ADF and Databricks to improve data quality and monitor pipeline failures.</p><p> Processed high-volume patient, provider, claims, and clinical datasets using PySpark, ensuring data consistency and adherence to healthcare business requirements.</p><p> Managed data organization in ADLS Gen2 by implementing optimized folder structures, partitioning strategies, and lifecycle management for efficient storage utilization.</p><p> Collaborated with Data Scientists, Business Analysts, and Reporting teams to deliver curated datasets for Machine Learning, predictive analytics, and operational reporting.</p><p> Secured sensitive healthcare data using Azure Key Vault, Managed Identity, and Role-Based Access Control (RBAC) to meet organizational security standards and HIPAA compliance requirements.</p><p> Published curated datasets to Azure Synapse Analytics and integrated them with Power BI to develop interactive dashboards for patient insights, claims analysis, and healthcare performance reporting.</p><p> Worked with Microsoft Fabric Lakehouse to centralize curated healthcare datasets and support enterprise-wide analytics and reporting.</p><p> Developed Microsoft Fabric Data Factory pipelines and Dataflows Gen2 to automate data ingestion, transformation, and orchestration across multiple healthcare systems.</p><p> Integrated Microsoft Fabric Lakehouse with Power BI to deliver self-service reporting solutions, enabling business users to analyze healthcare data through interactive dashboards.</p><p> Implemented incremental data loading using Watermarking and Change Data Capture (CDC) to reduce processing time and improve pipeline efficiency.</p>
Senior Associate
<p>Designed and developed ETL Pipelines in Azure Data Factory (ADF) for Batch and Incremental Data Ingestion from diverse Data Sources.</p><p> Built and optimized Data Transformation Workflows using Azure Databricks with PySpark for Large-Scale Data Processing.</p><p> Implemented Data Lake Architecture on Azure Data Lake Storage Gen2 (ADLS Gen2) to store Raw, Curated, and Consumption-Ready Datasets.</p><p> Developed and tuned complex SQL Server Queries, Stored Procedures, Functions, and Triggers for Transactional and Analytical Processing.</p><p> Integrated ADF Pipelines with Azure Databricks Notebooks for seamless Data Transformation and Workflow Orchestration.</p><p> Created Data Models in SQL Server and Azure Synapse Analytics to support Business Intelligence (BI) and Analytics Solutions.</p><p> Designed and implemented Parameterized Pipelines, Triggers, and Linked Services in Azure Data Factory (ADF) for Automation and Reusability.</p><p> Performed Data Migration from On-Premises SQL Server to Microsoft Azure using Azure Data Factory (ADF), Azure Databricks, and Azure Data Lake Storage Gen2 (ADLS Gen2).</p><p> Developed Power BI Dashboards and Reports integrating with SQL Server and Azure Synapse Analytics for Self-Service Analytics.</p><p> Applied Data Security and Data Governance best practices including Role-Based Access Control (RBAC) and Data Encryption.</p><p> Collaborated with Cross-Functional Teams to deliver End-to-End Data Solutions for Business Intelligence (BI) and Analytics.</p><p> Monitored, debugged, and optimized ADF Pipelines, Apache Spark Jobs, and SQL Queries to improve Performance, Scalability, and Cost Optimization.</p>
SQL Developer
<p>Analyzed source data and understood business requirements to develop reporting solutions.</p><p> Generated Weekly, Monthly, Quarterly, and Yearly Reports to support business decision-making.</p><p> Used Page-Level Filters and Report-Level Filters to create interactive visualizations in Power BI.</p><p> Gathered and analyzed business requirements to identify reporting and system needs.</p><p> Imported data from SQL Server into Power BI Desktop using the Get Data option.</p><p> Performed data analysis using SQL with SELECT, Aggregate Functions, and GROUP BY statements.</p><p> Developed multiple Scorecards, Reports, Visualizations, and Interactive Dashboards using Power BI Desktop.</p><p> Configured Scheduled Refresh and managed Automatic Data Refresh in Power BI Service.</p><p> Developed analytical reports using DAX Functions, including Table Functions, Aggregation Functions, and Iterator Functions.</p><p> Created reports using Time Intelligence Functions for daily, monthly, quarterly, and yearly business analysis.</p><p> Designed customized and interactive reports and dashboards to meet business reporting requirements.</p><p> Published reports to Power BI Service and managed user access and workspace permissions.</p>