About Me
Having 6.3 years of professional experience in Data Warehousing, ETL, Data Engineering, and Business Intelligence solutions using Databricks, PySpark, SQL, and Power BI. Experienced in designing and implementing end-to-e…
Having 6.3 years of professional experience in Data Warehousing, ETL, Data Engineering, and Business Intelligence solutions using Databricks, PySpark, SQL, and Power BI. Experienced in designing and implementing end-to-end data pipelines using Azure Databricks, PySpark, SQL, and cloud-based data platforms. Analyze source-to-target mapping documents and develop scalable Databricks notebooks and ETL pipelines to ingest, transform, validate, and load data from various source systems into target data platforms. Experienced in implementing Bronze, Silver, and Gold layer architecture using Databricks and Delta Lake for efficient data processing and analytics. Provide and manage pipeline configurations, parameters, environment-specific settings, and reusable components to support development, testing, and production deployments. Good exposure to ETL/ELT mechanisms using Databricks and PySpark, including data cleansing, transformation, joins, aggregations, filtering, deduplication, and incremental data processing. Strong experience working with relational databases, including SQL Server and Oracle, and writing complex SQL queries for data extraction, transformation, validation, and reporting. Good experience with SQL joins, views, indexes, stored procedures, functions, CTEs, subqueries, window functions, and performance optimization. Experienced in developing complex PySpark transformations and optimizing data processing using partitioning, caching, broadcast joins, and appropriate Spark configurations. Good experience working with heterogeneous data sources, including SQL Server, Oracle, CSV, Excel, and other file-based sources, and integrating data into cloud-based data platforms. Experienced in developing dynamic SQL and parameterized queries for flexible and reusable data processing solutions. Experienced in working with Delta Lake features such as ACID transactions, schema evolution, MERGE operations, and incremental data loads.
Experience
Databricks Data Engineer / Power BI Developer
<p>Developed and maintained Azure Databricks data pipelines for ingestion, transformation, validation, and processing of BMS and EPMS application data.</p><p> Created Databricks notebooks using PySpark and SQL to process structured and semi-structured data from CSV, Excel, and relational database sources.</p><p> Developed data workflows to ingest incoming CSV and Excel files into staging/raw Delta tables in Databricks.</p><p> Implemented Bronze, Silver, and Gold data processing layers to organize raw, cleansed, transformed, and business-ready datasets.</p><p> Developed PySpark transformations for data cleansing, filtering, joins, aggregations, deduplication, and business-rule implementation.</p><p> Created and maintained Delta Lake tables for staging, dimensional, and transactional data.</p><p> Developed SQL and PySpark logic to populate dimension tables and fact tables following data warehouse principles.</p><p> Created reusable transformation logic for loading master/reference data into dimension tables and transaction-level information into fact tables.</p><p> Developed complex SQL queries, CTEs, subqueries, joins, window functions, aggregations, and conditional logic for data extraction and validation.</p><p> Performed data quality checks and source-to-target validation to ensure completeness and accuracy of processed data.</p><p> Developed data extraction processes to generate CSV files and distribute processed datasets to required downstream locations.</p><p> Implemented incremental data processing logic to efficiently process new and modified records.</p><p> Used Delta Lake MERGE operations for handling inserts and updates in curated datasets.</p><p> Developed Power BI dashboards and reports using curated Databricks datasets to provide operational and management-level insights.</p><p> Created Power BI data models using appropriate relationships, calculated columns, measures, filters, and slicers.</p><p> Developed DAX measures and KPIs for analyzing BMS/EPMS operational data and supporting business reporting requirements.</p><p> Implemented interactive Power BI reports with drill-down, filtering, search, and export capabilities.</p><p> Worked with stakeholders and business teams to understand reporting requirements and translate them into Databricks data pipelines and Power BI dashboards.</p><p></p><p> Participated in release management, UAT, production deployment, troubleshooting, and post-production support activities.</p><p> Prepared technical documentation for data workflows, deployment activities, production issues, and release procedures.</p><p> Coordinated with application, infrastructure, database, and business teams to analyze issues and implement efficient solutions.</p>
Databricks Data Engineer / Power BI Developer
<p>Developed scalable Azure Databricks and PySpark ETL/ELT pipelines for sanctions-screening data processing.</p><p> Analyzed source-to-target mappings and existing ETL processes to implement business and transformation rules.</p><p> Consolidated Payee, Assured, Claimant, Broker, and Vendor data from multiple heterogeneous source systems into a unified sanctions-screening dataset.</p><p> Developed Databricks notebooks using PySpark and SQL for data extraction, cleansing, transformation, validation, and loading.</p><p> Implemented Delta Lake-based curated tables and incremental data processing using MERGE operations.</p><p> Developed de-duplication and entity matching logic using Entity Name, Entity Type, Address, and other critical attributes.</p><p> Created unique Hash Keys/Business Keys to identify entities and establish links between consolidated records and source-system records.</p><p> Implemented business rules for Activation and Deactivation feeds based on entity status and effective dates.</p><p> Developed data-quality checks for missing mandatory attributes, addresses, invalid records, and failed business rules.</p><p> Performed source-to-target reconciliation and data validation across raw, transformed, and consolidated datasets.</p><p> Supported REST API integration by preparing entity information for Compliance Link and capturing API success/failure responses.</p><p> Developed Power BI HitList reports enabling compliance users to search entities and view linked records across source systems.</p><p> Created Power BI data models, DAX measures, calculated columns, relationships, slicers, filters, and interactive dashboards.</p><p> Used Bitbucket and Jenkins for version control, CI/CD, and deployment of Databricks notebooks and related components.</p><p> Participated in UAT, production deployment, troubleshooting, reconciliation, defect resolution, and hypercare support.</p>
Power BI Developer / BI Developer
<p>Analyzed business requirements and existing SAP reports and data structures to understand reporting requirements and identify the required source data.</p><p> Worked on the migration of reporting data from SAP systems to SQL Server and supported the development of the enterprise reporting data warehouse.</p><p> Designed and implemented Star Schema data models consisting of fact and dimension tables to support analytical reporting.</p><p> Developed SQL queries, views, stored procedures, and data extraction logic required for reporting and data validation.</p><p> Worked with fact and dimension tables and established appropriate relationships to support Power BI semantic models.</p><p> Developed Power BI dashboards and reports for business users to analyze organizational performance and key business metrics.</p><p> Created interactive reports using DAX measures, calculated columns, Power Query, filters, slicers, drill-downs, and visualizations.</p><p> Developed business KPIs and analytical measures based on requirements from business stakeholders.</p><p> Connected Power BI to SQL Server data warehouse tables and views and developed optimized semantic models.</p><p> Performed data cleansing and transformation using Power Query before loading data into Power BI models.</p><p> Implemented appropriate relationships between fact and dimension tables and maintained star-schema-based Power BI data models.</p><p> Developed complex DAX calculations for aggregations, time-based analysis, percentages, trends, and business KPIs.</p><p> Performed source-to-target validation and reconciliation between SAP source data and SQL Server warehouse data.</p><p> Troubleshot data discrepancies, report issues, incorrect calculations, and data-refresh failures.</p><p> Optimized SQL queries and Power BI models to improve report performance and data-refresh efficiency.</p><p> Worked closely with business users and SMEs to gather requirements, clarify KPIs, validate reports, and incorporate feedback.</p><p> Participated in UAT, report validation, deployment, and production support activities.</p><p> Created technical and functional documentation covering data mappings, report specifications, business rules, and Power BI dashboards.</p><p> Supported migration from legacy SAP reporting to the SQL Server-based reporting platform, enabling centralized and standardized business reporting.</p><p> Provided production support and analyzed issues related to data loads, SQL Server datasets, Power BI refreshes, and report performance.</p><p> Collaborated with data engineering, database, SAP, and business teams to ensure successful delivery of reporting solutions.</p>
PROJECTS
Insurance Data Analytics and Reporting Solution
<p>Developed an end-to-end insurance analytics solution to process policy, customer, premium, and claims data using Azure Databricks and PySpark. Built interactive Power BI dashboards and reports to monitor business performance, analyze claims, track premium collections, and support data-driven decision-making.</p>