Renuka Gajula

<p>Worked as an Azure Data Engineer on a Healthcare Data Warehouse project, contributing to the development and maintenance of scalable data pipelines and DWH solutions.</p><p> Designed and developed end-to-end ETL/ELT pipelines using Azure Data Factory (ADF) to ingest healthcare data from multiple source systems into Azure data platforms.</p><p> Worked with healthcare source systems including ECW, eCares, PARCS, Actuarials, and Excel-based data sources for data ingestion and processing.</p><p> Developed ADF pipelines using Lookup, Get Metadata, ForEach, Execute Pipeline, Set Variable, Filter, Until, Copy Data, and Stored Procedure activities.</p><p> Implemented data ingestion and processing across Bronze, Silver, and Gold layers using Azure Data Lake Storage and Azure Databricks.</p><p> Developed PySpark and Spark SQL transformations in Azure Databricks to cleanse, standardize, transform, and process healthcare datasets.</p><p> Implemented incremental data loading strategies to efficiently process new and modified healthcare records.</p><p> Worked with fact and dimension tables and supported the development of data warehouse structures based on business requirements.</p><p> Worked with Slowly Changing Dimensions (SCD) concepts to maintain historical changes in dimension data.</p><p> Developed and optimized SQL/T-SQL queries, stored procedures, joins, CTEs, and window functions for data transformation and warehouse processing.</p><p> Performed source-to-target data validation and reconciliation to ensure data accuracy, completeness, and consistency.</p><p> Implemented data quality checks and investigated data discrepancies, pipeline failures, and transformation issues.</p><p> Monitored and troubleshot ADF pipelines and Databricks jobs, identifying root causes and implementing corrective solutions.</p><p> Worked with Azure Data Lake Storage (ADLS) for storing raw, intermediate, and curated healthcare datasets.</p><p> Used Databricks notebooks, PySpark, Spark SQL, DBUtils, and Widgets for data processing, parameterization, and automation.</p><p> Worked with Unity Catalog for data governance, access control, and management of Databricks data assets.</p><p> Implemented parameterized and reusable ADF pipelines to improve maintainability and reduce development effort.</p><p> Supported deployments using Azure DevOps and Git, following CI/CD practices across environments.</p><p> Used Azure Key Vault and RBAC to maintain secure access to Azure resources and data.</p><p> Participated in requirement analysis, development, testing, deployment, and production support activities.</p>

<p>Worked as an Azure Data Engineer on an Insurance Data Warehouse project, developing scalable ETL/ELT pipelines for insurance data integration, transformation, and reporting.</p><p> Designed and managed end-to-end data pipelines using Azure Data Factory (ADF) for extracting, transforming, and loading data into the data warehouse.</p><p> Orchestrated data ingestion from multiple sources including SQL Server, Teradata, Azure SQL, Excel, APIs, Azure Storage, and Data Warehouse systems.</p><p> Developed ADF pipelines using Get Metadata, Lookup, ForEach, Wait, Execute Pipeline, Set Variable, Filter, Until, Copy Data, and Stored Procedure activities.</p><p> Implemented ETL workflows to load data into Azure Data Lake Storage (ADLS), Azure Storage, Azure SQL, and Data Warehouse layers.</p><p> Developed PySpark and Spark SQL applications in Azure Databricks to cleanse, transform, aggregate, and process large volumes of insurance data.</p><p> Implemented incremental data processing using parameterized pipelines and Auto Loader for CSV, Parquet, XML, and JSON data formats.</p><p> Worked with fact and dimension tables to support insurance reporting and analytical requirements.</p><p> Worked with historical data processing and Slowly Changing Dimensions (SCD) concepts for maintaining changes in dimension records.</p><p> Developed and optimized SQL/T-SQL queries, stored procedures, functions, triggers, indexes, joins, CTEs, and window functions for DWH processing.</p><p> Prepared and transformed data from multiple source systems for consumption through Power BI reports and dashboards.</p><p> Worked on Databricks notebooks and scheduled batch jobs using Job Clusters for automated data processing.</p><p> Used Unity Catalog, DBUtils, and Widgets in Azure Databricks for data governance, parameterization, secure access, and automation.</p><p> Performed source-to-target data validation, reconciliation, and data quality checks to identify data discrepancies.</p><p> Investigated and resolved ADF pipeline failures, Databricks job failures, transformation issues, and data quality problems.</p><p> Optimized T-SQL queries, Spark jobs, and ADF pipelines to improve ETL performance and reduce processing time.</p><p> Used Azure Key Vault for secure credential management and implemented RBAC for controlled access to Azure resources.</p><p> Used Azure DevOps and Git for version control and CI/CD deployments across development, testing, and production environments.</p><p> Collaborated with business analysts, developers, QA teams, and stakeholders to understand requirements and deliver reliable data solutions.</p><p> Worked in an Agile/Scrum environment, participating in sprint planning, daily stand-ups, sprint reviews, and retrospectives.</p>