About Me
Data Engineer and data analyst with 5+ years of experience designing and optimizing scalable ETL/ELT pipelines, data migration, application modernization, regulatory reporting, batch processing, and enterprise data engin…
Data Engineer and data analyst with 5+ years of experience designing and optimizing scalable ETL/ELT pipelines, data migration, application modernization, regulatory reporting, batch processing, and enterprise data engineering solutions. Skilled in Oracle, PL/SQL, Hadoop, Spark, PySpark, Apache NiFi, Apache Airflow, HDFS, Hive, Splunk, and Cloudera Data Engineering (CDE). Experienced in historical data backloading, Hadoop monthly aggregate batch creation, Oracle batch migration, performance optimization, monitoring automation, and data quality controls.
Experience
Senior Project Engineer
<p>Contributed to the Lift & Shift modernization initiative by migrating legacy applications and data workloads to scalable modern platforms.</p><p> Designed and created 70+ Oracle database tables to support high-volume transaction processing, reporting, and reconciliation requirements.</p><p> Migrated and modernized 15+ Oracle batch processes for data ingestion, transformation, validation, and downstream reporting.</p><p> Developed PL/SQL scripts to support historical data migration, backloading, and reconciliation activities.</p><p> Backloaded 3–5 years of historical transaction data while ensuring data completeness, accuracy, and consistency across systems.</p><p> Performed Oracle query tuning, batch performance optimization, and large-scale data migration activities to improve processing efficiency.</p><p> Designed and developed an end-to-end Apache NiFi pipeline with MiniO, Spark, HDFS, API for regulatory reporting requirements, incorporating dependency-based workflow orchestration, secure encryption/decryption processing, automated validations, exception handling, and monitoring controls to support reliable daily production operations and compliance-driven reporting.</p><p> Delivered enhancements across Spark, NiFi, HDFS, MinIO, and API integration layers to improve pipeline reliability and maintainability.</p><p> Built automation solutions for audit controls, reconciliation validation, data quality checks, and operational efficiency improvements.</p><p> Contributed to platform migration and modernization initiatives involving Apache Airflow and Cloudera Data Engineering.</p><p> Led production deployments, validations, and release activities across multiple environments in alignment with enterprise change and release management processes.</p><p> Collaborated with business stakeholders, architects, support teams, and external partners to deliver regulatory compliance-driven data solutions.</p><p> Performed issue analysis, root-cause investigation, and production incident resolution to ensure platform stability and timely regulatory reporting.</p><p> Developed PySpark-based solutions for large-volume data processing, transformation, and validation workloads.</p><p> Designed Splunk-based monitoring, alerting, and operational dashboards to improve pipeline observability and supportability.</p><p> Implemented frameworks for retention management, purge automation, audit validation, and data quality monitoring.</p><p> Designed proactive threshold-based monitoring and alerting mechanisms to identify issues early and improve operational resilience.</p>