About Me
Data Engineering Specialist: ~5 years of experience architecting, streamlining, and maintaining enterprise ETL/ELT pipelines, distributed data systems, and cloud data lakes across AWS and Azure environments.
Distributed…
Data Engineering Specialist: ~5 years of experience architecting, streamlining, and maintaining enterprise ETL/ELT pipelines, distributed data systems, and cloud data lakes across AWS and Azure environments.
Distributed Big Data & Transformations: Engineered scalable batch and streaming pipelines using PySpark, Apache Spark, and Pandas for high-volume data ingestion, advanced transformations, and partitioning strategies.
Multi-Cloud Architecture: Built automated, event-driven ingestion frameworks utilizing AWS Glue, Lambda, Amazon S3, Step Functions, Azure Data Factory (ADF), ADLS Gen2, and Azure Databricks.
Modern Cloud Warehousing & Analytics: Modeled performant analytical schemas, data staging layers, and clustered tables in Snowflake and Azure Synapse Analytics using complex SQL queries and dimensional modeling.
Workflow Automation & Governance: Authored production-grade Apache Airflow DAGs with proactive monitoring, enforcing strict data quality, reconciliation, duplicate detection, and data governance practices in Agile/Scrum environments.
Enterprise Systems & AI Integration: Modernized legacy environments (Hadoop, Informatica PowerCenter) and enabled AI workflows, feature engineering, and LLM-ready contextual data preparation.
Experience
Sr. Data Engineer
<p>Architected serverless and hybrid cloud ingestion pipelines utilizing AWS Glue, AWS Lambda, Amazon S3, Azure Data Factory (ADF), and ADLS Gen2 to automate daily financial asset records and valuation ingestions.</p><p> Engineered distributed transformation workflows in PySpark and Azure Databricks, enforcing schema compliance, partitioning strategies, automated duplicate detection, and data reconciliation routines.</p><p> Implemented Generative AI-enabled data pipelines, curating structured financial knowledge bases and contextual datasets for automated document summarization and LLM search queries.</p><p> Automated multi-tier ELT processes using AWS Step Functions, Azure Data Factory triggers, and Apache Airflow DAGs, ensuring fault-tolerant task coordination and proactive incident alerts within an Agile framework.</p><p> Designed analytical layers across Snowflake and Azure Synapse Analytics, building performant dimensional tables, views, and SQL scripts to accelerate quantitative deal analyses and compliance reporting.</p><p> Prototyped data preparation pipelines in Google Colab and Pandas, implementing feature engineering routines aligned with future machine learning model deployments.</p>
Data Engineer
<p>Engineered end-to-end multi-cloud ingestion pipelines, aggregating multi-region order transactions, clickstream events, and inventory feeds into centralized Amazon S3 and Azure ADLS Gen2 storage layers.</p><p> Constructed scalable PySpark, Python, and Pandas jobs in Azure Databricks to compute critical business KPIs such as cart abandonment rates, SKU turnover, and customer session metrics.</p><p> Orchestrated workflow schedules via Apache Airflow, deploying dynamic DAGs with built-in sensors, automated failure retries, and data profiling checks to ensure uninterrupted data refreshes.</p><p> Optimized data delivery into Snowflake via staging areas and Snowpipe automated loads, significantly decreasing data latency for cross-functional business intelligence teams.</p><p> Enforced robust data validation, duplicate detection, and data governance standards across distributed workflows to guarantee schema consistency and regulatory compliance.</p><p> Conducted exploratory data analysis in Google Colab, preparing clean structured datasets to support upcoming predictive merchandising and forecasting models.</p>
Associate Data Engineer
<p>Developed enterprise data integration mappings utilizing Informatica PowerCenter and complex SQL to extract, transform, and clean clinical trial patient records.</p><p> Managed high-volume distributed batch jobs using Hadoop and Python, adhering to stringent healthcare privacy, HIPAA compliance, and data governance mandates.</p><p> Executed structured database migrations from legacy source systems into scalable relational databases and cloud staging stores, ensuring strict data reconciliation and zero data discrepancy.</p><p> Implemented automated data quality checks, data profiling scripts, and custom lookup logic to identify, isolate, and remediate anomalous metrics prior to warehouse loading.</p><p> Collaborated with clinical analytics teams in Agile/Scrum sprints, drafting clear source-to-target mapping documentation, pipeline lineage, and operational runbooks.</p>
PROJECTS
Financial records
<p></p><p>• Architected serverless ingestion pipelines using AWS Glue, AWS Lambda, and Amazon S3 to automate daily financial asset records and valuation data ingestion. • Built distributed data transformation workflows in PySpark and Apache Spark, implementing schema enforcement, partitioning strategies, and deduplication routines. • Automated multi-tier ELT processes with AWS Step Functions and Apache Airflow to ensure fault-tolerant task coordination and proactive incident alerting. • Designed analytical layers in Snowflake, developing high-performance dimensional tables, views, and SQL scripts to support quantitative deal analysis and compliance reporting. • Built Generative AI-enabled data pipelines to curate structured financial knowledge bases and contextual datasets for automated document search and summarization. • Prototyped data prep pipelines in Google Colab, implementing feature engineering routines aligned with machine learning model deployments.</p>
Migration and ingestion
<p>• Engineered end-to-end data ingestion pipelines aggregating multi-region order transactions, clickstream events, and inventory feeds into Amazon S3. • Developed scalable PySpark and Python jobs to calculate key business metrics, including cart abandonment rates, SKU turnover, and session analytics. • Orchestrated workflow schedules using Apache Airflow, deploying dynamic DAGs with automated sensors and retry logic for continuous data delivery. • Optimized automated data loading into Snowflake using staging areas and Snowpipe to reduce data latency for analytics teams. • Conducted exploratory data analysis in Google Colab, preparing structured datasets for predictive merchandising and forecasting models. • Implemented data validation checks across distributed pipelines to enforce schema consistency and regulatory audit standards.</p>
EHR
<p>• Developed enterprise data integration mappings using Informatica PowerCenter and complex SQL queries to clean and load clinical trial patient records. • Managed high-volume distributed batch data processing using Hadoop and Python in compliance with healthcare data regulations. • Executed structured database migrations from legacy source systems into relational data stores while verifying end-to-end data integrity. • Implemented automated data quality checks and validation routines to isolate anomalous records prior to warehouse loading. • Collaborated with analytics teams to author source-to-target mapping documentation, Azure, data lineage, and operational runbooks.</p>