Process-driven Data Engineer with 3 years of experience in designing, migrating, and optimizing scalable data pipelines...
<p>Engineered a financial data lakehouse on AWS S3 using Apache Iceberg and Trino, enabling time-travel queries, ACID transactions, and schema evolution—reducing reporting latency by 40%.<br>
Migrated terabytes of historical trading and order data from Hive tables to Iceberg, eliminating AWS Glue crawler dependencies and cutting ETL costs by 30%.<br>
Designed and orchestrated data pipelines in Apache Airflow to ingest and normalize 1.5B+ records of daily market, order, and execution data from multiple OMS and client sources (Succession, VTrader, VFlux), improving data reliability by 95%.<br>
Developed low-latency Spark Structured Streaming pipelines on Apache Kafka to process real-time trade and order events, leveraging ScyllaDB for high-throughput storage to generate Consolidated Audit Trail (CAT) and regulatory reports, cutting report turnaround time by 40%.<br>
Collaborated with compliance and analytics teams to ensure data accuracy, lineage, and FINRA reporting standards, establishing automated monitoring for critical market feeds and regulatory submissions.</p>
<p>Assisted in building batch pipelines in SQL Server for financial transactions, processing 2M+ daily rows with optimized indexing strategies.<br>
Developed Python ETL scripts to load customer & sales data into PostgreSQL, reducing manual workload by 70%.<br>
Supported migration of workflows to Spark-based processing, cutting runtime from 2 hours to 40 minutes.</p>
- Engineered a financial data lakehouse on AWS S3 using Apache Iceberg and Trino, enabling time-travel queries,
- Migrated terabytes of historical trading and order data from Hive tables to Iceberg, eliminating AWS Glue crawler
dependencies and cutting ETL costs by 30%.
- Designed and orchestrated data pipelines in Apache Airflow to ingest and normalize 1.5B+ records of daily
market, order, and execution data from multiple OMS and client sources (Succession, VTrader, VFlux),
improving data reliability by 95%.
- Developed low-latency Spark Structured Streaming pipelines on Apache Kafka to process real-time trade and
order events, leveraging ScyllaDB for high-throughput storage to generate Consolidated Audit Trail (CAT) and
regulatory reports, cutting report turnaround time by 40%.
- Collaborated with compliance and analytics teams to ensure data accuracy, lineage, and FINRA reporting stan-
dards, establishing automated monitoring for critical market feeds and regulatory submissions.
- Assisted in building batch pipelines in SQL Server for financial transactions, processing 2M+ daily rows with
optimized indexing strategies.
- Developed Python ETL scripts to load customer & sales data into PostgreSQL, reducing manual workload by 70%.
- Supported migration of workflows to Spark-based processing, cutting runtime from 2 hours to 40 minutes.
The AI-first Recruitment Operating System that unifies and automates the entire hiring lifecycle for modern enterprises worldwide.
Dr Job FZ LLC. 2026 © All Rights Reserved
