Experienced Data Engineer with 2+ years of expertise in end-to-end pipeline design, cloud transitions, and team leadership. Specialized in optimizing ETL pipelines and implementing CI/CD solutions for GDPR-compliant data…
Experienced Data Engineer with 2+ years of expertise in end-to-end pipeline design, cloud transitions, and team leadership. Specialized in optimizing ETL pipelines and implementing CI/CD solutions for GDPR-compliant data storage. Proficient in Marillion, Python, Apache Kafka, Snowflake, and more. Experienced in real-time data ingestion with Sqoop, Flume, Apache MiFi, and DVC For streamlined version control. Adept at optimizing database performance using SǪL Profiler. Leadership includes team management, cloud transitions (AWS), and collaboration with data scientists for efficient analytics. Achievements in reducing setup time by 25% through Git, Jenkins, Docker integration.
Managed data warehousing solutions using Azure Data Warehouse, reducing data storage costs by 20% through data compression and partitioning strategies
Managed end-to-end data pipelines using a combination of Kafka for real-time stream processing and Hadoop MapReduce for batch processing
Implemented data integration solutions using SSAS and Informatica to support OLAP and OLTP databases, enhancing reporting and analytics capabilities and followed Agile Methodology for application Implementation and Testing
Managed version control and issue tracking using GIT and JIRA, maintaining the codebase and enhancing collaboration within the data engineering team and implemented Jenkins for CI/CD, automating data pipeline updates and maintaining reliability
Designed, developed, and maintained ETL processes using SSIS to extract, transform, and load data from various sources into SQL Server and Azure data warehouses
Optimized data processing performance with fine-tuned Spark configurations and Hive optimizations
Implemented cloud-based ETL solutions, utilizing GCP Dataflow and AWS Lambda functions for real-time data processing, resulting in reduced data latency by 40%
Managed data warehousing solutions using Azure Data Warehouse, reducing data storage costs by 20% through data compression and
partitioning strategies
• Managed end-to-end data pipelines using a combination of Kafka for real-time stream processing and Hadoop MapReduce for batch
processing.
• Implemented data integration solutions using SSAS and Informatica to support OLAP and OLTP databases, enhancing reporting and
analytics capabilities and followed Agile Methodology for application Implementation and Testing.
• Managed version control and issue tracking using GIT and JIRA, maintaining the codebase and enhancing collaboration within the data
engineering team and implemented Jenkins for CI/CD, automating data pipeline updates and maintaining reliability.
• Designed, developed, and maintained ETL processes using SSIS to extract, transform, and load data from various sources into SQL Server
and Azure data warehouses.
• Optimized data processing performance with fine-tuned Spark configurations and Hive optimizations.
• Implemented cloud-based ETL solutions, utilizing GCP Dataflow and AWS Lambda functions for real-time data processing, resulting in
reduced data latency by 40%.
Assisted senior data engineers in designing and developing data pipelines, ensuring data integrity and accuracy
Collaborated with cross-functional teams to gather and define data requirements, aligning data engineering efforts with business needs and maintained Hadoop clusters, including HDFS, and MapReduce, for processing large-scale data
Utilized advanced analytics techniques, such as regression and clustering, to uncover insights from customer data, resulting in a 15% increase in marketing campaign effectiveness
Combined Apache Kafka for streaming and Apache Spark for real-time analytics, ensuring responsive data insights
Modify datasets using DAX query, Merge statements, Refresh Schedule, Row level security and Modelling within Power BI
Developed and maintained data integration solutions using SSIS and Informatica, facilitating data access for analytics and reporting purposes and Used GIT for version control with Data Engineer team and Data Scientists colleagues
Engineered data pipelines, improving processing efficiency by 30% through optimization techniques, including parallel processing and data partitioning
The AI-first Recruitment Operating System that unifies and automates the entire hiring lifecycle for modern enterprises worldwide.
Dr Job FZ LLC. 2026 © All Rights Reserved
