Data Engineer with 2+ years of experience in building scalable data pipelines, ETL workflows, and large-scale distributed data processing systems. Proficient in Python, SQL, and Big Data tools including Hadoop, Hive, Sqo…
Data Engineer with 2+ years of experience in building scalable data pipelines, ETL workflows, and large-scale distributed data processing systems. Proficient in Python, SQL, and Big Data tools including Hadoop, Hive, Sqoop, and Spark (PySpark & Scala Spark). Experienced in ingesting, transforming, and processing high-volume datasets across on-prem and cloud environments.
Hands-on experience with AWS cloud services such as S3, EMR, EC2, and Step Functions, along with workflow orchestration using Apache Airflow. Skilled in data lake and data warehouse integrations, performance tuning, debugging pipeline failures, and ensuring end-to-end data quality and reliability. A strong team player with the ability to understand business data needs and deliver analytical data solutions that support reporting, product insights, and machine learning initiatives.
Designed and developed distributed ETL pipelines using PySpark on AWS EMR.
Created and managed Hive external, managed, and partitioned tables for optimized query performance.
Imported and exported large datasets between Hadoop and RDBMS using Sqoop.
Integrated AWS S3, HDFS, and EMR to build scalable data lakes.
Applied partitioning, bucketing, and shuffling strategies to optimize Spark job performance.
Automated PySpark workflows using AWS Step Functions and EMR cluster orchestration.
Tuned Spark jobs for memory optimization and improved execution time.
Designed and developed distributed ETL pipelines using PySpark on AWS EMR., Created and managed Hive external, managed, and partitioned tables for optimized query performance., Imported and exported large datasets between Hadoop and RDBMS using Sqoop., Integrated AWS S3, HDFS, and EMR to build scalable data lakes., Applied partitioning, bucketing, and shuffling strategies to optimize Spark job performance., Automated PySpark workflows using AWS Step Functions and EMR cluster orchestration., Tuned Spark jobs for memory optimization and improved execution time.
Integrated data from multiple sources (AWS S3, HDFS, EMR, EC2) into analytical systems.
Developed and optimized Hive schemas, handling schema evolution with Avro file formats.
Designed and implemented ETL pipelines using Spark and Hive.
Managed Hive partitions and buckets to improve query efficiency.
Collaborated with data architects to design scalable storage and transformation solutions.
Monitored and debugged Spark workflows to ensure data quality and performance.
Integrated data from multiple sources (AWS S3, HDFS, EMR, EC2) into analytical systems., Developed and optimized Hive schemas, handling schema evolution with Avro file formats., Designed and implemented ETL pipelines using Spark and Hive., Managed Hive partitions and buckets to improve query efficiency., Collaborated with data architects to design scalable storage and transformation solutions., Monitored and debugged Spark workflows to ensure data quality and performance.
نظام التوظيف التشغيلي المعتمد على الذكاء الاصطناعي أولاً، الذي يوحّد ويؤتمت دورة التوظيف بالكامل للمؤسسات الحديثة حول العالم.
د.جوب منطقة حرة ذ.م.م. 2026 © جميع الحقوق محفوظة
