Over 5 years of professional Data Engineer experience with a strong focus on Big Data, Hadoop Ecosystem, Cloud Engineering, and Data Warehousing. Proficient in AWS and Azure services, data pipeline development, Spark, Py…
Over 5 years of professional Data Engineer experience with a strong focus on Big Data, Hadoop Ecosystem, Cloud Engineering, and Data Warehousing. Proficient in AWS and Azure services, data pipeline development, Spark, Python, Scala, SQL, ETL, and CI/CD, with experience across Hadoop distributions, streaming, batch processing, and cloud migration.
Performed data cleansing and applied transformations using Databricks and Spark data analysis.
Experience with Azure cloud platforms (HDInsight, Databricks, Data Lake, Blob, Data Factory, Synapse, SQL DB, and SQL
DWH).
Worked on Azure Synapse analytics service that brings together enterprise data warehousing and Big Data analytics.
Developed JSON Scripts for deploying the Pipeline in Azure Data Factory (ADF) that process the data using the SQL Activity.
Involved in the development of automated workflows for daily incremental loads, moving data from traditional RDBMSs to
data lakes
Performed data cleansing and applied transformations using Databricks and Spark data analysis.
Worked on Azure cloud platforms including HDInsight, Databricks, Data Lake, Blob, Data Factory, Synapse, SQL DB, and SQL DWH.
Worked on Azure Synapse analytics service that brings together enterprise data warehousing and Big Data analytics.
Developed JSON scripts for deploying the pipeline in Azure Data Factory (ADF) that process the data using the SQL Activity.
Involved in the development of automated workflows for daily incremental loads, moving data from traditional RDBMSs to data lakes.
Created pipelines in Azure Data Factory to extract, transform, and load data from different sources like Azure SQL, Blob storage, Azure SQL Data Warehouse, write-back tool, and backwards.
Created database objects such as tables, views, stored procedures, triggers, packages, and functions using T-SQL.
Developed ETL process using Spark.
Facilitated data for interactive Power BI dashboards and reporting purposes.
Used Azure Data Factory, SQL API, and Mongo API and integrated data from MongoDB, MS SQL, and cloud sources including Blob and Azure SQL DB.
Developed SQL scripts for automation purposes.
Used Airflow operators for data orchestration and related Python libraries.
Extensively used Databricks notebooks for interactive analytics using Spark APIs.
Designed and automated custom-built input adapters using Spark, Sqoop, and Oozie to ingest and analyze data from RDBMS to Azure Data Lake.
Analyzed SQL scripts and designed solutions to implement using PySpark.
Performed continuous integration and continuous deployment (CI/CD) of applications into Azure Cloud.
Leveraged Spark's in-memory capabilities to manage large datasets on the S3 Data Lake, loading data into S3 buckets and subsequently filtering and loading it into Hive external tables.
Demonstrated understanding of AWS services including S3, EC2, IAM, and RDS, along with experience in orchestration and data pipeline tools like AWS Step Functions, Data Pipeline, and Glue.
Utilized Python programming skills to construct robust data pipelines and dynamic systems.
Collaborated with the client team to transform data and integrate algorithms and models into automated processes.
Developed batch processing applications that involve functional pipelining utilizing Spark APIs.
Integrated data from various sources while ensuring compliance with data quality and accessibility standards.
Built a data pipeline and conducted analytics using the AWS stack including EMR, EC2, S3, RDS, Lambda, Glue, and Redshift.
Created and modified SQL stored procedures, functions, views, indexes, and triggers.
Executed ETL operations using Python, Spark SQL, S3, and Redshift to derive customer insights from terabytes of data.
Applied knowledge of Hadoop architecture, HDFS commands, and experience in designing and optimizing queries to construct data pipelines.
Developed batch and stream processing applications using Spark APIs for functional pipelining.
Built a data pipeline and conducted analytics using AWS services such as EMR, EC2, S3, RDS, Lambda, Glue, SQS, and Redshift.
Utilized Spark's in-memory capabilities to manage large datasets in S3 Data Lake, filtering and loading data into Hive external tables.
Created and modified SQL stored procedures, functions, views, indexes, and triggers.
Performed ETL operations using Python, Spark SQL, S3, and Redshift to derive customer insights from terabytes of data.
Involved in setting up the CI/CD pipeline using Jenkins, Terraform, and AWS for end-to-end architecture and implementation assessment of various AWS services.
Transformed data using AWS Glue dynamic frames with PySpark, cataloged the transformed data using Crawlers, and scheduled jobs and crawlers using workflow features.
Implemented continuous integration and deployment using CI tools like Jenkins and Bamboo, and worked with EC2 Container Service plugins to automate Jenkins master-slave configuration and deployment processes.
Played a crucial role in physical and logical data modeling, contributing to enhancement in data accuracy through diligent documentation.
Executed advanced SQL queries for data modeling and reporting, and crafted analytical reports with actionable recommendations that drove increase in profitability.
Used Python and Tableau for predictive analytics, generating reports with data visualization, model performance analysis, and predictive insights for informed decision-making.
نظام التوظيف التشغيلي المعتمد على الذكاء الاصطناعي أولاً، الذي يوحّد ويؤتمت دورة التوظيف بالكامل للمؤسسات الحديثة حول العالم.
د.جوب منطقة حرة ذ.م.م. 2026 © جميع الحقوق محفوظة
