Data Engineer with nearly 5 years of experience designing and delivering scalable data pipelines, cloud data platforms, and analytics solutions across Azure, Databricks, and AWS ecosystems. Specialized in Azure Data Fact…
Data Engineer with nearly 5 years of experience designing and delivering scalable data pipelines, cloud data platforms, and analytics solutions across Azure, Databricks, and AWS ecosystems. Specialized in Azure Data Factory, Databricks, PySpark, Spark SQL, SQL, Delta Lake, AWS S3, and Power BI, with hands-on expertise in end-to-end ETL/ELT development, data migration, data validation, reconciliation, KPI automation, and production-grade data delivery. Proven ability to translate business requirements into reliable, governed, and analytics-ready data solutions while collaborating with cross-functional teams across retail, healthcare, FMCG, and enterprise analytics projects.
<p>Modernizing manual Excel-based KPI reporting into automated Databricks pipelines, using Delta tables as the governed and traceable reporting layer.<br>
Building data workflows to ingest inputs from Aspen and SharePoint Excel files, apply business rules, and publish curated outputs to Power BI.<br>
Developing PySpark, Spark SQL, and SQL logic for SLA metric calculations, sanity checks, reconciliation, archive handling, and historical data loads.<br>
Implementing validation and reconciliation controls to detect incorrect manual changes, reduce data inconsistencies, and improve trust in business-critical reporting.<br>
Automating monthly data refresh, validation, archive handling, and SLA/KPI calculations, reducing dependency on offline Excel-based manual processing.<br>
Preparing architecture and unit testing documentation to support sprint-based delivery, controlled development, production readiness, and auditability.<br>
Coordinating releases and deployments using Bitbucket, Jenkins, VS Code, and Jira within a client-facing Agile delivery model.</p>
<p>Built scalable Databricks-based PySpark ETL pipelines to ingest and transform multi-source data into Delta Lake on AWS S3.<br>
Designed modular batch pipelines to deliver curated, analytics-ready datasets for reporting, analytics, and downstream consumption.<br>
Developed reusable transformation logic using PySpark and Spark SQL for data cleansing, standardization, validation, and business rules.<br>
Implemented Delta Lake features including schema enforcement, versioning, and ACID transactions to improve data reliability and consistency.<br>
Optimized Spark jobs using partitioning, query tuning, and efficient join strategies to improve processing performance and pipeline stability.<br>
Added logging, validation checks, and failure-handling mechanisms to improve production monitoring and reduce execution issues.<br>
Worked with business and analytics stakeholders to translate reporting needs into scalable and maintainable data engineering solutions.</p>
<p>Developed ETL pipelines using Azure Data Factory and Azure Databricks to process ECOM and EPOS data for business reporting and analytics.<br>
Built PySpark, Spark SQL, and SQL transformations to read, cleanse, validate, and write data across Azure Data Lake and downstream systems.<br>
Created ADF workflows using Linked Services, Datasets, Copy Activity, and orchestration activities to automate data movement.<br>
Developed SQL queries, joins, subqueries, and validation checks to support complex reporting and reconciliation requirements.<br>
Supported production validation, data cleanup, process documentation, and issue resolution to maintain reliable data delivery.<br>
Collaborated with business stakeholders in Agile sprint cycles to understand requirements and deliver data engineering solutions.</p>
نظام التوظيف التشغيلي المعتمد على الذكاء الاصطناعي أولاً، الذي يوحّد ويؤتمت دورة التوظيف بالكامل للمؤسسات الحديثة حول العالم.
د.جوب منطقة حرة ذ.م.م. 2026 © جميع الحقوق محفوظة
