Mohamed Mostafa

Leading the architecture and end-to-end delivery of an enterprise data ingestion and analytics platform on GCP, directing a team of 5 engineers.
Architected scalable pipelines ingesting 500+ APIs plus web scraping and automated file downloads, processing tens of millions of records daily.
Designed the platform's dimensional data model — SCD Type 2 and star schema in the silver layer, with use-case-driven data cubes in the gold layer to serve business reporting and analytics.
Drove architecture and technology-selection decisions for cost optimization — selected Dataform over Dataproc for transformations to avoid unnecessary cluster compute, and right-sized BigQuery editions and Cloud Composer environments to actual user and workload demand.
Established Apigee API Management as the centralized access layer for all external and internal sources, enforcing secure, governed API consumption.
Designed a metadata-driven ingestion framework automating execution, validation, scheduling, and data-quality gating across raw-to-curated layers, reducing onboarding time for new sources by ~50%.
Developed CI/CD pipelines on Azure DevOps with unit and integration testing (cutting release cycles by ~40%), plus Cloud Monitoring–based observability, alerting, and resilient error-handling/retry mechanisms.
Enforced security controls in BigQuery — DLP, RBAC, and column-level security — to protect and mask PII.
Designed and built a real-time streaming proof of concept on Databricks (GCP) using Autoloader and checkpointing.
Delivered technical presentations and live demos that secured stakeholder approval to move the solution toward production.
Implemented a Bronze–Silver–Gold Lakehouse with AI-driven transformations via the ai_query tool.
Integrated GenAI for natural-language analytics, enabling business stakeholders to query data without writing SQL.
Promoted to Senior Consultant after delivering several complex ETL pipelines — previously open for 3–4 months — and completing them to business requirements within two weeks.
Built and optimized the ETL pipelines feeding the warehouse's reporting and analytics layers, improving processing time by ~35% and ensuring timely, high-quality data delivery.
Established data validation frameworks and standardized governance practices across the platform.
Partnered with stakeholders to translate requirements into delivery, and coached junior engineers toward production-ready solutions.

Leading the architecture and end-to-end delivery of an enterprise data ingestion and analytics platform on GCP, directing a team of 5 engineers., Architected scalable pipelines ingesting 500+ APIs plus web scraping and automated file downloads, processing tens of millions of records daily., Designed the platform's dimensional data model — SCD Type 2 and star schema in the silver layer, with use-case-driven data cubes in the gold layer to serve business reporting and analytics., Drove architecture and technology-selection decisions for cost optimization — selected Dataform over Dataproc for transformations to avoid unnecessary cluster compute, and right-sized BigQuery editions and Cloud Composer environments to actual user and workload demand., Established Apigee API Management as the centralized access layer for all external and internal sources, enforcing secure, governed API consumption., Designed a metadata-driven ingestion framework automating execution, validation, scheduling, and data-quality gating across raw-to-curated layers, reducing onboarding time for new sources by ~50%., Developed CI/CD pipelines on Azure DevOps with unit and integration testing (cutting release cycles by ~40%), plus Cloud Monitoring–based observability, alerting, and resilient error-handling/retry mechanisms., Enforced security controls in BigQuery — DLP, RBAC, and column-level security — to protect and mask PII., Designed and built a real-time streaming proof of concept on Databricks (GCP) using Autoloader and checkpointing., Implemented a Bronze–Silver–Gold Lakehouse with AI-driven transformations via the ai_query tool., Integrated GenAI for natural-language analytics, enabling business stakeholders to query data without writing SQL., Delivered technical presentations and live demos that secured stakeholder approval to move the solution toward production., Promoted to Senior Consultant after delivering several complex ETL pipelines — previously open for 3–4 months — and completing them to business requirements within two weeks., Built and optimized the ETL pipelines feeding the warehouse's reporting and analytics layers, improving processing time by ~35% and ensuring timely, high-quality data delivery., Established data validation frameworks and standardized governance practices across the platform., Partnered with stakeholders to translate requirements into delivery, and coached junior engineers toward production-ready solutions.

Designed and deployed ETL workflows using Apache Airflow and Apache Spark to meet evolving business requirements.
Built a Redis-based caching layer for Apache Superset, significantly reducing database load and improving dashboard response time.
Mentored two junior data engineers and optimized workload distribution across Airflow and crontab servers, eliminating redundant jobs and improving query efficiency.
Improved data processing speed by 40% and reduced resource consumption by 30% through targeted workload optimization.
Monitored and stabilized server performance using Linux tools, proactively resolving CPU spikes and system bottlenecks.

Designed and deployed ETL workflows using Apache Airflow and Apache Spark to meet evolving business requirements., Built a Redis-based caching layer for Apache Superset, significantly reducing database load and improving dashboard response time., Mentored two junior data engineers and optimized workload distribution across Airflow and crontab servers, eliminating redundant jobs and improving query efficiency., Improved data processing speed by 40% and reduced resource consumption by 30% through targeted workload optimization., Monitored and stabilized server performance using Linux tools, proactively resolving CPU spikes and system bottlenecks.

Contributed to large-scale data integration initiatives including the ET Contact project, working with 30M+ customer records using Informatica PowerCenter and IBM DataStage.
Supported integration of customer data with Azure Synapse ML models, improving voucher eligibility prediction accuracy, and enhancing customer engagement outcomes.
Developed and maintained Oracle stored procedures to automate voucher offerings and streamline digital operations workflows for business teams.

Contributed to large-scale data integration initiatives including the ET Contact project, working with 30M+ customer records using Informatica PowerCenter and IBM DataStage., Supported integration of customer data with Azure Synapse ML models, improving voucher eligibility prediction accuracy, and enhancing customer engagement outcomes., Developed and maintained Oracle stored procedures to automate voucher offerings and streamline digital operations workflows for business teams.

Worked as a PLC and technical sales engineer, then founded and ran an independent business end-to-end — managing operations, sales, and customer relationships.
Self-studied data engineering (SQL, ETL, Python) out of strong interest in the field, leading to a deliberate transition into a full-time data engineering career at Etisalat Egypt in 2023.

Worked as a PLC and technical sales engineer, then founded and ran an independent business end-to-end — managing operations, sales, and customer relationships., Self-studied data engineering (SQL, ETL, Python) out of strong interest in the field, leading to a deliberate transition into a full-time data engineering career at Etisalat Egypt in 2023.