نبذة عني
Data Engineer with 2+ years of experience in data warehousing, data integration, and cloud-based ETL solutions. Experienced in designing and optimizing batch data pipelines, data migration workflows, and analytical data …
Data Engineer with 2+ years of experience in data warehousing, data integration, and cloud-based ETL solutions. Experienced in designing and optimizing batch data pipelines, data migration workflows, and analytical data stores using SQL, Python, Azure services, and enterprise ETL concepts. Strong understanding of cloud platforms, business intelligence reporting, and scalable data processing systems.
الخبرة
Azure Data Engineer
Designed and developed scalable ETL pipelines using Azure Data Factory, integrating on-premises and cloud-based data sources to streamline data ingestion from diverse batch file sources
Implemented robust data quality checks and validation rules aligned with business requirements, ensuring data accuracy and consistency across all ingestion processes
Developed data transformation notebooks in Azure Databricks using PySpark and Spark SQL, improving overall data processing performance and efficiency
Created parameterized and reusable Azure Data Factory pipelines to streamline deployment processes and enhance operational efficiency across multiple environments
Optimized SQL queries and stored procedures, significantly enhancing query performance and execution efficiency for critical banking operations
Developed comprehensive data operations and logging mechanisms enabling effective job monitoring, debugging, and issue tracking for improved system reliability
Worked on data warehousing and data integration concepts, including fact-dimension modeling and optimized analytical datasets for reporting use cases
Designed and maintained batch data ingestion pipelines, with exposure to real-time ingestion patterns using event-based triggers
Supported cloud data migration activities, moving structured data from on-premises systems to Azure Data Lake for analytics and reporting
Collaborated with reporting teams to ensure data readiness for BI tools and dashboards
Azure Data Engineer
Designed and developed scalable ETL pipelines using Azure Data Factory, integrating on-premises and cloud-based data sources to streamline data ingestion from diverse batch file sources, Implemented robust data quality checks and validation rules aligned with business requirements, ensuring data accuracy and consistency across all ingestion processes, Developed data transformation notebooks in Azure Databricks using PySpark and Spark SQL, improving overall data processing performance and efficiency, Created parameterized and reusable Azure Data Factory pipelines to streamline deployment processes and enhance operational efficiency across multiple environments, Optimized SQL queries and stored procedures, significantly enhancing query performance and execution efficiency for critical banking operations, Developed comprehensive data operations and logging mechanisms enabling effective job monitoring, debugging, and issue tracking for improved system reliability, Worked on data warehousing and data integration concepts, including fact-dimension modeling and optimized analytical datasets for reporting use cases., Designed and maintained batch data ingestion pipelines, with exposure to real-time ingestion patterns using event-based triggers., Supported cloud data migration activities, moving structured data from on-premises systems to Azure Data Lake for analytics and reporting., Collaborated with reporting teams to ensure data readiness for BI tools and dashboards.
المشاريع
End-t o-End Data Ingest ion Plat form
End-to-End Data Ingestion PlatformDesigned and developed scalable ETL pipelines using Azure Data Factory (ADF) to integrate on-premises and cloud-based data sources, streamlining batch data ingestion from diverse file sources.Implemented robust data quality checks and validation rules aligned with business requirements, ensuring data accuracy and consistency across ingestion processes.Developed data transformation notebooks in Azure Databricks using PySpark and Spark SQL, improving overall data processing performance and efficiency.Created parameterized and reusable Azure Data Factory pipelines, enabling seamless deployments and enhancing operational efficiency across multiple environments (Dev, QA, Prod).Optimized complex SQL queries and stored procedures, significantly improving query performance and execution efficiency for critical banking operations.Built comprehensive data operations and logging frameworks for effective job monitoring, debugging, and issue tracking, improving overall system reliability.Applied data warehousing concepts including fact-dimension modeling (Star Schema) and optimized analytical datasets for reporting and business intelligence use cases.Designed and maintained batch ingestion pipelines, with exposure to near real-time ingestion patterns using event-based triggers.Supported cloud data migration initiatives, moving structured data from on-premises systems to Azure Data Lake for analytics and reporting.Collaborated with reporting and BI teams to ensure data readiness for dashboards and business intelligence tools.