Data Engineer-Manager
Job Location:
Monthly Salary:
Posted on:
Vacancies:
Job Summary
Job Description:
-
Design develop and maintain data pipelines and ETL processes using Azure Databricks PySpark and Azure Data Factory/AWS/GCP.
-
Build and optimize scalable data models and data lakes/warehouses on Azure.
-
Develop high-performance Python/PySpark scripts for data transformation and processing.
-
Write tune and optimize SQL queries for large-scale datasets.
-
Work with structured and unstructured data sources (databases APIs files streaming data).
-
Collaborate with data architects business analysts and stakeholders to deliver high-quality solutions.
-
Ensure data governance quality and security standards are followed.
-
Monitor and troubleshoot data pipelines ensuring high availability and performance.
-
Stay updated on emerging Azure technologies and recommend best practices.
