Azure Data Engineer
Share
Job Location:
Monthly Salary:
Posted on:
Vacancies:
Job Summary
Role Summary
We are looking for a skilled Azure Data Engineer with 4 years of experience in designing building and optimizing data pipelines on Microsoft Azure. The candidate should have strong hands-on experience with Azure Data Factory Azure Databricks Azure Synapse and ADLS Gen2 along with good knowledge of Python SQL PySpark and Delta Lake. The role involves working on ETL development data migration and building scalable data platforms to support analytics and reporting.
Key Responsibilities
- Design develop and maintain end-to-end ETL/ELT data pipelines using Azure Data Factory and Azure Databricks.
- Build and implement Medallion architecture (Raw Staging Refined Enriched layers) for scalable data processing.
- Perform data ingestion transformation validation and loading from multiple sources such as REST APIs Excel files and SQL databases.
- Migrate legacy systems (SSIS-based solutions) to Azure cloud-native data platforms.
- Work with Azure Synapse Analytics to create and manage data pipelines and notebooks.
- Optimize pipeline performance and reduce cloud costs using best practices.
- Develop and manage Databricks notebooks workflows and clusters.
- Implement unit testing integration testing and audit logging to ensure data quality and reliability.
- Collaborate with business stakeholders analysts and reporting teams to support dashboards and Power BI reports.
- Ensure secure access to data using Azure Key Vault and Unity Catalog.
Required Skills & Technical Expertise
- Programming & Data Processing: Python SQL PySpark
- Azure Services:
- Azure Data Factory (ADF)
- Azure Databricks
- Azure Synapse Analytics
- Azure Data Lake Storage (ADLS Gen2)
- Azure SQL / MS SQL Server
- Azure Key Vault
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
