Azure Data Engineer – Databricks & Data Factory

Toronto - Canada

Monthly Salary: Not Disclosed

Experience Required: 5years

Posted on: 5 hours ago

Vacancies: 1 Vacancy

Job Summary

Azure Data Engineer Databricks & Data Factory

Experience: 68 years

Role Overview:

We are looking for a Senior Azure Data Engineer with strong expertise in Azure Data Factory Databricks and Azure SQL. The ideal candidate will design develop and maintain data pipelines ETL/ELT processes and data warehousing solutions while collaborating closely with business stakeholders and ensuring high-quality production-ready data solutions.

Key Responsibilities:

Develop optimize and troubleshoot Azure Data Factory pipelines and Databricks notebooks using PySpark Spark SQL and Delta Lake.
Design and implement ETL/ELT processes data ingestion patterns and data warehousing solutions (EDW/BDW).
Monitor and resolve pipeline failures cluster issues and performance bottlenecks.
Collaborate with stakeholders to gather requirements perform data mapping/profiling and document systems effectively.
Utilize Azure DevOps Git version control and CI/CD pipelines for deployment and release management.
Implement monitoring using Azure Monitor Log Analytics and Databricks logs to ensure system reliability.

Essential Skills:

Strong hands-on experience with Azure Data Factory Databricks and Azure SQL.
Expertise in PySpark Spark SQL Delta Lake and Python/Pandas.
Solid understanding of ETL/ELT concepts data ingestion and data warehousing.
Experience with Azure DevOps Git and CI/CD pipelines.
Ability to troubleshoot production issues and optimize pipeline performance.

Desirable Skills:

68 years of professional experience ideally in the insurance domain.
Strong Business Systems Analysis (BSA) skills: requirement gathering PIA completion stakeholder follow-ups data mapping/profiling and documentation.
Familiarity with EDW/BDW Python processes developed in F22/F23.
Self-starter with excellent analytical documentation and problem-solving skills.

Azure Data Engineer Databricks & Data FactoryExperience: 68 yearsRole Overview:We are looking for a Senior Azure Data Engineer with strong expertise in Azure Data Factory Databricks and Azure SQL. The ideal candidate will design develop and maintain data pipelines ETL/ELT processes and data warehou...