- Design Medallion architectures (Bronze Silver Gold) using Azure Databricks or Microsoft Fabric.
- Lead the transition of legacy architectures (SQL Server/Synapse) to OneLake and open formats (Delta Lake).
- Implement security catalog and data lineage policies using Microsoft Purview.
- Provide technical escalation for the engineering team on complex Spark cluster performance issues and query optimization.
- Ensure smooth communication between Azure Data Lake Storage (ADLS) Azure Data Factory and Power BI.
- Monitor CU (Capacity Units) and DBU (Databricks Units) usage to optimize processing costs.
Qualifications :
- Advanced experience with PySpark and Spark SQL.
- Deep knowledge of Azure Databricks (Unity Catalog Delta Live Tables).
- Solid experience with Azure Data Factory for complex pipeline orchestration.
- Hands-on experience with Microsoft Fabric (pipelines notebooks and Lakehouses).
- Familiarity with Infrastructure as Code (Terraform or Bicep).
- Experience in DataOps and CI/CD for data (Azure DevOps or GitHub).
- Previous experience as a Senior Data Engineer with interest in moving into Data Architecture.
Additional Information :
The Devoteam Group works for equal opportunities promoting its employees based on merit and actively fights against all forms of discrimination. We are convinced that diversity contributes to the creativity dynamism and excellence of our organization. All of our vacancies are open to people with disabilities.
Remote Work :
No
Employment Type :
Full-time
Design Medallion architectures (Bronze Silver Gold) using Azure Databricks or Microsoft Fabric.Lead the transition of legacy architectures (SQL Server/Synapse) to OneLake and open formats (Delta Lake).Implement security catalog and data lineage policies using Microsoft Purview.Provide technical esca...
- Design Medallion architectures (Bronze Silver Gold) using Azure Databricks or Microsoft Fabric.
- Lead the transition of legacy architectures (SQL Server/Synapse) to OneLake and open formats (Delta Lake).
- Implement security catalog and data lineage policies using Microsoft Purview.
- Provide technical escalation for the engineering team on complex Spark cluster performance issues and query optimization.
- Ensure smooth communication between Azure Data Lake Storage (ADLS) Azure Data Factory and Power BI.
- Monitor CU (Capacity Units) and DBU (Databricks Units) usage to optimize processing costs.
Qualifications :
- Advanced experience with PySpark and Spark SQL.
- Deep knowledge of Azure Databricks (Unity Catalog Delta Live Tables).
- Solid experience with Azure Data Factory for complex pipeline orchestration.
- Hands-on experience with Microsoft Fabric (pipelines notebooks and Lakehouses).
- Familiarity with Infrastructure as Code (Terraform or Bicep).
- Experience in DataOps and CI/CD for data (Azure DevOps or GitHub).
- Previous experience as a Senior Data Engineer with interest in moving into Data Architecture.
Additional Information :
The Devoteam Group works for equal opportunities promoting its employees based on merit and actively fights against all forms of discrimination. We are convinced that diversity contributes to the creativity dynamism and excellence of our organization. All of our vacancies are open to people with disabilities.
Remote Work :
No
Employment Type :
Full-time
View more
View less