Azure Data Platform Engineer (Databricks Focus)

Brussels - Belgium

Monthly Salary: Not Disclosed

Posted on: 21 hours ago

Vacancies: 1 Vacancy

Job Summary

Description

Key Responsibilities Azure & Databricks Platform Engineering

Engineer operate and evolve enterprisegrade Azure Databricks platforms across multiple subscriptions and environments (dev test prod).
Design and implement Databricks workspace configurations cluster policies pools and job orchestration with a strong focus on scalability reliability and cost efficiency.
Engineer and maintain Azure platform components supporting Databricks including networking identity storage and security services.
Monitor platform health performance and usage; proactively design improvements and resolve operational issues.

DatabricksCentric Data & Analytics Enablement

Engineer and optimize Databricks integrations with Azure services including:
- Azure Data Lake Storage (ADLS Gen2)
- Delta Lake and Unity Catalog
- Azure SQL Azure Synapse Event Hubs and other data sources
Support batch and streaming data pipelines and analytics workloads running on Databricks.
Partner with Data Engineering Analytics and AI teams to ensure performant reliable and secure production workloads.

Security Governance & Compliance Engineering

Design and implement security and governance controls across Databricks and Azure including:
- Azure Active Directory integration RBAC and SCIM provisioning
- Unity Catalog data access policies and cluster governance
Engineer secure integrations using Azure Key Vault Private Endpoints and network security constructs.
Ensure platform designs comply with organizational security governance and audit requirements.

Infrastructure as Code & Platform Automation

Design develop and maintain Terraformbased IaC for Databricks and supporting Azure infrastructure (workspaces permissions networking storage).
Build and evolve CI/CD pipelines using Azure DevOps YAML pipelines for platform and infrastructure deployments.
Promote engineering best practices for IaC including modularization versioning automated testing and environment consistency.

Collaboration & Platform Leadership

Act as a subject matter expert for Databricks on Azure guiding teams on platform usage architecture and operational best practices.
Work crossfunctionally with Cloud Architecture CICD Data AI and Application teams.
Create and maintain platform documentation reference architectures and operational runbooks.

Required Qualifications

5+ years of handson experience engineering solutions on Microsoft Azure with a strong focus on Azure Databricks platforms.
Deep expertise in:
- Azure Databricks (clusters jobs pools Delta Lake Unity Catalog workspace engineering)
- Azure core services (Storage Networking Identity & Access Management)
- Cost optimization monitoring and observability for data platforms
Strong experience with Terraform including multisubscription and multienvironment deployments.
Proficiency with CI/CD pipelines preferably Azure DevOps.
Solid understanding of cloud security governance and platform engineering practices.

Preferred Certifications Azure & Databricks Certifications

Microsoft Certified: Azure Administrator Associate
Microsoft Certified: Azure Solutions Architect Expert
Databricks Certified Data Platform Engineer or Administrator

HashiCorp Terraform Certifications

HashiCorp Terraform Associate (003 or later)
HashiCorp Terraform Engineer

Soft Skills

Strong communication and documentation skills.
Ability to explain complex Azure and Databricks platform designs to diverse technical audiences.
Engineering mindset with a strong focus on automation reliability and continuous improvement.
Collaborative proactive and adaptable in a fastevolving cloud and data landscape.

Required Experience:

DescriptionKey Responsibilities Azure & Databricks Platform EngineeringEngineer operate and evolve enterprisegrade Azure Databricks platforms across multiple subscriptions and environments (dev test prod).Design and implement Databricks workspace configurations cluster policies pools and job orchest...