Specialist Architecture

Responsibilities

• Lead the architecture design and implementation of advanced analytics solutions using Azure Databricks and Microsoft Fabric.
  
• Apply deep expertise in big data technologies data engineering cloud computing Azure Databricks and strong SQL skills to deliver scalable solutions.
  
• Work closely with business stakeholders and IT teams to understand requirements and deliver effective business-aligned solutions.
  
• Oversee end-to-end implementation of data solutions ensuring alignment with business requirements and industry best practices.
  
• Lead the development of data pipelines and ETL processes using Azure Databricks PySpark and other relevant tools.
  
• Integrate Azure Databricks with Azure services such as Azure Data Lake Azure Synapse Azure Data Factory and on-premises systems.
  
• Provide expertise in Spark; Scala experience is an added advantage.
  
• Demonstrate strong Microsoft Fabric expertise including pipeline development notebook development semantic modelling real-time intelligence and related scenarios.
  
• Provide technical leadership and mentorship to the data engineering team fostering a culture of continuous learning and improvement.
  
• Ensure proper documentation of architecture processes and data flows while maintaining compliance with security and governance standards.
  
• Promote best practices for code quality data security scalability and maintainability.
  
• Stay current with the latest developments in Databricks and associated technologies to support continuous innovation.
  

Essential Skills

• Mandatory French-speaking capability.
  
• Strong experience with Azure Databricks including cluster management notebook development and Delta Lake.
  
• Proficiency in big data technologies such as Hadoop and Spark along with data processing frameworks such as PySpark.
  
• Deep understanding of Azure services including Azure Data Lake Azure Synapse and Azure Data Factory.
  
• Experience with Power BI SSRS SSAS and SKU migration from P-SKU to F-SKU.
  
• Experience with ETL/ELT processes data warehousing and data lake development.
  
• Strong SQL skills and familiarity with NoSQL databases.
  
• Experience with CI/CD pipelines and version control systems such as Git.
  
• Knowledge of cloud security best practices.
  

Soft Skills

• Mandatory French-speaking capability.
  
• Excellent communication skills with the ability to explain complex technical concepts to non-technical stakeholders.
  
• Strong problem-solving skills and a proactive approach to identifying and resolving issues.
  
• Leadership skills with the ability to manage and mentor a team of data engineers.
  

Nice-to-Have Skills

• Power BI experience for dashboarding and reporting.
  
• Microsoft Fabric experience for analytics and integration tasks.
  
• Spark Streaming experience for processing real-time data streams.
  
• Familiarity with Azure Resource Manager templates for infrastructure as code practices.
  

Experience

• Demonstrated expertise of 12 years in developing data ingestion and transformation pipelines using Databricks/Synapse notebooks and Azure Data Factory.
  
• Solid understanding and hands-on experience with Delta tables Delta Lake and Azure Data Lake Storage Gen2.
  
• Experience using Auto Loader and Delta Live Tables for seamless data ingestion and transformation.
  
• Proficiency in building and optimizing query layers using Databricks SQL.
  
• Demonstrated experience integrating Databricks with Azure Synapse ADLS Gen2 and Power BI for end-to-end analytics solutions.
  
• Prior experience developing optimizing and deploying Power BI reports.
  
• Familiarity with modern CI/CD practices especially in the context of Databricks and cloud-native solutions.