Microsoft Fabric Architect

Job Summary:

We are looking for a highly skilled and experienced Microsoft Fabric Engineer / Architect to design build and maintain our enterprise-grade data and machine learning solutions. The ideal candidate will possess a strong background in data engineering with extensive hands-on experience across the Microsoft Fabric ecosystem. A deep understanding of MLOps principles and experience with Azure Databricks are highly preferred. This role will be instrumental in enabling our data scientists and analysts to efficiently develop deploy and manage data assets and machine learning models at scale within a robust and governed environment.

Responsibilities:

• Design implement and optimize scalable and secure data warehousing Lakehouse and analytics solutions within Microsoft Fabric (e.g. Lakehouse Data Warehouse Pipelines Dataflows Gen2 Notebooks).
• Develop and manage data ingestion transformation and storage solutions for analytical and machine learning workloads leveraging various Fabric components and Azure data services (e.g. Azure Data Lake Storage Gen2 Azure SQL Database).
• Collaborate closely with data scientists machine learning engineers and business stakeholders to understand data and analytical requirements and translate them into efficient technical designs within Fabric.
• Implement and maintain CI/CD pipelines for data solutions and where applicable machine learning models ensuring automated testing deployment and monitoring within Azure DevOps or similar tools.
• Optimize data processing and query performance within Fabric utilizing Spark SQL and KQL where appropriate.
• Implement data governance security and compliance standards (e.g. Microsoft Purview) across the Fabric environment ensuring data quality lineage and access control.
• Monitor the performance and health of deployed data solutions and underlying Fabric infrastructure implementing robust logging and alerting.
• Troubleshoot and resolve complex technical issues related to Microsoft Fabric components and data pipelines.
• Stay current with the latest Microsoft Fabric features Azure updates and MLOps best practices advocating for and implementing new technologies to enhance our data capabilities.
• If applicable design and implement data pipelines and machine learning workflows utilizing Azure Databricks optimizing for performance and cost.
• Provide technical guidance and mentorship to junior engineers and data analysts.

Required Qualifications:

• Bachelors or Masters degree in Computer Science Engineering Data Science or a related quantitative field.
• 5-10 years of hands-on experience in Data Engineering with a strong focus on building and maintaining large-scale data pipelines and data platforms.
• Extensive hands-on experience with Microsoft Fabric including a deep understanding and practical application of its various components (e.g. Lakehouse Data Warehouse Data Pipelines Notebooks Dataflows Gen2).
• Proficiency in SQL Python and/or PySpark for data manipulation scripting and development within Fabric notebooks.
• Strong understanding of cloud architecture principles and best practices on Microsoft Azure.
• Experience with Infrastructure as Code (IaC) tools such as Azure Resource Manager (ARM) templates or Terraform.
• Solid understanding of data warehousing concepts dimensional modeling and ETL/ELT processes.
• Excellent problem-solving skills and the ability to troubleshoot complex technical issues.
• Strong communication and collaboration skills with the ability to work effectively with cross-functional teams.

Preferred Qualifications:

• Hands-on experience with Azure Databricks including Spark optimization Delta Lake and Databricks notebooks for advanced data engineering and machine learning workloads.
• Experience with Power BI for data visualization and reporting and connecting to Fabric data sources.
• Familiarity with Azure DevOps for source control CI/CD and project management.
• Experience with other Azure data services such as Azure Data Factory Azure Synapse Analytics or Azure Stream Analytics.
• Microsoft Certifications relevant to data and analytics (e.g. Microsoft Certified: Fabric Analytics Engineer Associate Azure Data Engineer Associate Azure Solutions Architect Expert).