Lead Data Engineer

Responsibilities

• Lead the design build and optimisation of the organisations Microsoft Fabric-based data platform implementing a medallion architecture (bronze silver gold) to ensure data is structured governed and analytics-ready.

• Architect and manage scalable data pipelines within Microsoft Fabric including Data Factory pipelines with REST API integrations to automate and orchestrate multi-source ingestion workflows.

• Own the end-to-end delivery of enterprise-level Power BI reporting solutions including advanced DAX modelling Power Query transformations and performance tuning across large semantic models.

• Design and maintain Power BI Embedded integrations ensuring reliable report rendering token management row-level security and seamless application-level embedding for internal and external users.

• Establish and execute robust processes for rolling out Power BI reports into production including structured Dev/QA/Prod workspace strategies deployment pipelines and automated validation gates.

• Define and enforce data quality standards across the platform building testing frameworks that span ingestion transformation and semantic model layers to ensure reporting accuracy and trust.

• Use Python and PySpark notebooks within Fabric for data transformation automation data quality checks and AI-related scripting in collaboration with the data science team.

• Translate complex business requirements into scalable maintainable reporting and data platform solutions acting as a bridge between technical delivery and stakeholder objectives.

• Monitor and optimise platform and report performance to ensure reliability cost-efficiency and scalability across the Microsoft Fabric and Power BI environments.

• Mentor and guide data engineers establishing best practices for development standards code review documentation and deployment workflows.

• Maintain comprehensive documentation for data architecture pipeline logic semantic models and operational processes supporting team collaboration and business continuity.

• Manage and prioritise ad hoc data requests through structured intake processes ensuring high accuracy timely turnaround and clarity of deliverables.

Required Skills & Experience

• Strong hands-on experience with Microsoft Fabric including Lakehouses Warehouses Data Factory pipelines and Notebooks.

• Demonstrated experience designing and implementing medallion architecture (bronze/silver/gold) patterns in a modern cloud data platform.

• 5 years of hands-on development experience with Power BI including advanced DAX Power Query semantic model design and Power BI Service administration.

• Proven track record of building and executing production rollout processes for Power BI reports including deployment pipelines workspace strategies and validation gates.

• Strong experience with Power BI Embedded including capacity management token-based authentication RLS enforcement and troubleshooting embedding scenarios.

• Proficiency in T-SQL and SQL-based querying for cloud data platforms and relational databases with deep understanding of schema design indexing and optimisation best practices.

• Experience with SQL analytics endpoints within Microsoft Fabric or equivalent cloud-native SQL tooling.

• Proficiency in Python and/or PySpark for data transformation automation testing and AI-related development.

• Strong understanding of Row-Level Security (RLS) object-level security workspace governance and access control in enterprise Power BI environments.

• Experience with Azure DevOps or equivalent CI/CD tooling to automate testing deployment and release management of data solutions.

• Excellent analytical thinking problem-solving and stakeholder communication skills with a track record of influencing technical decisions.

• Experience leading or mentoring junior engineers in a data platform or BI team context.

Key Competencies

• Data Platform Architecture: Demonstrates ability to design build and govern a Microsoft Fabric-based data platform using medallion architecture principles.

• SQL & Data Modelling: Demonstrates advanced proficiency in SQL and T-SQL for querying modelling and optimising complex datasets within cloud-native environments.

• Power BI Mastery: Demonstrates deep expertise in DAX Power Query semantic model design Power BI Embedded and enterprise reporting governance including RLS.

• Production Delivery: Demonstrates proven ability to establish and execute structured rollout processes for Power BI reports from development through to production.

• CI/CD & Deployment: Demonstrates ability to manage and automate deployments using Azure DevOps Pipelines Fabric deployment pipelines and Power BI deployment workflows.

• Python & Notebook Development: Demonstrates capability in using Python and PySpark within Fabric for data transformation quality assurance and AI team support.

• Data Quality Ownership: Demonstrates a proactive approach to data quality building validation frameworks and testing processes across the full data lifecycle.

• Performance Engineering: Demonstrates attention to performance tuning capacity planning cost-efficiency and reliability across platform and reporting infrastructure.

• Technical Leadership: Demonstrates ability to mentor junior engineers set development standards and drive best practices across the data team.

• Cross-Team Collaboration: Demonstrates strong communication and delivery alignment with both technical and non-technical stakeholders at all levels.

• Lifecycle Ownership: Demonstrates end-to-end ownership across the full data lifecycle from ingestion and modelling to visualisation deployment and ongoing governance.

Qualifications

• Degree or equivalent qualification in Data Science Computer Science Information Systems Mathematics or a related field.

• Microsoft certifications such as Fabric Analytics Engineer Associate (DP-600) Power BI Data Analyst Associate (PL-300) or Azure Data Engineer Associate (DP-203) preferred or in progress.

• Proven experience leading data platform or BI initiatives in a professional environment with demonstrable production delivery outcomes.

Whats in it for You

• EMI company share scheme meaning you would own a percentage of shares turning into significant monetary value in time.

• Competitive salary.

• Westfield Health cash plan.

• 1-2-1 Sanctus coaching sessions to support personal and professional development with proactive mental wellbeing support.

• Flexible working.

• A paid day off on your birthday.

• Aviva Life insurance.

• Workplace Nursery Benefit.

• Long service awards to celebrate key employment anniversaries.

• Excellent discounts/wider wallet.

• 5% matched contribution pension scheme

• Pension salary exchange scheme

• 25 days holiday bank holidays (33 days).

• Paid wellbeing days volunteer days and study days.

• Quarterly All Star award.

• Annual company Summer event.

• Modern homeworking tech kit.

• Enhanced maternity & paternity pay.

• And being part of something AMAZING!

Cognassist is a Disability Confident Committed employer and we welcome your application even if you believe you do not meet all of the above criteria - your unique skills and experiences are valued and your contribution could be exactly what we need to grow together.