Principal Data Engineer

Were 1st Central a market-leading insurance company utilising smart data and technology at pace. Rapid growth has been based on giving our 1.4 million customers exactly what they want: great value insurance with an excellent service. And thats the same for our colleagues too; we won Insurance Employer of the Year at the British Insurance Awards 2024 and our Glassdoor score is pretty mega too!

Werecurrently recruiting for aPrincipal Data Engineerto join1st Central acting as a key bridge between business opportunities and technical solutions. This is a hands-on technical role whereyoullprovide leadership on data solutions across both project delivery and ongoing data products.

Youlltake responsibility for defining the technical direction of the data platform and solution design working closely witharchitectsto ensure alignment with overall platform patterns and standardsand willalso provide clarity and technical direction to engineers building solutions against agreed designs.

While this role does not include direct line management responsibilities youll playan important rolein supportingcoachingand mentoring data engineers ensuring delivery meets the required standards of quality and integrity.

Core skills were looking for to succeed in the role:

• Data Engineering:As a Principal Data Engineeryoullbringexpertiseacross all stages of the SDLC alongside strongcoachingleadershipandmentoringskills.

• Cloud Platforms:Youllhave deep experience withMicrosoftAzure data platforms ensuring best practices are followed tobuildandmaintaina well-managed high-performing platform.
• Technical Skills: Strong experience using Microsoft Azure Apache Spark and Databricks for scalable high-performance data processing.

• Agile Working:Youllunderstand how to develop solutions iteratively within agile teams always focusing on delivering business value.

• Business Intelligence:WithBIas the primary consumer of our data solutionsyoullhave a strong grasp of reporting dashboards and advanced modelling.

• Problem Solving & Thinking:A creative problem solver who thrives on creating simplicity out of complexitywith ability to bring together multiple different views and perspectives to create agreed designs and solutions.

This is aflexible hybrid workingrole with occasional visits to our offices whenrequiredin eitherSalford Quays Manchester Haywards Heath West SussexorGuernsey.We offer great flexibility in working patterns and a company-wide culture to be proud of.

Whatsinvolved:

• Youllbe responsible forcreating or guiding the low-level design of data solutions taking high level solution architectureartefactsand translating them into workable designs and work packages.

• Youllbe responsible forthe quality of the overall data platform(s) ensuring that data pipelines and database solutions are implemented effectively from a re-use and performance optimisation perspective.

• Youllbe responsible forcoding standards low leveldesignand ingestion patterns for the data platform(s) that all users including data engineers follow.

• Youlldevelop high complexity secure governed high quality efficient data pipelines from a variety of on and off premise internal and external data sources.

• Youllset the standards and ensure that data is cleansed mappedtransformedand optimised for storage to meet requirements for business and technical use cases.

• Youlldesign and build of data observability and data quality by design into all Data pipelines promoting self-testing pipelines that proactivelyidentifyprocessing issues or discrepancies.

• Youllbuild solutions that pipe transform data into data lake storage areas physical database models and reporting structures across data lake data warehouse business intelligencesystemsand analytics applications.

• Youllbuild physical data models that are appropriately designed to meet business needs and optimise storage requirements ensuring maximum re-use.

• Youllcarry out unit testing of own code peer testing of others code to ensureappropriate quality andbe responsible forcompleteness and integrity of solutions delivered on the data platform(s).

• Youllensure that effective andappropriate documentationthat brings transparency and understandability are in place for all content on the data platform(s).

• Youllcoach and mentor Senior Data Engineers DataEngineers & Associate Data Engineers.

• Youllcreate high complexity BI solutions including data mart semantic layer and reporting & visualisation solutions in recognised BI tools such as Power BI.

Experience & knowledge:

• Requires extensive experience of designing and building end to end data solutions (10 years ).

• Exceptional at building strong effective relationships with people from different disciplines.

• Experience of carrying out data engineering design and build activities using agile working practices (such as Scrum or Kanban).

• Databricks Designing and optimising scalable data pipelines on Databricks usingPySparkand Spark SQL.

• Data Lake / Lakehouse Experienceimplementing Delta Lake-basedlakehousearchitectures on ADLS Gen2 including optimised storage design hierarchical namespaceMedallionarchitectureand patterns for scalable batch and streaming workloads.

• SynapseAnalytics-Experience designing and implementingtablesschemasstoredproceduresanddata modelswithin Synapse dedicated SQL poolswithknowledge ofSynapsearchitecture and performance.

• Version Control & CI/CD Expertise in Git-based workflows and CI/CD for data platforms including branching strategies and automated deployments via YAML pipelines.

• Unit TestingExperience implementing unit testing for data pipelines to ensure code quality and reliability.

• Data Architecture Experiencedesigning and implementing data models within modernlakehousearchitectures including dimensional modelling (Kimball) for scalable analytics.

• DataQualityexperienceembeddingdata quality by design into data pipelines incorporating validation rules cleansing and transformation logic to deliver trusted high-quality data products.

• Azure Cloud ArchitectureExperiencedesigningand governingsecure scalable Azure environmentsleveragingnetwork isolation (Virtual Network Private Link Firewall) identity and access controls (RBAC PIM) and effective resource organisation (subscriptions resource groups tagging cost management).

• Microsoft Entra ID Experience in implementing identity and access management for cloud data platforms including security groups and service principals for secure authentication and authorisation.

• Infrastructure-as-Code Experiencedesigning and implementing infrastructure using declarative approaches (e.g. Terraform Bicep) enabling automated repeatable and consistent cloud deployments.

• Programming Expertise in Python and SQL for data engineering with languagesto support automation integration and platform operations.

• Keen and active interest in the use of data in the wider industry with practical knowledge and networks.

Skills:

• A creative problem solver who thrives on creating simplicity out of complexity.

• A passion for people and creating environments that enable others to flourish.

• Resilient and comfortable prioritising in demanding situations.

• Highly trustworthy and able tooperatewith integrity and discretion at all times.

• Energetic and proactive and someone who motivates others by their can do will do attitude.

• Able tooperatewith minimal brief and a fast-moving set of changing priorities.

• Ability to bring together multiple different views and perspectives to create agreed designs and solutions.

Ifyourelooking for a role where you can combine hands-on technicalexpertisewith real influence and impactwedlove you to join us and help shapewhatsnext.

What can we do for you

People first. Always. Were passionate about our colleagues and know the best people deserve an extraordinary working environment. We owe it to them so thats what we offer. Our workplaces are energetic inspirational supportive. To get a taste of the advantages youll enjoy take a look at all our perks in full here.

Intrigued Our Talent team can tell you everything you need to know about what we want and what were offering so feel free to get in touch.