Senior Data Engineer

Our data team has expertise across engineering analysis architecture modeling machine learning artificial intelligence and data science. This discipline is responsible for transforming raw data into actionable insights building robust data infrastructures and enabling data-driven decision-making and innovation through advanced analytics and predictive modeling. 

Youll design build and maintain data pipelines using a medallion architecture transforming operational data into trusted well-structured datasets for reporting and analysis.

The role combines hands-on development data modelling production support and technical leadership. Youll work closely with development and data teams product owners and business stakeholders to deliver reliable data products that meet reporting and operational needs.

Responsibilities

• Work with analysts and stakeholders to understand source systems data flows business processes and reporting requirements clarifying business rules and challenging unclear assumptions.
• Design build and maintain ETL/ELT pipelines in Azure Databricks using Python PySpark and SQL.
• Implement bronze silver and gold data layers covering raw data ingestion cleansing validation and preparation for reporting.
• Develop curated datasets and dimensional models and make them available through Microsoft Fabric for Power BI reporting.
• Implement incremental processing and manage schema changes duplicates missing values and late-arriving data.
• Provide technical guidance and mentoring lead code and design reviews and support consistent development testing and CI/CD practices.
• Contribute to product and delivery discussions by explaining technical trade-offs identifying data dependencies and helping prioritise work based on business value and technical constraints.
• Document data models transformations lineage dependencies and operational procedures.

Qualifications :

• 7 years of experience in data engineering including designing building deploying and supporting production data pipelines.
• Strong Python and SQL skills with strong hands-on experience using PySpark and Spark SQL in Azure Databricks.
• Experience with Microsoft Fabric particularly making curated data available for reporting through OneLake lakehouses or warehouses.
• Good understanding of data warehousing and dimensional modelling including table grain facts dimensions and historical changes.
• Experience orchestrating and monitoring data pipelines.
• Experience with automated testing data validation Git and CI/CD practices.
• Working knowledge of Azure storage access management and secure credential handling.
• Taking ownership of technical decisions reviewing designs and code and guiding other engineers.
• Ability to understand business processes and product priorities translate requirements into practical data solutions and explain how technical choices affect users reporting and operational decisions.
• Ability to work independently explain technical decisions clearly and collaborate effectively in English with business and technical colleagues.
  
  Useful additional experience
• Experience with Snowflake.
• Experience delivering data solutions in enterprise environments. Familiarity with logistics transportation or supply chain data is an advantage.
• Enterprise data modelling and aligning data definitions across source systems.
• Practical use of AI development tools.

Additional Information :

Discover some of the global benefits that empower our people to become the best version of themselves:

• Finance: Competitive salary package share plan company performance bonuses value-based recognition awards referral bonus;  
• Career Development: Career coaching global career opportunities non-linear career paths internal development programmes for management and technical leadership;
• Learning Opportunities: Complex projects rotations internal tech communities training certifications coaching online learning platforms subscriptions pass-it-on sessions workshops conferences;
• Work-Life Balance: Hybrid work and flexible working hours employee assistance programme;
• Health: Global internal wellbeing programme access to wellbeing apps;
• Community: Global internal tech communities hobby clubs and interest groups inclusion and diversity programmes events and celebrations.

At Endava were committed to creating an open inclusive and respectful environment where everyone feels safe valued and empowered to be their best. We welcome applications from people of all backgrounds experiences and perspectivesbecause we know that inclusive teams help us deliver smarter more innovative solutions for our customers. Hiring decisions are based on merit skills qualifications and potential. If you need adjustments or support during the recruitment process please let us know.

Remote Work :

No

Employment Type :

Full-time