Big Data Engineer
Big Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
dunnhumby is the global leader in Customer Data Science empowering businesses everywhere to compete and thrive in the modern data-driven economy. We always put the Customer First.
Our mission: to enable businesses to grow and reimagine themselves by becoming advocates and champions for their Customers. With deep heritage and expertise in retail one of the worlds most competitive markets with a deluge of multi-dimensional data dunnhumby today enables businesses all over the world across industries to be Customer First.
dunnhumby employs nearly 2500 experts in offices throughout Europe Asia Africa and the Americas working for transformative iconic brands such as Tesco Coca-Cola Meijer Procter & Gamble and Metro.
Most companies try to meet expectations dunnhumby exists to defy them. Using big data deep expertise and AI-driven platforms to decode the 21st century human experience then redefine it in meaningful and surprising ways that put customers first. Across digital mobile and retail. For brands like Tesco Coca-Cola Procter & Gamble and PepsiCo.
Were looking for a Big Data Engineer who expects more from their career. Its a chance to extend and improve dunnhumbys Data Engineering Team Its an opportunity to work with a market-leading business to explore new opportunities for us and influence global retailers.
Joining our team youll work with world class and passionate people which is part of Innovation Technology. You will be responsible for working with stakeholders in the development of data technology that meet the goals of the dunnhumby technology strategy and data principles. Additionally this individual will be called upon to contribute to a growing list of dunnhumby data best practices.
Key Responsibilities
- Build end-to-end data solutions including data lakes data warehouses ETL/ELT pipelines APIs and analytics platforms.
- Build scalable and low-latency data pipelines using tools like Apache Kafka Flink or Spark Streaming to handle high-velocity data streams.
- Automate data pipelines and processes end-to-end using orchestration frameworks such as Apache Airflow to manage complex workflows and dependencies.
- Develop intelligent systems that can detect anomalies trigger alerts and automatically reroute or restart processes to maintain data integrity and availability.
- Develop pipeline for real-time data processing.
- Implement data governance metadata management and data quality standards.
- Explore appropriate tools platforms and technologies aligned with organizational standards.
- Ensure security compliance and regulatory requirements are addressed in all data solutions.
- Evaluate and recommend improvements to existing data architecture and processes.
Technical Expertise
- Bachelors or masters degree in computer science Information Systems Data Science or related field.
- 3 years of experience in data architecture data engineering or a related field.
- Proficient in data pipeline tools such as Apache Spark Kafka Airflow or similar.
- Familiaritywith data governance frameworks and tools (e.g. Collibra Alation OpenMetadata).
- Good experience of cloud platforms (Azure or Google Cloud) especially with cloud-native data services.
- Familiarity of API design and data security best practices.
- Familiarity with data mesh data fabric or other emerging architectural patterns.
- Experience working in Agile or DevOps environments.
- Extensive experience with high level programming languages - Python Java & Scala
- Experience with Hive Oozie Airflow HBase MapReduce Spark along with working knowledge of Hadoop/Spark Toolsets.
- Experience working with Git and Process Automation
- In depth understanding of relational database management systems (RDBMS) and Data Flow Development
Soft Skills
- Problem-Solving: Strong analytical skills to troubleshoot and resolve complex data pipeline issues.
- Communication: Ability to articulate technical concepts to non-technical stakeholders and document processes clearly.
- Collaboration: Experience working in cross-functional teams
- Adaptability: Willingness to learn new tools and technologies to stay ahead in the rapidly evolving data landscape.
What you can expect from us
We wont just meet your expectations. Well defy them. So youll enjoy the comprehensive rewards package youd expect from a leading technology company. But also a degree of personal flexibility you might not expect. Plus thoughtful perks like flexible working hours and your birthday off.
Youll also benefit from an investment in cutting-edge technology that reflects our global ambition. But with a nimble small-business feel that gives you the freedom to play experiment and learn.
And we dont just talk about diversity and inclusion. We live it every day with thriving networks including dh Gender Equality Network dh Proud dh Family dh One and dh Thrive as the living proof. Everyones invited.
Our approach to Flexible Working
At dunnhumby we value and respect difference and are committed to building an inclusive culture by creating an environment where you can balance a successful career with your commitments and interests outside of work.
We believe that you will do your best at work if you have a work / life balance. Some roles lend themselves to flexible options more than others so if this is important to you please raise this with your recruiter as we are open to discussing agile working opportunities during the hiring process.
For further information about how we collect and use your personal information please see our Privacy Notice which can be found (here
What you can expect from us
We wont just meet your expectations. Well defy them. So youll enjoy the comprehensive rewards package youd expect from a leading technology company. But also a degree of personal flexibility you might not expect. Plus thoughtful perks like flexible working hours and your birthday off.
Youll also benefit from an investment in cutting-edge technology that reflects our global ambition. But with a nimble small-business feel that gives you the freedom to play experiment and learn.
And we dont just talk about diversity and inclusion. We live it every day with thriving networks including dh Gender Equality Network dh Proud dh Family dh One and dh Thrive as the living proof. We want everyone to have the opportunity to shine and perform at your best throughout our recruitment process. Please let us know how we can make this process work best for you. For an informal and confidential chat please contact to discuss how we can meet your needs.
Our approach to Flexible Working
At dunnhumby we value and respect difference and are committed to building an inclusive culture by creating an environment where you can balance a successful career with your commitments and interests outside of work.
We believe that you will do your best at work if you have a work / life balance. Some roles lend themselves to flexible options more than others so if this is important to you please raise this with your recruiter as we are open to discussing agile working opportunities during the hiring process.
For further information about how we collect and use your personal information please see our Privacy Notice which can be found (here)
Employment Type
Full Time
