ETL Data Lead Developer

At Circana we are fueled by our passion for continuous learning and growth we seek and share feedback freely and we celebrate victories both big and small in an environment that is flexible and accommodating to our work and personal lives.â Were a global company dedicated to fostering inclusivity and belonging. We value and celebrate the unique experiences cultures and viewpoints that each individual brings. By embracing a wide range of backgrounds skills expertise and beyond we create a stronger more innovative environment for our employees clients and communities. With us you can always bring your full self to work.â Join our inclusive committed team to be a challenger own outcomes and stay curious together. Circana is proud to be Certified by Great Place To Work. This prestigious award is based entirely on what current employees say about their experience working at Circana.
Learn more at .

What is the role & team:

Join our Global Professional Services team as a Lead Data Engineer where you will be instrumental in bridging the gap between client needs business objectives and cutting-edge technology solutions particularly supporting our new Private Cloud clients.

This role is for more than just a service provider; we are seeking aconsultant-minded go-getterwho isclient-obsessed aliquid thinkerwho thrives at pace and isbold and bravein challenging the status quo. We are a team that values being part of a successful business and building partnerships based ontrust deep knowledge respect and unwavering support.

We arepassionate about technology data and AIand how it can be leveraged to solve our clients most pressing business challenges. You will be a key player in understanding our clients world becoming their trusted partner and proactively identifying opportunities for innovation and efficiency.Curiosity accountability and a positive questioning mindsetare at the core of our teams DNA we dont just aim to meet expectations we strive to exceed them.

You will work collaboratively with global Circana teams acting as the vital link to ensure seamless unified support and strategic technological partnership for our clients.

In this role we are seeking a skilled and motivated Data Engineer to join a growing Global Team based in the UK. You will be responsible for designing building and maintaining robust data pipelines and infrastructure on the Azure cloud platform. You will leverage your expertise in PySpark Apache Spark and Apache Airflow to process and orchestrate large-scale data workloads ensuring data quality efficiency and scalability. If you have a passion for Data Engineering and a desire to make a significant impact we encourage you to apply!

Key Responsibilities:

Data Engineering & Data Pipeline Development

• Design develop and optimize scalable DATA workflows using Python PySpark and Airflow
• Implement real-time and batch data processing using Spark
• Enforce best practices for data quality governance and security throughout the data lifecycle
• Ensure data availability reliability and performance through monitoring and automation.

Cloud Data Engineering:

• Manage cloud infrastructure and cost optimization for data processing workloads
• Implement CI/CD pipelines for data workflows to ensure smooth and reliable deployments.

Big Data & Analytics:

• Build and optimize large-scale data processing pipelines using Apache Spark and PySpark
• Implement data partitioning caching and performance tuning for Spark-based workloads.

• Work with diverse data formats (structured and unstructured) to support advanced analytics and machine learning initiatives.

Workflow Orchestration (Airflow)

• Design and maintain DAGs (Directed Acyclic Graphs) in Airflow to automate complex data workflows
• Monitor troubleshoot and optimize job execution and dependencies

Team Leadership & Collaboration

• Lead a team of data engineers providing technical guidance and mentorship
• Foster a collaborative environment and promote best practices for coding standards version control and documentation.

Required Skills & Experience:

• Client facing role so strong communication and collaboration skills are vital (Ideally gained in a Consulting organisation)
• 8 years of experience in data engineering with expertise in Azure PySpark Spark and Airflow.
• Strong programming skills in Python SQL with the ability to write efficient and maintainable code
• Deep understanding of Spark internals (RDDs DataFrames DAG execution partitioning etc.)
• Experience with Airflow DAGs scheduling and dependency management
• Knowledge of Git Docker Kubernetes Terraform and apply best practices of DevOps for CI/CD workflows
• Excellent problem-solving skills and ability to optimize large-scale data processing.
• Experience in leading teams and working in Agile/Scrum environments
• A proven track record of working effectively global remote teams

Bonus Points:

• Experience with data modeling and data warehousing concepts
• Familiarity with data visualization tools and techniques
• Knowledge of machine learning algorithms and frameworks

What are we looking for

We are seeking an individual who embodies the following characteristics:

• Anauthentic optimistwith a genuine passion for solving problems and driving progress.
• Intenselycurious you love to understand the why and how behind things especially when it comes to technology and client challenges.
• Highlyaccountablefor your actions your deliverables and the success of your clients.
• Comfortable and adept atbuilding strong professional relationshipswith both external clients and internal technical and business teams.
• Deeplycommittedto giving your best for your team and ensuring our technology solutions exceed client expectations.
• Confident inmaking informed decisions weighing options and considering the technical and business implications.
• Highlyorganised with meticulous attention to detail in managing documentation communication and time.
• Passionate abouttechnology Data AI and their applicationin solving real-world business problems.
• Previous experience translating client requirements into technical specifications
• Previous experience building relationships with clients.

#LI-AM19