Senior Data Engineer python Java and SQL Data pipelines
Senior Data Engineer python Java and SQL Data pipelines
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Senior Data Engineer
Mission of the Role
You are a passionate and skilled data engineer who enjoys building scalable productiongrade data pipelines that enable machine learning and AI at scale. You thrive on solving complex data challenges and understand how critical a robust data pipeline is to the success of any predictive model.
As a Senior Data Engineer on the Forecasting team you will be responsible for designing implementing and maintaining the data pipelines that power deep neural network models used for global sales and revenue forecasting. You will work closely with ML engineers data scientists and product stakeholders to build automated secure and costefficient pipelines using GCPnative services.
You take pride in writing clean testable and productionready code and are comfortable operating in cloud environments. You know how to make data pipelines observable recoverable and performant. You are also comfortable taking architectural ownership and contributing to longterm platform improvements.
You will:
- Design and build robust scalable and maintainable data pipelines for training and inference of deep learning models used in forecasting and other Machine Learning based algorithms.
- Develop and orchestrate endtoend workflows using GCPnative tools such as Cloud Workflows Cloud Run BigQuery Cloud Functions and Pub/Sub.
- Automate and maintain the pipeline lifecycle including data extraction transformation loading (ETL/ELT) and triggering model runs.
- Monitor and Optimise performance costefficiency and data freshness across all stages of the pipeline.
- Collaborate with data scientists to productionise AI/ML models and integrate them seamlessly into business workflows.
- Own the data ingestion validation and transformation logic powering the forecasting models ensuring highquality consistent input data.
- Handle deployment versioning and configuration of pipeline components using Docker Git and CI/CD practices.
- Implement logging monitoring and alerting to ensure reliability and traceability in a highscale environment.
- Review code mentor junior engineers and help define best practices in our evolving data engineering stack.
Qualifications :
You have:
- 7 years of experience in data engineering or backend engineering roles.
- Strong expertise in Python and SQL with experience building productiongrade data pipelines.
- Solid understanding of Docker Git and shell scripting in Linux environments.
- Handson experience with GCP services
- Experience in building deploying and maintaining data workflows that feed AI/ML models.
- Familiarity with model lifecycle management and infrastructure challenges in ML pipelines.
- Proficiency in setting up CI/CD pipelines for data and ML systems using GitLab CI Cloud Build or similar tools.
- Exposure to Java for backend services or pipeline components (even if not primary language).
- A proactive collaborative mindset and strong communication skills across engineering and data science teams.
Nice to have:
- Exposure to forecasting or time series modelling pipelines.
- Experience with eventdriven architectures.
- Familiarity with infrastructureascode tools like Terraform
- Understanding of data quality frameworks and observability tools
- Knowledge of model versioning tools and experiment tracking systems
Additional Information :
Why Join us
- Youll be working on a highimpact platform that forecasts consumer demand and drives businesscritical decisions globally
- Youll be part of a tightknit collaborative and innovative Forecasting & AI team within a leading global data science organisation
- Youll have access to toptier infrastructure GCP services and challenging realworld problems in predictive modelling
- Flexible working hours remotefriendly culture and strong focus on personal and professional growth
- Competitive compensation and performancebased bonuses
Our Benefits
- Flexible working environment
- Volunteer time off
- LinkedIn Learning
- EmployeeAssistanceProgram (EAP)
About NIQ
NIQ is the worlds leading consumer intelligence company delivering the most complete understanding of consumer buying behavior and revealing new pathways to growth. In 2023 NIQ combined with GfK bringing together the two industry leaders with unparalleled global reach. With a holistic retail read and the most comprehensive consumer insightsdelivered with advanced analytics through stateoftheart platformsNIQ delivers the Full View. NIQ is an Advent International portfolio company with operations in 100 markets covering more than 90 of the worlds population.
For more information visit
Want to keep up with our latest updates
Follow us on: LinkedIn Instagram Twitter Facebook
Our commitment to Diversity Equity and Inclusion
NIQ is committed to reflecting the diversity of the clients communities and markets we measure within our own workforce. We exist to count everyone and are on a mission to systematically embed inclusion and diversity into all aspects of our workforce measurement and products. We enthusiastically invite candidates who share that mission to join us. We are proud to be an Equal Opportunity/Affirmative ActionEmployer making decisions without regard to race color religion gender gender identity or expression sexual orientation national origin genetics disability status age marital status protected veteran status or any other protected class. Our global nondiscrimination policy covers these protected classes in every market in which we do business worldwide. Learn more about how we are driving diversity and inclusion in everything we do by visiting the NIQ News Center: Work :
No
Employment Type :
Fulltime
Employment Type
Full-time
