We are seeking an experienced Senior Data Engineer to design build and operate enterprise-scale data pipelines and datasets that support advanced analytics enterprise reporting and machine learning use cases. You will play a key role in shaping the Enterprise Data Strategy and Common Data Model while working closely with data scientists analysts and business stakeholders in an Agile environment.
Design build and maintain datasets and data pipelines for analytics and data science use cases
Prepare troubleshoot and optimize data pipelines to ensure reliability performance and data quality
Co-design and evolve the Enterprise Data Strategy and Common Data Model
Implement and operate core Data Platform processes and services
Develop and maintain data pipelines tailored for Data Scientists and analytics teams
Maintain and govern model JSON schemas metadata and data definitions
Identify analyse and resolve data quality and data consistency issues
Support:
Enterprise reporting and analytics
Machine Learning operations (MLOps)
Collaborate with stakeholders across IT data science analytics and business domains using Agile delivery methods
Proven experience working with Big Data platforms and technologies including:
S3 Hive Spark Trino MinIO
Kubernetes (K8S) and Kafka
Strong experience with SQL-based and relational data systems including PL/SQL
Experience handling banking or financial data including governance and regulatory considerations
Hands-on experience with large-scale on-premises Data Lake migrations
Integration of Data Science workbenches such as:
KNIME Cloudera Dataiku (or similar platforms)
Experience working in Agile environments (Scrum SAFe)
Proven stakeholder management and cross-team collaboration skills
Strong understanding of enterprise data reference architectures
Expert-level SQL and data modelling skills
Python for:
Automation
Data processing
Notebooks and analytics workflows
Data preparation techniques for:
Reporting
Advanced analytics
Machine learning
Experience implementing and working with data quality frameworks
Familiarity with streaming and event-driven architectures using Kafka
Required Skills:
We are seeking an experienced Senior Data Engineer to design build and operate enterprise-scale data pipelines and datasets that support advanced analytics enterprise reporting and machine learning use cases. You will play a key role in shaping the Enterprise Data Strategy and Common Data Model while working closely with data scientists analysts and business stakeholders in an Agile environment. Key Responsibilities Design build and maintain datasets and data pipelines for analytics and data science use cases Prepare troubleshoot and optimize data pipelines to ensure reliability performance and data quality Co-design and evolve the Enterprise Data Strategy and Common Data Model Implement and operate core Data Platform processes and services Develop and maintain data pipelines tailored for Data Scientists and analytics teams Maintain and govern model JSON schemas metadata and data definitions Identify analyze and resolve data quality and data consistency issues Support: Enterprise reporting and analytics Machine Learning operations (MLOps) Collaborate with stakeholders across IT data science analytics and business domains using Agile delivery methods Required Experience Proven experience working with Big Data platforms and technologies including: S3 Hive Spark Trino MinIO Kubernetes (K8S) and Kafka Strong experience with SQL-based and relational data systems including PL/SQL Experience handling banking or financial data including governance and regulatory considerations Hands-on experience with large-scale on-premises Data Lake migrations Integration of Data Science workbenches such as: KNIME Cloudera Dataiku (or similar platforms) Experience working in Agile environments (Scrum SAFe) Proven stakeholder management and cross-team collaboration skills Technical Skills Strong understanding of enterprise data reference architectures Expert-level SQL and data modeling skills Python for: Automation Data processing Notebooks and analytics workflows Data preparation techniques for: Reporting Advanced analytics Machine learning Experience implementing and working with data quality frameworks Familiarity with streaming and event-driven architectures using Kafka
IT Services and IT Consulting