Data Engineer

Sheffield - UK

Monthly Salary: Not Disclosed

Posted on: Yesterday

Vacancies: 1 Vacancy

Job Summary

Anticipated Contract End Date/Length: November 30 2026
Work set up: Hybrid 60% office and 40% home (must be eligible for BPSS)

Our client in the Information Technology and Services industry is looking for a Data Engineer to design implement and maintain scalable data pipelines that ingest and process OpenShift telemetry including metrics logs and traces. This role focuses on streaming telemetry via Kafka engineering resilient consumer services for transformation and enrichment and integrating processed data into Splunk for advanced visualization dashboards alerting and analytics. The position plays a key role in advancing observability maturity toward proactive insights while ensuring governance data quality lineage and compliance across hybrid and multi-cluster environments.

What you will do:

Design implement and maintain data pipelines to ingest and process OpenShift telemetry at scale.
Stream OpenShift telemetry via Kafka using producers topics and schemas and build resilient consumer services for transformation and enrichment.
Engineer data models and routing for multi-tenant observability and ensure lineage quality and SLAs across the stream layer.
Integrate processed telemetry into Splunk for visualization dashboards alerting and analytics to achieve Observability Level 4.
Implement schema management using Avro or Protobuf including governance and versioning for telemetry events.
Build automated validation replay and backfill mechanisms to ensure data reliability and recovery.
Instrument services with OpenTelemetry and standardize tracing metrics and structured logging across platforms.
Use LLMs to enhance observability capabilities such as query assistance anomaly summarization and runbook generation.
Collaborate with platform SRE and application teams to integrate telemetry alerts and SLOs.
Ensure security compliance and best practices for data pipelines and observability platforms.
Document data flows schemas dashboards and operational runbooks.

Qualifications :

Hands-on experience building streaming data pipelines with Kafka including producers consumers schema registry Kafka Connect KSQL or KStream.
Proficiency with OpenShift or Kubernetes telemetry using OpenTelemetry Prometheus and CLI tooling.
Experience integrating telemetry into Splunk using HEC UF sourcetypes and CIM and building dashboards and alerting.
Strong data engineering skills in Python or similar languages for ETL ELT enrichment and validation.
Knowledge of event schemas using Avro Protobuf or JSON and understanding of backward and forward compatibility.
Familiarity with observability standards and practices and ability to drive toward Level 4 maturity with proactive monitoring and automated insights.
Understanding of hybrid cloud and multi-cluster telemetry patterns.
Knowledge of security and compliance for data pipelines including secret management RBAC and encryption in transit and at rest.
Strong problem-solving skills and ability to work in a collaborative team environment.
Strong communication and documentation skills.

Additional Information :

All your information will be kept confidential according to EEO guidelines.

Candidates must be legally authorized to live and work in the country where the position is based without requiring employer sponsorship.

HelloKindred is committed to fair transparent and inclusive hiring practices. We assess candidates based on skills experience and role-related requirements.

We appreciate your interest in this opportunity. While we review every application carefully only candidates selected for an interview will be contacted.

HelloKindred is an equal opportunity employer. We welcome applicants of all backgrounds and do not discriminate on the basis of race colour religion sex gender identity or expression sexual orientation age national origin disability veteran status or any other protected characteristic under applicable law.

Remote Work :

Employment Type :

Contract

Anticipated Contract End Date/Length: November 30 2026Work set up: Hybrid 60% office and 40% home (must be eligible for BPSS)Our client in the Information Technology and Services industry is looking for a Data Engineer to design implement and maintain scalable data pipelines that ingest and process ...

Anticipated Contract End Date/Length: November 30 2026
Work set up: Hybrid 60% office and 40% home (must be eligible for BPSS)

What you will do:

Design implement and maintain data pipelines to ingest and process OpenShift telemetry at scale.
Stream OpenShift telemetry via Kafka using producers topics and schemas and build resilient consumer services for transformation and enrichment.
Engineer data models and routing for multi-tenant observability and ensure lineage quality and SLAs across the stream layer.
Integrate processed telemetry into Splunk for visualization dashboards alerting and analytics to achieve Observability Level 4.
Implement schema management using Avro or Protobuf including governance and versioning for telemetry events.
Build automated validation replay and backfill mechanisms to ensure data reliability and recovery.
Instrument services with OpenTelemetry and standardize tracing metrics and structured logging across platforms.
Use LLMs to enhance observability capabilities such as query assistance anomaly summarization and runbook generation.
Collaborate with platform SRE and application teams to integrate telemetry alerts and SLOs.
Ensure security compliance and best practices for data pipelines and observability platforms.
Document data flows schemas dashboards and operational runbooks.

Qualifications :

Hands-on experience building streaming data pipelines with Kafka including producers consumers schema registry Kafka Connect KSQL or KStream.
Proficiency with OpenShift or Kubernetes telemetry using OpenTelemetry Prometheus and CLI tooling.
Experience integrating telemetry into Splunk using HEC UF sourcetypes and CIM and building dashboards and alerting.
Strong data engineering skills in Python or similar languages for ETL ELT enrichment and validation.
Knowledge of event schemas using Avro Protobuf or JSON and understanding of backward and forward compatibility.
Familiarity with observability standards and practices and ability to drive toward Level 4 maturity with proactive monitoring and automated insights.
Understanding of hybrid cloud and multi-cluster telemetry patterns.
Knowledge of security and compliance for data pipelines including secret management RBAC and encryption in transit and at rest.
Strong problem-solving skills and ability to work in a collaborative team environment.
Strong communication and documentation skills.

Additional Information :

All your information will be kept confidential according to EEO guidelines.

Candidates must be legally authorized to live and work in the country where the position is based without requiring employer sponsorship.

HelloKindred is committed to fair transparent and inclusive hiring practices. We assess candidates based on skills experience and role-related requirements.

We appreciate your interest in this opportunity. While we review every application carefully only candidates selected for an interview will be contacted.

Remote Work :

Employment Type :

Contract

Key Skills

Apache Hive
S3
Hadoop
Redshift
Spark
AWS
Apache Pig
NoSQL
Big Data
Data Warehouse
Kafka
Scala

Apply Now

About Company

HelloKindred

Who is HelloKindred?HelloKindred are specialists in staffing marketing, creative and technology roles, offering a range of talent solutions that can be delivered on-site, remotely or hybrid.Our vision is to make work accessible and people’s lives better. We do this by disrupting tradi ... View more

View Profile View Profile

AI AutoApply

Apply to 100+ jobs with one click