Senior Data Engineer

Madrid - Spain

Monthly Salary: Not Disclosed

Posted on: 30+ days ago

Vacancies: 1 Vacancy

Job Summary

Who we are

We are on a mission to make all science open enabling the research community to develop thesolutions we all need to live healthy lives on ahealthy planet.

Frontiers is one of the worlds largest and most impactful research publishers dedicated to making peer-reviewed quality-certified science openly accessible. Our articles have been viewed 4 billion times reflecting the power of research that is open for all.

Follow the links below to learn more about our work.

we can offer you

At Frontiers our working model for new roles includes a balanced approach that fosters collaboration through regular engagement at our office hubs in key locations. While we value flexibility we also believe that shared in-person time strengthens team culture trust and productivity.

We provide a range of benefits across our global locations including but not limited to:

4 additional wellbeing days in addition to existing annual leave allowance
Access to learning platforms and dedicated learning & development time
A range of wellbeing initiatives including free online yoga classes and an employee assistance plan
Employees can dedicate three days each year to volunteer
Additional benefits depending on your location (e.g. pension plan and private medical care)

About Us

We are one of the worlds leading open access publishers dedicated to advancing knowledge and making research accessible to everyone. Our engineering team builds and maintains high-quality data systems that are the backbone of our publishing and peer review pipelines.

Our data products are used by hundreds of internal users across editorial publishing and review workflows. These systems manage complex heterogeneous and high-volume datasets enabling smooth operations and supporting data-driven decisions across the organization. By joining us youll have a direct impact on advancing open science and accelerating research discovery.

Role Overview

As a Senior Data Engineer you will design build and optimize large-scale data pipelines and systems on Azure. Youll be responsible not only for delivery but also for architecture scalability testing and cost efficiency of our data platform. You will ensure our data is accurate secure and reliable while playing a strategic role in influencing technical direction. Beyond hands-on engineering you will mentor peers collaborate with data scientists and analysts and contribute to shaping the future of our data ecosystem.

Responsibilities

Architecture & Design: Shape data models define engineering standards and design pipelines optimized for performance scalability and cost efficiency.
Pipeline Development: Build and maintain scalable data pipelines using Databricks Delta Lake Python SQL and Spark.
Performance Tuning: Optimize pipelines for high-throughput workloads (billions of records) and explore near-real-time streaming where relevant.
Cloud Cost Optimization: Monitor and optimize Azure costs to ensure efficient resource usage.
Workflow Orchestration: Build and optimize pipelines in Azure Data Factory.
Quality & Testing: Apply dbt for data quality checks and implement testing frameworks (unit integration data validation) to ensure robust pipelines.
CI/CD: Implement CI/CD for data pipelines with Azure DevOps.
Collaboration: Work closely with data scientists and analysts to deliver pipelines that power advanced analytics and machine learning.
Strategic Input: Contribute to technical direction and roadmap planning beyond execution.
Documentation & Mentorship: Document complex systems mentor junior engineers and foster a collaborative culture.
Trade-off Evaluation: Weigh complexity costs scalability and coordination overhead when proposing solutions.

Requirements

48 years of experience in Data Engineering or related fields.
Strong expertise in SQL and Apache Spark.
Proven hands-on experience with Databricks and Delta Lake (most critical skills).
Experience with BigQuery (nice to have).
Experience with dbt (for data quality checks).
Good understanding of Docker and containerized environments.
Experience with CI/CD pipelines (Azure DevOps).
Experience with testing data pipelines (e.g. Great Expectations dbt tests unit/integration testing in Spark or Python).
Deep understanding of data engineering principles data governance and quality standards.
Strong problem-solving and analytical skills with attention to detail.
Ability to collaborate across teams and communicate with both technical and non-technical stakeholders.
Leadership and mentoring experience.
Ability to balance trade-offs and apply judgment in complex technical decisions.
Nice to have: Understanding of statistics for data validation and analysis; familiarity with graph databases (Neo4j).

Apply now

Please click the apply button on this page and submit your application in English.

Please inform us if you require any special accommodations to participate fully in our recruitment experience. You can contact us at for any additional support.

Equal opportunity statement

Frontiers actively embraces diversity and is a safe and welcoming workplace. Recruitment is free from discrimination including based on race national or ethnic origin age religion disability sex gender identity or sexual orientation. With employees from more than 50 different nations our diversity creates vibrant teams and constantly challenges us to appreciate multiple perspectives.

Required Experience:

Senior IC

Who we areWe are on a mission to make all science open enabling the research community to develop thesolutions we all need to live healthy lives on ahealthy planet.Frontiers is one of the worlds largest and most impactful research publishers dedicated to making peer-reviewed quality-certified scienc...

Key Skills

Apache Hive
S3
Hadoop
Redshift
Spark
AWS
Apache Pig
NoSQL
Big Data
Data Warehouse
Kafka
Scala

Apply Now

About Company

Frontiers

Open access publisher of peer-reviewed scientific articles across the entire spectrum of academia. Research network for academics to stay up-to-date with the latest scientific publications, events, blogs and news.

View Profile View Profile

AI AutoApply

Apply to 100+ jobs with one click