Sr Engineer – GenAI Quality Assurance

Barcelona - Spain

Monthly Salary: Not Disclosed

Posted on: 30+ days ago

Vacancies: 1 Vacancy

Job Summary

At Johnson & Johnsonwe believe health is everything. Our strength in healthcare innovation empowers us to build aworld where complex diseases are prevented treated and curedwhere treatments are smarter and less invasive andsolutions are our expertise in Innovative Medicine and MedTech we are uniquely positioned to innovate across the full spectrum of healthcare solutions today to deliver the breakthroughs of tomorrow and profoundly impact health for more at

Job Function:

R&D Product Development

Job Sub Function:

R&D Machine Learning

Job Category:

Scientific/Technology

All Job Posting Locations:

Cornellà de Llobregat Barcelona Spain Madrid Spain

Job Description:

Johnson & Johnson Innovative Medicine R&D Data Science and Digital Health is recruiting for a SeniorEngineer GenAI Quality Assurance. This role can be based in Madrid or Barcelona. Hybrid working.

J&J Innovative Medicine develops treatments that improve the health of people worldwide. Research and development areas encompass oncology immunology neuroscience cardiopulmonary and specialty ophthalmology. Our goal is to help people live longer healthier lives. We have produced and marketed many first-in-class prescription medications and are poised to serve the broad needs of the healthcare market from patients to practitioners and from clinics to hospitals. To learn more about Johnson & Johnson Innovative Medicine visit Were hiring a professional who specializes in designing testing and refining complex GenAI applications primarily in scientific or ML-driven environments. Youll be responsible for validating stress-testing and improving Gen-AI powered products (pipelines and AI-assisted workflows) across the product lifecycle from pre-deployment validation to post-release monitoring.

Key Responsibilities:

Design and implement automated and manual QA strategies for GenAI systems
Build regression tests generate synthetic test data build test harnesses and evaluation pipelines for accuracy grounding robustness safety
Validate prompt logic model behavior and evaluation metrics
Execute tests and debug pipelines
Support deployment of GenAI products
Lead system design reviews advocating best practices for architecture and reliability of GenAI applications
Collaborate closely with data scientists and developers to embed testing and quality checkpoints
Establish testing standards and workflows for continuous integration and deployment
Conduct code reviews with a focus on design testability and maintainability
Document system design decisions test cases and best practices
Close collaboration with domain experts and stakeholders to validate developed solutions

Technical Skills & Experience Needed:

Strong software engineering background with QA ownership
Familiarity with RAG & Graph frameworks
3 years of experience as a software engineer (preferably 5) or QA engineer in production systems preferably including at least 1 year experience in GenAI systems
Advanced Python skills
Experience working with LLM-based systems in real environments
Hands-on experience with:

LLM APIs
- AWS Bedrock (Anthropic & OSS)
- Azure Foundry (OpenAI)
- GCP (Gemini)
Cloud platforms (AWS Azure GCP)
Proven ability to implement test frameworks for GenAI applications

Knowledge of API testing data validation and data pipeline testing
Experience with version control (Git)
Experience with building and deploying AI/ML products through the full product lifecycle
Good understanding of scientific computing and ML frameworks
Solid understanding of safety bias and failure modes of GenAI systems
Comfortable working in a relatively fast-moving environment where priorities may shift and initiative is valued
Proactive problem-solver who can anticipate needs and find solutions
Comfortable working independently but also a strong team player

Nice-to-Have:

Experience with Jenkins Sonarqube
Monitoring and observability tools (Prometheus Grafana etc)
Devops experience (docker jenkins)
Experience deploying GenAI services to production at scale
Exposure to ML Ops or production ML systems

#LI-GR

#LI-Hybrid

#JRDDS
#JNJDataScience
#JRD

Required Skills:

Large Language Models (LLMs) Microsoft Azure AI foundry OpenAI Python (Programming Language) Software Quality Assurance (SQA)

Preferred Skills:

Artificial Intelligence (AI) Coaching Cognitive Computing Critical Thinking Cross-Functional Collaboration Curious Mindset Data Structures Distributed Computing Emerging Technologies Human-Computer Relationships Machine Learning (ML) Persistence and Tenacity Program Management Research and Development SAP Product Lifecycle Management Scripting Languages Technologically Savvy

Required Experience:

Senior IC