RC-REGULATORY COMPLIANCE-AI-Senior

EY- GDS Consulting Enterprise Risk (ER) Regulatory Compliance Senior GxP AI/Model Evaluation Consultant

As part of our EY-ER- Regulatory Compliance team you will help the clients by understanding their business needs and delivering solutions in accordance with the EY guidelines & methodologies. As a Regulatory Compliance senior actively establish maintain and strengthen internal and external line with our commitment to quality youll consistently drive projects to completion with high quality deliverables achieve operational efficiency proactively identify and raise risks with the client as well as EY senior management team and lead internal initiatives.

The opportunity

We are seeking a highly experienced and detailoriented AI Model Evaluation Consultant with strong expertise in validating AI/ML models Generative AI (GenAI) systems and multiagent workflows within GxPregulated and qualitycritical environments. The ideal candidate will ensure the reliability fairness robustness and regulatory compliance of AI solutions while supporting crossfunctional teams in designing testing and maintaining productionready inspectionready AI systems.

This role is responsible for leading AI model evaluation and validation activities across the full model lifecycle including model development training testing deployment and postdeployment monitoring in alignment with GxP principles riskbased validation and data integrity expectations. The consultant will play a key role in embedding Responsible AI qualitybydesign and governancebydesign practices enabling the safe trustworthy and explainable use of AI and agentbased solutions in regulated life sciences and pharmaceutical contexts.

Your key responsibilities

• Support development of AI/ML and Agentic AI strategies by providing analysis validation insights and readiness assessments.
• Perform AI/ML model evaluation and validation across development training testing and deployment phases.
• Evaluate model performance using precision recall accuracy F1score and confusion matrices.
• Assess model behavior across datasets edge cases and failure scenarios to identify bias drift and instability.
• Demonstrate strong understanding of supervised and unsupervised learning models including classification regression clustering and anomaly detection.
• Conduct handson evaluation of GenAI and LLMbased systems including prompt behavior response quality and consistency.
• Design and execute testing strategies for AI agents and multiagent workflows covering orchestration logic tool invocation and output validation.
• Use LangChain LangGraph and Langfuse for agent workflows observability traceability and evaluation logging.
• Collaborate with data scientists and engineers to identify model improvement opportunities and risk mitigations.
• Document evaluation results limitations risks and acceptance criteria in a clear auditready manner.

• Bachelors or masters degree in Life Sciences Engineering or related field.
• 5 years of experience in AI/ML GenAI software testing validation or quality engineering roles.
• Experience creating user stories test scripts validation plans or AI solution prototypes.
• Exposure to cloud ecosystems such as Azure AWS or GCP.
• Consulting or clientfacing experience preferred.
• Excellent communication documentation and stakeholder management skills.

Must-Have Skills And Attributes

• Strong understanding of ML models LLMs GenAI workflows agentic systems and evaluation techniques.
• Strong foundation in machine learning concepts and model evaluation techniques.
• Handson experience with: Precision Recall Confusion Matrix ROC/AUC Supervised and Unsupervised ML models
• Practical experience with LangChain LangGraph and Langfuse.
• Experience evaluating GenAI / LLMbased applications and agents.
• Strong documentation analytical and stakeholder communication skills.
• Experience designing test cases user stories and acceptance criteria for AI/MLbased systems.
• Ability to validate AI outputs evaluate accuracy detect bias and assess model reliability under varied conditions.
• Handson skills in building prototypes test harnesses and demo environments.
• Strong stakeholder management analytical thinking and structured communication skills.
• Familiarity with version control prompt engineering fundamentals and AIassisted testing tools is preferred.
• Strong knowledge of GxP regulations (FDA 21 CFR Part 11 EU Annex 11)
• GAMP 5 methodology and system development life cycle (SDLC)
• CSV documentation: URS FRS IQ/OQ/PQ Traceability Matrix Validation Plans
• Understanding of data integrity principles and electronic records/e-signatures compliance
• Ability to collaborate with cross-functional teams (QA IT Manufacturing Labs)
• Experience in client-facing roles managing expectations and delivering solutions
• Strong communication and presentation skills
• Experience in IT project management and validation planning
• Ability to troubleshoot application issues and recommend solutions
• Familiarity with change control deviation handling and CAPA processes
• Proven ability to mentor junior staff and lead validation teams
• Experience in training and cross-skilling resources
• Strong teamwork and collaboration mindset

Good-to-Have Skills And Attributes

• Exposure to business development activities and client engagement strategies
• Ability to create innovative insights and contribute to thought leadership
• Understanding of market trends and competitor landscape
• Experience in process optimization and operational efficiency initiatives
• Ability to align technology with business transformation goals
• High attention to detail and analytical thinking
• Adaptability to changing regulatory and business environments
• Strong problem-solving and decision-making abilities

What we look for

• A Team of people with commercial acumen technical experience and enthusiasm to learn new things in this fast-moving environment with consulting skills.
• An opportunity to be a part of market-leading multi-disciplinary team of 1400 professionals in the only integrated global transaction business worldwide.
• Opportunities to work with EY Consulting practices globally with leading businesses across a range of industries

What working at EY offers

At EY were dedicated to helping our clients from startups to Fortune 500 companies and the work we do with them is as varied as they are.

You get to work with inspiring and meaningful projects. Our focus is education and coaching alongside practical experience to ensure your personal development. We value our employees and you will be able to control your own development with an individual progression plan. You will quickly grow into a responsible role with challenging and stimulating assignments. Moreover you will be part of an interdisciplinary environment that emphasizes high quality and knowledge exchange. Plus we offer:

• Support coaching and feedback from some of the most engaging colleagues around
• Opportunities to develop new skills and progress your career
• The freedom and flexibility to handle your role in a way thats right for you