AI Quality Engineering & Assurance

Job Title: Director -- AI Quality Engineering & Assurance
Job Summary
We are seeking an experienced AI Quality Engineering Leader with 15 years of experience in Quality Assurance Test Strategy and Digital Transformation including leadership in AI/ML and Generative AI quality initiatives. This role will be responsible for defining and driving the enterprise AI Quality Assurance vision governance framework testing strategy and operational excellence across AI-powered products platforms and intelligent automation solutions.
The ideal candidate will lead global QA teams establish AI testing standards ensure Responsible AI compliance and partner with Engineering Product Data Science Security Risk and Business leaders to deliver trusted scalable and high-performing AI solutions.

Key Responsibilities
AI Quality Strategy & Leadership
Define and execute the enterprise-wide AI Quality Engineering and Testing Strategy.
Establish AI QA Centers of Excellence (CoE) quality governance models and AI testing best practices.
Lead quality transformation initiatives for AI Generative AI Agentic AI Machine Learning and Intelligent Automation programs.
Drive organizational adoption of AI quality standards and frameworks.
Build and mentor high-performing QA and Quality Engineering teams across multiple geographies.
AI Model & LLM Quality Assurance
Define quality metrics and evaluation frameworks for AI/ML and Generative AI systems.
Establish validation approaches for:
o Accuracy
o Hallucination detection
o Groundedness
o Bias and fairness
o Toxicity and safety
o Robustness and reliability
Lead testing of AI Assistants Copilots RAG solutions Chatbots and Agentic AI systems.
Implement continuous AI quality monitoring and model performance assessment.
Governance Risk & Responsible AI
Establish Responsible AI testing and compliance frameworks.
Ensure adherence to enterprise security privacy regulatory and ethical AI requirements.
Collaborate with Legal Compliance Security and Risk teams to manage AI-related risks.
Define audit readiness and governance processes for AI solutions.
Test Engineering & Automation
Drive adoption of next-generation test automation powered by AI.
Establish automation and quality engineering practices across cloud-native and AI ecosystems.
Define test data management and synthetic data strategies.
Lead implementation of AI-driven test optimization and predictive quality analytics.
Stakeholder & Executive Management
Partner with CIOs CTOs Product Executives and Business Leaders to define quality roadmaps.
Present AI quality metrics risks and strategic recommendations to executive leadership.
Influence enterprise technology decisions related to AI quality and governance.

Required Qualifications
Experience
15 years of experience in Quality Engineering Test Management and Quality Leadership.
5 years leading large-scale QA organizations and digital transformation programs.
Proven experience in AI/ML Generative AI and Large Language Model quality validation.
Experience managing global teams and multi-million-dollar quality portfolios.
Technical Expertise
AI/ML Testing Frameworks
Generative AI & LLM Evaluation
RAG Architecture Validation
Agentic AI Testing
Cloud Platforms (Azure AWS GCP)
Test Automation Platforms
API & Microservices Testing
Data Engineering & Data Quality
MLOps & Model Lifecycle Management
Security Privacy & Responsible AI Controls

Preferred Skills
Strong understanding of Azure OpenAI Microsoft Copilot OpenAI Gemini Claude and enterprise AI platforms.
Experience defining AI governance frameworks and responsible AI practices.
Knowledge of AI observability and evaluation tools such as LangSmith MLflow Prompt Flow TruLens DeepEval or similar platforms.
Experience leading AI quality initiatives in regulated industries such as Banking Healthcare Insurance Retail or Technology.