AI ML QA

AI/ML GCP QA Engineer
Fulltime
Hybrid (2 days a week)- Toronto ON

QA Engineer with specialization in AI/ML systems delivered on Google Cloud including Google ADK/Vertex AI stacks. This role combines traditional QA/testing skills with AI-specific validation strategies (LLM output quality performance testing regression checks hallucination detection and data integrity across agentic systems). The ideal candidate bridges QA and ML engineering to ensure reliability fairness and robustness of AI-powered products in production.

Key Responsibilities

• Develop and execute comprehensive test plans and automated frameworks for AI/ML components (models pipelines APIs agent workflows).
• Define QA strategies for AI outputs including correctness stability bias detection and edge case resilience.
• Partner with engineering teams to test AI model integrations with Vertex AI Endpoints ADK agent systems and cloud services.
• Implement regression tests performance and load testing for multi-agent and conversational systems.
• Collaborate on MLOps pipelines to include automated QA checkpoints as part of CI/CD.
• Track and surface issues with metrics dashboards logging and test reports; work with ML and cloud engineers to verify fixes.
• Contribute to QA documentation coverage matrices and testing guidelines for both deterministic and probabilistic behaviors in AI/ML.

Required Skills & Qualifications

• Bachelors degree in Computer Science Software Engineering ML/AI or related fields.
• 5-8 years of QA experience with strong exposure to AI/ML testing concepts.
• Solid programming skills in Python (test automation tooling scripts ML model test harnesses).
• Understanding of Google Cloud services especially Vertex AI and related AI/ML services and how they fit into QA pipelines.
• Experience with automated testing systems and CI/CD integrations.

Preferred Qualifications

• Experience with Google Agent Development Kit (ADK) or other agent frameworks.
• Exposure to RAG systems conversational AI evaluation and context/semantics QA.
• Familiarity with GCP networking IAM data security and compliance standards.
• Additional QA certifications or cloud certifications (e.g. Google Cloud ML Engineer).