AI Tester

Job Title: AI Tester
Location: Los Angeles CA (Onsite/Hybrid)
Duration: 12 Months

Role Summary

We are seeking a detail-oriented AI Tester to ensure AI-driven products perform accurately safely and reliably across real-world scenarios. This role will partner closely with Product Engineering and Subject Matter Experts (SMEs) to validate model behavior assess data quality and identify trust-related risks throughout the AI lifecycle.

Key Responsibilities

• Evaluate AI/ML and GenAI outputs for accuracy consistency edge cases and failure modes
• Validate training and inference data quality identifying risks such as bias and data leakage
• Design implement and execute model evaluation and regression testing strategies
• Maintain and expand Golden Datasets for regression testing across model updates prompt changes and releases
• Test AI behavior within end-to-end workflows and system integrations
• Support Responsible AI initiatives including fairness explainability and compliance validation
• Monitor model performance post-release identifying drift regressions and user feedback trends
• Develop and enhance test automation coverage for AI systems
• Track and report quality metrics such as accuracy precision and recall highlighting gaps errors and hallucinations
• Clearly communicate testing outcomes risks and recommendations to stakeholders

Required Skills

• Strong experience in QA/test planning exploratory testing and UAT
• Solid understanding of AI/ML fundamentals and evaluation metrics
• Ability to analyze data and reason about model behavior and outputs
• Experience testing GenAI prompts and responses (preferred)
• Strong domain understanding and user-centric mindset
• Excellent documentation and communication skills

Qualifications

• 37 years of experience in QA testing or quality engineering roles
• Experience working with AI/ML or data-driven products preferred
• Certifications such as ISTQB AI/Data certifications or Responsible AI training are a plus