Principal Quality Automation Architect

This is a remote position.

Reporting to Manager Quality Engineering & AI Validation designs and scales Accordions automated quality framework across products services and AI-enabled workflows. Owns the architecture behind automated validation test harnesses evaluation pipelines test data strategy and cross-pod tooling standards.

Key Responsibilities

Design and implement the core automation framework for functional API integration regression and workflow testing.

Define standards for reusable test assets environment usage data setup and evidence capture.

Create scalable approaches to testing complex multi-system workflows.

AI Evaluation Infrastructure

Build and maintain regression harnesses for AI-enabled features including prompt and workflow comparisons scored evaluation runs and benchmark execution.

Partner with AI quality analysts to support rubric-based testing and measurable release criteria.

Test Data & Environment Strategy

Define approaches for test data synthetic data masked data and scenario generation.

Improve reliability and repeatability of validation environments.

Engineering Partnership

Work closely with software engineers to improve testability observability CI/CD integration and defect prevention.

Help pods move quality checks earlier in the development lifecycle.

Requirements

Required Qualifications

7 years of experience in QA automation SDET or quality engineering roles.

Strong experience with test automation frameworks API testing CI/CD and integration validation.

Strong scripting or coding skills.

Experience working with data-intensive or workflow-heavy applications.

Exposure to AI or LLM validation tooling or strong interest in building in that space.

Ability to design frameworks not just individual tests.

Bachelors degree preferred.

Key Traits for Success

Systems-minded and deeply technical.

Focused on scale repeatability and leverage.

Comfortable setting standards others adopt.

Practical about balancing sophistication with delivery pace.

Benefits