AI Operations Analyst

This is a remote position.

Key Responsibilities

AI Workflow Monitoring

Monitor AI-driven workflows in production for abnormal behavior trust issues degraded outputs and recurring exception patterns.

Review signals related to output quality fallback use low-confidence patterns and workflow anomalies.

Escalate issues that could affect live user trust business quality or finance-sensitive outcomes.

Incident Investigation

Support investigation of output-quality incidents orchestration failures prompt or workflow regressions and tool-call problems in production.

Help distinguish true software defects from model behavior issues data-quality problems or process misunderstandings.

Partner closely with Engineering QA and business stakeholders on judgment-heavy incidents.

Operational Controls

Support human-review patterns fallback routes and safe-operating responses for AI-enabled features.

Help define and apply production review practices for higher-risk workflows.

Document evidence and issue patterns in ways that support repeatability and better release decisions.

Continuous Improvement

Identify recurring AI operations issues and recommend improvements to supportability monitoring and quality feedback loops.

Feed production learnings back into QA Product and Engineering.

Help improve operational trust in AI-enabled solutions over time.

Requirements

Required Qualifications

4 years of experience in production support AI operations business process operations analytics QA or a similar role.

Strong analytical skills and comfort evaluating nuanced workflow or output issues.

Ability to work across engineering and business teams in ambiguous situations.

Experience with AI-enabled workflows operational analytics business process validation or support for workflow-heavy systems.

Strong written communication and documentation discipline.

Bachelors degree preferred.

Key Traits for Success

Thoughtful precise and calm under pressure.

Strong at spotting subtle trust or quality issues others may miss.

Comfortable balancing evidence judgment and business impact.

Focused on safe operations and clear escalation.

Benefits

Base salary plus performance-based bonus.

Actual compensation packages are determined by evaluating a wide array of factors unique to each candidate including but not limited to skill set years and depth of experience education certifications cost of labor and internal equity.