AI Agent Evaluator Remote

Role Type: Contractor
Location: United States

We are seeking detail-oriented professionals to evaluate AI agents performing everyday tasks. Youll test two AI agents side by side assess their performance and provide feedback to improve their reliability and user experience.

• Test AI agents on everyday tasks including shopping travel bookings bill payments email follow-ups and forms.
• Approve or stop agents at key steps.
• Screen-record each testing session.
• Score task completion quality control issues user satisfaction and time.
• Provide clear written feedback on agent performance and usability.

• Based in the United States with a bachelors degree.
• Regularly use AI tools in daily life.
• 23 years of experience preferably in engineering business management or office operations.
• Strong attention to detail and ability to follow instructions.
• Excellent written communication and documentation skills.
• An active personal AI account (Free Go or Plus; not Pro) and a smartphone capable of screen recording.

Preferred: Relevant domain expertise. No coding or AI development experience is required.

1. Submit an application and complete screening questions.
2. Attend a 1530-minute interview.
3. Complete profile review and onboarding.

• Task duration: Approximately 1.5 hours per run including evidence and scoring.
• Kickoff: Monday October 12.
• Onboarding: Before the project begins subject to scheduling.