Audio Generalist Evaluator Expert

San Francisco, CA - USA

Monthly Salary: Not Disclosed

Posted on: 3 hours ago

Vacancies: 1 Vacancy

Job Summary

Mercor is seeking an in-person (San Francisco) detail-oriented transcribing/writing experts to contribute to a high-impact audio AI research project with a leading lab. Freelancers will listen to and transcribe audios annotate images and evaluate videos to help train advanced language models. This is a short-term flexible opportunity for professionals with strong academic backgrounds and a knack for instructional clarity. Ideal for those who enjoy distilling complex concepts into well-crafted text.

Job Details:

Transcribe and Optimize Audio/Video: Create detailed audio transcriptions with multiple constraints and instructions.
Define and Document Evaluation Standards: Establish high-level expectations for correct responses in general audio consumer contexts and develop comprehensive rubric.
Conduct Model Testing and Grading: Run prompts through models and assess preliminary outputs against expectations.
Support Benchmarking and Quality Assurance: Collaborate in QA review processes to ensure prompt tasks and rubrics meet rigor maintaining consistency and reliability before integration into official benchmarks.

Minimum Qualifications:

BS or BA from a reputable institution completed or in progress
Strong writing and critical thinking skills.
Ability to work independently and meet deadlines.
Available to work from office 20 hours a week on the clients location.

Preferred Qualifications:

College students/grads

Application & Onboarding Process:

Complete an AI-led interview this should take around 15 minutes.
If selected you will be invited to work on the project.

More Details About This Role:

This is a in person s role you can only work from the clients office in SF
Expect to contribute at least 20 hours per week.
Expect a commitment of around 2-4 weeks.
Youll be working in a structured project environment with clear goals and tools.