Taiwanese Audio Generalist Evaluator Expert

Mercor is seeking a Taiwanese Audio Generalist Evaluator Expert to contribute to a high-impact audio AI research project with a leading research this role you will work on transcription annotation and evaluation tasks that help train and benchmark advanced language models. This is a short-term structured engagement ideal for candidates with strong academic or analytical backgrounds who are fluent in Traditional Chinese (Taiwan Mandarin) and English and enjoy translating complex audio and visual information into precise well-structured text.

Job Responsibilities

Transcribe and Optimise Audio & Video

• Listen to analyse and transcribe audio and video content in Traditional Chinese (Taiwan Mandarin) following detailed constraints and instructions.

• Produce high-quality written outputs in Traditional Chinese with supporting work in English when required.

• Ensure clarity accuracy and strict adherence to formatting and stylistic guidelines.

• Capture tone intent and region-specific expressions common in Taiwanese Mandarin usage.

Define and Document Evaluation Standards

• Establish clear expectations for correct and high-quality responses in general consumer audio contexts.

• Develop detailed evaluation rubrics and grading guidelines in Traditional Chinese and English.

• Document standards to ensure consistency across reviewers and model evaluations.

• Identify linguistic nuances colloquialisms and edge cases specific to Taiwanese Mandarin.

Conduct Model Testing and Grading

• Run prompts through language models and assess generated outputs.

• Evaluate responses against predefined criteria for accuracy completeness fluency and instructional clarity.

• Provide structured feedback to improve model performance in Traditional Chinese audio tasks.

Support Benchmarking and Quality Assurance

• Participate in QA and review cycles to ensure tasks rubrics and outputs meet Mercors quality bar.

• Maintain consistency and reliability before datasets are integrated into official benchmarks.

• Collaborate with project leads to resolve ambiguities and improve task design.

Minimum Qualifications

• Strong writing editing and critical thinking skills.

• Ability to work independently manage time effectively and meet deadlines.

• Native or near-native fluency in Traditional Chinese (Taiwan Mandarin spoken and written) and professional fluency in English.

• Ability to accurately transcribe and analyse Taiwanese Mandarin audio content across general consumer contexts.

• Available to commit 1020 hours per week.

Preferred Qualifications

• College students or recent graduates.

• Background in linguistics humanities social sciences journalism or technical disciplines.

• Prior experience with transcription annotation localisation evaluation or research workflows in Traditional Chinese.

• Familiarity with Taiwan-specific vocabulary idiomatic expressions and cultural context.

• Interest in AI language models or applied research environments.

Application & Onboarding Process

• Complete a short AI-led interview (approximately 15 minutes).

• If selected you will be onboarded and invited to begin project work.

Additional Role Details

• Work in a structured goal-oriented project environment with clear tooling guidelines and support.

• Gain hands-on exposure to real-world AI research and evaluation workflows.

• Contribute directly to benchmarking and improving advanced multilingual language models.