Quality Analyst AI Business

Team: Client Services

Prolific

Prolific isnt just enabling AI innovation were redefining it. While foundational AI technologies are becoming commoditized Prolifics human data infrastructure provides the high-quality diverse data required to train the next generation of AI models. Through our platform we empower researchers and companies to access a global ethically curated participant base ensuring cutting-edge AI research and training grounded in inclusivity and precision.

The Role

As a Quality Analyst within Prolific AI Data Services you will play a critical role in ensuring the consistency reliability and integrity of the human data we deliver to customers. As task types and volumes grow quality cant rely on spot checks or tribal knowledgethis role helps operationalise quality through measurement early detection and clear feedback loops.
Youll work closely with Operations reviewers and Quality and Delivery leads to run day-to-day quality measurement surface issues early and help turn fixes into repeatable workflows that scale across studies and customers.

What Youll Be Doing

• Run day-to-day quality measurement for managed service studies including sampling rubric-based review and structured reporting.
• Apply rubrics and guidelines consistently flag ambiguities and propose clear actionable clarifications as task types evolve.
• Maintain and update golden sets and support calibration efforts by preparing materials tracking outcomes and documenting decisions.
• Track defects and failure modes by categorising issues assessing severity and keeping defect taxonomies current.
• Run automated quality checks using SQL and Python or R including completeness checks schema and format validation anomaly detection and label distribution monitoring.
• Support launch gates by preparing readiness artefacts such as rubric versioning gold status sampling plans and early pilot findings.
• Investigate quality drops by pulling examples quantifying scope and supporting root-cause analysis and corrective and preventive actions (CAPAs).
• Build dashboards and weekly updates covering defect rates rework agreement throughput versus quality trade-offs and SLA risk.
• Partner with Operations reviewers andwhen neededProduct and Engineering to improve task design reviewer workflows and escalation paths.

What Youll Bring to the Role

• 13 years of experience in analysis quality operations or data-focused roles or equivalent academic projects or internships.
• A background in computer science data science statistics engineering or a related quantitative discipline.
• Strong SQL skills (required) with the ability to independently query join and summarise datasets to answer quality and operational questions.
• Working proficiency in Python and/or R with experience writing scripts or notebooks to clean data compute metrics and automate checks.
• Strong analytical rigour with a high bar for precision consistency and clear documentation of decisions.
• Solid statistical fundamentals including intuition for sampling distributions confidence and agreement concepts.
• Clear and structured communication skills with the ability to explain issues using concrete examples and propose practical fixes.
• A collaborative mindset and willingness to partner cross-functionally to improve quality outcomes and delivery reliability.
  
  Even Better if you have
  
  
• Experience with human data workflows such as annotation or data labelling or ML/LLM evaluation (rubric-based evals preference data red teaming).
• Familiarity with QA concepts such as defect severity leakage root-cause analysis and CAPA processes.
• Experience building or maintaining dashboards in BI tools such as Looker Tableau or Mode.

Why Prolific is a great place to work

Weve built a unique platform that connects researchers and companies with a global pool of participants enabling the collection of high-quality ethically sourced human behavioral data and feedback. This data is the cornerstone of developing more accurate nuanced and aligned AI systems.

We believe that the next leap in AI capabilities wont come solely from scaling existing models but from integrating diverse human perspectives and behaviors into AI development. By providing this crucial human data infrastructure Prolific is positioning itself at the forefront of the next wave of AI innovation one that reflects the breath and the best of humanity.

Working for us will place you at the forefront of AI innovation providing access to our unique human data platform and opportunities for groundbreaking research.

Join us to enjoy a competitive salary benefits and hybrid working within our impactful mission-driven culture. At Prolific our compensation packages for eligible roles include base salary equity and benefits. Many roles also include the opportunity to earn a cash variable element such as a bonus or commission. Your recruiter will also be happy to share the specific salary range for your preferred location during the hiring process.

Links to More Information

Website

Youtube

Privacy Statement

By submitting your application you agree that Prolific may collect your personal data for recruiting and global organisation planning. Prolifics Candidate Privacy Notice explains what personal information Prolific may process where Prolific may process your personal information its purposes for processing your personal information and the rights you can exercise over Prolific use of your personal information.