Vendor Operations Manager – Safety Labeling
Share
Job Location:
San Mateo, CA - USA
Monthly Salary:
Not Disclosed
Posted on:
Yesterday
Vacancies:
1 Vacancy
Job Summary
At Roblox we are looking for a Senior Vendor Operations Manager for Safety Labeling to bridge the gap between human intuition and machine learning. You will manage the vendor ecosystem responsible for generating the high-quality ground truth data that powers our safety models. Reporting to the Senior Manager of Vendor Operations youll ensure our BPO partners translate complex safety policies into precise scalable labels that keep our community safe.
You Will:
- Lead Labeling Operations: Manage day-to-day vendor execution for safety labeling (RLHF and Ground Truth) ensuring accuracy policy alignment and clinical precision.
- Optimize Data Quality: Implement rigorous quality frameworksincluding Inter-Rater Reliability (IRR) and Gold Standard auditsto eliminate labeling bias.
- Scale the Pipeline: Partner with AI/ML Engineering and Policy to transform abstract guidelines into executable labeling handbooks and efficient workflows.
- Drive Vendor Performance: Own the operational roadmap unit costs and capacity planning for specialized labeling workforces globally.
- Ensure Resilience: Maintain business continuity and site redundancy plans to protect our data labeling pipelines from disruption.
You Have:
- Expertise in Data Labeling: 6 years in Ops/Trust & Safety with a proven track record managing large-scale vendor teams focused on data annotation or RLHF.
- Analytical Engine: Experience using data (SQL Looker or similar) to identify quality drift and operational bottlenecks.
- Cross-Functional Fluency: The ability to move seamlessly between technical ML requirements and nuanced Policy discussions.
- Operational Grit: A high tolerance for the complexity and emotional intensity of managing teams exposed to sensitive or high-risk content.
Required Experience:
Manager
Key Skills
- Presentation Skills
- Purchasing
- Procurement
- Vendor Management
- Process Improvement
- Pricing
- Project Management
- Relationship Management
- Contract Management
- Contract Negotiation
- negotiation
- Contracts
About Company
Roblox is the ultimate virtual universe that lets you create, share experiences with friends, and be anything you can imagine. Join millions of people and discover an infinite variety of immersive experiences created by a global community!
