AI Safety & Data Quality Analyst (HITL) NYC

Join the Human Intelligence Behind the Worlds Leading AI NYC

Are you sharp curious and obsessed with the details Welo Data is looking for Data Quality Analysts who can go beyond simple labels to help define how AI should behave in the real world.

This is a Human-in-the-Loop (HITL) role where your judgment is the product. You wont just be following guidelinesyoull be refining them identifying failure modes and ensuring that the next generation of AI is safe accurate and helpful.

The Mission: Beyond Annotation

• Model Evaluation: Critique AI outputs and provide structured high-level feedback that drives model improvement.
• Safety Engineering: Use your 2 years of GenAI safety experience to identify bias risks and edge cases.
• Trend Analysis: Spot patterns in how models fail and communicate these insights to stakeholders.
• Framework Architecture: Dont just execute workflowshelp improve the evaluation frameworks themselves.
• Collaborative Calibration: Partner with leads and cross-functional teams to set the gold standard for AI quality.

Project Details

• Job Title: Data Quality Analyst
• Employment Type: W2 Full-Time Employee
• Hours: 40 hour/week
• Location: NYC (Onsite)
• Pay Rate: 30$/hour

Benefits

Working on-site at our NYC office comes with premium benefits designed to let you focus on your best work:

• Gourmet Dining: Free breakfast lunch and dinner with a wide variety of cuisines.
• Endless Snacks: Micro-kitchens stocked with premium coffee beverages and healthy treats.
• Campus Life: Access to unique features like rooftop nature parks and modern collaborative spaces.
• Commuter Ease: Free transport shuttles and bike-to-work perks.
• Full Security: Comprehensive Medical Dental and Vision 401(k) and HSA.
• Time Off: 15 days of combined Paid Sick/Holiday time major holidays.

Who Were Looking For

• Education & Language: University degree (Bachelors or higher) and Native-level English proficiency are required.
• The Multilingual Edge: Proficiency in languages other than English is highly preferred with a particular interest in candidates demonstrating bilingual mastery in French Italian German Spanish or Brazilian Portuguese.
• Writing Mastery: 12 years of professional writing experience with the ability to articulate complex why justifications clearly.
• AI Safety Expertise: 2 years of experience focusing on Safety & Trust in GenAI data delivery.
• Critical Thinking: The ability to navigate grey areas and make sound judgment calls when guidelines are ambiguous or evolving.
• Technical Curiosity: A natural interest in LLMs and the confidence to speak up and ask questions when something feels off.
• Operational Mindset: Experience in data quality QA or annotation is helpful but a strong analytical mind is essential.