Sr. Product Mgr, AI & Human Eval Prog

At eBay were more than a global ecommerce leader were changing the way the world shops and sells. Our platform empowers millions of buyers and sellers in more than 190 markets around the world. Were committed to pushing boundaries and leaving our mark as we reinvent the future of ecommerce for enthusiasts.

Our customers are our compass authenticity thrives bold ideas are welcome and everyone can bring their unique selves to work every day. Were in this together sustaining the future of our customers our company and our planet.

Join a team of passionate thinkers innovators and dreamers and help us connect people and build communities to create economic opportunity for all.

About the Team and the Role:

The Senior Product Manager will be the connective tissue between category managers merchandisers and domain guides on one side and the scientists and engineers building AI models on the other. This role requires a deep understanding of how AI models are built and trained including how data curation human judgment and iterative feedback shape model behavior and the ability to actively contribute to that process. Your impact depends on execution trust and the ability to coordinate complex cross-functional work across a matrix organization.

What you will accomplish:

Run Human Evaluation and Annotation Programs

• Design and lead structured programs to evaluate AI model outputs across ranking classification generation and extraction tasks

• Coordinate distributed annotation teams across timezones running workflows maintaining quality consistency and ensuring continuous coverage and on-time delivery. Handle relationships with internal and external evaluator pools

• Translate evaluation results into clear actionable requirements for model and product teams
  

Lead AI Quality Monitoring Programs

• Track quality metrics surface issues proactively and run structured resolution processes to ensure AI-driven experiences consistently meet the bar

• Ensure quality signals from the live site feed back into model development priorities.

• Review what ships ask hard questions and hold teams accountable when AI-driven experiences fall short

• Help define what good data looks like for each focus category: coverage freshness labeling schema and quality thresholds

• Build the end-to-end processes to meet those standards from scoping and sourcing through annotation QA and delivery in partnership with science teams and business partners

• Define quality standards for training and evaluation data in partnership with business partners and science teams and build the processes to meet those standards consistently.

What you will bring:

• 5 years of PM experience with deep exposure to AI/ML or technical platform products

• Experience leading annotation or evaluation programs including working with distributed annotator pools external vendors or crowdsourced labeling at scale

• Analytical and data science fluency comfortable writing SQL queries to explore datasets track program metrics and validate quality signals independently

• Ability to work alongside Applied Scientists understands how DS workflows operate can scope analytical work clearly and knows when to roll up their sleeves.

• Working knowledge of how AI models are built and trained including the role of data human judgment and evaluation in shaping model behavior

• Direct exposure to LLM evaluation preference data benchmarking red-teaming or human feedback collection is a plus

• Experience with annotation vendors or crowdsourced labeling programs is a plus

• Familiarity with AI observability quality monitoring or anomaly detection in production is a plus

Additional Details

The base pay range for this position is expected in the range below:

Base pay offered may vary depending on multiple individualized factors including location skills and experience. The total compensation package for this position may also include other elements including a target bonus and restricted stock units (as applicable) in addition to a full range of medical financial and/or other benefits (including RRSP eligibility various paid time off benefits such as PTO and parental leave). Details of participation in these benefit plans will be provided if an employee receives an offer of employment.

This job posting relates to an existing vacancy within eBay.

eBay is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race color religion national origin sex sexual orientation gender identity and disability or other legally protected you have a need that requires accommodation please contact us at. We will make every effort to respond to your request for accommodation as soon as possible. View our accessibility statement to learn more about eBays commitment to ensuring digital accessibility.

We use cookies to enhance your experience and may use AI tools for administrative tasks in the hiring process. To learn how we handle your personal data and use AI responsibly please visit our Talent Privacy Notice Privacy Center and AI Hiring Guidelines.