Join the Seller AI team where youll lead benchmarking and evaluation of AI models that enhance the seller experience across Amazons global marketplace. Youll manage a team dedicated to validating testing and improving Artificial Intelligence (AI) and Large Language Models (LLMs) that power innovative seller tools. This role combines strategic leadership with hands-on technical oversight requiring exceptional communication skills team management and stakeholder engagement.
In this position youll drive the development and implementation of comprehensive benchmarking methodologies to evaluate AI model performance across accuracy robustness bias and reliability metrics. Your expertise will be crucial in translating technical findings into actionable insights that improve Seller Assistants performance and contribute to the growth of Amazons seller community worldwide.
Key job responsibilities
1. Plan and execute benchmarking exercises for AI models defining test plans metrics and acceptance criteria across accuracy robustness bias and reliability dimensions
2. Lead a team responsible for validating data based on specific annotation guidelines ensuring accuracy and quality while escalating potential regulatory risks
3. Prepare comprehensive audit and benchmarking reports including error ratings root-cause analysis and recommendations for senior stakeholders
4. Drive process efficiencies and explore automation opportunities to enhance the productivity of data generation initiatives
5. Mentor team members and help develop their skills while managing overall schedules proactively mitigating risks and keeping project scope under control.
A day in the life
Your day begins with prioritizing benchmarking tasks for your team based on current business requirements. Youll review progress on ongoing audits provide guidance where needed and ensure deliverables meet quality standards. Throughout the day youll analyze performance data from Seller Assistant tools identifying patterns and insights that could improve AI model effectiveness. Youll wrap up by preparing detailed presentations that showcase your teams findings and recommendations to leadership highlighting opportunities to enhance the seller experience.
About the team
The Seller AI team within International Seller Services is dedicated to creating Gen-AI/LLM powered tools and agentic solutions that accelerate business growth for Amazon sellers worldwide. We focus on handling annotations for training measuring and improving Artificial Intelligence and Large Language Models to deliver superior seller experiences. Our team combines technical expertise with a deep understanding of seller needs to develop innovative solutions that simplify complex tasks and drive business growth.
By joining us youll play a pivotal role in shaping the future of selling on Amazon working with cutting-edge AI technologies that directly impact millions of sellers across global marketplaces.
- Bachelors degree or MBA in business data science public administration finance engineering human resources or related field
- - Min. 5 to 8 years of exp. as a People Manager with good communication skills
- Experience in people management
Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process including support for the interview or onboarding process please visit
for more information. If the country/region youre applying in isnt listed please contact your Recruiting Partner.