AI Benchmarking Specialist, SFIX
Share
Job Location:
Monthly Salary:
Posted on:
30+ days ago
Vacancies:
1 Vacancy
Job Summary
The Seller AI team within International Seller Services organization is focused on helping sellers with the right set of Gen-AI/LLM powered tools and agentic solutions that can enable them to accelerate business growth on Amazon. Our primary focus lies in handling annotations for training measuring and improving Artificial Intelligence (AI) and Large Language Models (LLMs) enabling Amazon to deliver a superior seller experience to our sellers worldwide. The AI Benchmarking Associate supports the evaluation of AI systems by designing and executing benchmarking and audit activities to assess model quality compliance robustness and fairness. The role combines elements of AI auditing quality assurance and traditional audit-style documentation and stakeholder communication. By joining us you will play a pivotal role in shaping the future of selling on Amazon for sellers worldwide.
Key job responsibilities
Key job responsibilities
As part of your role you will have the opportunity to
Assist in planning and executing benchmarking exercises for AI models including defining test plans metrics and acceptance criteria across accuracy robustness bias and reliability
Support content accuracy relevancy and privacy checks by reviewing datasets model outputs and data handling practices escalating potential regulatory risks.
Validate data based on specific annotation guidelines ensuring the accuracy and quality of the collected information
Prepare clear audit and benchmarking reports including error ratings root-cause analysis and recommendations and contribute to presentations for senior stakeholders
Maintain organized audit documentation evidence and benchmarking datasets to support internal review
You will work closely with your team members and managers to drive process efficiencies and explore opportunities for automation
You will strive to enhance the productivity and effectiveness of the data generation by contributing to the development and continuous improvement of AI audit methodologies checklists and test frameworks as regulations and best practices evolve
We are open to hiring candidates to work out of the following location:
Bengaluru Karnataka IND
- Bachelors degree or equivalent
- 2 years of equivalent experience
Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process including support for the interview or onboarding process please visit for more information. If the country/region youre applying in isnt listed please contact your Recruiting Partner.
Key job responsibilities
Key job responsibilities
As part of your role you will have the opportunity to
Assist in planning and executing benchmarking exercises for AI models including defining test plans metrics and acceptance criteria across accuracy robustness bias and reliability
Support content accuracy relevancy and privacy checks by reviewing datasets model outputs and data handling practices escalating potential regulatory risks.
Validate data based on specific annotation guidelines ensuring the accuracy and quality of the collected information
Prepare clear audit and benchmarking reports including error ratings root-cause analysis and recommendations and contribute to presentations for senior stakeholders
Maintain organized audit documentation evidence and benchmarking datasets to support internal review
You will work closely with your team members and managers to drive process efficiencies and explore opportunities for automation
You will strive to enhance the productivity and effectiveness of the data generation by contributing to the development and continuous improvement of AI audit methodologies checklists and test frameworks as regulations and best practices evolve
We are open to hiring candidates to work out of the following location:
Bengaluru Karnataka IND
- Bachelors degree or equivalent
- 2 years of equivalent experience
Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process including support for the interview or onboarding process please visit for more information. If the country/region youre applying in isnt listed please contact your Recruiting Partner.
Required Experience:
IC
Key Skills
About Company
Free shipping on millions of items. Get the best of Shopping and Entertainment with Prime. Enjoy low prices and great deals on the largest selection of everyday essentials and other products, including fashion, home, beauty, electronics, Alexa Devices, sporting goods, toys, automotive ... View more
