Sr AI Language Engineer, Amazon Rufus

The Conversational Shopping team is looking for a Language Engineer to drive efficiencies and innovation in its efforts to deliver a seamless fluent and engaging experience for AI-assisted shopping. This is an opportunity to join the high-performing team behind Amazons Generative AI shopping initiatives such as Rufus AI Amazons Conversational Shopping assistant. Our objective is to make it easy for customers worldwide to find and discover the best products meet their personalized needs with product research providing comparisons and recommendations answering specific product questions and more. This role is inherently high-visibility and highly cross-functional requiring collaboration and influence across global product design science and engineering teams.

We are looking for candidates who are passionate about the intersection of language and technology and who are keen to use their technical abilities to develop automated scalable solutions to questions in the Large Language Model (LLM) space. Applying a combination of expertise in LLMs coding and linguistics (i.e. semantics syntax pragmatics) they will overcome complex problems in natural language processing (NLP) language understanding and automated AI evaluations.

In this role within the Editorial team they will act as one of the driving forces behind our evaluation-driven product development strategy. They will design processes to facilitate the production of high quality editorial data which will allow us to evaluate and improve the Shopping AI experience in different languages. To do so they will be tasked with the creation and development of LLM-assisted editorial tools automated verification scripts and automated annotations (e.g. LLM-as-a-judge) to support the humans-in-the-loop (HITL) work of the broader Editorial team. They will lead and drive the requirements behind data annotation tasks and tooling writing intuitive annotation guidelines and guiding the creation of the tools adapted to these workflows. They will employ their data processing and analysis skills to track team productivity and measure output quality. They will work in close collaboration with other Language Engineers AI Editors Product Managers Applied Scientists and Software Engineers on initiatives that drive editorial quality speed and consistency. By creating and synthesizing quality metrics they will also guide Conversational Shopping teams in delivering both internal stakeholder requirements and achieve the desired Amazon customer outcomes.

This role requires strong analytical and technical skills as well as experience in language technology to help us measure analyze and solve complex problems. They should have experience in creating technical solutions for automating and processing data workflows at scale and have the ability to do so while upholding the highest linguistic quality standards. They should also have exceptional writing and communication skills with the ability to interface between both technical and non-technical teams.

Key job responsibilities
* Produce process and manipulate different types of language data analyze and provide efficient solutions
* Automate operations and perform data analysis using coding/scripting language (e.g. Python)
* Develop LLM-assisted workflows and annotations solutions (e.g. LLM-as-a-judge) to support Human-in-the-loop evaluations
* Design and lead editorial data production/collection by defining scope with internal customer teams
* Define clear editorial workflows (SOPs) to meet or exceed the quality bar
* Adopt and design control mechanisms metrics and methodologies for editorial and annotation quality
* Maximize productivity process efficiency and quality through streamlined workflows process standardization documentation audits and investigations on a periodic basis.
* Collaborate with editors applied scientists engineers and product managers to deliver the optimal customer experience and define metrics guidelines and workflows to continue doing so
* Establish processes and mechanisms to onboard and train editors on an ongoing basis.
* Handle work prioritization and deliver based on business priorities.
* Be flexible in changes to conventions deployed in response to customers requests and change workflows accordingly.

- Knowledge of and proficiency in the use of Python scripting language
- Experience communicating technical concepts and processes using clear simple language and visuals
- Experience working in a fast-paced environment similar to a high-tech start-up
- Experience prioritizing and handling multiple assignments at any given time while maintaining commitment to deadlines
- Masters degree or a PhD and experience in building speech recognition machine translation and natural language processing systems (e.g. commercial speech products or government speech projects)
- Knowledge of command line tools to troubleshoot protocols analyze log outputs or automate basic tasks
- * Strong Experience in Natural Language Processing Machine Learning AI or Large Language Models
- * Knowledge of Regex SQL MS Excel Git.
- * Familiarity with annotation tools and workflows.

- PhD or a Masters degree and experience in building speech recognition machine translation and natural language processing systems (e.g. commercial speech products or government speech projects)
- PhD in a quantitative field or PhD and 4 years of data scientist experience
- Experience with language annotation and other forms of data markup
- Experience in several of the following areas: machine learning statistics deep learning natural language processing or information retrieval
- Knowledge of user experience principles and techniques
- Experience in online retail operations or similar field
- * Experience with AWS services (S3 Sagemaker ML language services etc.).

Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status disability or other legally protected status.

Los Angeles County applicants: Job duties for this position include: work safely and cooperatively with other employees supervisors and staff; adhere to standards of excellence despite stressful conditions; communicate effectively and respectfully with employees supervisors and staff to ensure exceptional customer service; and follow all federal state and local laws and Company policies. Criminal history may have a direct adverse and negative relationship with some of the material job duties of this position. These include the duties and responsibilities listed above as well as the abilities to adhere to company policies exercise sound judgment effectively manage stress and work safely and respectfully with others exhibit trustworthiness and professionalism and safeguard business operations and the Companys reputation. Pursuant to the Los Angeles County Fair Chance Ordinance we will consider for employment qualified applicants with arrest and conviction records.

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process including support for the interview or onboarding process please visit for more information. If the country/region youre applying in isnt listed please contact your Recruiting Partner.

Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $84000/year in our lowest geographic market up to $185000/year in our highest geographic market. Pay is based on a number of factors including market location and may vary depending on job-related knowledge skills and experience. Amazon is a total compensation company. Dependent on the position offered equity sign-on payments and other forms of compensation may be provided as part of a total compensation package in addition to a full range of medical financial and/or other benefits. For more information please visit This position will remain posted until filled. Applicants should apply via our internal or external career site.