Senior Applied Scientist, Multilingual AI Evaluation

AI systems are only as trustworthy as the methods used to evaluate them. At Apple where AI powers experiences for billions of people around the world getting evaluation right is not a support functionit is a foundational science. Our team part of Apple Services Engineering is building that scientific foundation: rigorous scalable evaluation methodology for LLMs agentic systems and human-AI looking for a senior applied scientist to make the evaluation tooling we build work across every language and culture Apple serves. This is a role for someone who is fluent in both modern AI and the science of language and who can set direction and drive initiatives independently not just execute them. Youll do this on a deeply interdisciplinary team working alongside ML researchers measurement scientists and platform engineers.n

In this role youll help ensure Apples AI features work well across languages and cultures. Your goal is to make our evaluation tooling multilingual from the start so that engineers building AI features can design test and ship across the world from day one. Its a broad applied science role: youll shape how Apple evaluates AI wherever the hardest questions are and youll have the opportunity to publish novel scientific challenge is real. How do we ensure we consistently evaluate AI features across different grammar script or cultural norms and how do we do this at scale Youll bring linguistic judgment to questions like these and working with measurement scientists and ML researchers turn it into validated methodology that holds across dozens of is a hands-on role. Youll design and implement your own methods in Python working closely with research and engineering partners while staying focused on the science of getting evaluation right.n

Extend Apples AI evaluation methodology and tooling to new languages and locales so our AI experiences are equally capable accurate and culturally appropriate not just translated and validate evaluation methods benchmarks and metrics that capture language- and culture-specific phenomena: grammar morphology script register dialect code-switching and cultural and curate high-quality multilingual datasets and human evaluation protocols partnering with linguists and native-speaker how LLMs and agentic systems behave across languages identifying systematic failure modes capability gaps and quality disparities between high- and low-resource with engineers to productionize your methods so they run reliably and at scale implementing your own work in findings clearly translating results into actionable guidance for model and product teams and publishing novel work where appropriate.

MS in Linguistics Computational Linguistics NLP Computer Science or a related field or equivalent research/work expertise in linguistics with working fluency in the structure of multiple languages beyond English. nStrong proficiency in understanding of LLMs and AI evaluation fundamentals including how language models process and generate across experience shipping or evaluating features across multiple languages or designing benchmarks datasets or human evaluation protocols with attention to statistical rigor and to drive initiatives independently and collaborate across a cross-functional interdisciplinary written and verbal communication skills.

PhD in Linguistics Computational Linguistics or NLP with a focus on multilingual or cross-lingual in NLP multilingual evaluation or evaluation -on experience with modern ML frameworks (PyTorch JAX) and with fine-tuning or evaluating with low-resource languages dialectal variation or with localization/internationalization workflows and quality with LLM-as-judge approaches rubric design or bias and fairness evaluation across or professional proficiency in one or more languages in addition to English.