AIML Sr Engineering Specialist, Evaluation

Apple is where individual imaginations gather together committing to the values that lead to great work. Its the diversity of our people and their thinking that inspires the innovation running through everything we do here youll do more than join something youll add something. nThe AIML Evaluation team measures how well Apples intelligence features including Siri AI actually work in our customers hands. nWere looking for an organized critical problem solver who thinks beyond what a feature is supposed to do and finds creative ways to measure the qualitative

As a Sr Engineering Specialist on the Evaluation team youll keep our evaluation projects running clean from design through delivery. Youll be responsible for understanding how well our products perform advocating for the changes that matter and representing how our customers will experience them. Were a nimble fast-paced team that runs on a tight partnership between Feature/ Tooling Engineering Program Management Operations and Data Science and this role sits at the center of this role you will:nOwn device configuration for evaluation projects and coordinate with the Operations team to roll out configurations to all in-progress projects and flag quality issues early before they affect downstream with Evaluation Data Science to review evaluation design for quality and alignment ahead of triage with the evaluation tooling team when technical issues surface mid-project and follow through to recurring problems into durable fixes checklists guardrails and process improvements that hold up across future in this role looks like projects that launch on a sound design run without avoidable surprises and produce results the team can act on with

BA/BS in Management Information Systems Computer Science or a related technical field or equivalent practical experience.n7 years of relevant experience in roles such as technical program manager consultant QA/device tester or evaluation -on manual testing experience including designing and executing test passes on physical devices. Highly organized with a track record of balancing multiple concurrent efforts and meeting firm written and verbal communication skills. This role involves a high level of interaction with engineering teams management and other organizations across Apple and we expect you to build and navigate those relationships autonomously.n

10 years of related experience including 3 years leading quality evaluation projects evaluating LLM-based assistants voice assistants or other generative AI track record of building and scaling efficient quality processes across cross-functional managing device configurations at scale MDM profiles or build rollouts in coordination with an operations data literacy: SQL dashboards or light scripting to spot anomalies in evaluation data yourself rather than waiting on a partnering with data science on evaluation or experiment design or supporting multi-locale/international evaluation testing or evaluating AI-powered features and comfort using AI tools in your own day-to-day debugging and troubleshooting skills with the analytical judgment to assess project quality and prioritize which issues to fix first.n