Software Engineer, Evaluation

At Apple great ideas have a way of becoming great products services and customer experiences very quickly. Bring passion and dedication to your job and theres no telling what you could accomplish. Do you want to make Siri and Apple Intelligence features smarter for our users Contribute to the next revolution in AI and Machine Learning. Were a dynamic team seeking engineers who are enthusiastic about influencing the future of Apple Intelligence.

As a Software Engineer in AIML Evaluation you will:nn- Work closely with product managers backend engineers data scientists and other stakeholders to improve the overall quality and user experience of numerous Apple products.n- Design and develop evaluation system and framework to enhance the quality of Apple Intelligence ensuring our AI/ML models meet the highest standards.n- Tuning evaluation infrastructure tightening operational controls improving auditability and debug trails and scaling the workflows .n- Youll play a pivotal role in developing Apple Intelligence driving the next generation of groundbreaking products across all Apple platforms.

Benchmark Design u0026 Development: Design and implement evaluation benchmarks metrics and test suites that rigorously measure model capabilities across reasoning knowledge code and agentic -Aligned Evaluation: Develop evaluation methods that capture how models behave in real product settings and validate that evaluation metrics predict user-perceived quality and product Methodology Research u0026 Tooling: Research and apply state-of-the-art evaluation techniques including scoring frameworks model-based judging and contamination-resistant benchmark design. Build reusable tools scorer libraries and analysis frameworks that scale across the teams benchmark Analysis: Design and execute rigorous experiments comparing model capabilities engage with third-party vendors on benchmarking and perform detailed gap analysis to guide model development -Team Collaboration: Work closely with model training training data and product teams to ensure evaluation insights inform training strategies data decisions and product quality improvements.

3 years of professional software engineering experience building and operating production systemsnAbility to tackle complex challenges think critically and deliver innovative use of AI-assisted development tools (coding agents code review assistants etc.) to work effectively at scalenExperience designing and scaling distributed systems (load balancing active-active topologies capacity planning throughput-bound services)nExperience with frontend web development (React and modern JavaScript frameworks)nExperience with RESTful API design SQL and NoSQL databasesnBS MS in Computer Science Computer Engineering or equivalent degree with applicable experience

Eager to explore and apply cutting-edge tech working on large cross-functional and multi-team ability to operate independently on ambiguous open-ended problems where the right answer is not to elevate every aspect of the user experience.