Software Engineer Evaluation Engineering

Join the team redefining what a deeply personal and integrated assistant can be. nnAs part of the Siri organization you will help shape one of the worlds most widely used AI assistants powered by our next-generation of Apple Intelligence with capabilities like personal context understanding and on-screen awareness built with privacy from the ground up. Your work will have direct meaningful impact for users across iOS iPadOS macOS watchOS and is a rare opportunity to build at the intersection of cutting-edge AI and human-centered design shipping technology that is centered around users and their needs.

We are looking for a Software Engineer to help build the foundational on-device execution harness and runtime infrastructure for agentic evaluation. As Apple Intelligence evolves validating agent performance requires reliable deterministic execution of multi-step agent trajectories directly on heterogeneous user hardware. Our team focuses on building the core harnesses environment control layers and trace collection pipelines that enable downstream teams to evaluate agentic reasoning tool use on-screen awareness and personal context actions across the entire Apple this role you will design the low-level execution harnesses and orchestration tooling that drive dynamic agent simulations on target devices. You will solve complex systems engineering challenges around local device state management process isolation determinism and high-fidelity trace generation across varying hardware constraints (iPhones Macs iPads Apple Watch and Apple Vision Pro).n

Architect and build high-reliability execution harnesses to drive dynamic multi-step agentic trajectories and tool interactions directly on target Apple hardware (iOS macOS watchOS visionOS).nImplement lightweight and event-capture layers to record high-fidelity execution traces (tool inputs/outputs state mutations and runtime events) for downstream scoring and CLI tools local automation scripts and integration hooks that allow software and ML teams to run on-device agent harnesses seamlessly within local development environments and continuous testing harness performance and resource overhead ensuring test execution remains stable and lightweight without interfering with on-device AI runtime behavior thermal limits or memory profiles.n

Strong programming proficiency in compiled languages (Swift C or Objective-C).nExcellent communication skills and ability to collaborate across and automation skills in Python for .S. or M.S. in Computer Science Machine Learning or equivalent computer science and systems fundamentals including process management and local runtimes.n

Hands-on experience developing on Apple platforms (iOS macOS visionOS) or working directly with low-level OS runtimes and designing or operating environment-staging systems such as state reset mechanisms device provisioning or sandboxed test building telemetry trace-capture or event-logging infrastructure for complex multi-threaded or asynchronous with on-device AI/LLM runtime environments function calling/tool execution mechanics or agentic system architectures.n