Software Engineer Manager Agentic Evaluation Platform

Join the team redefining what a deeply personal and integrated assistant can be and lead the platform that defines how we measure part of the Siri Agentic Evaluation Engineering organization you will help shape one of the worlds most widely used AI assistants powered by our next generation of Apple Intelligence with capabilities like personal context understanding and on-screen awareness built with privacy from the ground up. Your work will have direct meaningful impact for users across iOS iPadOS macOS watchOS and the engineering leader of the evaluation platform and services youll own the systems that run and scales Siris evaluations leveraging the best Apple platform capabilities and unlocking new ones to create novel evaluations. Your team applies large scale infrastructure apple platforms and service and AI agents to steer validate and judge evaluation scenarios at generate the signals that leadership relies on to make ship decisions. When we say a capability is ready its your platform that backs the is a rare opportunity to build scalable software and services for real world products at the intersection of cutting-edge AI and human-centered design shipping technology that is centered around users and their needs.n

In this role youll lead and grow a team of engineers building the backend services compute orchestration and evaluation harnesses that let us evaluate Siri reliably and at scale. Youll own the technical direction and the roadmap and youll be accountable for through-line of the work is interfaces. Your team owns the APIs that expose evaluation as a platform capability the harness contracts that determine how devices and models are driven through a run and the data workflows that carry results from execution through scoring to reporting. Those boundaries are the product: they decide how much compute the organization can actually use how quickly a failure can be traced to its cause and how many teams can build on evaluation without coming to you first. Get them right and the platform scales past your team; get them wrong and everything routes through you.n nYoull also have meaningful autonomy in how you get there. Both the evaluation platform and the underlying Siri architecture are evolving so the specific frameworks runtimes and components will change over time. Were looking for a leader who operates well in that ambiguity someone who can reshape team scope deliberately as the domain matures decide what to invest in versus retire and keep the teams momentum through the Just as importantly youll build the team itself: recruiting mentoring and developing engineers including senior ICs with a focus on engineering rigor technical depth and clear ownership.

Lead and grow a team of engineers building backend services compute orchestration and evaluation harnessesnSet the teams technical direction and roadmap and own delivery against itnOwn the backend services and APIs that expose evaluation as a platform capability job submission execution and results retrievalnOwn compute orchestration across heterogeneous device and host fleets: scheduling capacity allocation and the throughput and utilization targets that follownOwn harness engineering the layer that drives devices and models through a run captures output and hands off to scoringnDefine and steward the interfaces and contracts between harness pipelines and consuming teams including versioning backward compatibility and migration pathsnOwn the data workflows that move results from execution through scoring to storage and reporting and the schemas they depend onnDeliver and evolve reliability and health dashboards and release-readiness signals that leadership relies on for ship decisionsnSet the bar for diagnosability across the stack from environment provisioning through pipeline execution to scoring prioritizing automated triage and durable fixes over recurring manual investigationnPartner with QE engineers feature teams architecture leads and infrastructure teams to align on interfaces priorities and shared standards and to hold a defensible quality barnRecruit mentor and develop engineers including senior ICs with a focus on engineering rigor technical depth and clear ownershipnOperate effectively in ambiguity reshaping team scope as the evaluation platform and the underlying Siri architecture evolven

5 years of experience developing production software (e.g. Python Java Go or Swift)n3 years of engineering management experience including owning roadmap and delivery mentoring setting clear expectations and providing consistent honest feedbacknStrong technical leadership and the ability to set architectural direction and inspire a teamnExperience with test infrastructure CI/CD or evaluation/ML platformsnProactive and self-motivated with demonstrated creative and critical-thinking abilitiesnExcellent spoken and written communication skills and the ability to collaborate across teamsnM.S. or B.S. in Computer Science Machine Learning or a related field or equivalent experiencen

Experience building or leading teams that apply LLMs / AI agents to real engineering workflows (e.g. data generation code generation or agentic pipelines with human-in-the-loop review)nDepth in one or more of: distributed systems and backend services (REST/gRPC) cloud infrastructure (Docker/Kubernetes) data platforms or large-scale test and release infrastructurenFamiliarity with LLM and Agentic evaluation conceptsnExperience implementing model evaluation frameworks and experiment workflows including offline and online experiments to measure how model or platform changes affect key metrics such as accuracy latency and engagementnExperience building machine learning platforms and infrastructure used by multiple teams for experimentation and deploymentnExperience designing and integrating monitoring logging and alerting solutions dashboards anomaly detection and real-time alerts to track reliability and performance in production systemsnExperience developing data processing pipelines with big data frameworks (such as Apache Spark Kafka or Flink) to ingest transform and analyze large-scale log or event datanDemonstrated cross-functional leadership and the ability to drive coverage and quality decisions with senior stakeholdersnExperience hiring and growing a team including senior engineersnComfort operating in ambiguity and reshaping team scope as the domain matures