Systems Engineer Evaluation Engineering

Join the team redefining what a deeply personal and integrated assistant can be. nnAs part of the Siri organization you will help shape one of the worlds most widely used AI assistants powered by our next-generation of Apple Intelligence with capabilities like personal context understanding and on-screen awareness built with privacy from the ground up. Your work will have direct meaningful impact for users across iOS iPadOS macOS watchOS and is a rare opportunity to build at the intersection of cutting-edge AI and human-centered design shipping technology that is centered around users and their needs.

We are looking for a Distributed Systems Engineer to own the infrastructure powering our core Siri Agentic Evaluation Platform. Evaluation is no longer just a static test suiteit is a highly dynamic massive-scale distributed problem. Our platform enables teams to run high-throughput agentic simulations orchestrate multi-model judging pipelines and generate real-time observability dashboards across billions of tokens and complex data this role you will design the execution engine that coordinates these complex evaluation loops. You will build systems that remain deterministic fault-tolerant and cost-efficient even when coordinating massive parallel requests across heterogeneous device types(iPhones Mac iPads etc).n

Distributed Execution Engine: Architect and scale the core asynchronous engine responsible for orchestrating thousands of parallel agent simulations validation tests and LLM-as-a-judge Developer Platform (IDP): Design and build self-service infrastructure CLI tools and internal APIs that allow ML and product teams to easily integrate evaluation pipelines into their CI/CD API u0026 Service Architecture: Design build and maintain highly performant type-safe APIs (gRPC/REST) capable of serving complex evaluation pipelinee trace data and real-time generation Processing u0026 Lineage: Build robust data pipelines to ingest and transform high-volume execution traces. Ensure immutable data lineage so that every evaluation metric can be perfectly traced back to its raw generation for granular error -as-Code u0026 GitOps: Own the deployment topologies of the evaluation platform across multi-tenant clusters using declarative infrastructure and continuous delivery Observability u0026 Guardrails: Implement deep observability (distributed tracing structured metrics and alerting) across the platform. Design smart scheduling layers token buckets and circuit breakers to prevent downstream API rate-limiting or cascading cluster failures.

MS in computer science or equivalentn7 years of experience as distributed systems engineer platform engineer or equivalentnSystems u0026 Backend Languages: Strong proficiency in languages optimized for concurrency and enterprise scale such as Python (asyncio) or JavanAPI Design u0026 Ecosystems: Deep expertise in designing robust versioned production APIs using gRPC/Protobuf GraphQL or REST (FastAPI)

Relational u0026 Analytical Databases: Strong experience modeling complex relational data and trace hierarchies using PostgreSQL combined with high-throughput analytical query u0026 Messaging: Experience designing asynchronous event-driven architectures using Kafka AWS SQS/SNS RabbitMQ or Redis Native Architecture: Advanced experience with Kubernetes (orchestration custom operators service meshes like Istio or Linkerd) and cloud providers (AWS GCP or Azure).nExperience building Agentic RAG platforms or developer-facing infrastructure -as-Code (IaC): proficiency with Terraform to manage infrastructure u0026 DevEx: Experience building automated containerized deployment pipelines (GitHub or ArgoCD) with an emphasis on keeping developer feedback loops fast and reliable.