Sr. Machine Learning Engineer, Foundation Models Inference Cloud OS & Inference

We are the Foundation Model Inference team within Cloud OS and AI Inference organization. We are on a mission to build the most highly performant secure and private inference stack that powers Siri AI Apple Intelligence and Apps that are powered with the largest foundation systems serve billions of queries daily across Siri AI Apple Intelligence Apple Search Apple Music Apple TV App Store iMessage Photos Camera Spotlight u0026 Safari at remarkably low latency with every ounce of compute extracted from the hardware beneath them. We optimize language vision and speech models with billions of parameters using state-of-the-art techniques and ship them at Apple is a rare opportunity to directly shape how AI reaches billions of people worldwide.

You will work at the intersection of research and production partnering closely with the Foundation Model Research team and our external partners to bring cutting-edge model architectures from prototype to planetary-scale deployment. You will own hard problems in inference efficiency hardware/software codesign systems architecture and tooling and help set the technical direction for the engineers around role sits within CloudOS and Private Cloud Compute (PCC) Apples purpose-built privacy-preserving cloud infrastructure for AI workloads. PCC represents a first-of-its-kind approach to running foundation models in the cloud with verifiable privacy guarantees and CloudOS is the systems foundation that makes it possible. You will be building and optimizing inference systems on top of this infrastructure working closely with platform and security teams to deliver both performance and trust at scale.

Partner with the Foundation Model Research team and our external partners to optimize inference for the latest model architectures across language vision and and ship production-grade inference systems serving millions of customers in real profiling tools and simulators to identify and resolve performance bottlenecks across different hardware configurations and use technical decisions on high-throughput low-latency serving at supercomputing and grow engineers across the organization.

5 years of experience leading complex ambiguous technical projects from end to -on experience with LLM inference knowledge of GPU or TPU programming with PyTorch JAX or building and operating high-throughput services at large distributed deploying applications on cloud platforms (AWS GCP or equivalent) using Kubernetes and in Computer Science Machine Learning Artificial Intelligence Data Science or a related field.

Experience building productions systems in Go or knowledge of deep learning architectures including Transformers encoder/decoder models and multimodal with inference optimization frameworks such as TensorRT-LLM vLLM SGLang TGI or Nvidia Triton authoring custom CUDA kernels using CUDA C or OpenAI in Computer Science Machine Learning Artificial Intelligence Data Science or a related field.