Apple Ray Inference Engineer
Apple Ray Inference Engineer
Posted on :
20-08-2025
Employer Active
1 Vacancy
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Monthly Salary
Not Disclosed
Salary Not Disclosed
Vacancy
1 Vacancy
Posted on :
20-08-2025
Job Description
Apple Ray leverages open-source Ray to offer a unified framework for processing and deployment of complex dataML pipelines. It enables the next generation of intelligent experiences for Apple products and services by combining data and processing layers as well as a model inference platform into one unified end-to-end workflow that eliminates the complexity of running multiple independent jobs while significantly improving the hardware resource efficiency and development speed. Tight integration of Apple Ray with Apple Data services makes it the go-to solution when serving complex and large-scale data and ML pipelines. The team enables future Apple intelligent products by making a cutting edge ecosystem of dataML technologies for large-scale and efficient systems for all data and ML engineers within Apple. As a member of the Apple Ray team your responsibilities will include:* Designing implementing and maintaining distributed systems to build world-class ML platforms/products at scale* Experiment with deploy and manage LLMs in a production context* Benchmark and optimize inference deployments for different workloads e.g. online vs. batch vs. streaming workloads* Diagnose fix improve and automate complex issues across the entire stack to ensure maximum uptime and performance * Design and extend services to improve functionality and reliability of the platform* Monitor system performance optimize for cost and efficiency and resolve any issues that arise* Build relationships with stakeholders across the organization to better understand internal customer needs and enhance our product better for end users
- 5 years of experience in distributed systems with deep knowledge in computer science fundamentals
- Experience managing deployments of LLMs at scale
- Experience with inference runtimes/engines e.g. ONNXRT TensorRT vLLM sglang
- Experience with ML Training/Inference profiling and optimization for different workloads and tasks e.g. online inference batch inference streaming inference
- Experience with profiling ML models for different end use cases e.g. RAG vs. code completion etc.
- Experience with containerization and orchestration technologies such as Docker and Kubernetes.
- Experience in delivering data and machine learning infrastructure in production environments
- Experience configuring deploying and troubleshooting large scale production environments
- Experience in designing building and maintaining scalable highly available systems that prioritize ease of use
- Experience with alerting monitoring and remediation automation in a large scale distributed environment
- Extensive programming experience in Java Python or Go
- Strong collaboration and communication (verbal and written) skills
- B.S. M.S. or Ph.D. in Computer Science Computer Engineering or equivalent practical experience
- Understanding of the ML lifecycle and state of the art ML Infrastructure technologies
- Familiarity with CUDA kernel implementation
- Experience with inference optimization and fine-tuning techniques (e.g. pruning distilling quantization)
- Experience with deploying optimizing ML models on heterogenous hardware e.g. GPUs TPUs Inferentia etc.
- Experience with GPU and other type of HPC infrastructure
- Experience with training framework like PyTorch Tensorflow JAX
- Deep understanding of Ray and KubeRay
Employment Type
Full-Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
