Machine Learning Systems Engineer

Our client is building a new generation of highly efficient AI models designed to dramatically improve the speed and economics of large language model inference.

The company has pioneered diffusion-based language models that generate responses in parallel rather than relying exclusively on traditional sequential token generation. This approach enables significantly faster and more efficient AI inference while maintaining competitive model quality.

The company launched one of the first commercially available diffusion-based language models in early 2025 and is now deploying large-scale AI models with Fortune 500 organizations.

The team is small highly technical and research-driven with engineers working directly alongside world-class researchers and founders. The organization places a strong emphasis on technical depth experimentation performance optimization and production-scale AI infrastructure.

As a Machine Learning Systems Engineer youll work on the infrastructure that enables large-scale model training and inference contributing directly to systems that make advanced AI models faster more efficient and more reliable.

This is an opportunity to join an elite AI team where you can work at the intersection of machine learning distributed systems GPU infrastructure and high-performance model serving.

• Design build and operate infrastructure supporting large-scale ML training and inference systems
• Develop high-performance systems for serving and deploying large language models
• Optimize model inference for latency throughput memory utilization and cost efficiency
• Build and maintain production ML infrastructure across GPU and cloud environments
• Work with inference engines such as vLLM TensorRT ONNX Runtime and SGLang
• Develop and optimize GPU-accelerated ML workloads using CUDA
• Build scalable training and inference pipelines using PyTorch and/or TensorFlow
• Deploy and manage ML workloads across Kubernetes and containerized environments
• Design distributed systems capable of supporting high-volume model inference
• Improve model serving performance across different hardware and infrastructure configurations
• Build reliable systems for model deployment monitoring evaluation and production operations
• Work closely with research teams to translate new model architectures into production systems
• Optimize infrastructure for emerging diffusion-based language models and other generative AI architectures
• Develop tooling and automation for ML experimentation and deployment
• Build and maintain cloud infrastructure across AWS Azure or comparable environments
• Work with Kubeflow and other ML orchestration platforms
• Investigate performance bottlenecks across compute networking memory and model-serving layers
• Develop systems that make model training and inference faster more efficient and more reliable
• Contribute to system architecture and technical strategy across the ML infrastructure stack
• Operate with high ownership in a fast-moving deeply technical AI environment
• Work closely with founders and researchers on highly technical infrastructure challenges

• 25 years of professional experience in ML Systems Engineering ML Infrastructure AI Infrastructure or related engineering roles
• Experience building and operating production ML systems
• Experience working on infrastructure for model training and/or inference
• Experience deploying machine learning models into production environments
• Experience working with GPU-based computing infrastructure
• Experience building scalable ML or distributed systems
• Experience working with modern deep learning frameworks
• Experience operating in technically demanding engineering environments
• Experience collaborating closely with research and engineering teams
• Strong ownership mentality with demonstrated execution ability
• Comfortable working on complex technical problems with limited precedent
• Strong interest in machine learning systems and AI infrastructure
• Ability to operate effectively in a fast-moving research-driven environment

• Strong Python engineering experience
• Strong experience with PyTorch TensorFlow or comparable ML frameworks
• Experience with GPU computing and CUDA
• Experience with ML inference systems such as vLLM TensorRT ONNX Runtime or SGLang
• Strong understanding of model serving and inference optimization
• Experience with Docker and containerized ML workloads
• Experience with Kubernetes
• Experience working with AWS Azure or other major cloud platforms
• Experience building distributed systems or scalable infrastructure
• Experience with ML training and inference pipelines
• Experience with Kubeflow or comparable ML orchestration platforms preferred
• Strong understanding of performance optimization and system reliability
• Experience debugging production ML infrastructure
• Strong understanding of computer science and software engineering fundamentals
• Ability to reason about GPU utilization memory constraints latency and throughput
• Strong debugging and performance analysis capabilities

• Bachelors degree or higher in Computer Science Computer Engineering Electrical Engineering Mathematics or related technical field preferred
• Advanced degree in Machine Learning Computer Science or related field is a plus
• Strong computer science systems and machine learning fundamentals
• Equivalent practical engineering experience accepted

• Exceptional technical ownership
• Strong analytical and problem-solving ability
• Deep technical curiosity
• Comfortable working on difficult and ambiguous infrastructure problems
• Strong communication skills
• Comfortable collaborating with researchers and highly technical engineers
• High execution velocity
• Strong attention to system performance and engineering quality
• Bias toward experimentation and continuous improvement
• Low-ego collaborative mentality
• Comfortable receiving and incorporating technical feedback
• Strong ability to reason from first principles
• Comfortable working in a small high-performing team
• Strong interest in cutting-edge AI systems
• Willingness to work five days per week in Palo Alto

• Base Salary: $200000 $300000
• Competitive Equity Package
• Opportunity to work on cutting-edge diffusion-based language models
• Direct collaboration with world-class AI researchers and founders
• Opportunity to work on large-scale ML training and inference infrastructure
• Exposure to advanced GPU optimization and AI systems engineering
• Significant technical ownership in a small elite engineering organization
• Opportunity to influence foundational ML infrastructure and model-serving architecture
• High-growth AI company environment
• Opportunity to work on AI systems being deployed by Fortune 500 organizations

This is an opportunity to join a highly technical AI company working on one of the most important challenges in modern machine learning: making advanced language models significantly faster and more efficient.

Youll work directly on the infrastructure powering next-generation AI models solving difficult problems across distributed systems GPU computing model inference and production ML infrastructure.

As part of a small and technically elite team youll work closely with researchers and founders and have meaningful influence over how AI systems are designed optimized and deployed.

If you enjoy systems engineering machine learning infrastructure GPU optimization high-performance inference and solving difficult technical problems at the intersection of research and production this role offers exceptional technical depth and impact.