Enter a job title or keyword

Inference Optimization Engineer

Hedra


Job Location:

San Francisco, CA - USA

Monthly Salary: Not provided by the employer
Posted: 15 November 2026 (3 days ago)
Application Deadline: 12 February 2027
Vacancies: 1 Vacancy

Job Summary

About Hedra

Hedra is a research and infrastructure lab advancing post-training and inference for multimodal and physical AI. Our ambition is to bring recursive self-improvement to models that learn from and interact with the real world.

We combine reinforcement learning research with GPU compute and optimized inference to help teams post-train serve and continuously improve models they own.

Were backed by Index Ventures a16z and Abstract Ventures. Our team works fully in person in San Francisco.

Were looking for people with startup experience who take ownership of their work and care deeply about the problems our customers need to solve. Youll help build the technology products and business that turn this research into a lasting company.

What Youll Do
  • Work directly with research scientists and engineers to make new visual models fast and efficient at inference time.

  • Profile model architectures and workloads to understand bottlenecks across compute memory communication and model execution.

  • Develop and implement new approaches to improving inference latency throughput memory efficiency and GPU utilization.

  • Explore algorithmic optimizations including quantization sparsity caching compilation attention optimizations and alternative execution strategies.

  • Build or optimize GPU kernels using CUDA Triton or similar technologies when existing implementations leave performance on the table.

  • Optimize model execution across single-GPU multi-GPU and multi-node environments.

  • Reason about the interaction between model architecture and hardware and work with researchers when architectural changes can unlock meaningful performance improvements.

  • Investigate communication memory movement parallelism and distributed execution strategies for large visual models.

  • Build rigorous benchmarking profiling and performance-regression infrastructure to understand performance and evaluate new optimization ideas.

  • Evaluate new inference runtimes compilers frameworks optimization techniques and accelerator hardware.

  • Stay close to advances in efficient inference GPU programming model architectures compilers and ML systems research and rapidly test promising ideas.

  • Help turn research breakthroughs into models that can be deployed and served efficiently at scale.

What Were Looking For
  • Deep technical ability in efficient ML inference ML systems GPU computing or adjacent research demonstrated through research production engineering open-source contributions or unusually ambitious independent work.

  • Strong understanding of how modern deep learning models execute on hardware including the relationship between compute memory communication and performance.

  • Experience profiling ML workloads identifying bottlenecks forming hypotheses and driving measurable performance improvements.

  • Strong programming fundamentals in Python C or another systems-oriented language.

  • Experience with some combination of PyTorch CUDA Triton TensorRT vLLM SGLang or comparable technologies.

  • Ability to reason across abstraction layers rather than treating model architecture framework runtime kernel and hardware boundaries as fixed.

  • Strong intuition for performance tradeoffs across latency throughput memory numerical precision model quality and complexity.

  • Curiosity about how models work internally and a willingness to modify or rethink existing approaches when the performance problem calls for it.

  • Comfort working on ambiguous problems where the bottleneck and sometimes even the right question is not known in advance.

  • Ability to communicate technical ideas clearly and collaborate closely with research scientists and engineers.

We dont expect every candidate to have experience across the entire stack. Exceptional depth in one or more relevant areas combined with the ability and curiosity to reason across the others matters more to us than checking every box.

Nice to Have
  • Experience optimizing large generative multimodal vision or video models.

  • CUDA Triton CUTLASS or other GPU kernel development.

  • Deep knowledge of GPU architecture memory hierarchy and hardware-aware optimization.

  • Experience with attention optimization kernel fusion memory-efficient execution or custom operators.

  • Model compilation or graph optimization experience.

  • Quantization sparsity caching speculative execution or other efficient inference techniques.

  • Experience optimizing diffusion autoregressive transformer or other large generative architectures.

  • Multi-GPU or multi-node model execution including tensor pipeline sequence or other forms of parallelism.

  • Experience optimizing communication or data movement between accelerators.

  • Experience with profiling tools such as Nsight Systems or Nsight Compute.

  • Contributions to ML systems inference runtimes compilers GPU libraries or performance-focused open-source projects.

  • Research or publications in efficient ML ML systems GPU computing compilers or related areas.

Benefits:
  • Competitive compensation and equity

  • 401k

  • Healthcare (Silver PPO Medical Vision Dental)

  • Lunch and snacks at the office

This role is based in San Francisco and we work together in person five days a week.


Required Experience:

IC


About Company

Company Logo

Hedra is a next generation multimodal AI content creation platform.

View Profile View Profile