Robotics Machine Learning Engineer Platforms for Vision Language Action Foundation Models

Los Altos, CA - USA

Yearly Salary: $ 176000 - 264000

Posted on: 6 days ago

Vacancies: 1 Vacancy

Job Summary

At Toyota Research Institute (TRI) were on a mission to improve the quality of human life. Were developing new tools and capabilities to amplify the human experience. To lead this transformative shift in mobility weve built a world-class team in Automated Driving Energy & Materials Human-Centered AI Human-Interactive Driving and Robotics.

We are looking for a machine learning engineer to develop our infrastructure and support researchers in the development of foundation models for robotics.

The Mission

We are working to create general-purpose robots capable of accomplishing a wide variety of dexterous tasks. To do this our team is building general-purpose machine learning foundation models for dexterous robot manipulation. These models which we call Large Behavior Models (LBMs) use generative AI techniques to produce robot action from sensor data and human request. To accomplish this we are creating a large curriculum of embodied robot demonstration data and combining that data with a rich corpus of internet-scale text image and video data. We are also using high-quality simulation to augment real world robot data with procedurally-generated synthetic demonstrations.

The Team

The Robotics Machine Learning Teams charter is to push the frontiers of research in robotics and machine learning to develop the future capabilities required for general-purpose robots able to operate in realistic environments such as homes or factories.

The Job

We have several research thrusts under our broad mission and we are looking for a machine learning engineer to contribute to some of the following objectives:

Hardware Infrastructure: Develop our hardware platform making sure the robots and software stack are state-of-the-art operational and continuously improved with new functionalities. This includes the robot hardware (YAM Franka and custom) the sensors (monocular stereo depth etc) the robot/computer interface the human/robot interface the data logging and the controls.

Inference & Deployment: Build APIs and systems for high-throughput inference and logging in simulation and on real robot platforms. Enable low-latency model serving and robust policyenvironment communication.

Evaluation & Monitoring: Design metrics pipelines for quantitative and qualitative evaluation. Build tools for experiment tracking logging visualization and leaderboard management using systems like Weights & Biases MLflow or ClearML.

Data Infrastructure: Build scalable pipelines for heterogeneous multimodal data (images text video touch depth proprioception). Work with data storage versioning streaming and visualization systems optimized for throughput and accessibility.

The machine learning engineer who joins our team will be expected to create working code and interact frequently with researchers. They will run experiments with both simulated and real (physical) robots and participate in publishing the work to peer-reviewed venues. Were looking for an engineer who is comfortable working with multiple robotic embodiments and stacks as well as a growing dynamic corpus of robot data.

Qualifications

Hardware experience on robots
Communication protocol experience (ROS WebSocket RPC)
Strong software engineering skills in Python PyTorch and distributed systems.
Experience with large-scale data handling including streaming preprocessing and storage of video or sensor data.
A make it happen attitude and comfort with fast prototyping.
A passion for robotics and development grounded in important fundamental problems.
Continuous integration

Bonus Qualifications

Familiarity with modern ML efficiency frameworks (e.g. FSDP DeepSpeed XLA Ray Hugging Face Accelerate).
Experience with machine learning and familiarity with large multi-modal datasets and models.
Experience working in a research environment published research papers open-source projects

The pay range for this position at commencement of employment is expected to be between $176000 and $264000/year for California-based roles. Base pay offered will depend on multiple individualized factors including but not limited to business or organizational needs market location job-related knowledge skills and experience. TRI offers a generous benefits package including medical dental and vision insurance 401(k) eligibility paid time off benefits (including vacation sick time and parental leave) and an annual cash bonus structure. Additional details regarding these benefit plans will be provided if an employee receives an offer of employment.

Please reference thisCandidate Privacy Noticeto inform you of the categories of personal information that we collect from individuals who inquire about and/or apply to work for Toyota Research Institute Inc. or its subsidiaries including Toyota A.I. Ventures GP L.P. and the purposes for which we use such personal information.

TRI is fueled by a diverse and inclusive community of people with unique backgrounds education and life experiences. We are dedicated to fostering an innovative and collaborative environment by living the values that are an essential part of our culture. We believe diversity makes us stronger and are proud to provide Equal Employment Opportunity for all without regard to an applicants race color creed gender gender identity or expression sexual orientation national origin age physical or mental disability medical condition religion marital status genetic information veteran status or any other status protected under federal state or local laws.

It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability. Pursuant to the San Francisco Fair Chance Ordinance we will consider qualified applicants with arrest and conviction records for employment.

We may use artificial intelligence (AI) tools to support parts of the hiring process such as reviewing applications analyzing resumes or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed please contact us.