At Roche you can show up as yourself embraced for the unique qualities you bring. Our culture encourages personal expression open dialogue and genuine connections where you are valued accepted and respected for who you are allowing you to thrive both personally and professionally. This is how we aim to prevent stop and cure diseases and ensure everyone has access to healthcare today and for generations to come. Join Roche where every voice matters.

The Position

We believe its urgent to deliver medical solutions right now even as we develop innovations for the future. We are passionate about transforming patients lives and we are fearless in both decision and action. And we believe that good business means a better world. We commit ourselves to scientific rigor unassailable ethics and access to medical innovations for all. We do this today to build a better tomorrow. Pharmaceutical Sciences (PS) is a global function within Roche Pharma Research and Early Development (pRED).

As a team member in the Prediction Modelling team of PS you will work in close collaboration with (computational) toxicologists as well as other scientists in pRED using state-of-the-art bioinformatics and biostatistics tools and methods and gaining toxicological insights from experts in the field.

The Opportunity

Roche has accumulated a vast collection of curated preclinical and clinical study reports spanning the full drug development lifecycle. Current commercial LLMs still underperform in specialized biomedical and pharmaceutical contexts. This internship focuses on advancing Roches internal LLM post-training capabilities to build domain-specialized LLMs for clinical study use by applying Supervised Fine-Tuning (SFT) Direct Preference Optimization (DPO) and Reinforcement Learning from Human Feedback (RLHF). Youll work side-by-side with Roche ML scientists to fine-tune open-source models optimize them for pharmacological translational-science and drug-discovery tasks and help build the infrastructure and workflows needed to support model deployment in a drug-discovery context.

Youll have access to high-performance GPU infrastructure (A100 clusters) and unique domain-specific data. Youll start with smaller-sized LLMs to build the methodology then iterate toward larger models and production-scale workflows.

• Data Creation: Approximately 10% of this interns work will be data collection and dataset creation.

• Model Post-Training: Fine-tune and align open-source LLMs (e.g. Llama Mistral Qwen) using Roches curated clinical and preclinical datasets through SFT DPO or RLHF.

• Pipeline Development: Implement and iterate on training pipelines using frameworks such as (including but not limited to) Hugging Face Transformers TRL NV Megatron-LM or HF Smol.

• Evaluation: Design evaluation protocols for factual accuracy safety and alignment with biomedical domain knowledge.

• Experimentation: Start with small-scale LLMs to establish scalable training and evaluation workflows progressing toward larger foundation models.

• Documentation: Maintain experiment logs model cards and reproducible training setups for internal knowledge transfer.

Who you are

• Enrolled in a Masters or PhD program in Computer Science Computational Linguistics or a related field.

• Strong coding skills in Python and PyTorch; experience with large-scale model training and distributed computing in Linux-based clusters.

• Hands-on experience with at least one LLM training or fine-tuning framework (e.g. Hugging Face Transformers/TRL Megatron-LM HF Smol).

• Familiarity with RLHF/DPO workflows and human-in-the-loop alignment methods.

• Understanding of evaluation metrics for LLMs (factual consistency hallucination detection preference modeling).

• Interest in applying AI to clinical and biomedical domains and working with sensitive data responsibly.

• You have very good interpersonal and communication skills are able to build good working relationships and are an outstanding teammate. Your experience and investigative attitude allow you to work independently to design perform and interpret experiments and to embark on new scientific methodologies.

Start: asap/ February 2026

Duration: 12 Months

Workload: 100%

Due to regulations Non-EU/EFTA citizens must provide a certificate from the university stating that an internship is mandatory as part of the application documents. Furthermore they need to be enrolled during the entire duration of the internship.

We are looking forward to your application!

Who we are

A healthier future drives us to innovate. Together more than 100000 employees across the globe are dedicated to advance science ensuring everyone has access to healthcare today and for generations to come. Our efforts result in more than 26 million people treated with our medicines and over 30 billion tests conducted using our Diagnostics products. We empower each other to explore new possibilities foster creativity and keep our ambitions high so we can deliver life-changing healthcare solutions that make a global impact.

Lets build a healthier future together.