Research Engineer – LLM Training (fmd)

Heidelberg - Germany

Monthly Salary: Not Disclosed

Posted on: 30+ days ago

Vacancies: 1 Vacancy

Job Summary

Aleph Alpha Researchs mission is to deliver category-defining AI innovation that enables open accessible and trustworthy deployment of GenAI in industrial applications. Our organization develops foundational models and next-generation methods that make it easy and affordable for Aleph Alphas customers to increase productivity in finance administration R&D logistics and manufacturing processes.

Our model releases and research bio can be found on our company Hugging Face.

We are growing our Frontier LLM Research Engineering team with an experienced model trainer who will be responsible for conducting experiments that drive novel research across the entire lifecycle of model training. This involves proposing novel LLM architectures datamixes and evaluations to advance our offerings. This role sits at the intersection of research and engineering and requires a strong coding and AI science background.

The goal of our Frontier team is to own the entire lifecycle of model training. This role will contribute to research across a wide range of topics including (continuous) pre-training post-training and synthetic data generation with a strong emphasis on scalable training and data pipelines. Since the model artifacts produced by Frontier will power our product offerings we expect the successful candidate to be excited about language model applications and their real-world use cases.

Your responsibilities:

Research and develop novel approaches and algorithms that improve training of foundation models for practical use in real-world applications
Develop large-scale robust distributed training and data generation pipelines
Analyze and benchmark state-of-the-art as well as new approaches in LLM research
Collaborate with scientists and engineers at Aleph Alpha Aleph Alpha Research external industrial and academic partners and directly with customers
Publish own and collaborative work on machine learning venues and release code and models for use by the broader research community

Your profile:

Basic Qualifications

Recent experience addressing complex cutting-edge AI challenges with expertise in at least one of: distributed training training data model architectures
Advanced knowledge of transformers deep learning concepts and practices and ideally experience coding and pretraining LLMs from scratch
Strong software engineering skills with expertise in Python and related deep-learning frameworks (PyTorch)
Experience with shipping production-ready models building on open-source AI libraries
Proven ability to apply advanced scientific methods to novel problems resulting in impactful outputs such as publications or projects
Willingness to work from Heidelberg Berlin or in a hybrid setup within Germany; we value in-person collaboration and will cover all travel expenses to our Research HQ in Heidelberg for occasional onsite work

Preferred Qualifications

PhD in machine learning or related fields with publications in top tier ML/AI venues (eg NeurIPS ICML ICLR EMNLP NAACL ACL etc)
Experience writing kernels for GPUs (with CUDA Triton etc.)
Production-level skills with at least one other programming language especially systems languages (Rust C/C Go etc.)
Fluency in writing scientific documentation and proposals with strong public speaking skills in scientific contexts
Strong collaborative and interpersonal skills with a track record of contributing to a multidisciplinary teams technical and strategic success

What you can expect from us

Become part of an AI revolution contribute to Aleph Alphas mission to provide technological sovereignty
Work with international industry and academic experts
Share parts of your work via publications and source-available code
An inspiring working environment with short lines of communication horizontal organization and great team spirit
30 days of paid vacation
Access to a variety of fitness & wellness offerings via Wellhub
Mental health support through
Substantially subsidized company pension plan for your future security
Subsidized Germany-wide transportation ticket
Budget for additional technical equipment
Flexible working hours for better work-life balance and hybrid working model
Virtual Stock Option Plan
JobRad Bike Lease