Research Scientist, Open Source Technical Safeguards

Societal Resilience:

Societal Resilience is a multidisciplinary team that studies how advanced AI models can impact people and society. We research the prevalence and severity high-impact societal risks caused by frontier AI deployment and develop mitigations to address these risks. Core research topics include the use of AI for assisting with criminal activities preventing critical overreliance on insufficiently robust systems undermining trust in information jeopardising psychological wellbeing or for malicious social engineering. We are interested in both immediate and medium-term risks.

Why this team matters

One emerging risk area we are concerned with is the use of open weight models to drive risks like child sexual abuse material (CSAM) and non-consensual intimate imagery (NCII) generation. AISI has previously published research on methods for making open weight models more robust against malicious this role youll join a strongly collaborative technical research team to help design and develop technical safeguards for open weight models that will reduce the risks of CSAM NCII and other risk. We do not expect this role to handle this kind of content directly.

About the role:

This is a research scientist positionfocused on developing technical safeguards against tampering with open weight model. This role will focus on mitigating AI-generated CSAM and NCII by targeting the real-world supply chain driving harm: open-weight models adaptation artifacts (LoRAs guides) and downstream distribution infrastructure (hosting platforms app stores operating systems).

Our approach prioritises downstream mitigations and actors beyond frontier model developers. This role will build technical tools protocols and evidence that platforms and OS/app ecosystems can adopt.

This work belongs inside UK government because effective mitigation requires cross-agency coordination (Home Office DSIT Ofcom) engagement with regulated platforms under the Online Safety Act and credible evidence to inform policy trade-offs across innovation competition and child protection.

This role willsynthesise threat intelligence on howAI generated CSAM and NCIIare developed createscalable screening methodologies that platforms can realistically run and publish best-practice protocols with NGOs to raise the floor across the ecosystem.

Youllwork closely with engineers and domain experts across AISI as well as external research collaboratorsat Home OfficeInternet Watch Foundation and Ofcom. Researchers on this team have substantial freedom to shape independent research agendas lead collaborations and initiate projects that push the frontier of what evaluations can reveal.

Example Projects:

• Publish a Problem Book framing the technical challenges and research directions for preventing CSAM/NCII misuse across model and hosting layers.

• Develop threat models for how AI generated CSAM and NCII are created and shared.

• Design and pilot scalable automated screening methodologies platforms can run pre-publication on uploads (topic-general prototypes that avoid exposure to illegal content).

• Develop approaches foridentifyingand tracking known or novel CSAMLoRAsto enable platform blocking at upload.

• Co-develop best-practice protocols with NGOs (e.g. Thorn/IWF) for hosting app store and OS enforcement.

• This is an individual contributor role with no line management responsibilities. You will report into a senior Research Scientistoverseeing our teams misuse workstream.

Impact

Your work will raise safety standards across hosting and distribution layers reduce the availability of CSAM/NCII-generating artifacts () on major platforms informindustry protocols andpossibly standards and provide actionable evidence for government decisions

Crucially we do not expect this role to handle NCII or CSAM material.

Role Requirements:

Wereflexible on the exact profile and expect successful candidates will meet many (but not necessarily all) of the criteria below. Depending on experience we will consider candidates at either the RS or Senior RS level.

Essential

• At least 3years of relevant experience in applied ML trust & safety tooling content moderation security engineering or adjacent technical fields; we also welcome strong earlier-career applicants (23 years) withdemonstratedimpact in open-source technical work.

• Deep familiarity with open-weight image/video models (diffusionLoRA) model hosting ecosystems (e.g. Hugging Face GitHub) and the limitations of pre-deployment safeguards.

• Strong methodological rigor and creativity;able to design automated scalable evaluations and detection methods that generalise and avoid reliance on illegal content.

• Strong Python and ML stack (PyTorch/JAX) data engineering and systems skills; experience building pipelines and tooling that run at platform scale.

• Knowledge of fingerprinting and detection approaches (e.g. perceptual hashing embedding-based similarity behavioural signatures) and their privacy and robustness trade-offs.

• Excellent writing and communication for technical and policy audiences; ability to translate evidence into practical governance guidance.

• High agency ethical judgment and safe-working practices for sensitive topics.

• Committowork from our London office in Whitehall for parts of the week with flexibility for remote work.

• Werelooking for full-time commitment but are opento part-time arrangements.

Preferred

• Experience collaborating with hosting platforms app stores OS vendors or regulators (e.g. Ofcom) on safety-by-design initiatives.

• Familiarity with Online Safety Act requirements and platform trust & safety operations; prior work with NGOs such as IWF Thorn or .

• Expertisein diffusion models and adaptation techniques (LoRA) model evaluation and secure tooling for sensitive domains.

• Experience with privacy-preserving computation metadata-poor detection and standardization efforts (RFCs protocols).

• Open-source contributions (tools libraries) and evidence of leading cross-sector technical projects.

Example backgrounds

• Senior trust & safety engineer who built automated content integrity pipelines for a large platform; strong OSStrack record; experience with model hosting ecosystems.

• Applied ML researcher with a PhD/postdoc in computer vision or ML safety; hands-on with diffusion/LoRA; led evaluations and published tooling used by industry.

• Security/data engineer with 3 years building scalable detection systems; experience in fingerprinting hashing and privacy-preserving methods; collaborated with regulators/NGOs.

What we offer:

Impact youcouldnthave anywhere else

• Incredibly talented mission-drivenand supportive colleagues

• Direct influence on how frontier AI is governed and deployed globally

• Work with the Prime Ministers AI Advisor and leading AI companies
• Opportunity to shape the first & best-resourced public-interest research team focused on AI security

Resources & access

• Pre-release access to multiple frontier models and ample compute

• Extensive operational support so you can focus on research and ship quickly
• Work with experts across national security policy AI research and adjacent sciences

Growth & autonomy

• Ifyouretalented and drivenyoullown important problems early.

• 5 development days per year an annual L&D budget and travel support for conferences and external collaborations.

• Freedom to pursue research bets without product pressure

• Opportunities to publish and collaborate externally

Life & family

• Modern central London office (cafes food court gym) oroptionto work in similar government offices in Birmingham Cardiff Darlington Edinburgh Salford or Bristol

• Hybrid working with opportunities for occasional remote work abroad

• At least 25 days annual leave 8 public holidays and extra team-wide breaks

• Generous paid parental leave (36 weeks of UK statutory leave shared between parents 3 extra paid weeks option foradditionalunpaid time)

• Plus: 27% government-funded pension contribution on top of salary work from home equipment and dental insurance

Annual salary is benchmarked to role scope and relevant experience. Most offers land between 65000 and 145000 (base plus technical allowance) with 27% employer pension and other benefits on top (details on the what we offer section on ourcareers page).

This role sits outside of theDDaTpay frameworkgiven the scope of this role requires in depth technicalexpertisein frontier AI safetyrobustnessand advanced AI architectures.

The full range of salaries are available below:

• Level 3 - Total Package 65000 - 75000 inclusive of a base salary 35720 plusadditionaltechnical talent allowance of between 29280 - 39280

• Level 4 - Total Package 85000 - 95000 inclusive of a base salary 42495 plusadditionaltechnical talent allowance of between 42505 - 52505

• Level 5 - Total Package 105000 - 115000 inclusive of a base salary 55805 plusadditionaltechnical talent allowance of between 49195 - 59195

• Level 6 - Total Package 125000 - 135000 inclusive of a base salary 68770 plusadditionaltechnical talent allowance of between 56230 - 66230

• Level 7 - Total Package 145000 inclusive of a base salary 68770 plusadditionaltechnical talent allowance of 76230

In accordance withtheCivil Service Commissionrules the following listcontainsall selection criteria for the interview process.

The interview process may vary candidate tocandidatehowever you should expect a typical process to include some technicalproficiencytests discussions with a cross-section of our team at AISI (including non-technical staff) conversations with your team lead. The process will culminate in a conversation with members of the senior team here at AISI.

Candidates should expect to go throughsome orallofthe following stages once an application has beensubmitted:

• Initial interview

• Technical take home test

• Second interview and review oftake hometest

• Third interview

• Final interview with members of the senior team

Nationality requirements

We may be able to offer roles to applicant from any nationality or background. As such we encourage you to apply even if you do not meet the standard nationality requirements (opens in a new window).

Working for the Civil Service

Diversity and Inclusion