Control Red Team - Research Engineer/Research Scientist

London

Negotiable

About this role

Phone numbers and emails in this ad are masked until you log in.

auto_translated_note

<div class="content-intro"><h2>About the AI Security Institute</h2> The AI Security Institute is the world's largest and best-funded team dedicated to understanding advanced AI risks and translating that knowledge into action. We’re in the heart of the UK government with direct lines to No. 10 (the Prime Minister's office), and we work with frontier developers and governments globally. We’re here because governments are critical for advanced AI going well, and UK AISI is uniquely positioned to mobilise them.

With our resources, unique agility and international influence, this is the best place to shape both AI development and government action.</div><h2>The deadline for applying to this role is 30th September 2026, end of day, anywhere on Earth. </h2> <h2>Team Description</h2> Control measures - monitors, permission systems, sandboxing, resampling, escalation protocols - are designed to detect and prevent misaligned behaviour from advanced AI systems. Though the measures are already critical to safety, whether they would in fact catch a capable model attempting to cause harm is an empirical question that remains largely untested.   The Control Red Team at AISI is stress-testing control monitors (<a href="https://www.aisi.gov.uk/blog/how-our-new-control-red-team-is-stress-testing-frontier-monitors">including from GDM and Anthropic</a>) - finding attacks that beat monitors and attempting to design experiments to measure real-world risk. We’re based within the Red Team, about a dozen people who have spent the last two years breaking developer’s alignment and misuse safeguards, and grew out of <a href="https://www.aisi.gov.uk/research?category_equal=%5B%22Control%22%5D">AISI’s previous research into control evaluations and safety cases.</a>  We're opening roles on the Control Red Team, and we think it's an unusually good place to do this work.

You'd join early, with real ownership over the team's direction; you'd have frontier model access, serious compute and strong infrastructure support from across AISI; and you'd get privileged insight into control measures across several frontier developers, working alongside some of the most experienced red teamers in the field.  Our current bet is to focus our effort on monitoring: the measures frontier companies lean on most heavily, and the ones where the science of evaluation is not yet settled.   <h2>

About the Role

</h2> <h2>What You'll Be Doing</h2> You’ll spend your time across two tracks of work:   1. Research: How and what should we measure to understand the efficacy of control measures?   How can we gather empirical evidence about how likely a monitor is to prevent harm - and what can we legitimately conclude from it? How do you estimate a monitor's recall against dangerous behaviours nobody has seen yet? These are difficult questions - conceptually and empirically. Day to day this looks like: designing and running ML experiments (including RL and other optimisation-heavy work), building the adversarial attacks that generate the evidence, writing arguments, and arguing them out with the rest of the team.

We intend to publish this work.  2. Testing: Running evaluations of frontier labs' monitors, and reporting the implications.  Turning our research into concrete assessments of real systems: threat modelling how an AI attacker would actually operate in a frontier internal deployment; breaking monitors, sandboxes and the surrounding infrastructure; conducting security analyses; and producing reports that are decision-relevant and action-guiding for the companies and for government.  Underpinning both: <ul> <li data-leveltext="•" data-font="Arial" data-listid="2" data-list-defn-props="{"*** *** 41":1,"*** *** 85":720,"*** *** 91":360,"*** *** 42":[8226],"*** *** 03":"left","*** *** 04":"•","*** *** 15":"multilevel"}" data-aria-posinset="1" data-aria-level="1">Building tooling and experimental pipelines that let us go from question to result fast, at a quality bar that makes the results reusable. </li> </ul> <ul> <li data-leveltext="•" data-font="Arial" data-listid="2" data-list-defn-props="{"*** *** 41":1,"*** *** 85":720,"*** *** 91":360,"*** *** 42":[8226],"*** *** 03":"left","*** *** 04":"•","*** *** 15":"multilevel"}" data-aria-posinset="2" data-aria-level="1">Heavy use of LLMs to automate our own attack, evaluation and analysis loops - and getting faster as models improve. </li> </ul> <ul> <li data-leveltext="•" data-font="Arial" data-listid="2" data-list-defn-props="{"*** *** 41":1,"*** *** 85":720,"*** *** 91":360,"*** *** 42":[8226],"*** *** 03":"left","*** *** 04":"•","*** *** 15":"multilevel"}" data-aria-posinset="3" data-aria-level="1">Building and running the infrastructure for training and serving models at the scale our experiments need. </li> </ul> <h2>Research Scientists and Research Engineers </h2> We're looking for research science and engineering skills, and we're excited to hear from strong scientists, strong engineers, and people who are a bit of both.  We’re deliberately open on seniority. For exceptional candidates, with experience leading research teams, we’ll grow the scope to match.  <h2>

What We're Looking For

</h2> The experiences listed are examples of the expertise we're looking for, rather than a list of everything we expect to find in one applicant.  <h2>Essential

Requirements

</h2> <ul> <li>Demonstrated ability to design, build and run ML experiments on frontier models, and to work autonomously on complex research projects involving substantial engineering. This includes black-box work (API-based evaluations and attacks) and ideally some white-box work (e.g. fine-tuning open-weight models). </li> <li>Strong software engineering and ML experience: writing clean, documented, reusable code for machine learning experiments - beyond one-off research scripts - including experience with LLM finetuning and inference frameworks, or evaluation frameworks like Inspect.  </li> <li>The ability to understand and critique how an experiment does and does not support a safety claim - including an understanding of why AI safety and control are hard problems, or a clear appetite to get up to speed fast. </li> <li>Impact-driven mindset and a collaborative team player: motivated by the work that most reduces risk rather than what is superficially impressive, flexible about what needs doing, and high velocity with a high-quality bar for outputs. </li> </ul> <h2>Highly Desirable</h2> We don't expect candidates to have all of these - they're additional signals that help us identify exceptional fits for specific aspects of the role.  <ul> <li>A good working model of frontier AI companies' internal deployments: what their ML infrastructure and dev practices look like, the kinds of experiments they run internally, and where the security weak points and easiest escape routes would be. </li> <li>An exceptional red-teaming mindset - instinctively finding the path a capable adversary would actually take, whether against a model, a monitor or a sandbox. </li> <li>Experience with ML optimisation: RL, SFT, evolutionary methods, or similar. Experience optimising hard against a defined metric and making (and justifying) careful measurement choices. </li> <li>Strong written communication and argumentation: high-quality research write-ups in any medium - a paper, a blog post, an internal report, an unusually good thread - where the reasoning, not just the result, is the point. </li> <li>Willingness and ability to construct and defend arguments for safety claims, and to think about which claims are worth making in the first place. </li> <li>Experience building or operating ML research infrastructure at a large organisation: GPU management, running experiments at scale, securing evaluation environments. </li> <li>Experience in cybersecurity or security analysis, including attacking LLM-based applications and agent scaffolds.</li> <li>Familiarity with the AI control and adversarial ML literature, and existing relationships with researchers working on control at labs or in the wider safety community.  </li> <li>Participation in an AI safety research or fellowship programme, or equivalent evidence of independent research output. </li> <li>Broad evidence of strong mathematical, scientific or analytic ability (for example, highly competitive courses or programmes, or olympiad-level results.</li> <li>Proficient use of LLM coding tools and agents. </li> </ul> We are less interested in credentials as such: a first-author conference paper or a CS degree is welcome evidence, but neither is required, and neither substitutes for the signals above.  <h2>Selection process </h2> The interview process may vary from candidate to candidate; however, you should expect a typical process to include some technical proficiency tests, discussions with a cross-section of our team at AISI (including non-technical staff), and conversations with your team lead.

The process will culminate in a conversation with members of the senior leadership team here at AISI.  Candidates should expect to go through some or all of the following stages once an application has been submitted:  <ul> <li>Initial assessment </li> <li>Initial screening call</li> <li>Technical assessment</li> <li>Behavioural interview</li> <li>Research interview</li> <li>Final interview with members of the senior leadership team </li> </ul><div class="content-conclusion"><h2>What We Offer </h2> Impact you couldn't have anywhere else  <ul> <li data-leveltext="" data-font="Symbol" data-listid="52" data-list-defn-props="{"*** *** 41":1,"*** *** 85":720,"*** *** 91":360,"*** *** 26":"Symbol","*** *** 42":[8226],"*** *** 03":"left","*** *** 04":"","*** *** 15":"hybridMultilevel"}" data-aria-posinset="1" data-aria-level="1">Incredibly talented, mission-driven and supportive colleagues. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="52" data-list-defn-props="{"*** *** 41":1,"*** *** 85":720,"*** *** 91":360,"*** *** 26":"Symbol","*** *** 42":[8226],"*** *** 03":"left","*** *** 04":"","*** *** 15":"hybridMultilevel"}" data-aria-posinset="2" data-aria-level="1">Direct influence on how frontier AI is governed and deployed globally. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="52" data-list-defn-props="{"*** *** 41":1,"*** *** 85":720,"*** *** 91":360,"*** *** 26":"Symbol","*** *** 42":[8226],"*** *** 03":"left","*** *** 04":"","*** *** 15":"hybridMultilevel"}" data-aria-posinset="3" data-aria-level="1">Work with the Prime Minister’s AI Advisor and leading AI companies. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="52" data-list-defn-props="{"*** *** 41":1,"*** *** 85":720,"*** *** 91":360,"*** *** 26":"Symbol","*** *** 42":[8226],"*** *** 03":"left","*** *** 04":"","*** *** 15":"hybridMultilevel"}" data-aria-posinset="4" data-aria-level="1">Opportunity to shape the first & best-resourced public-interest research team focused on AI security. </li> </ul> Resources & access  <ul> <li data-leveltext="" data-font="Symbol" data-listid="52" data-list-defn-props="{"*** *** 41":1,"*** *** 85":720,"*** *** 91":360,"*** *** 26":"Symbol","*** *** 42":[8226],"*** *** 03":"left","*** *** 04":"","*** *** 15":"hybridMultilevel"}" data-aria-posinset="5" data-aria-level="1">Pre-release access to multiple frontier models and ample compute. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="52" data-list-defn-props="{"*** *** 41":1,"*** *** 85":720,"*** *** 91":360,"*** *** 26":"Symbol","*** *** 42":[8226],"*** *** 03":"left","*** *** 04":"","*** *** 15":"hybridMultilevel"}" data-aria-posinset="6" data-aria-level="1">Extensive operational support so you can focus on research and ship quickly. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="52" data-list-defn-props="{"*** *** 41":1,"*** *** 85":720,"*** *** 91":360,"*** *** 26":"Symbol","*** *** 42":[8226],"*** *** 03":"left","*** *** 04":"","*** *** 15":"hybridMultilevel"}" data-aria-posinset="7" data-aria-level="1">Work with experts across national security, policy, AI research and adjacent sciences. </li> </ul> Growth & autonomy  <ul> <li data-leveltext="" data-font="Symbol" data-listid="52" data-list-defn-props="{"*** *** 41":1,"*** *** 85":720,"*** *** 91":360,"*** *** 26":"Symbol","*** *** 42":[8226],"*** *** 03":"left","*** *** 04":"","*** *** 15":"hybridMultilevel"}" data-aria-posinset="8" data-aria-level="1">If you’re talented and driven, you’ll own important problems early. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="52" data-list-defn-props="{"*** *** 41":1,"*** *** 85":720,"*** *** 91":360,"*** *** 26":"Symbol","*** *** 42":[8226],"*** *** 03":"left","*** *** 04":"","*** *** 15":"hybridMultilevel"}" data-aria-posinset="9" data-aria-level="1">5 days off and annual stipends for learning and development, and funding for conferences and external collaborations. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="52" data-list-defn-props="{"*** *** 41":1,"*** *** 85":720,"*** *** 91":360,"*** *** 26":"Symbol","*** *** 42":[8226],"*** *** 03":"left","*** *** 04":"","*** *** 15":"hybridMultilevel"}" data-aria-posinset="10" data-aria-level="1">Freedom to pursue research bets without product pressure. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="52" data-list-defn-props="{"*** *** 41":1,"*** *** 85":720,"*** *** 91":360,"*** *** 26":"Symbol","*** *** 42":[8226],"*** *** 03":"left","*** *** 04":"","*** *** 15":"hybridMultilevel"}" data-aria-posinset="11" data-aria-level="1">Opportunities to publish and collaborate externally. </li> </ul> Life & family*  <ul> <li data-leveltext="" data-font="Symbol" data-listid="52" data-list-defn-props="{"*** *** 41":1,"*** *** 85":720,"*** *** 91":360,"*** *** 26":"Symbol","*** *** 42":[8226],"*** *** 03":"left","*** *** 04":"","*** *** 15":"hybridMultilevel"}" data-aria-posinset="12" data-aria-level="1">Modern central London office, or where applicable, option to work in similar government offices in Birmingham, Cardiff, Darlington, Edinburgh, Salford or Bristol. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="52" data-list-defn-props="{"*** *** 41":1,"*** *** 85":720,"*** *** 91":360,"*** *** 26":"Symbol","*** *** 42":[8226],"*** *** 03":"left","*** *** 04":"","*** *** 15":"hybridMultilevel"}" data-aria-posinset="13" data-aria-level="1">Hybrid working, flexibility for occasional remote work abroad and stipends for work-from-home equipment. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="52" data-list-defn-props="{"*** *** 41":1,"*** *** 85":720,"*** *** 91":360,"*** *** 26":"Symbol","*** *** 42":[8226],"*** *** 03":"left","*** *** 04":"","*** *** 15":"hybridMultilevel"}" data-aria-posinset="14" data-aria-level="1">At least 25 days’ annual leave, 8 public holidays, extra team-wide breaks and 3 days off for volunteering. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="52" data-list-defn-props="{"*** *** 41":1,"*** *** 85":720,"*** *** 91":360,"*** *** 26":"Symbol","*** *** 42":[8226],"*** *** 03":"left","*** *** 04":"","*** *** 15":"hybridMultilevel"}" data-aria-posinset="15" data-aria-level="1">Generous paid parental leave (36 weeks of UK statutory leave shared between parents + 3 extra paid weeks + option for additional unpaid time). </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="52" data-list-defn-props="{"*** *** 41":1,"*** *** 85":720,"*** *** 91":360,"*** *** 26":"Symbol","*** *** 42":[8226],"*** *** 03":"left","*** *** 04":"","*** *** 15":"hybridMultilevel"}" data-aria-posinset="16" data-aria-level="1">On top of your salary, we contribute 28.97% of your base salary to your pension. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="52" data-list-defn-props="{"*** *** 41":1,"*** *** 85":720,"*** *** 91":360,"*** *** 26":"Symbol","*** *** 42":[8226],"*** *** 03":"left","*** *** 04":"","*** *** 15":"hybridMultilevel"}" data-aria-posinset="17" data-aria-level="1">Discounts and

Benefits

for cycling to work, donations and retail/gyms.   </li> </ul> *These

Benefits

apply to direct employees.

Benefits

may differ for individuals joining through other employment arrangements such as secondments.  <h2>Salary</h2> Annual salary is benchmarked to role scope and relevant experience. Most offers land between £65,000 and £145,000 made up of a base salary plus a technical allowance (take-home salary = base + technical allowance). An additional 28.97% employer pension contribution is paid on the base salary.  This role sits outside of the <a href="https://ddat-capability-framework.service.gov.uk/">DDaT pay framework</a> given the scope of this role requires in depth technical expertise in frontier AI safety, robustness and advanced AI architectures.  The full range of salaries are available below:  <ul> <li>Level 3: £65,000 - £75,000 (Base £39,850 + Technical Allowance £25,150 - £35,150)</li> <li>Level 4: £85,000 - £95,000 (Base £47,355 + Technical Allowance £37,645 - £47,645)</li> <li>Level 5: £105,000 - £115,000 (Base £61,620 + Technical Allowance £43,380 - £53,380)</li> <li>Level 6: £125,000 - £135,000 (Base £74,605 + Technical Allowance £50,395 - £60,395)</li> <li>Level 7: £145,000 (Base £74,605 + Technical Allowance £70,395)</li> </ul>   <hr> <h4>Additional Information</h4> <div class="vac_display_field"> <div class="vac_display_nullclass"> <h3>Use of AI in Applications</h3> Artificial Intelligence can be a useful tool to support your application, however, all examples and statements provided must be truthful, factually accurate and taken directly from your own experience.

Where plagiarism has been identified (presenting the ideas and experiences of others, or generated by artificial intelligence, as your own) applications may be withdrawn and internal candidates may be subject to disciplinary action. Please see our <a href="https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.civil-service-careers.gov.uk%2Fartificial-intelligence-and-recruitment%2F&data=05%7C02%7CLynn.Tan%40dsit.gov.uk%7Ccf5d753b7f19427afc8f08de4c307014%7Ccbac700502c143ebb497e6492d1b2dd8%7C0%7C0%7C639031968075235036%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=jCrYoy62YfXN%2FLIB%2FhtHxdLGbCOgropbkZXDyQsUMC4%3D&reserved=0" target="_blank" data-outlook-id="02db05e7-03ca-4ca6-bcd9-73ff8bf258f2" data-auth="NotApplicable" data-linkindex="1">candidate guidance </a>for more information on appropriate and inappropriate use. <h3>Internal Fraud Database </h3> The Internal Fraud function of the Fraud, Error, Debt and Grants Function at the Cabinet Office processes details of civil servants who have been dismissed for committing internal fraud, or who would have been dismissed had they not resigned. The Cabinet Office receives the details from participating government organisations of civil servants who have been dismissed, or who would have been dismissed had they not resigned, for internal fraud.

In instances such as this, civil servants are then banned for 5 years from further employment in the civil service. The Cabinet Office then processes this data and discloses a limited dataset back to DLUHC as a participating government organisations. DLUHC then carry out the pre employment checks so as to detect instances where known fraudsters are attempting to reapply for roles in the civil service.

In this way, the policy is ensured and the repetition of internal fraud is prevented.  For more information please see - <a href="https://www.gov.uk/government/publications/internal-fraud-register-privacy-notice/internal-fraud-register-privacy-notice">Internal Fraud Register.</a> <h3>Security</h3> Successful candidates must undergo a criminal record check and get <a class="c-link" href="https://www.gov.uk/government/publications/government-baseline-personnel-security-standard" target="_blank" data-stringify-link="https://www.gov.uk/government/publications/government-baseline-personnel-security-standard" data-sk="tooltip_parent">baseline personnel security standard (BPSS) clearance</a> before they can be appointed. Additionally, there is a strong preference for eligibility for <a class="c-link" href="https://www.gov.uk/government/publications/national-security-vetting-advice-for-people-who-are-being-vetted" target="_blank" data-stringify-link="https://www.gov.uk/government/publications/national-security-vetting-advice-for-people-who-are-being-vetted" data-sk="tooltip_parent">counter-terrorist check (CTC) clearance</a>. Some roles may require higher levels of clearance, and we will state this by exception in the job advertisement. <a class="c-link" href="https://www.gov.uk/government/publications/vetting-explained-and-our-vetting-charter/the-vetting-charter" target="_blank" data-stringify-link="https://www.gov.uk/government/publications/vetting-explained-and-our-vetting-charter/the-vetting-charter" data-sk="tooltip_parent">See our vetting charter here.</a></div> </div> <div class="vac_display_field"> <h3>Nationality

Requirements

</h3> We may be able to offer roles to applicant from any nationality or background. As such we encourage you to apply even if you do not meet the standard <a href="https://www.gov.uk/government/publications/nationality-rules" target="_blank">nationality

Requirements

(opens in a new window).</a> </div> <div class="vac_display_field"> <h3>Working for the Civil Service</h3> <div class="vac_display_field"> <div class="vac_display_field_value">The <a href="http://civilservicecommission.independent.gov.uk/code/" target="_blank">Civil Service Code (opens in a new window)</a> sets out the standards of behaviour expected of civil servants. The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria. The Civil Service also offers a Redeployment Interview Scheme to civil servants who are at risk of redundancy, and who meet the minimum

Requirements

for the advertised vacancy.</div> </div> </div> <div class="vac_display_field"> <h3>Diversity and Inclusion</h3> <div class="vac_display_field_value">The Civil Service is committed to attract, retain and invest in talent wherever it is found. To learn more please see the <a href="https://www.gov.uk/government/publications/civil-service-people-plan-2024-2027/civil-service-people-plan-2024-2027-html" target="_blank">Civil Service People Plan (opens in a new window)</a> and the <a href="https://www.gov.uk/government/publications/civil-service-diversity-and-inclusion-strategy-2022-to-2025" target="_blank">Civil Service Diversity and Inclusion Strategy (opens in a new window)</a>. As part of the application process, we monitor statistics on D&I.

You can see how we process this data here: <a href="https://www.gov.uk/government/publications/government-digital-service-recruitment-privacy-notice/recruitment-privacy-notice" target="_blank">Recruitment privacy notice - GOV.UK</a>.</div> </div></div>Find more English Speaking Jobs in United Kingdom on Arbeitnow

Community Q&A

Anyone worked here? Ask before you apply.

No threads yet for this job or company.

Ask the community Work discussions

Report ad

About this role

About the Role

What We're Looking For

Community Q&A

Create a free account