When AI Leaves the Lab: Testing Frontier Models in Government Cyber Defence - Case study - GOV.UK

">

Cookies on GOV.UK

We use some essential cookies to make this website work.<br>We’d like to set additional cookies to understand how you use GOV.UK, remember your settings and improve government services.<br>We also use cookies set by other sites to help us deliver content from their services.

You have accepted additional cookies. You can change your cookie settings at any time.

You have rejected additional cookies. You can change your cookie settings at any time.

Accept additional cookies

Reject additional cookies

View cookies

Hide cookie message

Skip to main content

Case study

When AI Leaves the Lab: Testing Frontier Models in Government Cyber Defence

The Government Cyber Action Plan aims to boost cyber resilience across the UK public sector by using emerging technologies to manage risk. The Government Cyber Coordination Centre (GC3) - a partnership between the NCSC and the Department for Science, Innovation and Technology - is leading this work, exploring how frontier AI can be applied safely to cyber defence across government.

From:

Department for Science, Innovation and Technology and National Cyber Security Centre

Published<br>12 June 2026

From frontier models to front-line impact

We know AI is disrupting the cyber threat landscape. Recently released frontier AI systems such as Claude Mythos and GPT-5.5 brought a step-change in cyber capabilities, and the UK AI Security Institute (AISI)’s evaluations show these models getting better at cyber tasks very quickly.

However, evaluation in synthetic environments gives a limited understanding of real-world use. A high score on a benchmark does not necessarily translate into finding and fixing real vulnerabilities.

What we did

The Government Cyber Coordination Centre led a weekly, in-person series of hackathons which used frontier AI to scan public code repositories across government. Working closely with specialists from the AISI and NCSC, our goal was to find and mitigate previously unidentified vulnerabilities before they could be exploited. Rather than mandate a single approach, we gave teams model access and let them build their own tooling, noticing what worked each week and building on the best approaches.

The UK Government encourages new source code to be open by default, with specific and justified exceptions. In practice, that creates a degree of shared visibility that attackers can also exploit. However, this openness also limits duplication and leads to cleaner, more easily maintained code.

Code published in the open has also already passed extensive prepublication scrutiny, meaning it can be shared with frontier model providers with minimal additional review. This means that government departments can deploy new capabilities quickly and with confidence.

An adversarial chain that challenges itself.  One team ran each public repo through a six-stage AI agent pipeline: triage, validator, auditor, tracer, judge, summary. Each stage reads and challenges the last. In one case, the agent downgraded a finding once it established that a backup mechanism was in place. The pipeline was agentic, but the escalation was manual. This means a member of the team checked every line, re-verified exposure, and handled false positives.

Deterministic scanners feeding a model.  Another team ran traditional scanning tools first (including Gitleaks, Trivy, Semgrep and Hadolint) to generate a ranked findings document. Three model stages were then layered on top: a discovery stage that treated the scanner output as leads and read the source against OWASP and CWE frameworks, a chain-investigation stage that composed individual findings into attack paths via per-chain sub-agents, and a triage stage that confirmed the finding viability.

Codifying a multi-service audit into reusable skills.  Another department developed five domain-specific Claude Skills. The Skills distil an organisation wide audit across hundreds of services into something repeatable. Skills enabled a reusable, scoped, and consistent approach across every repository and operator.

What we found

Participants identified 407 findings in total, including critical weaknesses exposing services to authentication bypass, data exposure and remote code execution. Some were already understood and mitigated by compensating controls while others were previously unknown. All critical weaknesses have been remediated, and no evidence of exploitation was identified for any finding.

AI models traced vulnerabilities across service boundaries, which traditional scanners can’t do, and linked business logic with technical detail. Departments prioritised validation and remediation through existing frameworks, patching critical and high-risk issues assessed as exploitable.

It cost us £13,000 in tokens to find these weaknesses, working across nine government organisations for the...