Finding Security Bugs with LLMs for 1% of the Cost | Etive Mòr

Skip to main content

Nicholas Carlini did a talk a few weeks ago at [un]prompted 2026. In it, he describes how Anthropic have been using their Claude 4.5 & 4.6 series models to automate searching for security issues in repositories. He’s had some success, and managed to get a few new CVEs to his name with the technique. While novel, Carlini’s approach, as described, is an expensive way to search for bugs. I think that using a heuristics based approach to pre-search the codebase for interesting files before hunting, we can reduce the cost of his technique by 99% or more, while retaining most of the benefits. A link to the prompts used in this article are organised in the companion repository at https://github.com/Etive-Mor/language-model-look-kit.

Note: This article isn’t about Anthropic’s Mythos series.

In his talk, Carlini describes writing a bash script which loops through every ${file} in a repository. The script launches an instance of Claude Code, tells it that it’s playing a Capture The Flag (CTF) discovery game, asking it to search for potential vulnerabilities in the codebase, focussing on ${file} as a starting point. For each file, he invokes the process five times. He found a few security vulnerabilities and lots of regular bugs too, including a long-standing heap buffer overflow in the Linux Kernel (that’s a super rare find).

That’s a lot of tokens

Carlini’s approach is fairly “brute force”, and definitely super expensive. The Linux Kernel has 80k+ files containing tens of millions of lines of code. If we set a conservative average Claude Code invocation cost of $0.10 per file, and five invocations per file, this process would cost $40k across the Linux Kernel. The Anthropic Fellows program offers 4 months of $15k/mo in compute funding, so this mid-five-figure price is within the expected range for an internal research experiment.

Carlini is embedded in Anthropic, so had access to basically unlimited compute for the project. So, for him, looping through every file in Linux and spawning a Claude Code was a viable plan of action. However, even for very well funded open source projects, this would be financially ruinous. I wanted to see how much of the capabilities I could reproduce in a smaller open source repository, on a lower budget.

Heuristics-based vulnerability discovery

I think having a language model generate a heuristics-based list of interesting components first, and then scanning through those is a low-cost way to get a lot of the same benefits as Carlini’s technique. Maybe you don’t get 100% coverage, but hitting the most interesting 80% of files (especially those on the code hot-path) should reveal any critical bugs for less than $100.

I’ve worked with Umbraco CMS and .NET for more than a decade now, and I sit on the project’s Community Security & Privacy Team, so I thought that the best place to test out a cheaper variant of this approach was in that project’s main repository at Umbraco/Umbraco-CMS. The project has 450k lines of code across 8,500 files, so while it’s smaller than Linux, it’s still large enough to demonstrate the concept. Using the estimates above, this project would cost around $4k to run through Carlini’s technique.

With my approach, I was able to spend less than $20 in GitHub Copilot tokens, while discovering 20 potential vulnerabilities. After some human review, I whittled these down to four which were interesting enough to address. The process found no severe vulnerabilities worthy of a CVE, but did reveal a handful of (now patched) issues & bugs which required a change in the repository. This somewhat mirrors Carlini’s experience, that he found lots of issues, and then spent a lot of time doing manual review to demonstrate them before disclosing. The process, and a deep-dive on one of the bugs is outlined below.

Everything described in this article has been responsibly disclosed, patched, or mitigated appropriately. If you recreate any of the work below in any repository, make sure you stick to the principles of responsible disclosure. If you find any issues in any Umbraco application, please follow the “How to report a vulnerability in Umbraco” process, and report by emailing security@umbraco.com.

Building the tool

When selecting models, I wanted to take a pluralist approach, rather than limiting or coupling myself to Anthropic’s models. To achieve this, I decided to work with GitHub Copilot via VSCode’s chat features.<br>Umbraco-CMS uses a lot of Claude Code tooling, which means the repository has lots of CLAUDE.md files. While that documentation is useful if you’re running Claude Code, it’s distributed sporadically throughout the app. Models like Codex and Mistral only access those files intermittently, if at all. I want to consolidate this info into a single directory, and decouple it from Claude.

Model onboarding prompt

I’ve got a template prompt which will help a language model...