Vulnerabilities and Best Practices in an LLM World - Internet Security Research Group

Blog

Vulnerabilities and Best Practices in an LLM World

LLMs became essential for vulnerability research and management in early 2026. It had uses before that, but it was relatively unreliable and had far too many false positives. If an open source project got LLM vulnerability reports in 2025, there was a good chance most of it was slop. If they get LLM vulnerability reports in 2026, the odds are good that they're relatively high quality. Not perfect by any means, it's still common for them to get severity or exploitability wrong for example, but there's probably a real bug there worth looking at. Overall, it's doubtful that human security researchers would be more effective at reducing vulnerabilities in a code base.

When it comes to vulnerabilities, almost everything is different now and nobody knows how it'll all play out. Are we going to be able to keep software sufficiently secure going forward? Will the overall advantage from LLMs go to the attackers or the defenders? What will best practices for keeping software secure look like five years from now? What architectural changes will come out of all this?

I don't have the answers, but those of us responsible for securing software can't just stand around shrugging our shoulders for the next couple of years. We have to act with some outlook in mind. Here's my best guess as to how things play out over the next five or so years.

Projects with the means and motivation will be able to use LLMs to drastically reduce the number of existing vulnerabilities by the end of 2026. Maybe by as much as 95%?

We don't know how many total vulnerabilities exist in a given piece of software, but for all practical purposes it's some finite number. There are X vulnerabilities, and if you fix one the count becomes X-1. Theoretically you could get it to zero, though you probably couldn't know if you got there. It would also be hard to keep it at zero since any future changes could introduce vulnerabilities, including vulnerability fixes themselves!

LLMs are going to help a lot of software projects drastically reduce the number of existing vulnerabilities in their code. Firefox is a great example - they are cleaning house. We don't know how many existing security bugs are lurking in Firefox, but there will be drastically fewer at the end of 2026 than there were at the start. This will be true for a lot of the most important software we use every day. It will be harder, though not impossible, for attackers to find vulnerabilities in thoroughly LLM-audited software.

This won't happen to all software, though.

The vast majority of projects without the means and motivation to use LLMs to extensively audit existing code for vulnerabilities will be extremely vulnerable.

Some projects don't have the means. LLM resources can be expensive, and some projects lack sufficient developer time and energy. Some projects, even if they can perform LLM audits and fix vulnerabilities, lack mechanisms to update their users in a reasonable amount of time. Some projects may have the resources to audit their own code, but not their dependencies.

Some don't have the motivation - maybe security is not the project's highest priority, or perhaps the maintainers are sceptical or opposed to using LLMs for this purpose.

This software will be a big problem because attacking it will become trivial. It used to take a long time to figure out what software wasn't keeping up in terms of security, but that's about to change. Software that isn't keeping up with best practices will get identified and shunned faster than has happened previously because it will get exploited much faster and more frequently.

The best thing we can do to help here is bring down the cost of LLM audits (e.g. tokens, but also the necessary tools to do the job) so that the vast majority of developers have the means to thoroughly audit their software.

I say that this applies to the vast majority of projects, not all, because there are some projects out there that have done such a good job with technology choices, testing, verification, and scoping, that LLMs don't find much. These projects are rare, but it's worth noting that they do exist.

LLMs will help to level the playing field when it comes to vulnerability management.

As mentioned previously, some software is going to really benefit from LLM security audits and some will get left behind. In the average case though, LLMs will level the playing field amongst defenders.

Some projects have always had more resources for vulnerability research and prevention than others. LLMs will bring vulnerability counts down further for even the best resourced defenders, but other projects will be able to come a lot closer to what the well-resourced are doing.

Firefox is a good example again. Firefox and Chrome probably introduced vulnerabilities at roughly the same rate, but Chrome has had...