Welcome to the strip mining era of open source security

All posts

May 14, 2026 in Engineering

8 min read

Welcome to the strip mining era of open source security

Sameer Al-Sakran

‧ May 14, 2026 in Engineering

‧ 8 min read

Share this article

Copied to clipboard

Open source software is in for a rough 2026 summer. If you’re an Open Source maintainer, there’s something afoot you should already know about. If you’re an OSS user, you should be aware of it as it’ll explain some behavior around you that might otherwise seem odd.

TL;DR: High volume, LLM-powered scanning for security vulnerabilities is going to uncover lots of security issues in anything with public source code.

This all started a few months ago

Historically, Metabase averaged 10 submissions per month to our security@metabase.com, most of which were trivial or not actually vulnerabilities. Many were false positives from scanning tools, and we spent most of our time explaining to the reporter that what they found wasn’t actually a problem.

At the turn of the year, things changed. Starting in January, we’ve been averaging 10 submissions per week, and many of these are legit. Most are not serious, and we’ve quietly fixed them, thanked the researcher, and went our merry way. However, it was a step change in both volume and quality of reporting. These come from a wide variety of locations and people, and sometimes, but not always, are looking for bug bounties. More often than not, the reports are in markdown, and read like they’re LLM generated.<br>Others are seeing this as well.

It doesn’t take too insightful an eye to realize we’re seeing a remarkable improvement in automated code scanning. We’ve since tried out a few vendors in the space, and what do you know — more (thankfully minor) issues found. There’s no one vendor or model that’s the root cause. While we originally thought it could be Claude Security, that was only announced in February, after things had already picked up. And OpenAI is also getting into the game. Does this mean there’s another wave coming after everyone gets access to these? Likely. But regardless of specific foundational models, this is just a consequence of coding agents in general getting better at scanning codebases for flaws.

Historically, we tended to get two styles of vulnerability research:

Superficial scanners run in bulk : Running an OWASP scanner or other out-of-the-box vuln scanner. These tended to be mostly false positives.

Motivated deep digging : This was typically a serious user paying researchers, often with a knowledge of the application area or framework deeply and knowing where to probe. These tended to find a cluster of similar issues, often related to the style or speciality of the researcher.

Vulnerabilities are now being strip mined

If your code is available, and someone is willing to spend tokens, they can scan the code in bulk. As coding agents get better at understanding code bases (and as a sane person — don’t take the other side of that bet), we’ll be getting layer after layer of progressively deeper vulnerabilities uncovered.

The Bright side with some dark implications

Now, if there’s any bright side to all of this, it’s that so far, there seems to be an alignment of incentives between ethical research and open core companies. If you’re a budding security researcher, the play is:

1) Wrap up claude/codex/etc in a bunch of skills and turn that into a SaaS offering.<br>2) Bulk scan commercial open source repositories<br>3) Send out every finding with a footer advertising your automated scanning SaaS service to the Commercial OSS company and any big users you can scrape from their website.

Needless to say, there are now approximately 1000 of these SaaS offerings, so it’s a competitive game, but thankfully, there is a pro-social path that pays. With non-commercial open source, it’s a bit less lucrative - you’re stuck mining for bounties that users of the OSS project offer.<br>I’m not sure I want to know what’s happening in the unethical side of security research these days.

For our broader security posture:<br>https://www.metabase.com/security

What this means for OSS maintainers

Now, in the Long Run, this is great. We have a lot of third parties burning tokens to help you find any possible exploitable flaws in your software. And once they’re fixed, any software you’re running is more secure and less likely to have issues. These scanners will proliferate and eventually probably make their way into your CI workflows.

In the short term, it’s gonna be rough.

To start with, you should assume that any vulnerability that was disclosed to you is trivially discoverable now, regardless of how buried it was in your code. While some researchers are doing novel work, the majority of this will be low-ish effort bulk code scans. If one person running Claude Code on your codebase found it, you can expect someone else using e.g., Codex to find it soon enough.

What this means is that even if you have an...