Security analysis is finally reaching software's long tail - Lalit Maganti
This post was formerly titled “17 bugs in 10 weeks from AI security scanning”.<br>It was changed upon feedback from trusted readers who suggested, while accurate,<br>it did not give a good sense of what I’m actually arguing in the post.<br>Over the last several weeks, I’ve been receiving more security bug reports for<br>Perfetto’s trace processor than I ever have before, all of them found by AI. And<br>I’m very happy about it! These are bugs that would almost certainly not have<br>been found a year ago and it feels good to close these loopholes even though<br>trace processor is by no means security critical.<br>For years, security researchers concentrated their time on the highest-stakes<br>targets: kernels, cryptography libraries, password managers. But there’s a lot<br>of code out there which is security-relevant but not truly security-critical. In<br>my experience, these sorts of projects didn’t draw much attention. Now systems<br>in the long tail can get that attention which they wouldn’t have before.<br>Why is this happening<br>Trace processor is a project which sits squarely in that long tail. It’s a C++<br>library (yes, Rust would be the obvious choice today but it’s not practical to<br>rewrite, see footnote 1) for processing recorded traces of various<br>formats. These are typically traces you collected yourself or in your test infra<br>and process offline so “untrusted input” isn’t much of a concern.<br>However, some people do process traces they didn’t collect themselves (e.g.<br>user bug reports, automated collection from dogfood users). For those cases<br>we’ve strongly recommended sandboxing trace processor (e.g.<br>gvisor,<br>sandbox2, or<br>minijail) or, for even more sensitive use<br>cases, a VM.<br>Beyond sandboxing, for catching issues proactively, we mainly relied on fuzzing<br>running internally in Google. These fuzzers occasionally surfaced real,<br>actionable bugs: we set them up to pass in arbitrary trace bytes (as this is the<br>main “attack surface”) but over time these became quite rare as they discovered<br>much of the low hanging fruit, which we quickly fixed. The bugs that remained<br>tend to live deep in the internals, reachable only with a very precisely crafted<br>sequence of bytes that a fuzzer is unlikely to hit by mutation alone.<br>Apart from this, there has rarely been any bandwidth or resources for a human,<br>either a security expert or someone from my team, to spend lots of time finding<br>security issues in trace processor. There were always other parts of Perfetto<br>more worth spending security time on (e.g. the tracing service, on-device<br>profilers) as they’re actively running in production systems.<br>All of this changed as of a couple of months ago. We started receiving bug<br>reports filed by some central team which appears to be running AI-based security<br>scanning against various projects throughout Google. Unfortunately, I have to be<br>hand wavy about what exactly they’re doing as their work doesn’t appear to be<br>public.<br>Starting in early April, we had a slow drip of 1 bug a week, but since the end<br>of April this increased to a rate of several a week, with some days having 3 or<br>4 being opened in quick succession. This lasted until mid-May, at which point it<br>started tapering back to 1-2 a week with some weeks having none.<br>I also want to say that the quality of the bugs is high. They’re well-described,<br>often with the relevant attacker model already worked out and even minimal fixes<br>proposed: basically everything I could ask for from a bug report. This matches<br>what both curl and<br>Linux kernel<br>maintainers have noted about security bugs they’ve received, especially how<br>sharply quality has improved in the last few months.<br>As I can only see the bugs that get filed against me, not the raw output of the<br>AI scanner, I don’t know exactly how much triage happens upstream. My guess is<br>there’s a human doing a light pass to drop obvious noise before reports reach<br>client teams, but judging from the rate at which bugs are opened and the way<br>they’re filed, I doubt anyone is deeply triaging each one.<br>In total, we’ve received 21 bugs (17 real issues and 4 not actionable),<br>which can be broken down into the following categories:<br>10 bounds checking : arbitrary trace data flowing into fixed-size buffers<br>or unchecked array indices, leading to out-of-bounds reads or writes.<br>5 use-after-free : back-pointers, pointer snapshots, or hashmap keys<br>outliving the object they refer to.<br>1 stack overflow : unbounded recursion when input is deeply nested.<br>1 access control : not enforcing allowlists on some rare codepaths.<br>4 closed as not actionable : either where the chance of exploit was purely<br>hypothetical or where fixing would have required fundamental design changes<br>which were not worth the tiny security risk.<br>All 17 real issues have been fixed, almost all shipping in Perfetto...