The Vulnerability Identity Crisis | Empirical Security
Open Menu<br>Close Menu
Open Menu<br>Close Menu
The Vulnerability Identity Crisis
Jun 29
Written By Jay Jacobs
Jay Jacobs and Art Manion presented "The Vulnerability Identity Crisis" at the 38th Annual FIRST Conference, Denver, June 14–19, 2026.<br>Art Manion and I have been spending roughly an hour a week for the past couple of years working through what sounds like a simple question: What are the minimum viable elements needed to define and identify a vulnerability? We had a few early attempts at frameworks, and talked through several fields and options, but we couldn’t find something that worked in every case. As we kept talking, we found that we had to be more precise, we had to stop and talk about concepts like “weakness” and we got wrapped up in questions like “what makes a vulnerability different from a regular bug?”. All of this leads us to a deceptively complex question: “What is a vulnerability?” But not in a “make a glossary” sense, we wanted to get at what a vulnerability is in the structural sense. We needed to understand the concept of a vulnerability. We needed to understand what a vulnerability is so we could separate vulnerabilities from bugs and define what is vulnerable, how it’s vulnerable and what happens when it’s exploited. After nearly two years of our weekly discussions we realized we need a working definition that captures the concept of vulnerability in order to know how to accurately describe and identify a vulnerability when we see it.<br>We've presented pieces of this at VulnCon (2025 and 2026). We've been forced to change our minds more than once (which we consider a really good sign). This talk at the FIRST 2026 Annual conference was our latest iteration, and we think we've arrived somewhere worth talking about.<br>The Problem We Keep Hitting<br>The problem we kept running into is that for every possible set of elements we would come up with (to describe and define a vulnerability in a record), we would find exceptions. Art has been studying vulnerabilities for years and can generally remember some vulnerability that didn’t fit into whatever mold we attempted to craft.<br>As an example of this in practice, go flip through the vulnerabilities with a “disputed" tag. Every disputed record is an example of where the system fails. Probably the best example of this is CVE-2020-19909: an integer overflow in cURL's --retry-delay argument parser. It prompted the cURL maintainer, Daniel Stenberg, to write a blog post where he said, “A bug, sure. Security problem? No.” The argument against it being a vulnerability is that in order to set the value of a command line argument, the actor must have access to run something at the command line. Exploiting this integer overflow would then allow them to… run something at the command line. In that scenario, nothing is gained by the attacker, or to rephrase that, there is no security property (or security policy/boundary) violated during exploitation.<br>The point here is that after reading a small handful of disputed CVEs (and then searching to figure out exactly why they are disputed), an obvious question is raised: if we cannot agree on what exactly a vulnerability is, how can we be sure we are collecting the right data? If two security professionals can look at an integer overflow and one says it is just a bug and the other says it is a vulnerability, clearly we’ve got a serious misalignment and any definition we may have for a vulnerability is insufficient. We need a working definition.<br>To move forward on this, we found 29 definitions of "vulnerability" across academic papers, frameworks, and standards bodies. They share many surface features: some notion of weakness, some notion of an affected system, some mention attackers and many included probabilistic language (“can”, “may”, “could”) around the outcome/impact or exploitation. But they are all generally superficial and do not cover the edge cases, the disputed cases. Another interesting discovery is that most definitions limit to “vulnerability”. In order for us to clearly communicate a vulnerability we actually needed to define six different terms and concepts. We found that the other terms around a vulnerability are equally vague and undefined.<br>The Dispositional Definition<br>A vulnerability is a disposition.<br>The word “disposition” is not exactly common, and that is intentional. We need a shock, we need a shift in our perspective and we need to be humble enough to accept it. Disposition has been heavily discussed and developed in philosophical circles.<br>A disposition, in the philosophical sense, is a property of a thing that exists whether or not it ever manifests, but when triggered, it produces characteristic outcomes. Glass has a disposition to shatter. It isn't shattered now but under the right conditions (the right force, the right location, the right angle), it will shatter. The disposition exists prior to any manifestation. A vulnerability is a...