Musings on Information Security and Data Privacy: Dunning-Kruger After AI: the Gap That No Longer Closes

skip to main |<br>skip to sidebar

Where facts are few, experts are many.

Home

Menu

About

My Publications

Academic Citations

Vulnerability Disclosure Policy

Old archived Blog

Get in touch

Contact Me

Subscribe to Feed

Dunning-Kruger After AI: the Gap That No Longer Closes

comments

or Dunning-Kruger doesn't self-correct anymore.

> TL;DR. The Dunning-Kruger effect, that is, the difference between what people think they can do<br>and what they can actually do, used to close and self corrects with experience. My hypothesis that I introduce in this post is that AI keeps it open: it increases confidence and splits real capability into "with the tool" and "without the tool." For boards and practitioners, that turns intrinsic capability from a productivity question into a governance one, and it is the capability that quietly erodes.

The ending that used to be guaranteed (more or less)

About the Dunning Kruger curve

What AI changes

The Gap that no longer closes

Does it really matter?

What it means for companies

1. The ending that used to be guaranteed

Everyone knowns Mount Stupid . The colleague who read one thread and now wants to redo the whole plan. The new hire who watched a video and is sure everyone before them was doing it wrong. Believe me, I have been there.

The nice part of the Dunning-Kruger story is how it ends. Experience wins over the overconfident, and the gap between what you think you can do and what you actually can do closes on its own. The thing doing the work is boring but reliable: you try, you fail in front of people, you learn. Reality keeps sending the bill until what you think you can do matches what you can.

Figure 1. The classic picture. Perceived capability runs ahead, crashes, then converges on actual capability. The gap closes.[Thierry ZOLLER]

2. About the Dunning-Kruger curve

One warning first : the chart everyone has in mind stands is an . The famous Dunning-Kruger curve, with its confidence peak and valley of despair, did not come from Dunning and Kruger. It is not in their 1999 paper or in any of Dunning's later work. It spread through management training and the internet in the mid-2000s, the real study compared what people thought of their ability with how they actually scored, in four bands, and the real line climbs steadily rather than peaking [1].

The effect itself has no real concensus, several papers say the pattern is mostly a statistical mirage, a mix of regression to the mean and the plain fact that most people rate themselves above average [2, 3]. Others point out that the basic pattern still turns up even when critics rerun the study, and that the numbers in the original look too big to be a trick alone [4]. The fair summary: the pattern in the data is real (and that it what matters most in my hypothesis), what it means is still disputed.

I use the popular chart here on purpose, because you know it, not because I am claiming it is the 1999 data. The argument does not need the curve to be literally true. It needs only the one thing nobody argues about: people are bad at judging their own ability, and the gap between what they can do and what they think they can do now runs through a tool that makes it wider.

My hypothesis only needs the one thing nobody argues about: people are bad at judging their own ability, and the gap between what they can do and what they think they can do now runs through a tool that makes it wider.

3. What AI changes

Two things change according to me. Let's go through them one at a time.

1. First, confidence goes up. A beginner with a good assistant turns out work that looks expert.

The work is the proof, and the proof says "good," so the early peak of overconfidence climbs higher than it ever did on its own. The valley isn't as deep either. The moment of getting caught comes later and lands softer, because the tool papers over the gaps that used to show you up. And the line never really drops back. There is no longer a reliable point where harsh reality (failure, mistakes etc) forces what you think to meet what is true, because the output keeps looking fine. Figure 2a shows the one line changing.

Figure 2a. Same chart, one line changed. Perceived capability peaks higher, dips less, and no longer comes back to meet actual. The gap that used to close stays open. [Thierry ZOLLER]

2. Second, "what you can really do" stops being one thing. Before AI, your ability was a single number. Now it splits in two (Figure 2b).

Let me explain, there is what you can produce with the tool in hand, which is high and comes fast. And there is what you can do if the tool is taken away (I call it "Intrinsic"), which is lower, and which only grows through the practice the tool now does for you.

Figure 2b. The single "actual" line splits. Assisted capability rises with the tool. Intrinsic capability, the part left when the tool is...