Never Trust A Number - by Steve Newman - Second Thoughts

Second Thoughts

SubscribeSign in

Never Trust A Number<br>They're shortcuts to understanding, and there are no shortcuts

Steve Newman<br>Jun 30, 2026

Share

This is a repost of one of my favorite essays, which I originally published back in 2022. It’s as relevant to AI as it was to the topics I was focusing on back then, as I explain in a new section at the end.)<br>My topic for today: All numbers are wrong .<br>Like, seriously. Whenever you see a number – in a tweet, newspaper headline, office email, technical report, textbook, anywhere – assume it is wrong. Treat it as enemy misinformation, deliberate sabotage of your understanding of the world, and disregard it.<br>You’re thinking, ha ha, I’m exaggerating for effect. I’m not. Seriously I am not. I mean, of course not all numbers are literally incorrect; but it happens so very, very much more often than your intuition, that I do literally mean it is a good practice to treat all numbers as incorrect by default.<br>I’ve come to this position slowly, over the years, one screwup at a time. But now that I’ve really started paying attention, the mistakes are everywhere.<br>It’s so easy to find examples

The day I sat down to write this post, I skimmed the New York Times headlines for examples. I didn’t have to look far:

“Half a million known virus cases”, huh? During the Omicron peak, the US alone exceeded that many cases every day. Clearly they meant half a billion.<br>Think about this. The New York Times, in a prominent location – a headline! – used a number that was off by a factor of one thousand. One thousand. Utterly, colossally, absurdly incorrect.<br>How could that happen? Is their research department using a flawed methodology? Did they fall victim to a misinformation campaign? Did someone accidentally post a headline from way back in early 2020? Were they hacked?<br>Of course not. Obviously, it’s just a typo. No big deal. But the fact that a factor-of-1000 error is no big deal, is a big deal. It undermines the very idea that numbers mean anything. It rubs our nose in the fact that an incorrect number looks exactly like a correct number . In this case, the number is so wildly incorrect as to reveal itself on casual inspection. But most mistakes aren’t so obvious; and even this “obvious” mistake made it to the front page.<br>Imagine you’re in line at a restaurant buffet, and you see an employee drop a piece of meat. They pick it up from the floor, brush off the dirt, and plop it into the serving platter. Obviously you’re not going to eat it. Are you going to carefully choose a different piece of meat? Or, having seen their standard of hygiene, are you going to leave that restaurant and never come back? You wouldn’t put that food into your mouth; and you just saw a mainstream news source do the metaphorical equivalent of serving meat that had been dropped on the floor. Don’t put these facts into your mind.

Subscribe

Even when a number is right, it’s wrong

Seven-layer cake? I thought you said seven layer cakes!<br>Mixing up “million” with “billion” isn’t even the worst part about that Covid statistic. It’s a big error, but it’s a one-off. The deeper problem is the phrase “most likely an undercount”. It is not likely an undercount, it is certainly a small fraction of the true value. The statistics the Times is citing – the statistics everyone always cites – are based on officially reported cases. If someone takes a home test instead of a lab PCR test, or if they never bother to get tested at all, or if they live in a part of the world where lab tests are unavailable, or if they have a mild case and don’t even know they’re sick – none of those infections are counted.<br>I couldn’t easily find an up-to-date estimate of total worldwide infections, but this article from way back in October 2020 cites a World Health Organization estimate of around 760 million. At the time, the number of reported cases was only 35 million. Or the CDC estimates that through September 2021 – i.e. before Omicron – 146 million Americans were infected. By now, well over half of the country must have had the virus. I imagine the same applies in most of the world, China (though this may be about to change) being the one large exception. “Half a billion” is completely disconnected from the actual infection count.<br>At least the Times specified “known” cases. Often, that distinction isn’t even mentioned. But anyone reading the headline is going to gloss over the word “known”. They’re liable to mentally compare half a billion cases with the world population of around 8 billion, and conclude that roughly one person in 16 has had Covid – when the truth is more like in two.<br>So saying that 500,000,000 worldwide Covid cases is most likely an undercount, is like saying “most human beings live at least seven years, possibly longer”. Technically, it’s a true statement; in practice, it anchors your mind on a number that means something so different than what you think as...