I fact-checked 15 stats from a 1.2M-subscriber creator. Six failed.
The Second Brain Dispatch
SubscribeSign in
I fact-checked 15 stats from a 1.2M-subscriber creator. Six failed.<br>How true statistics get bent: a primary-source audit of a creator I trust, the STRIP framework, and the autopsy of "85% of jobs are filled through networking."
Hemant Patil<br>Jul 09, 2026
Share
Issue #3 of The Second Brain Dispatch — real builds and systems for learning with AI, written by someone who actually ships them.<br>GPT-5 “exceeded expert-level performance in 50% of cases across 40+ occupations.” That claim aired ten months ago, from a creator I trust. It went into my notes, and from my notes into how I thought about my own career exposure. Last week I traced it to its source.<br>Thanks for reading The Second Brain Dispatch! Subscribe for free to receive new posts and support my work.
Subscribe
There is no source. The claim aired in August 2025 — a month before any benchmark measuring that even existed. When OpenAI’s GDPval landed that September, it scored GPT-5 at 38.8% “as good as or better than” industry experts. The number closest to 50%? It belonged to a competitor’s model, and it counted ties.<br>I didn’t hear this claim from a grifter. I heard it from an ex-CEO with 1.2M subscribers who cites more named research than almost anyone in the genre. He’s also the creator whose 58 videos became my second brain in issue #1 — meaning my notes, and the decisions downstream of them, run on his digits. So this week’s build was an audit: 15 headline statistics from his catalog, each chased back to its primary source.<br>The build: an audit with teeth
The method matters, because “I fact-checked it” usually means “I googled it and found a blog agreeing.”<br>Sample honestly. I took headline stats across his whole range — learning science, career advice, AI economics — not just the ones that smelled wrong. Cherry-picking your targets is its own distortion. (Issue #1’s spot-check had already caught three errors; those aren’t recounted here — these are fifteen fresh claims.)<br>Primary sources only. A creator citing a study is not a source. A journalist citing the study is not a source. The rule from issue #2 applies to the auditor too: chase the digits to the paper, the pricing page, the transcript.<br>Four verdicts, no vibes. TRUE (matches the source within rounding), DISTORTED (real study, materially bent number or framing), UNVERIFIABLE (no primary source findable), FALSE (the source contradicts it). Three research agents ran in parallel — the fan-out from issue #2, pointed at its own supply chain.<br>Check yourself before you check him. My notes are AI distillations, and issue #2 established that unsupervised summarizers hallucinate. Before pinning the worst verdict on Swadia, I queried the raw transcript. His exact words: “GPT-5 exceeded expert-level performance in 50% of the cases in 40-plus occupations... it’s happening already.” No attribution, no hedge. The error was his. But I had to prove it wasn’t mine first.<br>The scoreboard
9 TRUE. 5 DISTORTED. 1 FALSE. A 40% failure rate — from one of the best-sourced creators on the platform.<br>The clean ones were genuinely clean. “Tested learners retained 80% vs. 34% for re-readers” — that’s Karpicke & Roediger, Science, 2008, and the real numbers (81% vs. 33–36%) are almost exactly as quoted. “Story-wrapped words recalled at 93% vs. 13%” — Bower & Clark, 1969, and yes, actually Stanford. Federer really did win only 54% of his points.<br>The failures had a fingerprint. Look at what actually broke:<br>“$75 to $1.25 per million words, a 98% drop in a year.” Real prices — but they’re per million tokens, the gap was 5.5 months, and the $75 baseline was GPT-4.5, an admitted research-preview outlier. Compare flagship to flagship and it’s ~96% over 2.4 years. Still a collapse; no longer his sentence.
“Top performers deliver 800% more impact (McKinsey).” McKinsey’s figure is 400% — the 800% applies only to high-complexity roles. He quoted the ceiling as the average.
“AI hit 85% accuracy on the toughest cases vs. 20% for doctors.” Both numbers exact (Microsoft’s MAI-DxO, 304 NEJM cases). Omitted: the physicians were barred from search engines, textbooks, colleagues, and AI — conditions no working doctor practices under.
“Only 4% of students can monetize their passion.” The real study (Vallerand, 2003 — genuinely Canadian) found related to work or education. It never tested whether anyone could monetize anything.
“85% of jobs are filled through networking.” A 2016 non-scientific LinkedIn poll, whose own author got to 85% by adding two survey categories together. The only rigorous study anyone can point to — Granovetter, 1974 — says ~56%.
The framework: STRIP
Here’s the finding that reorganized how I read everything now: only one claim out of fifteen invented a number. The other five failures took a true number and bent it. Fabrication is rare. Degradation is the industry standard. And it degrades in exactly...