The Trouble With AI - Jakob Schwichtenberg’s Newsletter

Jakob Schwichtenberg’s Newsletter

SubscribeSign in

The Trouble With AI<br>Neither energy consumption, nor human unemployment or paper clip maximization

Jakob Schwichtenberg<br>Jun 01, 2026

Share

Imagine an alternative reality where everyone was happily living in a pre-ChatGPT world except for one guy. This guy secretly had access to a powerful state-of-the-art LLM. No one else would even suspect that such technology exists.<br>Would people like reading his essays and books?<br>They probably would. He would most likely win literary prizes and become a bestselling author given his sheer output would make James Patterson and Danielle Steel look lazy.<br>But why then, is everyone increasingly allergic to anything that even remotely smells like it was created using AI?<br>As the little thought experiment above illustrates, the issue certainly isn’t that AI is producing bad writing per se.<br>I still vividly remember when I first prompted a fourth-generation model to rewrite something I had written and thought “This is really good. This is much better than anything I ever could have written.”<br>I felt excitement and a tiny bit of existential angst.<br>If you never encountered something written by AI before it’s pure magic. It doesn’t seem like AI-slop.<br>But now when I look at the same AI output I can only think “Ugh”. The flaws and patterns are so, so obvious now. It doesn’t matter what model I use or what prompt I try.<br>Everyone loves creating with AI. No one loves consuming what AI creates.<br>But why? Why don’t we get excited when we read something written by this superhuman alien intelligence? Why do we feel disgust?<br>AI is full of shit

If you use AI tools regularly, you quickly figure out that they have zero shame telling you bullshit.<br>Hallucinations always creep in. You have to keep your guard up.<br>Most frustrating, there is no sign this is getting any better. Also getting angry at an LLM or explaining that this is bad behavior has zero effect. All you ever get is a cheery “You’re absolutely right!”.1<br>AI hallucinations are a huge issue because, famously, the “amount of energy needed to refute bullshit is an order of magnitude bigger than that needed to produce”.<br>This was an estimate from the pre-LLM era. Now with AI the cost of producing bullshit has gone to virtually zero while the effort required to refute it has remained mostly constant. So the ratio is more like 100x or 1000x now.<br>AI is full of fluff

Besides straight up lies, LLMs can’t stop producing vacuous phrases like “It’s not gradient, it’s texture”<br>The human mind has a strong tendency to maintain a sense of coherence.<br>When we look at an impossible object, we do not realize immediately that they don’t actually make sense. It takes effort to spot the impossibility of these objects.

source<br>Analogously, when you’re reading AI-slop full of vacuous phrases you do not immediately notice this. They do read plausible-enough for your mind to not stumble upon them on a first reading. But that doesn’t change the fact that they do not make any sense.<br>The situation is effectively like the one Isaac Asimov describes in Foundation when Mayor Hardin reveals that he secretly recorded the imperial envoy Lord Dorwin and ran his days of reassurances through logical analysis:<br>“That,” replied Hardin, “is the interesting thing. The analysis was the most difficult of the three by all odds. When Holk, after two days of steady work, succeeded in eliminating meaningless statements, vague gibberish, useless qualifications—in short all the goo and dribble—he found he had nothing left. Everything canceled out. Lord Dorwin, gentlemen, in five days of discussion didn’t say one damned thing, and said it so that you never noticed.

Just like with other types of bullshit doing the hard work of staring at the phrases long enough for them to evaporate into thin air is a ton of effort. So a much saner approach is to simply stop consuming anything that contains them.<br>AI has no grip on reality

Reading is rarely just about information transfer. Everyone knows that books don’t work. And yet, we’re clearly getting something out of reading them.<br>Who wrote something and how they wrote it is just as important as the set of facts they are writing about.<br>Relevance realization is the dynamically adaptive process of making and breaking frames to find an optimal grip. But finding that optimal grip typically requires more than a mere statement of the facts.<br>Knowledge is four-dimensional. Besides propositional knowledge, we need perspectival, procedural, and participatory knowledge to fully grasp something.<br>We need stories and analogies that provide alternative frames until it all “clicks” and starts to make sense to us.<br>The trouble with current AI models is that they have no grip on the world. They have no experiences of their own. It’s all Fugazi.<br>State-of-the-art LLMs might do a reasonably good job at parroting propositional knowledge. But whenever they are...