Contra "AI 2040 and the Cult of Intelligence"

Matthew Tromp

SubscribeSign in

Contra George Hotz on "AI 2040 and the Cult of Intelligence"

Matthew Tromp<br>Jul 20, 2026

Share

George Hotz is an American security hacker, entrepreneur, and software engineer (thanks Wikipedia). He also has a blog. That blog has a post about AI 2040. I don’t think the post is intended to be an especially thoughtful critique, and I apologize to Hotz for subjecting his off the cuff writing to the level of autism it is about to endure. But I could use the practice.<br>First off, -1 point for being mean for no reason:<br>I would like to see the authors of this document try to change a bike tire. Even with a superintelligent ChatGPT, I suspect they would struggle.

I think the world would be better if people were less mean, but I know I’m just yelling into the void on this.<br>Takeoff

Hotz is very skeptical of hard takeoff.<br>In The Metamorphosis of Prime Intellect, the hard takeoff works because AI discovers the correlation effect, some quantum trick to manipulate matter. In reality, there is no correlation effect.

(The Metamorphosis of Prime Intellect is great by the way, and you should totally read it.)<br>Confronting why these people are wrong requires confronting deep beliefs I hold about myself. Intelligence is not the end all be all, it’s just the current bottleneck for a few things. You cannot take over the world with tokens. […] [T]here’s no magic tricks machines can do. They are subject to the same laws of the universe and ecology. And there’s still no hard takeoff.

You know who else is skeptical of hard takeoff? The authors of AI 2040. None of the scenarios in AI 2040 (or AI 2027) involves what I would think of as a hard takeoff. They expect the transition from a fully automated coder to something much smarter than the smartest humans to take around a year, which is certainly fast (much faster than I’m comfortable with), but a far cry from what we traditionally think of as hard takeoff (hours to weeks), and doesn’t require Metamorphosis of Prime Intellect style magical reality manipulation powers.<br>After that year of self-improvement, the ASI takes over the world pretty easily. If you want to know how a superintelligence would take over the world with tokens, AI 2027 is a great description of how that could happen! If there is some part of AI 2027 that strikes anyone as unrealistic I’d like to hear it, but it seems very reasonable to me. There are no magical physics glitches. There are no nanobot swarms. They even assume the AI wouldn’t be capable of extremely effective persuasion. All it has to do is pretend to be a good, well behaved, aligned superintelligence, until the humans have it build them a fully automated supply chain, because of course once we have powerful AI and decent robotics that’s what we’d want to do with it, especially with our super nice super aligned machine god telling us what a good idea it would be.

AI 2040 includes this picture of a datacenter in the ocean. Just like vaporware, you can generate a picture easily. But in reality, you have to deal with supply chains. You have to deal with them shipping you the wrong part, the thing not meeting the spec, it randomly failing after 20 minutes, the chip warping in the reflow oven. Did you consider the barnacles?<br>All these things are managable, but it’s generally not the speed of humans that limits them. Are you paying for air shipping from China? Or cheaping out for the 3 week boat (Claude chanting by the engine won’t make the boat move faster). Or take a chip fab. It takes 3 months to make a chip, and humans are barely in the loop. It just takes 3 months.

I genuinely don’t understand the point Hotz is making with this. The context for this image in AI 2040 is that it’s something they propose happening in 2034, under the Plan A slowdown. I don’t think floating datacenters are really all that crazy of an engineering project. People have already built them. What’s shown in this image strikes me as something we could totally build 8 years from now, even without any AI assistance. We already build oil rigs in the ocean. This just doesn’t seem that hard? What point is he making here?

Subscribe

Plan A, for Actually not a whole lot of other options

Their crap Consortium is just world government with sci-fi characteristics. You aren’t gonna get the million dollars, you aren’t gonna get the datacenters in the ocean, but you are going to get a massively expanded nanny state that steals your GPUs like how FDR stole the gold. No hoarding!

I would like to thank George Hotz for introducing me to Executive Order 6102, because wow what a truly rancid policy.<br>Personally, I am not a huge fan of a lot of the regulations we have today. I can understand being resistant to regulation. But the authors of Plan A didn’t set out with some plan to make onerous regulations that restrict everyone’s freedom. They just sat down, thought about the problem really hard, and came up with...