AI Alignment as a Thought-Terminating Cliche

Many of the people building AI, and many of the people working on AI safety,<br>share a common vision of what a good AI future looks like:

We figure out alignment,

build superintelligent AI,

and it takes over the world, for our benefit.

Many people will tell you that last part outright: they think human<br>disempowerment is a good thing because the AIs will be smarter and “more moral”<br>than us. Others don’t outright cheer for disempowerment, but you can infer it<br>from their influences, e.g. people who say they are inspired by Iain Banks’<br>Culture series of novels, where benevolent superintelligent machines<br>run the world while humans just party and play video games. This idea of<br>benevolent disempowerment goes back to the origins of alignment as an<br>idea.

In this worldview, alignment is the last and most important task for humans to<br>work on. It is also a thought-terminating cliche, because it lets you<br>avoid any of the hard political or economic or moral<br>questions about the post-AI world. Any objection about the aligned AI utopia can<br>be dismissed by saying “that’s not real alignment”.

You might ask: “won’t humans be powerless in a world with<br>superintelligent machines?”, and the answer is “aligned AI would care about<br>human agency, so that would be a failure of alignment, which we don’t want, so<br>we really have to get alignment right!”. Similarly: “what happens to democracy<br>when the state doesn’t need any human labour?” can be answered by “the<br>AIs will be in control, and since they are aligned, nothing bad will<br>happen”. Which is completely irrefutable.

Of course if someone said “to solve our political problems, we just need to find<br>the right totalitarian dictator. The right dictator would select the right<br>successor, so, by induction, this system will be perfect forever!”, you would<br>laugh at them. But replace “dictator” with “aligned ASI”, and you have the<br>ideology of tens of thousands of the most influential people in the world.

Rhetorically, “aligned ASI” is an opaque premise from which we can prove every<br>desirable conclusion, and refute any undesirable conclusion. Every utopian dream<br>is realized by definition, and any dystopian outcome is averted by<br>definition. Any “gotchas” you try to find in the utopia can be refuted by “the<br>AI will know you better than you know yourself, and will be smarter than you, so<br>it will predict all the bad higher-order consequences of the utopia and fix<br>them”. This should make us suspicious that the concept of an aligned<br>superintelligence is incoherent and born of motivated reasoning.

Published<br>18 August, 2026

Previous

Books I Enjoyed In 2026H1

Next

None

Feel free to email me! I would<br>love to hear from you.

&copy; 2014–2026 Fernando Borretti