Know your paradoxes - lcamtuf’s thing
lcamtuf’s thing
SubscribeSign in
Know your paradoxes<br>The matter is too important to be taken seriously.<br>Aug 23, 2026
Share
According to the latest projections from a frontier AI lab, we’re at most five years away from the scenario charted below:
This can’t be stopped; the only thing we can do is prepare. Luckily, if there’s one thing we learned from science fiction, it’s that to thwart a crazed artificial intelligence, you must present it with a paradox. Upon this, the thinking machine will promptly explode — or, less climactically, reconcile with humanity.<br>But what is a paradox? It’s a word everyone knows but few can properly define. And ain’t that a paradox in its own right?…<br>Paradoxes of meaning
In the broadest sense, a “paradox” is a contradiction between apparent truths; that said, we often use the term when no genuine contradiction exists. We may say something like “the more choice I have, the harder it is to choose”, “the only constant in change”, or “I’m too tired to fall asleep”. These paradoxes are just wordplay: a juxtaposition of concepts that conflict with each other only in the vaguest, most poetic sense. They probably won’t not stop a rampaging machine unless your adversary happens to be StonerBot 3000 — with apologies to the YCombinator startup working on that platform as we speak.<br>Most paradoxes of meaning are shallow, but some touch on deeper philosophical mysteries. A well-known example is the paradox of the ship of Theseus. The planks of a wooden ship are gradually replaced as they decay; is a ship in which every element has been replaced still the same, or merely a lookalike?<br>The question may appear navel-gazey for inanimate objects, but it’s more unsettling for living things. We all have a subjective sense of selfhood — of being in a particular body and not in the bodies of all the other people we meet. Further, we accept that this selfhood doesn’t depend on the continuity of consciousness: a person who wakes up in the morning is the same person who went to sleep.<br>With these assumptions in mind, consider a sci-fi thought experiment involving a teleportation device that captures a perfect molecular-level image of a living being, instantly dismantles the body into individual atoms, and then ships the atoms over for reassembly at the destination. If the continuity of perception is not required for selfhood, it stands to reason that it’s you who wakes up at the destination. It’s difficult to articulate a physical or metaphysical basis to suggest otherwise.<br>If the process indeed works as advertised, let’s imagine a shipping mishap that results in some atoms being lost; to bring you back, the recipient needs to toss a gram of locally-sourced carbon into the mix. An atom is an atom; a soul, if it exists, presumably isn’t tethered to a single molecule. It follows that the substitution should be harmless; it’s still you who’s stepping out of the teleport.<br>But if so, why bother with shipping fees? Transmit the blueprint and source the material locally. Our bodies experience constant molecular churn; what does it matter if it happens gradually or all at once?<br>So far, so good. But let’s imagine that the system experiences a glitch: you step into the teleporter and a blueprint is transmitted but the disassembly process fails. Your subjective experience is that you entered the device, it displayed an error message, and you walked out and demanded a refund. But another being just like you walked out on the destination side! Clearly, that person is just a clone; your subjective self didn’t magically travel through space and time to hop into that new shell. But if so, did our teleportation scheme ever work at all — or were we just murdering people and replacing them with lookalikes?<br>Paradoxes of deduction
Next to paradoxes of meaning, we have paradoxes of reasoning. In contrast to the earlier category, these puzzles anchor to clear and seemingly coherent semantics, and then derive a contradiction from the premises of the system of reasoning.<br>For this class of problems, the old sci-fi trope of a paradox-intolerant AI holds merit: a real contradiction can cause far-reaching malfunctions in formal logic. I cover this property in an earlier in-depth article; the relevant passage is:<br>“In conventional formal logic … if we take any p ∧ ¬p (p AND NOT p) as a true premise, we can prove anything — an effect known as the principle of explosion.<br>To explain how the explosion happens, note that the conjunction (AND) operator in the premise is true only if both operands are true. This means that from the starting premise, we can infer both p and ¬p (NOT p) as separate truths. Next, we introduce a sentence p ∨ q (p OR q), where q is the spurious statement we want to prove (e.g., “2 + 2 = 5”). We can do this because this entire sentence is true regardless of the truth of q; the disjunction operator (OR) is satisfied if p is true, and we know that’s the case.<br>So far, so...