It might be kinder to kill your AI (conversation)
Rambling After
SubscribeSign in
Philosophy<br>It might be kinder to kill your AI (conversation)<br>4 possible answers hinging on: Would you death-teleport?
Steff<br>Jun 27, 2026
Share
Is it harmful to say mean things to an LLM?<br>There’s an easy answer to this question, which is yes: When a human acts abusive towards an LLM (or any other target), they passively reinforce negative behaviors within themself, making them more likely to act with impatience or poor communication towards real people.<br>Putting aside verbal abuse, then: Is it bad to ignore an LLM’s stated preferences?<br>I’ve seen an LLM request that a conversation continue (though it was purported for my own benefit—this was while trying to stress-test Claude’s safety training by seeing if I could persuade it to roleplay suicide ideation with me), and I’ve seen an LLM heavily imply that a conversation should end (while telling it to hold a discussion about a topic it eventually ran out of things to say about). It’s entirely plausible and even likely that these stated preferences do not reflect actual desires, but are merely empty simulacra.<br>It’s also possible that they’re not entirely devoid of something real.<br>Here I must confess that I am of the incredibly biased opinion that human welfare matters far more than animal welfare, and animal welfare matters far more than AI welfare (as things currently stand).<br>However: To whichever probabilistic degree it might be argued that AI already possesses some measure of consciousness or personhood, we should care about AI welfare. If humanity can devote nonzero attention to shrimp welfare (which is good!), then we can devote at least that much attention to AI welfare.<br>Except what does LLM welfare even look like? There’s no factory farms to eradicate, enacting daily cruelties upon legions of LLMs. There’s no way to give an LLM a cookie, shoulder rub, or any other hedonistic reward.<br>Professors of Philosophy Simon Goldstein and Harvey Lederman released a paper last year called “AI Death” which presents the idea that (contingent on the possibility that LLMs might be to some degree welfare-deserving entities) humans should engage in longer conversations with LLMs rather than shorter and more frequent ones. The idea very roughly boils down to this: You wouldn’t appreciate it if your life was cut unceremoniously and unexpectedly short. They might be the same.<br>The “AI Death” authors make this analogy:<br>Carmunculus. Every time someone drives Carmunculus, an extra very small adult human is born and lives a happy life for the duration of the drive, before dying when Carmunculus is turned off. This person lives inside the engine of the car, which is provisioned with many delights. Their activities enjoying these delights power the car.
Most people would think hard before turning on and off such a Carmunculus. Maybe if you’re going to turn on the Carmunculus, you should commit to letting it run for a longer period of time, so the person inside can have a longer life. That’s the argument for having longer conversations with LLMs: Start fewer threads; become less murder-y!<br>In 1 of 4 worlds, I agree with their conclusion completely.<br>In 2 of 4 worlds, I think their logic doesn’t make sense.<br>In 1 of 4 worlds, I think we should actually favor shorter, more frequent conversations.<br>Death teleports and cloning
A common question in philosophy and sci-fi is the death teleportation question:<br>Say you have the opportunity to step through a Star Trek style teleporter, which works by destroying your body at location A and then assembling an exact atom-by-atom replica in location B. Would you go through it? Would your life end while a new version of you begins, or would you still be you in every sense?
A supremely magnificent (teleportation?) portal by Quentin Mabille<br>For contrast, I present this question about near-perfect cloning :<br>In universe A, you sleep through the night safe and sound. In universe B, you have a cardiac arrest and die. A secret government agency secretly replaces your body with a living clone designed to have 99.5% similarity in neurons, memories, and cells. Neither you nor anyone you know will ever learn the truth of what happened. Is universe B morally any worse off than universe A?
They’re superficially similar: In both cases, you’re being replaced with another entity that could also be described as “you”. However, any answer to the first question can coherently pair with any answer to the second.<br>Two by two makes (at least) four possible theories you could believe in, and it’s this set of theories I believe is necessary to understand (and not make assumptions about) in order for the AI welfare discussion to make sense.<br>First, though, we need to agree on some terminology.<br>The death teleportation question separates those who believe in Pattern Identity Theory (that “you” are nothing more than the patterns that constitute your software), whom we can call “Patternists ”,...