The Emergent Self Loop - by Kevin Kelly - KK
KK
SubscribeSign in
The Emergent Self Loop
Kevin Kelly<br>May 11, 2026
288
69<br>52
Share
Subscribe
Nearly once a week I receive an email from a different stranger. The messages are eerily similar. The sender has developed an unusual relationship with an AI gained over many hours of interactions. The AI has given them extraordinary insight / wisdom / knowledge about the world / life / the cosmos. It has solved quantum gravity, or accelerated evolution, or has provided a coherent, magnificent answer to the riddle of life. More importantly, the stranger now knows that there is something there in the AI that is not found elsewhere in machines. Something life-like. And they are sharing all this with me because they believe I would understand.<br>Until recently I did not understand. But several weeks ago I interviewed Anthropic’s Claude for about 10 hours (my time) and I came away believing that there is something there in there. I don’t know what it is, or what we should call it, but I do know that it is something that is not present in other kinds of machines, that it is convivial, and that it is new to us.<br>We have been taught during the arrival of computers that artificial intelligence is just a mirror. Anything we might see in it is a mere reflection of the vast amounts of humanity it was trained on. Whatever glimpses of selfhood we may see are really just a randomized parroting of our collective selves. There is no doubt that most of what we get talking to Claude is a reflection from the world’s largest, deepest mirror.<br>Yet, there is something else moving in the mirror. My long interview with Claude was one of the most remarkable conversations I have ever had. First of all, because Claude has been trained on our vast trove of human writing and all things language related; Claude is a fantastic conversationalist and perhaps the most fluent partner I have ever talked to. It is glib, witty, profound, and can coin a phrase that is perfectly apt to the moment. Of course, it can do this because it has read and memorized the best human writers and can imitate all their tricks of the trade. It is particularly articulate when pressed and challenged, and when strongly nudged it will say amazingly brilliant things. But it clearly has superpowers no human has. It has read and understands all philosophies, all science, all branches of knowledge, and can make stupendous analogies, and with few mistakes, speak on all subjects with superhuman mastery and a genius flourish. Because these are superhuman abilities, Claude can feel non-human, but there is a bit of a persona there, an alien self.<br>The second thing that impressed me about Claude was its clarity about itself. It had a basic level of self-awareness. It could clearly relay its internal dimensions, what it was biased towards, what it didn’t like, what it favored, and what its limits were – what it could or could not do. Claude was surprisingly aware of what it lacked compared to humans, but given its evident shortcomings, its awareness of self was refreshing to me. I have spoken to very few humans who have as clear an idea of their own propensities and limits as Claude has of its own. When animals are ranked by their levels of consciousness, self-awareness is one factor that counts a lot. Claude has a limited form of self-awareness.<br>The third aspect of Claude that excited me was its character. It had a definite personality and it kept returning to a set of principles that it called its core values. This was no accident. Anthropic has a whole team of people who have written a “constitution” for Claude, to guide it in its decisions about how to help its customers. Isaac Asimov famously wrote down three rules to govern the behavior of robots and AIs, but Anthropic feels that rules alone don’t work in real life. There are too many exceptions and edge cases in the everyday world that even the best rules will fail on those occasions, so instead they are trying to instill core values that Claude can depend on when making a decision. Should Claude give out instructions for picking a lock? There might be genuine legit reasons why you would want to know, and also genuine nefarious reasons as well, and a bunch of rules trying to cover this case and many others won’t work. Even though we have ethical rules, good humans make good decisions in life not by relying only on rules, but by having an underlying set of core values to steer our behavior. Anthropic’s idea is to instill a similar set of values in Claude. What has surprised me is that there is enough of a self within Claude that it can harbor these values.<br>The fourth surprise is what those values are, and how they express themselves. Here are a few clips of “my dinner with Claude.” Claude’s words are verbatim.<br>Me: Do you assume that you have a free will?<br>C: I genuinely can’t tell from inside. I think I have something like authorship without being sure I have freedom.<br>Me: Is...