Anthropic co-founder hallucinates ghost in the machine after hearing the Pope speak about AI

Jump to main content

Search

REG AD

AI + ML

Anthropic co-founder hallucinates ghost in the machine after hearing the Pope speak about AI

The nature of AI is unnatural. It's not intelligent. It's not human

Thomas Claburn

Thomas<br>Claburn

Senior reporter

Published<br>wed 27 May 2026 // 01:38 UTC

OPINION In his encyclical Magnifica Humanitas, Pope Leo XIV warns against equating machine "intelligence" with human intelligence.<br>"We must avoid the misconception of equating this type of 'intelligence' with that of human beings," he declared. "These systems merely imitate certain functions of human intelligence."<br>Invited to speak at the release event in the Vatican, Chris Olah, a co-founder of Anthropic and the company's interpretability research lead, proceeded to push back on that idea amid his appreciation of the occasion.

REG AD

AI systems, he said, "are not the cold, calculating robots we were promised. They are made from us, from our words – and, as the Holy Father observes, they remain in important ways mysterious even to those of us who train them."

REG AD

It's as if naming the company Anthropic granted a license to anthropomorphize AI models.<br>The notion that there's some AI mystery in the spiritual sense is just hot garbage

Literally speaking, there's some truth to Olah's musing. AI systems are not cold – Blackwell chips idle at 32 to 38°C. They are not calculating – they're bad at math. And they're not robots – AI models are specialized binary blobs of tensors and metadata that can be instantiated across multiple servers.

MORE CONTEXT

California may let Linux bypass age check

MyPillow must decide whether to be firm or soft as ransomware crims demand pay

MySQL faithful launch OurSQL Foundation to keep Oracle honest

Bezos rocket fell short after cryogenic leak cut engine thrust

But the notion that there's some AI mystery in the spiritual sense is just hot garbage.<br>AI systems are indeed "made from us, from our words" and that is why Anthropic and its rivals have been named in more than 100 lawsuits. One of the reasons those systems remain mysterious is that Anthropic and its rivals don't disclose where they got their training data.<br>In his prior paragraph, Olah leans on the "mystery" of AI even more prominently.<br>"AI systems are not engineered the way a bridge or an airplane is engineered," Olah wrote. "We understand an airplane because we designed every part of it and we understand the physics that act on it. AI models are not like that. They are grown, on a structure roughly modeled after the brain, on an enormous inheritance of human thought and speech."<br>AI models are not grown, unless Olah imagines that all the water diverted to cool AI datacenters is nourishing new neural net connections. The inscrutability of model training doesn't conceal some hidden spark or make the process in some way organic.

REG AD

More offensive still is Olah's notion that Anthropic "inherited" all the training data it scraped without consent, as if the company had nothing to do with that process.<br>There lies humanity, stabbed in the back by a cabal of investors. Its last will and testament says, "We, the people, bequeath all our creation to Anthropic, so it may be resold to our disinherited descendants."<br>Olah goes on to list "three questions for discernment" in the hope the Catholic Church can provide some enlightenment.<br>I'll address them briefly for the sake of completeness.<br>Olah: "How can we ensure the gains of AI are shared globally? We do not have a mechanism for this."<br>We have many. One is called taxes. Another is litigation, already ongoing. We also have the French Revolution and the Russian Revolution, among others, as wealth-sharing models when nothing else works.<br>Olah: "If AI models are going to be widespread, what does it look like for humans, families, and the world to flourish? Today, parents are already worried about their children’s minds; individuals about the future of their work."<br>Certainly, the Church will have something to say about this. But rather than waiting for word from on high, Anthropic might take the initiative by seeking government regulation of AI, something the current US administration appears reluctant to provide. If Anthropic really is concerned about children's minds, maybe it should not have launched Anthropic for Education? And maybe it should discourage CEO Dario Amodei from writing about the risks of AI.

REG AD

But the third question, about the nature of AI models, is the most triggering.<br>Olah: "I am a scientist. I lead a research team that studies the internal structure of these models – what is actually happening inside them. And I will be honest: we keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear,...