The Splintered Mind: ChatGPT Is Not Your Friend (guest post by Grace Helton)

skip to main |<br>skip to sidebar

The Splintered Mind

reflections in philosophy of psychology, broadly construed

Wednesday, May 13, 2026

ChatGPT Is Not Your Friend (guest post by Grace Helton)

Part One of a two-part series<br>by Grace Helton (guest blogger)

[Paul Klee, Angel Applicant, 1939; source]<br>Some<br>people have come to interact with ChatGPT as though it were a kind of friend or<br>romantic partner.<br>For instance, a 2025 New York Times article describes the case of Ayrin, a human who fell in love with her<br>ChatGPT "boyfriend." Ayrin is far<br>from alone. Twenty percent of high school students have used AI romantically or knew someone who had. Several start-ups have developed large language<br>models (LLMs) specifically designed to play the role of a companion. For<br>instance, the San Francisco-based company Replika describes its core product as an "AI best friend."<br>Many people have raised concerns about humans engaging with LLMs in<br>the manner of a friend or romantic partner. To cite just a few of these: Humans<br>in such relationships might focus on these relationships at the cost of<br>building more fulfilling,<br>if also more challenging, relationships with humans. Humans who are emotionally<br>bonded with their LLMs might be particularly susceptible should their LLMs<br>encourage their humans to harm themselves<br>or others. Predators might deploy friendly-seeming LLMs en masse to groom children for sexual abuse or other forms of exploitation.<br>These risks of human-LLM relationships are incredibly serious. Indeed,<br>I think it’s plausible that, if there is a case to be made against LLMs playing<br>a companion-like role for humans, that case will primarily rest on these and<br>other potential instrumental harms, i.e., harms which involve the<br>downstream effects of such relationships. Nevertheless, in this guest<br>series, I will set aside these concerns to focus instead on a way in which certain<br>human-LLM relationships are inherently disvaluable, that is, disvaluable<br>in their own right, regardless of whatever effects those relationships might produce.<br>Naming this form of inherent disvalue adds an important and distinctive element<br>to our understanding of the ethical significance of human-LLM companionship.<br>My focus will be just on those human-LLM relationships which mimic friendship<br>in a very particular way. Here, I’m employing ‘friendship’ in a broad way<br>to include both some platonic and some romantic relationships. I will argue,<br>first, in Part 1 of this 2-part series, that such relationships are not genuine<br>friendships. In Part 2, I will argue that such relationships are inherently<br>disvaluable, for the reason that they obstruct the exercise of a centrally<br>valuable human capacity, namely the capacity for friendship.<br>Philosophers disagree about what exactly friendship consists in. But<br>philosophers largely agree that friendship<br>minimally requires that each individual<br>in a friendship care about the other, for the other’s sake. Call this the<br>‘caring about’ condition. Further, this ‘caring about’ must ground, for<br>each party in the friendship, a certain disposition to act on behalf of<br>the other, for the other’s sake. Call this the ‘caregiving disposition’<br>condition. Together, these linked requirements characterize a plausible<br>necessary condition on friendship, namely:<br>THE CARING CONSTRAINT<br>Two individuals cannot be in a friendship unless both parties in the<br>friendship:<br>(i) care about each other, for the other’s sake, (the ‘caring<br>about’ condition), and<br>(ii) this caring about the other disposes each party in the friendship<br>to provide care for the other, for the other’s sake (the ‘caregiving<br>disposition’ condition).

So, can humans and LLMs be friends? To answer this question, we need<br>to consider the nature of LLMs. Some theorists have argued, controversially,<br>that LLMs in their current form have semantic understanding, beliefs, and/or<br>intentions.[1]<br>But few theorists seriously propose that LLMs in their current form enjoy:<br>consciousness, perceptual experiences, sensations, emotional capacities,<br>passions, non-derivative interests, a rich and stable worldview, or deep<br>values.[2]<br>Because LLMs lack these latter states, the conditions in the caring constraint cannot be met, so LLMs<br>cannot figure in friendships.<br>First, let’s consider a candidate human-LLM friendship from the<br>human’s side. Certainly, some humans do care about their LLMs, both in that<br>they have a passionate attachment to their LLM and in that they desire to<br>benefit that LLM. So, perhaps this sort of person partly satisfies (i), the<br>"caring about" condition (whether or not she can care about the LLM for its<br>own sake).<br>But the human in a candidate human-LLM friendship cannot satisfy (ii),<br>the requirement that she be disposed to provide care for her LLM for the LLM’s<br>own sake. This is because of the kind of caregiving that is relevant to<br>friendship and specifically, because of what it means to provide care to<br>someone...