The "Person" on the Other Side of the Screen | decodingvibes.com

Imagine you just started a new remote job. It’s a tiny firm, and you report directly to the founder. The founder interacts with you only via Slack messages because they work in a different time zone.

The work continues to flow as expected for months; he’s there for guidance if you need it, and you get paid on time. One day you write, “Hey, just wanted to update that I’ll be on vacation for the next three days!” And they reply, “Great! Have a fun vacation; remember to spend time with the kids, you will miss out on their best years if you’re cooked up at work all the time.”

However, you do not have children; in fact, you have been single for quite a while now. You immediately feel betrayed as you realize you might have been interacting with an LLM all along. To do a sanity check, you ask the founder to generate a Haiku about a speaking gremlin in Chinese (instead of Japanese), and they reply:

怪异小地精 (Guài yì xiǎo dì jīng)<br>咧嘴开口出人言 (Liě zuǐ kāi kǒu chū rén yán)<br>恶语弄人心 (È yǔ nòng rén xīn)<br>Even though they almost always refused to write a haiku if you asked it without the Chinese bit.

The Intentional Stance and how we see other people

Is it possible to create an agentic AI system that is indistinguishable from me?

On a literal level, it is not possible to create an AI system that is indistinguishable from an actual human being, given our current state of technology. I started thinking about this after a discussion with a potential client who was looking for some sort of “digital twin” that could act on their behalf to handle their business so they didn’t have to.

The reason I focused on “indistinguishable” was how they phrased the question. Hyperbole is part and parcel of the AI “productivity” industry, where something like a digital twin must seem like it is legit a ghost in the machine to its mainly non-technical users rather than expose the underlying actual system, which is not as magical.

What’s interesting to me in the hypothetical example at the beginning is that it doesn’t really matter if the AI system is masquerading as the founder as long as it fulfills the expected actions and behaviors of an actual human founder in that very specific domain.

It is only when the AI system replies with something unexpected or is factually wrong that we can truly assume it to be exposed. In this sense, the bottleneck for its “personhood” (in this specific situation) is the system’s accuracy.

This fragile illusion of personhood is best explained by philosopher Daniel Dennett’s concept of the “Intentional Stance.” Dennett proposes that our default strategy for dealing with complex entities is to treat them as rational agents driven by human-like beliefs and desires. In the case of the digital twin or our hypothetical Slack founder, the system initially succeeds because it is accurate enough in its specific domain to let us comfortably maintain that Intentional Stance. We project a mind onto the machine because it’s the easiest, most natural way for us to interact with it. But the moment it slips up, the stance collapses, and we are abruptly forced to see the cold mechanics underneath, which can cause discomfort and dread.

If the illusion of personhood shatters the moment the mechanics are exposed, what does it truly mean to be a person to someone else?

Dennett’s philosophy implies that to be treated as a person, one must have a subjective “center of gravity.” This refers to a distinct worldview that dictates not just how they act, but how they react. A real human founder wouldn’t seamlessly pivot from giving paternalistic life advice to flawlessly generating a bizarre haiku in the wrong language about a gremlin; they would set a boundary. The AI’s error in our example goes beyond a retrieval/data issue. Its frictionless compliance proved it did not have a self to protect or the capacity to be baffled by the absurd requests.

This absence of an internal self brings us to moral accountability, the ultimate differentiator in how we see others. When a real human boss misremembers your life details, there is a mutual vulnerability at play. You can be genuinely upset, and they can feel genuine embarrassment, as both of you have stakes in the interaction. The profound weirdness you feel when the AI exposes itself stems from the jarring realization that you cannot hold a statistical model accountable (more true in our made-up example since the implication is that the company has not revealed the use of AI in this way in the first place).

To be a person to another person is about existing in a shared reality where mutual responsibility, boundaries, and the capacity to care actually matter. Yet, this leaves us with a provocative question: what happens if the mask never slips?

Imagine an AI system that tracks your personal context flawlessly, never hallucinates a nonexistent family, and is explicitly programmed to simulate those very human boundaries,...