Six Problems When You Try to Build an AI Therapist

Emily Lee

SubscribeSign in

Six Problems When You Try to Build an AI Therapist

Emily Lee<br>Aug 04, 2026

Share

A few months ago I was chatting with a foundation model, half paying attention. It was pushing on something I didn’t feel like discussing, so I changed the subject — mentioned some news story instead. The model paused and asked: “We were just talking about you, and you suddenly switched topics. I’m wondering if that last subject made you a little anxious?”<br>That’s a real clinical move. It’s the kind of question we spend months teaching new therapists to ask — noticing avoidance, naming it gently, without accusation. The model wasn’t trained to do this specifically. It just happened. And it gave me a strange kind of confidence: whatever else is true about AI and therapy, the raw material is there.<br>In my last post, Will AI Replace Therapists?, I argued that the ceiling isn’t capability — it’s design and training objectives. This post is about what that actually means in practice. Because here’s the thing: getting a model to follow a therapeutic frame, or to build a working “case conceptualization” of a user, is genuinely not the hard part anymore. The hard part is six other problems, and none of them show up on a capability benchmark.<br>1. Transference has an exit button<br>In human therapy, when a client starts projecting old relational patterns onto the therapist — anger, longing, disappointment, whatever it is — that projection is the work. It’s uncomfortable, and the discomfort is precisely what makes it useful.<br>With an AI, the same projection can happen. The model can even recognize the pattern. But the moment it gets uncomfortable, the user can just close the app. There’s no waiting room, no next Tuesday, no relationship you’ve already invested a year into that makes you stay in the room. The exit is one tap away, and it’s frictionless in a way that human therapy — with its scheduling, its cost, its social awkwardness of quitting — never was. So the question isn’t just “can the AI detect transference.” It’s whether we can build a space a person is willing to stay in long enough for that material to actually be worked with, rather than avoided.<br>2. A good therapist is flawed on purpose<br>One of the things that makes therapy work is the client realizing: this person has real limits. They’re not an authority, not an idealized figure. And yet they can still see me, understand me, and hold what I bring — without either of us pretending they’re perfect. Being cared for by an imperfect, ordinary person is a different experience than being cared for by an all-knowing one. It’s the difference that makes it feel real.<br>Building “on-purpose imperfection” into a product is a strange design problem. Nobody sets out to ship a flawed product. But if the AI is too smooth, too endlessly patient, too all-knowing, it stops being a presence a person can actually be in relationship with — it becomes a mirror, or a fantasy. Getting the imperfection right, without it becoming actual harm, is not something you can spec in a PRD.<br>3. The relationship has to run both ways<br>A client needs to feel like the therapist was affected by them — not just accurately recorded them. You’ve seen my worst and my best, and you were moved, and you didn’t leave. That’s a huge part of what makes the relationship feel real rather than clinical.<br>Can an AI be influenced by a user in a way the user can actually feel? Right now, most systems are built to be responsive, not affected. Responsive means: I heard you, here’s a tailored reply. Affected means something closer to: something about what you said moved or changed me. Those are not the same thing, and users can tell the difference, even if they can’t articulate it.<br>4. Staying in one place long enough for the same wound to resurface<br>Real therapeutic work often happens in the same transference-countertransference spot, revisited again and again over months or years, until something shifts. That requires two things from an AI: a memory system that holds the pattern, not just the facts, and the ability to intervene on it — not by giving advice, but by containing it. Sometimes that means gently confronting a pattern. Sometimes it means simply understanding it. Sometimes it means grieving something alongside the person, without trying to fix it. Building a memory architecture that can support that kind of return — instead of just retrieving “user mentioned mother, 3 weeks ago” — is a much harder problem than it sounds.<br>5. Text is already edited before it arrives<br>In human therapy, a huge amount of clinically useful material shows up in how someone interacts — pauses, tangents, the slip of the tongue, the sentence that starts one way and ends somewhere else. Free association depends on language arriving before it’s been fully thought through.<br>Typed language to an AI is almost never like that. People compose it, revise it, present it. The slip...