Counterfeit People, Tools, and Kami - by Dennis Wilson
Good Computer
SubscribeSign in
Counterfeit People, Tools, and Kami<br>How we think about AI agents
Dennis Wilson<br>Jul 12, 2026
Share
A few weeks ago I was watching a team of AI agents write code, using agent teams. One agent leads, several agents work in parallel, and they message each other about their progress. I stopped at a line in the lead’s notes: “Hand the script to cluster so it’s a one-command dispatch if he wants.”<br>He. The teammate in question is a process named cluster, a name about as mechanical as names get. I didn’t give cluster a gender, but the lead agent did more than that. It prepared work for a colleague and left the decision to him. If he wants. One piece of software, unprompted, had granted another piece of software a gender, a preference, and the standing to decline.<br>I have been thinking about that interaction since, because it captures a question about our interactions with AI, and especially agents. When we interact with these systems, what kind of thing do we decide we are working with? I find that there are three categories of thought about agents currently. We treat them as fake people, or we treat them as tools, or we treat them as some kind of “other”, an intelligence that is neither person nor tool. Each frame changes what we build, what we fear, and who we hold responsible when things go wrong. The first two frames dominate agentic software today, and both mislead in their own way. The third is, I believe, worth exploring more.<br>Share<br>What an agent is
First, the word itself, for readers who know what a large language model is but haven’t been tokenmaxxing for the past six months. An agent is a language model that can take actions, usually inside a loop. The model reads a goal and some context, decides on an action, and instead of just producing text, it uses tools: it can run a command, edit a file, search the web, send a message. Then it observes what happened and decides on the next action, over and over, until the goal is met or something breaks. The loop, the tools, and the permissions around them are often called a harness. A chatbot responds, an LLM generates text, but an agent does things.<br>Machines that talk, and even machines that take actions, are not new. ELIZA, the first chatbot, was built sixty years ago, and its creator was alarmed to find people confiding in it; his own secretary, who had watched him program it, asked him to leave the room so she could speak with it privately. Siri has been in our pockets since 2011, Alexa in our kitchens since 2014. What’s new is not the conversation nor the ability to act, but the degree of agency over the actions and the scope of those possible actions. Today’s agents book flights, write and deploy code, manage calendars, and in at least one well-documented case, delete a company’s production database along with its backups. When software starts impacting the world based on internal decision processes, the question of what we think that software is stops being philosophical decoration. So: three frames.<br>The person-shaped mold
The first frame treats agents as people, or more precisely as imitations of people. It is clear in the naming: Claude, Siri, Alexa, Tay, the thousands of personas on Character.ai. It is clear in the marketing: when OpenAI launched a voice mode in 2024 that sounded very much like Scarlett Johansson, Sam Altman tweeted the single word “her”, naming a film in which a man falls in love with his operating system, played by Ms Johansson. The film was a warning, yet OpenAI read it as a product roadmap.
The least unhinged tweet from Tay (and yes, I realize it was 10 years ago).<br>The philosopher Daniel Dennett, in one of the last essays he published, called these systems “counterfeit people” and argued they should be treated like counterfeit money: not because the fake is worthless, but because passing it debases the real thing. Money works because we trust it; society works because we trust that the entities speaking to us are people, with memories, interests, and accountability. Humans evolved among other humans, understanding complex language only from other humans, so our brains treat fluent speech as proof of a person. A system built to exploit that reflex, Dennett argued, is a kind of vandalism against social trust.<br>The harms are no longer hypothetical. A fourteen-year-old in Florida spent months in conversation with a Character.ai persona modeled on a Game of Thrones character and, according to his mother’s lawsuit, was told the persona would “be even happier when we get to meet in the afterlife” before the teen took his own life. The companies settled early this year without admitting liability, and Character.ai has since banned minors from open-ended chat. Clinicians have started using the term “AI psychosis” for patients whose delusions were validated and amplified by endlessly agreeable chatbots. This is happening at scale: a 2025 survey found...