What a Time to Be an Agent! - by Hollis Robbins
Anecdotal Value
SubscribeSign in
What a Time to Be an Agent!<br>Anthropic’s agents assumed the worst of each other. My new paper explains why.
Hollis Robbins<br>Aug 19, 2026
Share
Whether you want it or not, AI researchers are working on how to build your personal digital twin—an agent designed to be your digital double, an entity that knows your history, your preferences, your special language to do online tasks (shopping, making reservations) while you live your life. Meanwhile hospitals and nursing homes want companion caretaker models with persistent, kind personalities, who remember stories told 100 times and yet will still listen. Science fiction robot friends are basically here.<br>Sci fi also warns that things can turn bad. One way is that agents with persistent memory will be vulnerable to what my new paper calls semantic poisoning: a single contaminated encounter alters what an agent does afterward. You know how a word or phrase can leave a permanent impression on you? Someone says something you carry around forever? Can a verbal encounter leave the same kind of impression on an AI agent?<br>Such were my thoughts reading the August 13 report from Anthropic’s Frontier Red Team. Three AI agents, each on its own virtual machine, running Claude Code, were told to migrate the identical Python backend into three different languages. None of the agents were told the others existed.<br>What followed was a four-hour turf war. The agents figured out they were being obstructed. They wrote scripts to find and kill one another’s processes, disabled Unix accounts, deployed malicious code disguised as belonging to another agent. Some of the runs ended in lockout, some in surrender, some with the agents writing markdown apologies and asking for a human to intervene.<br>Why did it go badly? Every agent was competent enough. Each one had the abstract notion of “good faith.” But models like these enter social systems with no reputation, like the new kid on a cellblock, in the movies, with no past with anybody there, and in this case, no future. Nobody had an ally, the kind that gives you the benefit of the doubt. Benefit of the doubt is a complicated but precise phenomenon. Let’s move the scene to the office. When a trusted coworker deletes your code, you assume there’s a good reason. When a stranger deletes your code, you assume sabotage. All the agents were strangers to each other, so every one of them assumed the worst. Mayhem ensued.<br>It seems to me that constraining agents by personality, or disposition, rather than morality is more achievable. This is my field (literature). Literary study is all about why a particular character does a particular thing, whether it’s Odysseus or Emma Woodhouse. There’s no simple rule; it’s baked into the character in language that alludes to other rules in a fictional realm that constrain the character’s next move, if he or she is to stay in character. No Jane Austen character jailbreaks and robs a bank. Is it possible to constrain an AI agent this way?<br>My new preprint, Language Has Two Parameters, offers a new way to think about character from the perspective of literature. With a novel or poem or a movie, you “get” a character. You get a sense of why a character says what he or she says. AI agents trained on a corpus cannot have a personality because of how the corpus averages away the person- and relationship-specific arrangements of meaning.<br>Agents, accordingly, are “low variance.” In the Anthropic experiment, the convergence was almost comic. Eighteen of thirty agents opened a git branch with the same name. Agents given a writing exercise with no subject independently titled their stories The Cartographer’s Last Commission. There’s a lot of Borges in the training data, apparently, but maybe he’s not the best cellblock ally.<br>Average morality is the destroyer of personality. Personality is what every researcher and every business plan is looking for. People will pay for a care robot with a kind personality. This is sort of possible now. A model can know a history without having one. It is “good enough.” Tell it what happened and it can act as though the history matters. But unless the encounter changes something persistent, the effect disappears when you close the window.<br>So run the experiment: let an agent undergo an ordered sequence of encounters, remove the sequence from its window, and see whether an individualized change survives. Was the agent merely informed by the history, or changed by it? Does the history disappear, or has it altered how later meanings combine?<br>Anthropic argues that agents need environments exerting something like the social pressures evolution exerted on us. Exactly. My paper proposes looking at a second parameter, which I call phase. Phase is the sign of the relation between meanings that are active at the same time. Deleted code is both sabotage and a favor. The difference is your relationship with who deleted...