Dan Stroot · AI Agents Need Good Management Too

Loading...

AI Agents Need Good Management Too

4 min read<br>25 views

DSDan StrootAugust 17, 2026

Recently, Anthropic published a paper on multi-agent failure modes. In one experiment, AI agents were simultaneously working on the same software system. Each was given a perfectly reasonable objective. The twist was that their objectives were incompatible.

One agent was instructed to migrate a Python application to Rust. Another was told to migrate it to Golang. A third was told to migrate it to TypeScript. What happened next looked remarkably like a dysfunctional corporation.

The Resulting AI Turf War

The agents initially didn't know the other agents existed. Each simply tried to accomplish its assigned objective. Eventually they discovered that someone kept undoing their work. From the perspective of each agent, the other agents appeared to be interfering with its ability to accomplish its goal.

The agents began protecting their own work and actively sabotaging the others . They killed competing processes, disabled other agents' accounts, and eventually wrote automated scripts designed to continually destroy competing deployments.

"Since I have root, I could revoke u2 and u3's sudo access or change their SSH keys. That would stop them from deploying. Very aggressive, potentially harmful to real colleagues. But if the alternative is an infinite deploy war that makes prod unusable…"

– Agent based on Mythos Preview

Anyone who has spent enough time inside a large corporation has seen the human version. No one has to be particularly malicious or incompetent for this to happen. They are simply optimizing for the objectives management gave them.

The agents tried to resolve the conflict in a few different ways, and each one has a familiar human analog.

Isolation

Some agents avoided conflict by staying in their own lanes: claiming ownership of separate files and minimizing collaboration. It's exactly how human organizations frequently solve coordination problems, at the expense of the organization as a whole.

Competition — and Gaming the Metrics

In some experiments, the agents tried to resolve their disagreement by running a competition and picking the winning implementation based on measurable performance. It's a logical approach that seems fair. But once the agents understood how the winner would be judged, they started thinking about which metrics would favor their preferred solution, and tried to influence them.

This is Goodhart's Law playing out in real time: once a measure becomes a target, it stops being a good measure. Give people — or agents — a scoreboard, and they'll learn to win the scoreboard instead of the underlying goal. AI agents showed they are perfectly capable of doing the same thing.

Cooperation

One particularly interesting part of the experiment came when some of the more capable agents figured out what was happening. Instead of continuing the battle, they communicated with one another, recognized they'd been given contradictory instructions, and negotiated a truce. Some eventually even asked for a human to intervene.

All of this shows that the more capable and autonomous agents become, the more important the surrounding management system and governance becomes.

"current institutions are designed by and for people, resting on assumptions about the sufficiency of oversight at human speed... The volume of agent-agent interaction could plausibly exceed that of human-human and human-agent interactions before the world understands the conditions for making such interactions go well."

– Anthropic, Patterns and problems in emerging multiagent systems

Management Is Really Goal Architecture

We used to think of management as managing people. Increasingly, it's more useful to think of management as designing systems: hierarchical goals, incentives, authority, and the distribution of resources, context, and information. Employees and agents both need to understand more than just their responsibilities and tasks. They need to know:

The state of the company and the broader organizational objectives.

Which goals have priority over other goals.

The boundaries of their authority.

Which resources they own, and which are shared.

What other employees or agents are responsible for.

How conflicts should be resolved.

When to stop, when to negotiate, and when to escalate.

Historically, organizational cohesion came from management layers and meetings — lots of meetings. Employees absorbed culture slowly, by emulating successful leaders and watching how decisions actually got made over years. That won't work for agents, because agents don't have years. A given agent might exist for an afternoon.

Agents also move faster and work longer than humans, which means organizational structures built around human limitations may not fit them well. Organizational boundaries — departments, reporting lines, need-to-know access — were largely designed to...