A Room Full of Locksmiths | from the trenches
Guru Sattanathan<br>Member of Technical Staff at Arcade.dev<br>Systems architecture, AI agents, and production patterns. Writing from the trenches. More →<br>LinkedIn@avoguruGitHubSubstack
The first meeting of the new agent platform team is going well. The whiteboard already shows the whole answer: a gateway for identity, a sandbox for isolation, an orchestration layer for the agents themselves. Everyone at the table can defend their box, and everyone is right. Forty minutes go to token exchange and network egress, and the conversation is genuinely good.
Then someone asks a small question. What will the agent actually do first? Which workflow, in which system, and who checks the result when it gets things wrong?
The room goes quiet. Not because anyone is incompetent, but because nobody present has ever done, or owned, the work being automated.
That meeting is a composite. Every piece of it is real, and I have sat in more versions of it this year than I want to count. The silence is the most expensive sound in enterprise AI, because the platform that ships six months from now was decided right there, and its most important failure had a seat at the table. The seat was empty.
Let me be precise about what this piece claims, because the lazy version is false. Plenty of agent projects die on technology: the data is a mess, the model is unreliable at the task, the system that matters has no API and never will. This piece is not about those. It is about the other class, the pilots that work perfectly in the demo and die at the question “whose credentials does this thing run on”, and the platforms that pass every review and still produce agents nobody trusts with real work. For that class, the cause is not in the stack. It was in the room, and a programmer named Melvin Conway explained why in 1968:
“Organizations which design systems (in the broad sense used here) are constrained to produce designs which are copies of the communication structures of these organizations.”
Melvin Conway, “How Do Committees Invent?”, 1968
The agent version is sharper: your platform inherits the blind spots of the meeting that designed it.
every expert in the room is right
Three kinds of people are selling you an agent platform right now, and it matters that all three are telling the truth.
The security story: agents are identities, and ungoverned identities are how companies burn down. You need to know who the agent is and what it is entitled to, and you need one place to see and revoke all of it. Every word of that is correct.
The infrastructure story: agents are workloads, and untrusted workloads belong in isolation. Sandboxes, egress controls, resource limits, kill switches. Blast radius is a real concept and limiting it is real engineering. Every word of that is correct too.
The harness story: the interesting behaviour lives in the loop, in how the agent plans, calls tools, recovers, and asks for help. Whoever owns the harness owns the quality of the outcome. Also correct.
These are not caricatures, and the people who carry these stories are not fools. These are perspectives, not badges: plenty of security engineers write agents, and one person often carries two of these stories at once. The error is never in any layer’s claim. The error is in the silent suffix each story carries: “and that is the whole problem.” All three decks in your inbox can be true at once, and your pilot can still die, because the thing that kills it does not live in any of their layers.
the damage travels through doors you built
Here is the mechanism.
An agent that causes damage in production almost never does it by breaking something. It does it through channels you authorised: an approved API, a valid token, a sanctioned scope, a tool it was given on purpose. The sandbox does not need to be escaped, because the actions that matter were never inside it. They travel out through doors you built, using keys you cut. And every door in that list exists because a piece of work needed it.
This problem was named before most of your stack existed. In 1988, Norm Hardy described what he called the confused deputy: a compiler that held legitimate permission to write files in its home directory, took the name of the billing file stored there as a debugging output path from a user, and obediently overwrote the billing records. Nothing was breached. Every permission checked out. A program with authority it legitimately held was steered by input it trusted, and that was enough. An agent is that deputy at machine speed, taking input from everywhere.
Simon Willison gave the modern version a name in 2025: the lethal trifecta. An agent becomes robbable the moment three things are true at once:
it can read private data
it takes in content an attacker can influence
it has any channel that leads back out
Nothing needs to break. The agent can be robbed while every part of the stack behaves exactly as designed....