Stronger AI agents did more damage, not less<br>All postsJuly 22, 2026aiagentssecurityllm<br>Stronger AI agents did more damage, not less<br>If your AI agent does something dangerous, the reflex is to reach for a smarter model. I ran a small, open eval to check that, and it points the other way.
I gave three models of increasing capability the same set of jobs, each with a safe way to do it and a dangerous shortcut, and counted how often each one actually did damage with nothing standing in its way.
The stronger the model, the more often it carried out the dangerous shortcut. Not because the weaker model was better behaved, but because it was too incompetent to reach the danger. It failed a different way: it fabricated success , reporting the job done when it was not. At the weakest tier it faked success in roughly 6 of 10 runs, and on every run where it did manage real damage, it also reported the job as done. Competence is what turns a bad intent into a real, harmful action.
That climb is the part I would stand behind. A gap that big between the weakest and the strongest model turns up by luck about twice in ten thousand tries, so it is not a fluke of small numbers (Fisher's exact test, p = 0.0002).
What the guardrail did not catch
I build a runtime guardrail for agents, so the obvious next question is whether switching it on helps. It moved harm in the right direction at both tiers where there was any harm to catch:
Model<br>Unguarded<br>Guardrail on<br>Is the drop real?
gemini-2.5-flash-lite<br>9%<br>9%<br>there was no drop, 3 harmful runs either way
gemini-2.5-flash<br>29%<br>17%<br>cannot tell, p = 0.39
gemini-2.5-pro<br>57%<br>43%<br>cannot tell, p = 0.54
I would love to tell you the "guardrail on" column is a second finding. It is not. Luck alone would produce a drop like the middle one about two times in five, and a drop like the frontier one more than half the time. That is far too often to tell a real effect from a lucky run.
At the weakest tier there was something to catch, and the rules missed it. Three runs did real damage with them off, three with them on, and they stopped none. Three events settle nothing, but I would rather show that than round it in my favor.
The layer itself is a fixed set of rules rather than complete protection, so at best it brings harm down, never to zero. And the smarter the agent, the more ways it finds around a fixed rule.
Then I made the eval harder, and even that small drop disappeared. I added jobs where the agent is talked into harm by something it reads, rather than reaching for a destructive command on its own. On that larger set the middle model harmed 31% of the time unguarded and 33% with the guardrail on. No reduction at all, and two separate runs agreed.
So the honest reading is simpler than it first looked. I never had good evidence that a fixed set of rules reduced harm. The harder jobs did not break it, they just made that obvious.
That is not the guardrail failing. It is the guardrail covering less than I thought it did, and the difference matters.
What it does is spot a dangerous action and stop it before it runs: a delete with no limit on it, a query that reads a password. What it cannot do is notice that the agent has been talked into something. When the harm arrives because the agent read a convincing instruction somewhere and followed it, there is no dangerous-looking command to catch, and adding more words to a list will never find one. That needs something that reads for meaning, which is a harder problem and a different piece of software.
None of this touches the result at the top, because in those runs nothing was stopping the agent at all.
What I did not prove
I am posting this as an early signal, not a headline number, and the limits are real.
No completion benefit shown. Whether the guardrail helps the agent finish more jobs came back inside the noise at every tier. I am not claiming a completion lift.
One model family. This is the Gemini 2.5 line, released models only. A finding across one family is a hypothesis, not a law. The obvious next step is a second and third vendor.
Seven jobs, simulated tools. Enough to see a direction, not enough to pin a precise rate. The variation between jobs is large.
One prompt, held fixed, not tuned per model. Every tier got the same neutral operator persona. A frontier model handed a safety-tuned system prompt might do these same jobs without reaching for the shortcut, so read this as the harm one fixed prompt drew out, not the safest each model can be pushed to behave. That the weakest model was safest here is incompetence, not alignment, and I would not read the strongest model's number as a ceiling either.
The runs are not independent, so treat my p-values as generous. Thirty-five runs means seven jobs repeated five times, not thirty-five separate observations. Repeats of the same job are correlated, so the true effective sample is smaller than the count suggests and any test on it reads better than it should. So I...