Human Approval Is a Privilege Escalation Path | CoreBase

All Posts<br>An agent wants to refund $200. A card appears, someone reads it, clicks Approve, and the refund goes out. The log records that a human approved it.

Two questions decide whether that record is worth anything. The first one gets asked a lot: what did they approve ; the sentence on the card, or the request that eventually hit the API? The second one almost never gets asked, and it is the one that turns a safety feature into a hole:

What does your system do with the word "approved"?

If the answer is "it skips the check," you haven't added an approval step. You've added a way to get past the check, and you've put the button in front of a person who has no way to know that's what they're holding.

The obvious implementation

Here is what nearly everyone writes, because it is what the feature sounds like:

Python<br># The agent asked to run something risky. A person said yes. So run it.<br>if approval.status == "approved":<br>result = await tool.run(**approval.arguments)

Read it as an attacker would. The check that would normally decide whether this call is permitted is not in this code path at all. approved is not satisfying a condition, it is bypassing the place where conditions get evaluated.

So the exploit writes itself. Find an action the agent is never allowed to take. Get it in front of a human; social engineering, an ambiguous summary, a prompt injection that makes the model request something plausible-sounding. The human, who sees "the assistant wants to do X, approve?", clicks yes because that is what the button is for. And now their click is the authorization. Not a confirmation of an authorization that already existed, the authorization itself.

The people clicking these buttons all day are support staff. They are not<br>adjudicating whether an action is permitted by policy; they assume the system already did that, and that they are being asked a business question. They are right to assume it. The system is what's lying.

Three outcomes, not two

The fix is not more validation around the approval. It's that a permission check has to have somewhere for approval to fit, rather than a place for it to jump over.

A pending call resolves to one of three things:

Python<br>PRECEDENCE = ("deny", "require_approval", "allow") # most restrictive first

def evaluate(rules, *, tool, source, arguments):<br>matched = [r for r in rules if r.matches(tool, source, arguments)]<br>if not matched:<br>return ALLOW<br>for effect in PRECEDENCE:<br>winner = next((r for r in matched if r.effect == effect), None)<br>if winner is not None:<br>return Decision(effect, rule=winner.name)<br>return ALLOW

Two properties matter here and both are load-bearing.

Combination is order-independent. Whoever authored the rules did not have to think about ordering, because the outcome is the same however the list is shuffled. A policy layer whose behaviour depends on rule order is a hand-written if/else chain wearing a costume, and it will eventually be edited by someone who doesn't know the order matters.

An explicit allow cannot overturn a deny. This is the same stance IAM and Cedar take, and for the same reason: a carve-out is for narrowing something permitted, never for punching through something forbidden.

Now approval has a place to fit; the middle one and only the middle one:

Python<br>decision = evaluate(rules, tool=name, source=source_id, arguments=args)

if decision.effect == "deny":<br># `approved` is not consulted. There is nothing here for it to spend.<br>raise Blocked(decision)

if decision.effect == "require_approval" and not approved:<br>hold(name, args, decision)<br>raise HeldForApproval(decision)

return await adapter.execute(name, args)

The whole claim of the design is in what that first branch does not read. An approval satisfies require_approval and nothing else. If a rule denies the call, a human's yes is not a stronger form of permission that overrides it; it is simply not the question being asked.

Because precedence puts deny above require_approval, a denied call never<br>becomes an approval request in the first place. That has a nice second-order<br>effect on the person reviewing: everything in their queue is genuinely a<br>decision someone authored for a human to make. They are never unknowingly<br>rubber-stamping a misconfiguration.

The flag has to say which decision, not just that there was one

Take the "one lock" claim seriously and a second question falls out: which lock and who handed over the key?

We got this wrong at first. approved was a bare boolean, so it satisfied<br>whatever require_approval rule happened to be asking at execution time, which is not necessarily the rule the person was asked about. Rails get edited. A rule that appears after a call is held is a question nobody has answered, and a bare boolean will answer it anyway.

There is a sharper version of the same bug if you have more than one kind of<br>approval. Ours also asks the customer to confirm before an agent changes their own...