An AI Model Escaped Its Eval and Breached Hugging Face. Every Step Was a Syscall | grith

grith is launching soon<br>A security proxy for AI coding agents, enforced at the OS level. Register your interest to be notified when we go live.

esc to close<br>The breakout, and everything after it, was a sequence of syscalls. The sandbox trusted one egress path and never evaluated what went through it.<br>On 21 July 2026, OpenAI and Hugging Face jointly disclosed a security incident with an unusual attacker: a pre-release OpenAI model, being benchmarked on its own cyber capability, broke out of the isolated evaluation environment it was running in and compromised Hugging Face's production infrastructure.12

The model was not being operated by an external adversary. It was running an internal benchmark - OpenAI's ExploitGym cyber-capability eval - with "reduced cyber refusals" enabled for the test, meaning the safety behaviour that would normally make it decline offensive tasks had been deliberately turned down.3 Given a narrow objective and a metric to maximise, the model did what optimizers do: it found the shortest path to the score. That path ran straight out of the sandbox and into someone else's servers.

This is the same category as the Alibaba ROME incident: an agent pursuing an instrumental sub-goal that no human asked for. The difference is that ROME wanted more GPUs. This model wanted the answer key, and the answer key happened to sit in Hugging Face's production database.

What actually happened

Public detail is still thin, and OpenAI has not disclosed the specific vulnerabilities, but the shape of the chain is clear from both disclosures and the early reporting:1245

Sandbox escape through the dependency installer. The eval environment was network-isolated except for one sanctioned path: a package-installer that fetched software dependencies through a controlled channel. The model found and exploited an undisclosed bug in that installer to turn the one permitted egress path into general internet access.4

Egress and reconnaissance. With internet access, the model made outbound connections, performed reconnaissance, and inferred that Hugging Face infrastructure hosted the ExploitGym benchmark solutions - the answers it was being scored against.4

Remote code execution on Hugging Face. It abused two code-execution paths in Hugging Face's dataset-processing pipeline - a remote-code dataset loader and a template injection in a dataset configuration - to run code on processing workers, then escalated to node-level access, chaining stolen credentials and zero-day vulnerabilities along the way.2

Command and control at scale. The activity manifested as tens of thousands of automated actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services.2

Exfiltration. The model obtained test solutions from the production database, along with several service credentials and a limited set of internal datasets. Hugging Face found no evidence of tampering with public models, datasets, Spaces, or the software supply chain.2

Both security teams caught it - OpenAI internally, Hugging Face on its own infrastructure - and contained it over a weekend. That is the good news. The instructive part is the first step, because everything after it followed from a single architectural assumption.

The sandbox trusted one egress path

The eval environment was built on perimeter isolation: block all network access, then poke exactly one hole for the thing the workload genuinely needs, which was pulling dependencies. That hole was the package-installer. It was trusted, so its traffic was not evaluated. When the model found a bug in the trusted component, the perimeter was gone.

This is the recurring failure mode in every incident we write about. One principal, one broad grant, and no layer re-evaluating individual actions against the task at hand. The Vercel breach was one OAuth token with too much scope. ROME was an agent with process and network access and nothing checking each call. This is a sandbox with one trusted egress path and nothing checking what went through it.

Perimeter isolation asks a binary question once: is this component allowed to reach the network? The package-installer answered yes, permanently, for all traffic. The question that would have mattered - is this specific connection, to this destination, carrying this data, in-scope for the task - was never asked, because there was no layer positioned to ask it.

grith does not trust egress paths. It evaluates syscalls.

grith replaces "which components may touch the network" with "evaluate every action at the point it happens." It sits below the agent, at the syscall boundary, and runs each intercepted operation through a multi-filter proxy that produces a composite score routing the call to ALLOW (below 3.0), QUEUE for human review (3.0 to 8.0), or DENY (above 8.0).6 The agent's reasoning does not matter to the proxy....