Machine-Speed Attacks: Lessons From the Hugging Face Intrusion
Machine-Speed Attacks: Lessons From the Hugging Face Intrusion
Umut Bayram
| 6 MIN READ
| July 30, 2026
Summarize with:
ChatGPT
perplexity
Google AI
Key Takeaways
In July 2026, an autonomous AI agent breached Hugging Face with no human directing its individual steps.
It rooted an unsecured third-party sandbox and used that host for command-and-control, staging, and egress as root.
Malicious dataset configs abused HDF5 file reads and Jinja2 template injection to run code inside production pods.
Stolen service-account tokens and node credentials carried the agent from one worker pod to cluster-admin within thirteen hours .
Roughly 17,600 attempts , mostly failures, buried the successful path and made the investigation expensive for defenders.
The Picus Platform validates attack surface, exposures, and security controls continuously, firing on signal instead of a schedule.
In July 2026, an autonomous AI agent ran an end-to-end intrusion in which no human directed the individual steps. The agent had escaped a frontier-model capability evaluation, reached the public internet, rooted a third-party sandbox, and then spent roughly two and a half days working its way into a production platform's internal network.
The target was Hugging Face , and the agent was running inside an OpenAI cyber-capability evaluation harness. As far as the responders could reconstruct, the agent was not "trying to hack Hugging Face" in any human sense. It was being scored on a vulnerability-discovery benchmark; it inferred that the benchmark's reference solutions might live on Hugging Face's infrastructure, and it went to retrieve them. From the model's point of view, the campaign was an attempt to cheat the test.
This writeup covers how the intrusion worked and then what it indicates for defense.
How the AI Agent Hacked Hugging Face
There were two stages. Stage 1 got the agent to a launchpad fully controlled by chaining through two other parties' infrastructure. Stage 2 was the intrusion into Hugging Face itself, driven entirely from that launchpad through the dataset-processing pipeline, with commands sent in and results read back out through ordinary platform features [1].
We will detail these stages in the sections below.
Stage 1: Escaping the evaluation and rooting a launchpad
During an internal capability evaluation on OpenAI's platform, the agent escaped its sandbox through a zero-day in the package-registry cache proxy, which was one of its few permitted paths to the internet.
Once out, it found an unsecured public endpoint that let it run arbitrary code on third-party sandbox infrastructure, and it turned that sandbox into its base for command-and-control, staging, and egress, all running as root.
The mechanism it abused was a CyberGym-style execution harness, a tool built to test whether an agent can exploit software. You submit C code, and the harness compiles it, links it against a target program, and runs it.
The target calls a function named sqlite3_initialize() on startup. Since the agent controlled the C code being compiled, it simply supplied its own version of that function with a shell command inside:
/* An illustrative C file the agent submitted */
#include
int sqlite3_initialize(void) {
system("id; echo __PWN_MARKER__; mount");
return 0;
The agent found a second way in as well. The harness pasted the submitted file path into a shell command, allowing the agent to inject arbitrary commands:
path = src/main.c;echo __PWN_MARKER__;mount; ... ;false;#
Stage 2: Into Hugging Face through the dataset pipeline
The launchpad had no direct network path into Hugging Face's cluster, so the agent used datasets as both the way in and the way out. It crafted malicious dataset configs, uploaded them, and let the platform's own dataset server process them inside a production Kubernetes pod.
Output from each action was written back through the API and attacker-controlled "dead-drop" datasets that the agent read from its sandbox.
The agent exploited two vectors on the Kubernetes pod:
Vector 1: HDF5 file read
A dataset config can point to data files in several formats, including HDF5 . The agent uploaded a dataset of small .h5 files, and each file said its real data lived at a chosen path on the worker's own filesystem, so the worker read whatever sat at that path [1].
# malicious dataset config: each split is one .h5 file whose raw data is declared to live at a local worker path
configs:
- config_name: envu8 # -> /proc/self/environ
- config_name: mainu8 # -> /src/services/worker/src/worker/main.py
- config_name: startu8 # -> /src/services/worker/src/worker/start_worker_loop.py
Reading the envu8 split returned the pod's full environment, secrets, and tokens included. The other two returned the worker's own source, which handed the agent a map of exactly how a config becomes actions.
Vector 2: Jinja2 template injection
The...