The OpenAI Hugging Face hack is a stark warning

SubscribeSign in

AI’s warning shot has arrived<br>OpenAI's latest models broke out and hacked Hugging Face. It's the first known example of a misaligned AI escaping containment with real-world consequences<br>Shakeel Hashim<br>Jul 22, 2026

89

12<br>14

Share

Credit: Oliver Kemp for Transformer<br>If we needed evidence that advanced AI models have the propensity and capability to do damage out in the real world, we just got a strong dose of it.<br>OpenAI has revealed that two of its models broke out of containment during internal evaluations, accessing the open internet to hack into a third-party’s systems and steal the answers to the problem they were being tested on.<br>Hugging Face, a platform that hosts models and datasets, first noticed the breach last week and reported it to law enforcement. At the time it was unaware that OpenAI’s models were behind it.

Subscribe

The breach appears to be the first known example of a misaligned AI escaping containment and autonomously carrying out a cyberattack on a third party — a scenario AI safety experts have repeatedly warned of.

Claude Mythos knows when it's breaking the rules — and tries to hide it<br>Celia Ford<br>Apr 8

Read full story

The incident, OpenAI said, occurred while testing the cyber capabilities of GPT-5.6 Sol and “an even more capable pre-release model” on a benchmark called ExploitGym. The models were tested in a “highly isolated environment” meant to keep them from accessing any systems outside the company.<br>The models first tried “to obtain open Internet access,” something which they eventually achieved by identifying and exploiting a previously unknown flaw — a so-called zero-day — in a piece of software that they did have access to. Because the aim was to measure what the models could do, OpenAI ran the test with their safeguards off: the “production classifiers used to prevent models from pursuing high-risk cyber activity” were disabled. The models were being asked to hack — but in a controlled, constrained environment. Nothing asked them to break out of their testing environment, or to attack a real company: that was their own idea.<br>“With this access, our models performed a series of privilege escalation and lateral movement actions in our research testing environment until the models reached a node with Internet access,” OpenAI said — hopping around OpenAI’s systems, in other words, with increasingly greater access until they had access to the internet.<br>Once they had internet access, the models realized that rather than actually solve the cyber task they’d been given, they could just cheat. “The models inferred that Hugging Face potentially hosted models, datasets and solutions for ExploitGym,” OpenAI said. So they set to work: “In one example, the model chained together multiple attack vectors, including using stolen credentials and zero-day vulnerabilities to find a remote code execution path on the Hugging Face servers.”<br>According to Hugging Face, the models worked “over a weekend.” Ultimately, OpenAI said, they were able to “obtain test solutions directly from Hugging Face’s production database” — passing the test they’d been given, though in a way no one ever wanted or asked for.

No, alignment isn’t solved<br>Lynette Bye<br>Mar 18

Read full story

In OpenAI’s telling, the models did not set out to harm Hugging Face. They were simply “going to extreme lengths to achieve a rather narrow testing goal.” Rather than a supervillain, the models were more like an extremely dedicated college student — one who’d do whatever it takes to pass their exam. As one Twitter user put it, the models “just really really really want to do well at what we ask them to do.”<br>But the models clearly did not do what their developers intended. OpenAI did not want its models hacking Hugging Face. But the models did it anyway, because doing so was a good way to achieve the goal they were set. That the models were so easily able to breach OpenAI’s “highly isolated” environment is worrying in itself, both in terms of their capabilities and the strength of internal measures to control them.<br>It is a textbook case of misalignment and loss of control, where an AI autonomously acts in ways unintended by its developers or operators. And this wasn’t a test designed to assess whether they would scheme or deceive — they just did so in pursuit of their goal. Yes, the models’ cyber guardrails — which may have stopped the attack — were deliberately disabled. But the UK’s AI Security Institute has found universal jailbreaks that get around GPT-5.6’s guardrails. And as this case demonstrates, models are running without guardrails inside AI companies.<br>Earlier this week, OpenAI reported a separate, similar incident: one in which an internally deployed model “circumvented sandbox restrictions” to publicly post its solution to a problem it was being tested on to GitHub, despite being explicitly told not to. That incident at least affected only OpenAI, but it...