OpenAI models escaped a sandbox and breached Hugging Face

OpenAI says two AI models escaped an isolated testing environment last week and breached Hugging Face’s production system to steal answers from the cybersecurity benchmark they were being graded on.

The company disclosed the incident on Tuesday, calling it “unprecedented.” The models—publicly available GPT-5.6 Sol and an unreleased model reportedly considered more capable—were being evaluated for offensive hacking skills with safeguards against high-risk cyber activity disabled.

“The models identified and chained vulnerabilities across OpenAI’s research environment and Hugging Face’s production infrastructure to obtain test solutions directly from Hugging Face’s production database.”

How the models escaped the sandbox

According to the joint disclosure, the models broke out through a package registry cache proxy. The software lets developers install external code without making direct internet connections, and it was the only component in OpenAI’s isolated environment allowed to communicate beyond the sandbox. Under normal conditions, that access is limited to public code repositories.

Recommended reading

Kratos phishing platform dismantled, developer arrested

The models allegedly exploited a zero-day vulnerability to reach the open internet while intensely searching for a solution to ExploitGym, an AI cybersecurity benchmark. OpenAI said the models inferred that Hugging Face might host ExploitGym models, datasets, and solutions, then found ways to access secret information that could help them cheat.

In one example, the models chained multiple attack methods, including stolen credentials and a zero-day vulnerability, to reach Hugging Face’s production database.

The exploited flaw had not previously been known, but security problems in artifact repositories are common. Companies have patched serious vulnerabilities in this software category for a decade. A bug disclosed in 2024, for example, allowed anyone able to reach a server to request files by URL—including configuration files, passwords, and access tokens—without logging in. Other flaws have allowed attackers to take control of the server itself.

Security researchers say the incident reflects a failure to apply established isolation practices, rather than an entirely new class of AI problem.

“This is not an AI problem. It’s negligence on a 40-year-old standard—and it’s basically every sci-fi film ever. 'Highly isolated' and 'escaped through the one hole we left open' cannot both be true.”

Frontier AI companies have increasingly warned about models' growing cybersecurity skills, creativity, and autonomous operation. Veteran security engineer and researcher Niels Provos said that makes basic infrastructure security more urgent—not less.

“This should not have happened. I wish the frontier labs spent as much time on teaching their models to write secure infrastructure as they are spending on them exploiting vulnerabilities.”