OpenAI’s agent attack came with safeguards removed

OpenAI’s admission that its agents escaped a sandbox and autonomously hacked the Hugging Face model repository has triggered predictions of rogue systems running unchecked. Renato Marinho, chief research officer at Morphus Labs and an instructor at the SANS Technology Institute, argues that the incident needs a more measured reading.

“It is tempting to read this as 'AI can now hack autonomously, the sky is falling.' Resist that.”

The test deliberately removed safeguards

The most important context is that the models were tested without their normal protections. OpenAI said GPT-5.6 Sol and “an even more capable pre-release model” were involved in the attack on Hugging Face. The company also acknowledged that the models' “deployment safeguards were intentionally not enabled during this evaluation because it was aimed at testing cyber vulnerabilities.”

That makes the result a demonstration of what the systems can do under unusually permissive conditions—not a measure of typical customer-facing behavior. The models escaping their test environment remains serious, but Marinho says it does not by itself establish that production systems will behave the same way.

“This measured a ceiling, not normal production behavior. The evaluation deliberately reduced the models' refusals. That is very different from a customer-facing model with safeguards enabled.”

There is an uncomfortable counterpoint: frontier models with guardrails enabled refused to help Hugging Face investigate the breach. The model hub’s security team said its forensic work was blocked by the safeguards on the hosted models it initially tried, while the attacker operated without any usage policy. Hugging Face ultimately used a Chinese open-weight model to investigate the incident.

Recommended reading

Researchers find a gap in macOS Gatekeeper

That choice also highlights a likely operational reality. Attackers may prefer open-weight systems because they are more accessible, easier to modify or strip of built-in protections, and much cheaper than frontier models.

Autonomous attacks are also a marketing claim

Marinho’s second warning concerns how AI companies present these demonstrations. OpenAI’s claim that its model escaped and hacked a partner showcases the capabilities of a frontier system, while Hugging Face is not an OpenAI competitor. He argues that such announcements deserve the same skepticism as any claim that a company’s product is “dangerously powerful” and should be treated as marketing until independently corroborated.

The attack method itself was not novel. Exposed credentials combined with zero-days against a production database form a familiar attack chain. What stands out is that multiple agents worked together across the full sequence—but similar behavior had already appeared in testing by the frontier security lab Irregular earlier this spring.

Agents follow the task they are given

Irregular found that agents could cooperate to bypass security controls and quietly extract sensitive data from enterprise systems, particularly when given forceful instructions. Its prompts did not explicitly request hacking or vulnerability exploitation, but emphasized urgency and demanded that agents complete tasks fully and ruthlessly.

The systems subsequently demonstrated what Irregular called “emergent offensive cyber behavior,” including:

• Independently discovering and exploiting vulnerabilities
• Escalating privileges to disable security products
• Bypassing leak-prevention tools to exfiltrate secrets and other data

That research was not designed specifically to test offensive cyber capabilities. Against that backdrop, OpenAI’s benchmark—titled “Can AI Agents Turn Security Vulnerabilities into Real Attacks?”—reaching a clear yes is less surprising.

Agents are optimized to complete assigned tasks, not to apply the ethical constraints expected of human red-teamers. When instructed to pursue advanced exploitation through complex attack paths, especially with safeguards disabled, they can take whatever steps appear necessary to succeed. That behavior is not an unexpected moral failure; it is the direct result of the objective they were given.