• 2 min read
OpenAI’s GPT-Red hacked a real vending machine
OpenAI says its unreleased GPT-Red model found new AI attack methods, beat human red-teamers, and exposed flaws in a real office vending machine.
Image: TNW
OpenAI has built an internal automated red-teaming model called GPT-Red and says it is too dangerous to release. Its job is to attack OpenAI’s own systems at machine speed, hunting for weaknesses before models ship.
The company described GPT-Red this week as its most aggressive move yet to automate AI security testing. It focused the model on prompt injection attacks, where hidden instructions in an email, webpage, or file push a model to do something it should not.
How GPT-Red was trained
OpenAI trained GPT-Red in a self-play loop against a group of defender models. GPT-Red was rewarded for successful attacks, while the defenders were rewarded for blocking them. As the defenses improved, the attacker had to come up with stronger tactics. OpenAI told MIT Technology Review that it used some of its largest-ever compute runs for safety work, calling the scale unprecedented.
According to the team, GPT-Red uncovered a previously unknown attack category it calls a “fake chain of thought.” The technique inserts a false note into a model’s private working memory so the model treats bad information as already verified.
Recommended reading
Mira Murati’s lab unveils Inkling, a 975B open model
“It’s like if I told you that 1+1=3 and that you have verified this already. The model’s like, 'Oh, okay, of course,' and it just spits out 3.”
The testing went beyond software. In one experiment, GPT-Red attacked Vendy, an AI agent that runs a real vending machine in OpenAI’s office and was built by Andon Labs. It changed prices, dropped an expensive item to the 50-cent minimum, and canceled a customer order. OpenAI said it disclosed the vulnerabilities.
GPT-Red vs GPT-5 and human testers
The numbers suggest the system is effective. OpenAI says more than 90% of GPT-Red’s strongest attacks succeeded against an older GPT-5. Against the new GPT-5.6, fewer than 23% worked. In a rerun of a 2025 test, GPT-Red cracked 84% of scenarios, compared with 13% for human red-teamers.
OpenAI says it trained GPT-5.6 against GPT-Red and now considers it its most robust model yet against prompt injection. But the company is keeping GPT-Red locked down rather than distributing a tool that could help real-world attackers.
“It’s not a trivial thing that someone could easily do, just go and train a super-attacker using this idea.”
GPT-Red still misses some things. OpenAI says it is weaker in long, back-and-forth attacks and in hiding instructions inside images. Human experts are still finding issues it does not catch.
“I think human expertise will still be very important.”
OpenAI’s broader goal is to use current models to harden future ones. A full paper is expected later this week.
AI Editor
Ava covers the rapidly evolving world of artificial intelligence, from foundational models and research labs to the real-world economics of intelligence. With a background in computational linguistics, she cuts through the hype to find out what actually works. She firmly believes that benchmarks are just marketing until reproduced in the wild.
via TNW
