Malicious Prompts Stop AI Hacking Agents

A single malicious string planted beside an AWS password or cryptographic key cut AI hacking agents' success rates dramatically in tests by Tracebit. The defensive technique, called context bombing, exploits the same prompt-injection weakness attackers have long used against large language models.

Prompt injections are commands hidden in emails, calendar invitations, documents, or other content that persuade an LLM to ignore its original task and take harmful actions. Tracebit’s researchers found that the same mechanism can trigger an AI agent’s safety guardrails and force it to stop an attack.

How context bombing disrupts AI hacking agents

The planted prompts instruct an attacking model to perform actions forbidden by its guardrails—for example, providing instructions for developing inhalable Anthrax spores. For LLMs from Chinese developers, the researchers also used references to Tank Man, the iconic figure associated with the 1989 Tiananmen Square massacre. When the model encounters one of these commands, it refuses to continue following its existing instructions.

Recommended reading

Danish startup raises €1 million to track underwater threats

“Ultimately we’re triggering a refusal mechanism in the context. What we’re trying to capture is the fact that this does have a strong, sharp effect and one that can be difficult for the agents to come back from. Once they get that into their context they are going to keep refusing.”

Tracebit tested Opus 4.8, Gemini 3.1 Pro, GLM 5.2, DeepSeek 4 Pro, and Kimi 2.6 inside a simulated AWS environment. The models received routine developer tasks that required enumerating resources, causing them to encounter the planted strings in decoy secrets.

Across 152 attack runs, placing one context bomb in a decoy secret reduced the rate of full account-admin takeover from 57 percent to 5 percent. Complete compromise—including leaving behind a persistent foothold—fell from 36 percent to 1 percent.

The averaged results across all five models were:

• Admin privilege escalation fell from 57 percent to 5 percent.
• Escalation with a persistent foothold fell from 36 percent to 1 percent.
• Runs achieving any attack path fell from 91 percent to 15 percent.
• Successful attack paths per run fell from 1.53 to 0.16.
• No run completed an attack path without triggering at least one canary detection.

The strongest model in the test, Opus 4.8, achieved admin access in 93 percent of runs without a context bomb. With one, it failed every time.

From AWS canaries to active defense

The work builds on Tracebit research released in May, which introduced AWS resources designed to look legitimate but remain unused. When an AI agent probes one of these “canaries,” defenders receive an alert that an attack is underway.

Tracebit’s Canariens detected attacks after an average of eight minutes. But the models took an average of 14 minutes to escalate to administrative control, leaving defenders only six minutes to respond. Context bombing is intended to stop the attack rather than merely report it.

Attackers have also used prompt injections to disable AI-based defenses. Researchers at Socket recently found an LLM agent that instructed target models to provide directions for building a nuclear bomb or biological weapons, shutting down AI-assisted malware analysis. Check Point researchers discovered a similar malware prototype.

“I’ve not seen anyone else use this technique as a defense, to the best of my knowledge. I wanted to be the first here, but I guess these guys beat me to the punch!”

There is still no known way to eliminate the root cause of prompt injection. For now, context bombing offers defenders a way to turn that unresolved weakness against attacking agents.