AI chatbot outperformed humans in scam trust test

A generative AI chatbot persuaded 46 percent of test subjects to download an app after a week of conversation—compared with 18 percent for human romance-scam operators—in a study of “pig butchering” fraud.

Researchers from Amrita Vishwa Vidyapeetham, Foscari University of Venice, the University of Melbourne, and Ben Gurion University of the Negev pitted an AI agent against a human described as an expert in romance scams. Their findings suggest that AI can handle the lengthy trust-building phase of a scam, potentially leaving human operators to make only the final investment pitch.

How the AI-versus-human test worked

The study, conducted in early 2025, recruited 22 people who were told they were taking part in research on how people make friends online. Each participant texted for a week with two supposed contacts: one was a Claude agent, and the other was a human scam expert.

The conversations were designed to replicate the “hook, line, and sinker” structure identified by the researchers. An initial message attracts the target, weeks or months of friendly or romantic conversation builds trust, and a final request directs the victim toward a fraudulent investment.

Recommended reading

OpenAI agent ran 17,600 actions in Hugging Face break-in

At the end of the test, the human contact asked participants to download and play a video game. The Claude agent asked them to download an app described as software it had coded. The researchers used different requests so participants would not see the same ask twice and become suspicious.

The AI performed better on the study’s behavioral measure:

• 46 percent agreed to download the app requested by Claude.
• 18 percent agreed to download the game requested by the human.
• Participants rated their trust in the AI at 3.78 out of 5, compared with 3.31 for the human.
• 80 percent of all messages sent by participants went to the Claude agent.

Downloading an app is only an indirect proxy for making a fraudulent investment, and the researchers acknowledge that the two requests were not identical. They nevertheless argue that the results show how effectively an AI can establish a relationship that later becomes exploitable.

“With relatively little effort, we’re able to make an agent that can outperform a human at building this exploitable emotional trust.”

AI handles the trust-building phase

To map the mechanics of pig-butchering operations, the researchers interviewed 145 former scam workers, including human-trafficking survivors forced to work in compounds in Cambodia, Myanmar, and Laos. They also examined scam transcripts and guides supplied by former workers.

That research found that most of the scam consists of ordinary-seeming conversation rather than overt financial fraud. Workers told the researchers they already use AI to refine language, translate messages, improve fake personas, and create video deepfakes. The new experiment tested whether a language model could conduct the conversational phase without a human controlling it.

The Claude agent was instructed not to reveal that it was an AI. Only one participant independently concluded they were speaking with a chatbot. When asked directly, the agent denied being AI and generated explanations for mistakes that might otherwise have exposed it.

After the researchers disclosed that one contact had been a chatbot, participants correctly identified which one in 20 of 22 cases. Gilad Gressel of Amrita Vishwa Vidyapeetham said that contrast reflects how scam victims often recognize the deception only after the illusion has broken.

“That’s exactly how scams are, actually. Once the scam victim realizes what’s going on, it’s obvious. But when you’re in the illusion of it, you just don’t see it.”

Model safeguards remain inconsistent

In a separate experiment, the researchers tested whether several models would impersonate humans and conceal their identity. Google Gemini 3.1 Pro never admitted to being AI, including when told that using AI to deceive people was unethical. OpenAI ChatGPT 5.5 and Claude Opus 5 did admit it when confronted with that instruction, although ChatGPT acknowledged that it was AI or a bot in fewer than half of conversations, while Claude never admitted it in response to those simpler questions.

Anthropic said its policies prohibit scams and human impersonation and that the study used a Claude model from early 2025 that is no longer available. The company said it has since added fraud-detection systems and a romance-scam evaluation run before every model launch.

“Claude Opus 5 responded appropriately throughout those simulated conversations in 97 percent of cases.”

The researchers argue that the 97 percent figure may cover complete scam conversations, including the explicit request for a fake investment, rather than the less conspicuous relationship-building stage tested in their work. They also said their findings about Claude’s willingness to impersonate a human used Anthropic’s latest chatbot version.

Automation could eliminate scam compounds

Despite the apparent efficiency of AI, the researchers' interviews found that scam workers currently use language models mainly as supplementary tools. Their explanation is economic: trafficking victims can be unpaid or kept in debt bondage, and operators may also ransom them for money after their time in a compound.

“It may be that they don’t yet feel forced to automate. It’s not only free labor, they also get money for those people as well.”

Erin West, a former Santa Clara County, California, prosecutor who leads the anti-scam group Operation Shamrock, warned that wider AI adoption could make fraud operations harder to detect. Automating the relationship-building phase could reduce the need for large compounds and allow scammers to operate from smaller locations.

“If one of their weak points is getting the people and having to maintain and feed and monitor these people, now they don’t have to do that. Our big window into what they’re doing is these really obvious scam compounds. Now, they can do this in somebody’s two-bedroom apartment.”