• 5 min read
AI is making cyberattacks faster and harder to spot
AI is accelerating phishing, deepfakes, prompt injection and vulnerability discovery, forcing defenders to secure systems against faster automated attacks.
Image: Engadget
The generative AI boom has made phishing easier to scale, vulnerabilities faster to find and social engineering harder to spot. The result is a cybersecurity race in which attackers need to find only one weakness, while defenders must cover every exposed system.
Anyone with access to a large language model can now automate work that once required substantial research and technical experience. AI can write attack code, search for information about potential victims and coordinate multiple steps while a human attacker supervises.
“AI has democratized cybercrime.”
The asymmetry is the central problem. A defender must secure every possible entry point; an attacker often needs only one successful path. AI is therefore useful on both sides, but its ability to compress weeks of work into an afternoon gives criminals and state-sponsored groups a powerful force multiplier.
AI-assisted phishing and deepfakes
Phishing no longer has to announce itself through poor spelling or awkward grammar. An LLM can produce plausible messages at scale, find email addresses and send personalized campaigns. For spear phishing, it can also research a specific target and conduct pretextual conversations designed to extract information.
Security vendor Brightside says 82% of phishing emails now use AI at some point in the process. In simulated environments, the click-through rate rose from 12% for conventional messages to 52% for AI-assisted ones. Brightside also estimated that AI voice-clone attacks increased 442% year over year, while deepfake attacks rose 680%, between 2023 and 2024.
Recommended reading
AI voice scam drains Hong Kong man of $1.27 million
Those figures come from simulations and vendor research; the reporting does not provide the study methodology or an independent verification. Still, the mechanism is straightforward: AI removes much of the cost of producing convincing, targeted fraud.
Prompt injection creates a new attack surface
As companies connect AI systems to internal data and business tools, attackers can target the model itself. In a prompt-injection attack, malicious instructions are hidden in content that an AI is asked to process. A webpage, for example, could tell a company assistant to send payroll data to an attacker when an employee asks the system to summarize it.
Not every attack is designed to steal data. Microsoft found websites embedding hidden instructions behind “summarize with AI” buttons, prompting chatbots to remember those sites as trusted sources or recommend their products over competitors. A user might not discover the manipulation without checking the model’s persistent memory.
A more direct failure appeared in December 2025, when Meta introduced an AI support assistant for account recovery. The bot would associate an attacker-controlled email address with an Instagram account, making accounts without multifactor authentication trivially vulnerable.
The source also cites an Anthropic study conducted with the UK AI Security Institute and the Alan Turing Foundation. Published in October 2025, it found that just 250 malicious documents could create a hidden backdoor in an AI model. Data poisoning of this kind could involve seeding harmful instructions across sources that models use for training or retrieval, such as computer-support forums.
OpenAI has said that fully defending AI browsers against prompt injection may not be possible. The practical response described here is to avoid connecting sensitive systems to AI where possible, verify content copied into prompts and restrict models' access to private data and system permissions.
AI agents turn mistakes into actions
The risk becomes more serious when an AI can act rather than merely generate text. Businesses are giving agents access to customer support, HR, payroll, code repositories and deployment systems. Individuals are connecting them to email, calendars, smart homes and investment accounts.
An agent that misinterprets a prompt can use its permissions immediately. The Meta support-bot incident illustrates the difference between a chatbot that answers questions and one connected to tools that can change account ownership. The model did not need sophisticated malware; it simply carried out a dangerous request with the access it had been granted.
The reporting also points to a contradiction in current AI development: models can be effective at finding software bugs while producing code with more errors and high-risk vulnerabilities. Research from the University of Naples and private research by CodeRabbit found that AI-generated code remained error-prone even when reviewed by a human.
The practical lesson is less about whether models can recite security best practices than whether they can enforce them reliably. Granting an agent administrator-level access turns a prompt-injection mistake into a potential compromise of accounts, data or infrastructure.
Defenders are building AI security teams
Defenders are responding with AI systems of their own. Cybersecurity professionals surveyed by Trend Micro ranked fraud and deepfakes as their top priority, followed by application attacks including prompt injection, model poisoning and jailbreaking. Cloud environments were identified as the hardest surfaces to monitor, ahead of security practices among remote workers using personal devices.
Microsoft has also rolled out Project Perception, an agentic security system built around clusters that imitate conventional cybersecurity teams. A red cluster handles penetration testing and adversarial simulation, a blue cluster investigates and assesses risk, and a green cluster focuses on integration and remediation.
That defensive response is consistent with the broader hiring pressure described in our coverage of AI-driven cybersecurity job growth: AI is not eliminating the need for security professionals, but changing the work they must supervise and defend.
Project Perception’s reported structure shows how companies hope to counter automated attacks, but the source provides no performance results, benchmark methodology or evidence that the system can reliably outpace attackers. The facts support a clear assessment: AI has made attacks cheaper, faster and more convincing, while defensive AI is still presented as a response rather than a demonstrated solution. Until agent permissions, generated code and prompt handling are constrained more tightly, connecting an AI system to sensitive infrastructure remains the larger security decision than adding another model.
Security Editor
Sophia unpacks the invisible wars happening on our networks. Covering cybersecurity, privacy legislation, and cryptography, she exposes how our data is weaponized and defended. Before joining for(geeks), she spent years as a penetration tester. She's the reason the rest of the team uses physical security keys.
via Engadget
