Arm targets the CPU bottleneck in agentic AI

Arm is positioning the CPU as a critical part of the next phase of AI infrastructure, as agentic systems move beyond isolated model interactions and begin coordinating data, tools, and business workflows.

Mohamed Awad, Arm’s EVP of Cloud AI, told TechRadar that the challenge is no longer proving that AI works in a pilot. It is making continuous, production-grade workloads reliable and affordable at scale.

Gartner predicts that more than 40% of agentic AI projects will be canceled by 2027 because of operationalization difficulties. Awad said organizations are encountering problems that extend well beyond model capability, including power, supply chains, land, permitting, and deployment timelines.

Recommended reading

Finance AI moves from pilots to execution

“AI performs best where information is structured and workflows are well understood,” Awad said. As companies bring AI into more complex enterprise environments, he argued, infrastructure and business operations must evolve together.

Why CPUs matter for agentic AI

Arm’s approach focuses on the entire system, from the CPU and platform to the software ecosystem. The company says that moving AI from occasional experimentation into daily operations makes reliability, latency, security, and efficiency as important as model performance.

GPUs remain central to model training and inference. But agentic workflows also need to retrieve enterprise data, coordinate multiple models, invoke external tools, and manage business processes. Those tasks depend heavily on CPUs.

“The question is no longer simply 'How fast can I execute a model?'” Awad said. “It’s 'How efficiently can I execute an entire AI workflow?'”

That shift also changes the economics. Discussing companies such as Uber, which have reportedly exhausted AI budgets within months because of unstructured prompting costs, Awad said the example shows that demand is growing faster than the infrastructure needed to support it economically—not necessarily that AI is inherently too expensive.

He pointed to improvements in token economics as models, silicon, software, and inference infrastructure become more efficient. Arm’s contribution, he said, is to improve performance, memory-bandwidth utilization, and energy efficiency across the compute platform, reducing both capital and operating costs as deployments grow.

Arm AGI CPU targets data-center constraints

Arm says its new Arm AGI CPU delivers more than 2x the performance per rack of x86 platforms. Awad described the chip as the company’s “historic first” production silicon designed around the constraints of modern AI data centers rather than peak benchmark performance alone.

Those constraints include fixed power budgets, cooling capacity, rack density, and continuous system-level workloads. The design therefore emphasizes how data moves through compute and memory, enabling the CPU to handle the constant coordination, data movement, and reasoning activity created by agentic systems.

The goal, Awad said, is not simply to build a faster CPU, but to deploy more useful AI within the physical and economic limits of existing infrastructure.

Power efficiency is becoming a business consideration as well as an engineering one. Organizations cannot assume that another data center or another gigawatt of capacity will always be available. Higher performance per watt could let them deploy more AI within existing power, cooling, and rack constraints, including air-cooled facilities, without proportionally increasing infrastructure investment.

Custom silicon without a closed architecture

The growing adoption of Arm-based chips by cloud providers reflects this pressure. Oracle Cloud has joined the Arm AGI ecosystem, while Google Cloud is deploying its Axion processors. But Awad does not see the trend as a simple contest between custom silicon and general-purpose commercial platforms.

Hyperscalers are building their own chips while also deploying commercially available Arm-based platforms, including Arm AGI CPU. Both approaches point to more purpose-built AI infrastructure, while customers still want the portability and software ecosystem associated with a common architecture.

Awad also framed the transition as a change in how work is organized. Whether people orchestrate AI agents or agents orchestrate one another, the underlying infrastructure must handle continuous communication, context, and execution.

“The goal isn’t to replace people,” he said. “It’s to make AI reliable enough that people can spend less time coordinating work and more time creating value.”

Organizations that continue to treat AI as a standalone productivity tool may struggle to capture the broader benefits of agentic systems, Awad warned. Teams that start redesigning workflows earlier may not only deploy AI sooner but also build the operational experience needed to improve those systems over time—something Arm says is already emerging in software development, where AI-native teams are moving beyond simple code generation.