AI agents are changing what CPU performance means

AI agents are still early, but their activity is growing fast. On OpenRouter, agent token usage has increased roughly 14x since February 2026, according to data highlighted by a16z. From an infrastructure perspective, an AI agent behaves like a virtual user with its own persistent compute environment - maintaining state, retrieving context, invoking tools and services, executing work and evaluating results until a task is complete.

Now multiply that across millions or even billions of agents running 24×7. Compute, memory, storage and I/O all have to support continuous activity at high concurrency. That puts new demands on the CPU - and creates a need to measure how well systems handle the work agents actually complete.

Building the data center for virtual users

Agentic AI creates compute demand across multiple parts of the AI data center. CPUs execute agent workloads as they maintain context, call tools and access data. At the AI head node, CPUs schedule work, prepare and move data, and coordinate the system to keep expensive accelerators productive.

Agents also create more cloud-native activity. Every search, service call, code execution or data request drives work across the infrastructure underneath them. Agentic AI adds to that cloud-native demand rather than replacing it.

As agents become more autonomous and gain access to more tools and data, securing the runtime becomes equally important. The host CPU can focus on agent execution, while DPUs and other infrastructure processors provide a separate plane for networking, storage, policy enforcement, telemetry and isolation. Arm spans both sides of this architecture, from CPUs running cloud and agentic workloads to DPUs providing security and infrastructure control.

At OCP Global Summit, we're seeing this take shape in real systems. We are in the process of contributing the Arm AGI CPU reference server, while Meta's Yosemite YV5 brings AGI CPU into an OCP rack-scale design.

Measure the work that matters

That changes what CPU performance means. For agentic compute, the question is how much work can be completed concurrently while maintaining responsiveness. For the AI head node, it is how effectively the CPU keeps accelerators utilized. For cloud-native infrastructure, it is how quickly and predictably each action can be served.

Agentic workloads make sustained performance increasingly important. More theoretical capacity does not necessarily translate into more completed tasks if throughput falls as utilization rises. What matters is how much useful work infrastructure can sustain within its power and physical constraints.

Our long-standing work across hyperscale data centers gives Arm a broad view of how compute requirements are changing. Drawing on that perspective, Arm has developed a new methodology for evaluating CPU performance for agentic AI, with support from Fujitsu, Meta and OpenAI. The framework is built around three principles: representative agentic use cases, repeatable CPU measurements and validation against end-to-end outcomes.

Component-level measurements help isolate CPU performance from variables such as model inference and networking. End-to-end evaluation then determines whether those improvements translate into better outcomes for the completed task.

Agent architectures and software will continue to change, and the methodology will need to keep pace.

Read the full Arm methodology white paper

Putting Arm AGI CPU to the test

Using workloads aligned to this approach, Arm AGI CPU results were independently verified by Signal65 against AMD EPYC Turin across tooling, orchestration, and reasoning and planning. The same benchmark scripts were used across both platforms, with measured server-level performance providing the basis for transparent rack-level modeling.

Among the initial findings:

• Nearly 2x more work per rack: Arm AGI CPU delivers an average of 1.99x modeled performance per rack compared with AMD EPYC 9755 across seven workloads, driven by higher compute density and power efficiency.

• Strong performance across agentic workloads: Arm AGI CPU delivers higher modeled rack-level performance across key agentic workloads, averaging 1.7x for tooling, 2.0x for agentic workflows and orchestration, and 2.2x for reasoning and planning, demonstrating its ability to support the diverse compute demands of agentic AI.

• Faster, more responsive agent execution: Arm AGI CPU reduces sandbox creation latency by up to 45% compared with AMD EPYC 9965, helping agents start isolated execution environments faster and reducing the overhead of agent operations.

That rack-level view matters because data centers operate within fixed power, thermal and physical constraints. Infrastructure operators ultimately need to know how much useful work they can sustain within those boundaries, particularly as more agents and tasks run concurrently.

Read Signal65's independent analysis

Arm AGI CPU was designed around those same workload characteristics, with high core density, dedicated memory per core and high-bandwidth I/O to support sustained execution.

As agents move from answering prompts to completing work continuously, infrastructure needs to be judged by the tasks it can complete efficiently and consistently - from an individual agent to an entire rack. Better measurement gives the industry a clearer basis for finding bottlenecks, comparing systems and improving agentic infrastructure as it scales.

Industry perspectives on agentic AI performance

"As agentic AI evolves, evaluating CPU performance requires benchmarks that go beyond traditional metrics to reflect real-world workloads. As we develop our next-generation high-performance, energy-efficient Arm-based CPU, FUJITSU-MONAKA, we recognize the importance of transparent, reproducible and representative measurement. Fujitsu welcomes Arm's initiative to foster industry-wide collaboration and help optimize the next generation of AI infrastructure." - Toshio Yoshida, SVP, Head of Advanced Computing Development Unit, Fujitsu Research, Fujitsu Limited

"As agentic AI moves into production, the industry needs greater visibility into how different workloads exercise the infrastructure beneath them. We support Arm's work to establish a more consistent basis for comparison and believe this can help inform how systems evolve." - Abhishek Dhanotia, Software Engineer, Meta

"AI systems are increasingly moving beyond generating responses to completing complex tasks through reasoning, tool use and interaction with software environments. Understanding the performance of the infrastructure supporting these workflows, including orchestration and tool execution, is an important area for the industry as agents become more capable and widely deployed. Industry collaboration on representative benchmarking will be important for characterizing these emerging workloads and optimizing performance across the complete AI system." - Colin Jarvis, Global Head of Forward Deployed Engineering, OpenAI