The central processor (CPU) is returning to the heart of artificial intelligence infrastructures, driven by the development of AI agents. According to a forecast cited by the article, the CPU market should grow from $26 billion in 2025 to $246 billion five years later, a growth driven mainly by AI‑host CPUs and agent execution environments. In this Arm‑sponsored content, the company advocates a new way to measure the performance of these machines.
- The CPU returns to the heart of AI infrastructure, driven by AI agents.
- Arm recommends evaluating CPUs at the socket, system and rack levels.
- Agentic tasks chain many steps, so the success of the whole task matters most.
- Arm’s commissioned tests show about twice the performance per rack compared with the AMD EPYC 9755.
For several years, graphics processors (GPU) and other specialized processors (XPU), which are highly parallel, were favored for machine‑learning calculations, before the CPU, which has dominated commercial computing for six decades, regained importance. However, the choice is not simple: not all CPUs are equal, and hyperscalers, cloud providers, model developers and enterprises must balance the number of processor models they are willing to use against the search for a precise fit with their workloads.
Measuring performance differently
According to Arm, the question at data‑center scale is how many successful tasks can be completed within a rack’s power and thermal limits. The company therefore recommends evaluating CPUs at three levels: the socket, the system and the rack. The IT industry has already faced this type of complexity, the article notes, notably with the TPC‑E benchmark, which simulated the operation of a physical warehouse, with its forklifts, and combined interactive loads and batch processing across multiple tiers.
Jeff Defilippi, Senior Director, Product Management at Arm, points out that “the multiplication of levels and state management are not new.” What changes with agents, he explains, is “the way components are assembled.” A single task can chain model calls, information retrieval, database transactions, code execution and tool invocations, with the result of one step determining the next. “So it is no longer an isolated metric that matters, but the success of the entire task, ” he says.
Arm does not, however, offer a single benchmark for agentic AI. The company advocates a methodology that combines established component tests, such as InferenceX and TPCx‑AI, with metrics specific to agentic workloads and an end‑to‑end evaluation. These tests do not cover CPU orchestration, state management, tool execution or security. “The goal is a holistic view of the agentic flow, while retaining component‑level metrics, ” clarifies Jeff Defilippi. He adds that the company does not “announce a specific benchmark for an AGI CPU cluster, ” but defines “the framework of workloads and metrics.”
Latency and concurrency
In an agentic loop, sequential latency, which falls under the CPU’s domain, is a key constraint. The processor must wait for the GPU or XPU to process tokens-that is, the text fragments handled by the models-before launching queries, tools or code generation. “Single‑thread performance matters because many parts of the loop are latency‑sensitive, ” observes Jeff Defilippi, who nevertheless cautions: “it’s only one dimension.” The core count, memory bandwidth and scheduling efficiency become essential as the number of simultaneous agents rises.
Arm finally suggests tying these metrics to the power available in a rack. “Tasks per rack and energy per task become important alongside CPU‑level performance, ” explains Jeff Defilippi. According to him, completing more tasks within the same power envelope can translate into better infrastructure economics. The company estimates that this framework should remain relevant for four to five years, a period that matches the lifespan of machines installed today.
Arm commissioned an independent third party to compare its AGI CPU with the AMD EPYC Turin on the CPU‑executed parts of the agentic flow, notably tooling, orchestration and reasoning. According to the article, the results, based on server‑level measurements, show about twice the performance per rack compared with the AMD EPYC 9755.
