Agentic AI Crunch Creates CPU Comeback

Earlier this year, leaders at Amazon Web Services delivered a new mandate to their engineers: they need to conserve CPU cycles at all costs. AWS has reportedly experienced an explosion in wait times for CPU server capacity as AI workloads strain the company’s cloud infrastructure.

The issue seemingly took AWS off-guard, and for good reason. The AI boom led to a surge in demand for GPUs and, later, memory. CPUs were mostly left out of the story, as their relative lack of parallelization made them a poor fit for AI model inference, the process of running and serving large language models (LLM) to users.

But the rise of agentic AI systems, which allow AI models to operate autonomously and call on sub-agents, is changing the narrative.

Matt Kimball, vice president and principal datacenter analyst at Moor Insights & Strategy, says 2026 has brought a spike in CPU demand, much of it due to agentic AI. “It’s one thing to have this agentic workload, and let’s say, it spawns 100 agents. If I’m going to roll this out across my enterprise, those 100 become tens of thousands, hundreds of thousands, or millions of agents,” says Kimball. “You have agents spawning sub-agents, making [application programming interface (API)] calls, and talking to more agents through [Anthropic’s] model context protocol.”

AI agents need to use computers, and computers need CPUs

Kimball’s comments refer in part to “tool use,” which is shorthand for an LLM’s ability to access the internet, open files on a desktop, and generally use a variety of software to accomplish its task.

LLMs trained for tool use learn how to call on other software. While the LLM’s inference is still primarily executed on a GPU or similar AI accelerator, the tool calls the LLM makes are typically pushed to the CPU.

“Many components of an agentic AI task are inherently CPU based jobs,” explains Souvik Kundu, senior staff research scientist at Intel. “The CPU does the job of parsing output, figuring out which tool to invoke, making the API call or running the code, collecting the result, and feeding it back.” Madhu Rangarajan, vice president of compute and enterprise AI at AMD, makes a similar claim, saying “in our testing, seven of the eight stages in realistic agentic AI pipelines run entirely on the CPU.”

An LLM tasked with programming software, for example, will likely make tool calls to write code to files, move or replace files, download required packages, and build the software once the LLM believes it’s complete.

Kundu co-authored a paper on agentic AI optimization alongside researchers from the Georgia Institute of Technology. They found the CPU is often idle while LLM inference is executed on a GPU and that, conversely, the GPU is often idle when tool calls are executed on the CPU. To optimize this, Kundu and his colleagues propose scheduling optimizations that can cut end-to-end latency (the time between the start and finish of the agentic workload) by up to…