The AI hardware story has been dominated by GPUs because training and inference are dominated by matrix math.
Agents are exposing a different bottleneck.
An autonomous coding or research agent does not spend every second inside a neural network. It installs packages, searches files, compiles code, launches processes, manipulates repositories, runs tests and waits on tools.
That work lands heavily on CPUs.
Modal, an AI infrastructure company that runs large numbers of isolated sandboxes for agents and reinforcement-learning workloads, published early benchmarks on October 19 showing Nvidia’s new Vera CPU delivering as much as twice the concurrent task throughput of an x86 comparison system under heavy load.
That is a much more useful proof point than a generic CPU benchmark because it tests the messy workload agents actually generate.
Agent Workloads Are Bursty and Hard to Size
A traditional server workload is often predictable enough to provision around steady CPU, memory or I/O needs.
An agent sandbox can sit idle while waiting for a model and then suddenly run an npm install, compile a project, search a large repository or execute a test suite.
Modal says hundreds of those heterogeneous workloads can share one host, creating contention patterns that are difficult to model in advance.
The infrastructure problem is therefore not only making one task fast. It is keeping many unpredictable tasks fast at the same time.
Vera’s Advantage Appeared Under Contention, Not in Every Single Task
Modal’s engineers explicitly say Vera did not dominate every serial benchmark.
On quiet-host, straight-line task latency, the CPU was often faster but not always by a dramatic margin.
The difference appeared when more concurrent containers were packed onto the system.
Modal reports that Vera continued increasing throughput while alternative systems plateaued earlier, producing up to about twice the task throughput of the compared x86 platform without the usual latency deterioration.
That nuance matters because it avoids the lazy conclusion that Vera is simply ‘2x faster.’ The reported advantage is workload- and concurrency-dependent.
The CPU Is Keeping Expensive GPUs From Waiting
There is a second-order economic effect.
In reinforcement learning, GPUs can sit idle while CPU-based environments execute code and return results. In production agents, the model can finish reasoning and then wait for a sandbox to complete an external task.
A faster agent CPU can therefore improve utilization of much more expensive accelerator infrastructure.
Nvidia has been making exactly that argument since Vera launched: CPUs increasingly sit on the critical path between model calls, tool execution, data processing and simulation.
This Is Why Nvidia Built a CPU Around Agents
Vera uses 88 custom Olympus cores and a high-bandwidth memory subsystem designed for sustained per-core performance under load.
Nvidia says the processor targets agentic AI, reinforcement learning and data-processing workloads rather than competing only on conventional enterprise-server core count.
Optimisus previously covered Nvidia’s data-center business reaching $89 billion in a single quarter as AI infrastructure demand broadened beyond hyperscalers. Vera expands the company’s ambition beyond the accelerator itself into more of the system surrounding the accelerator.
Modal’s Benchmark Is Valuable and Still Vendor-Adjacent Evidence
The benchmark should be read with appropriate restraint.
Modal is evaluating Vera because it intends to support the processor in its production infrastructure, and Nvidia is actively trying to establish a new CPU category around agent workloads.
The results are not an independent industry-wide CPU benchmark covering every architecture, cost configuration and workload.
They are still valuable because Modal published the shape of the workloads, explained where Vera did and did not win and focused on the contention behavior relevant to its own product.
Agentic AI Is Reopening a CPU Market GPUs Had Overshadowed
The broader implication is architectural.
The last wave of AI infrastructure made the GPU the center of the data center. Agents create more orchestration, code execution, sandboxing and stateful external work around every model call.
That gives the CPU a larger share of the end-to-end latency and cost problem.
Optimisus has been tracking the same shift at the software layer, where persistent agents increasingly need their own computers, permissions and long-running environments.
Vera is Nvidia’s hardware answer to that software trend.
If Modal’s production results hold at scale, the important AI metric may stop being tokens per second alone. It may become how quickly the entire loop — think, act, run, inspect, repeat — completes.
Sources
- Modal — NVIDIA Vera CPUs double throughput for Modal Sandboxes — Primary benchmark and detailed explanation of the agent-sandbox workload.
- Reuters — Nvidia wins fast-growing AI startup Modal for new central processors — Independent reporting on Modal’s adoption plans and business context.
- NVIDIA — Vera CPU for agents — Primary architecture and positioning details for Vera.
