← AUTOMATER NEWSROOM

Vera Rubin Debuts in MLPerf — and the First Agentic-Inference Benchmark Is the Real Story

Vera Rubin NVL72 makes its first MLPerf appearance, sweeping Training v6.1 and leading MLPerf Endpoints v1.0, the first benchmark built for agentic inference. What the numbers say.

Illustration of MLPerf Endpoints v1.0 measuring agentic inference as tokens of agent work delivered per kilowatt of power, with a multi-turn agent loop feeding an efficiency metric.Illustration of MLPerf Endpoints v1.0 measuring agentic inference as tokens of agent work delivered per kilowatt of power, with a multi-turn agent loop feeding an efficiency metric.
MLPerf Endpoints v1.0 reframes inference benchmarking around agentic work per watt: multi-turn histories, tool calls and reasoning all generate tokens, and power becomes the capacity ceiling. Original illustration; data from NVIDIA's MLPerf Endpoints submissions.

NVIDIA’s Vera Rubin architecture made its first public MLPerf appearance today, and it arrived in two places at once: a sweep of MLPerf Training v6.1, and the debut round of MLPerf Endpoints v1.0 — the first MLPerf benchmark built specifically for agentic inference. For anyone planning capacity for agent fleets, the second one matters more than the first.

NVIDIA blog post 'NVIDIA Sweeps MLPerf Training and Endpoints as Vera Rubin Debuts With Leading Performance' with its headline, intro text and November 16, 2026 publication visible.
NVIDIA's announcement of Vera Rubin's MLPerf Training v6.1 sweep and its first MLPerf Endpoints submission, published November 16, 2026 on the NVIDIA Blog. · Original source

The new yardstick: agentic work per watt

MLPerf Endpoints measures something that will feel familiar to anyone paying an inference bill for agents: how much agentic work a system delivers per watt. Agents carry growing conversation histories across turns, generate a mix of reasoning, tool calls and user-facing responses, and call tools mid-task. Each step generates tokens, and because AI data centers are power-limited, tokens per second per watt becomes the capacity ceiling for how many agents a facility can serve. The benchmark tests coding and enterprise workflows through API endpoints and checks generated tool calls and workflow intent against reference values, with separate SWE-bench Verified evaluations confirming that faster serving does not degrade task completion.

In its first submission, NVIDIA reports provisional results showing Vera Rubin NVL72 delivering more than 10x the throughput per kilowatt of GB300 NVL72 on Kimi K3, at a shared minimum end-to-end interactivity of 50 tokens per second per user. Across the three submitted models the provisional ratios are:

Model Throughput per kW ratio E2E interactivity ratio
Kimi K3 10x 3x
DeepSeek-V4.1-Flash 9x 5x
Qwen3.6-35B-A3B 5x 3x

One detail in the footnotes deserves as much attention as the ratios. The headline Kimi K3 comparison pairs a Vera Rubin result with 1,024 concurrent queries against a GB300 result with 48 concurrent queries, both normalized to a 50 tokens/sec/user interactivity floor. At matched concurrency of 1,024 queries, the same footnotes show Vera Rubin at 41,414 tokens/sec system throughput versus GB300’s 13,905 — roughly 3x at the same concurrency, on much newer silicon at higher provisioned power. The 10x figure is a per-kilowatt comparison across very different operating points, not a same-concurrency speedup.

Diagram comparing Vera Rubin NVL72 and GB300 NVL72 on Kimi K3 in MLPerf Endpoints v1.0 with proportionally scaled bars: headline throughput per kilowatt of 410.4 versus 36.8 tok/s/kW, and matched 1,024-query concurrency showing 41,414 versus 13,906 tokens/sec with GB300 falling below the 50 tok/s/user interactivity floor, alongside operating power and accelerator counts of 32 versus 48.Diagram comparing Vera Rubin NVL72 and GB300 NVL72 on Kimi K3 in MLPerf Endpoints v1.0 with proportionally scaled bars: headline throughput per kilowatt of 410.4 versus 36.8 tok/s/kW, and matched 1,024-query concurrency showing 41,414 versus 13,906 tokens/sec with GB300 falling below the 50 tok/s/user interactivity floor, alongside operating power and accelerator counts of 32 versus 48.
Two readings of the same Kimi K3 MLPerf Endpoints v1.0 data: the headline per-kilowatt ratio compares different operating points, while matched concurrency shows a roughly 3x system-throughput gap with GB300 falling below the interactivity floor. Bar lengths are proportional to the labeled values. Provisional Vera Rubin figures; verified GB300 baseline, per NVIDIA's footnotes.

Read the provisional label

There is a second caveat NVIDIA states plainly, and this article keeps it attached to every number: the Vera Rubin Endpoints results are provisional, pending MLCommons verification and peer review, and subject to change. The GB300 baselines they are compared against are verified. That inversion — new platform unverified, old platform verified — is unusual and worth tracking until MLCommons publishes its review.

NVIDIA Technical Blog post 'NVIDIA Vera Rubin NVL72 Leads Agentic Inference Performance on MLPerf Endpoints v1.0' dated Nov 16, 2026, with its title and opening text visible.
NVIDIA's technical blog detailing the MLPerf Endpoints v1.0 submissions, the 5x-10x provisional throughput-per-kilowatt ratios and the serving optimizations behind them, published November 16, 2026. · Original source

Training: a generational sweep, led by the hardest workload

In MLPerf Training v6.1, Vera Rubin NVL72 was the only platform to submit results on all seven benchmarks and posted the fastest time to train on six of them, at 1.4x to 1.6x the per-GPU performance of an optimized GB300 NVL72 baseline across most of the suite. The gap widens on the most communication-heavy workload: DeepSeek-V3 671B, where expert parallelism turns every iteration into a burst of all-to-all traffic. Vera Rubin NVL72 trained it in 4.14 minutes on 1,024 GPUs versus 8.03 minutes for GB300 NVL72 at the same scale — close to 2x per GPU.

Scaling held up too: above 97% efficiency from 256 to 1,024 GPUs, backed by a copper NVLink 6 spine at 3.6 TB/s per GPU and ConnectX-9 SuperNICs over Quantum-X800 InfiniBand or Spectrum-X Ethernet. The round’s absolute fastest result still belongs to GB300 — 1.38 minutes on DeepSeek-V3 across 10,240 GPUs, a scale Microsoft Azure matched with GB200 systems.

The round also added the first agentic RL post-training benchmark, Qwen 3.5 397B GRPO, where Vera Rubin led at 10.44 minutes on 1,024 GPUs — submitted with both TensorRT-LLM and vLLM inference backends, which says as much about software maturity as silicon. The Vera CPU sustained more than 22,500 concurrent RL environments across 256 CPUs in a single rack, feeding the GPU-side rollout generation and weight updates.

What actually moved the number

The MLPerf Endpoints technical blog attributes the gains to a stack, not a single chip: HBM4 for fast access to weights and conversation state, Tensor Cores for matrix work, sixth-generation NVLink for cross-GPU communication, then context-aware serving — prefix caching, state reuse, Dynamo routing, disaggregated prefill/decode — and model-specific kernels such as DeepSeek’s MegaMoE and Qwen’s multi-token prediction with Tensor Core verification. For operators, the useful takeaway is that the techniques (cache the shared prefix, reuse state, split prefill from decode) are portable ideas, even if the ratios are not. NVIDIA has described the same substrate pattern — scheduling idle agent sessions onto shared worker pools — in its agent substrate work.

What this means for procurement

If these results hold after MLCommons verification, the practical consequence is straightforward: within a fixed power budget, a facility on this generation can serve roughly 5x to 10x more agentic work per kilowatt than the previous one at comparable responsiveness. That reshapes the cost-per-completed-task math for self-hosted agent fleets, the same way token-plan pricing reshapes it for hosted ones. Power, not GPU count, is the binding constraint in most AI data centers, and Endpoints is the first MLPerf yardstick that measures the thing that constraint actually limits.

Two verification steps before planning around any of this: wait for MLCommons to publish verified Vera Rubin Endpoints records, and read the full benchmark table at endpoints.mlcommons.org rather than any single headline ratio. Independent third-party corroboration of these figures does not exist yet — every number above is NVIDIA’s own submission or provisional result. This matters more than usual in a debut round, and our guide to reading agent benchmarks in 2026 covers why harness and methodology choices move scores as much as hardware does.

Vera Rubin NVL72 systems are reported as generally available through cloud infrastructure partners, so the availability claim — unlike the benchmark verification — is already in motion.

Further reading: the MLPerf Training v6.1 technical blog and NVIDIA’s announcement post.

Sources

  1. NVIDIA Sweeps MLPerf Training and Endpoints as Vera Rubin Debuts With Leading Performance | NVIDIA Blog
  2. NVIDIA Vera Rubin NVL72 Leads Agentic Inference Performance on MLPerf Endpoints v1.0 | NVIDIA Technical Blog
  3. NVIDIA Vera Rubin NVL72 Achieves up to 2x Blackwell Ultra Performance in MLPerf Training v6.1 | NVIDIA Technical Blog