Menu

Endpoints › Agentic Model Results

Agentic: Throughput vs. E2E Average Interactivity

Systems

Remember to review the Messaging Guidelines when using this material.

100%
0.020k40k60k80k100kSystem Throughput (Tokens/s) · higher is better0.0100200300400E2E Average Interactivity (Tokens/sper User) · constraint: minimum orequalMLPerf Endpoints v1.0, deepseek-v4_1-flash,Standardized division: GoogleGB300-NVL72_GB300-288GB_aarch64x4_vLLM-Dynamo(Available, published November 4, 2026, 8.43kW provisioned power (MLC Estimated Power));NVIDIA GB300 NVL72 (4-GPU SUT) (Available,published November 4, 2026, 7.5 kWprovisioned power); Krai Dell PowerEdgeXE9680 (8x MI300X-OAM-192GB, stock vLLM)(Available, published October 27, 2026, 9.9kW provisioned power); Krai Dell PowerEdgeXE9680 (8x MI300X-OAM-192GB, vLLM + Krai MoEkernels) (Available, published October 27,2026, 9.9 kW provisioned power); NVIDIA VeraRubin NVL72 (4-GPU SUT) (Available, publishedOctober 19, 2026, 12.61 kW provisionedpower); Krai HPE Cray XD670 (8xH200-SXM-141GB, vLLM) (Available, publishedOctober 13, 2026, 9.73 kW provisioned power(MLC Estimated Power)). Retrieved fromendpoints.mlcommons.org on November 7, 2026.4 of 6 results (square markers) areprovisional. Pending MLCommons Associationverification, results are subject to changepending peer review. Other results verifiedby MLCommons Association. The MLPerf name andlogo are registered and unregisteredtrademarks of MLCommons Association in theUnited States and other countries. All rightsreserved. Unauthorized use strictlyprohibited. See www.mlcommons.org for moreinformation.

Results shown

Model Division Status Total Provisioned System Power MLPerf Result ID
GoogleGB300-NVL72_GB300-288GB_aarch64x4_vLLM-Dynamodeepseek-v4_1-flashStandardized2026-11-04 Available8.43 kW (MLC Estimated Power)—
NVIDIANVIDIA GB300 NVL72 (4-GPU SUT)deepseek-v4_1-flashStandardized2026-11-04 Available7.5 kW—
KraiDell PowerEdge XE9680 (8x MI300X-OAM-192GB, stock vLLM)deepseek-v4_1-flashStandardized2026-10-27 Available9.9 kW—
KraiDell PowerEdge XE9680 (8x MI300X-OAM-192GB, vLLM + Krai MoE kernels)deepseek-v4_1-flashStandardized2026-10-27 Available9.9 kW—
NVIDIANVIDIA Vera Rubin NVL72 (4-GPU SUT)deepseek-v4_1-flashStandardized2026-10-19 Available12.61 kW—
KraiHPE Cray XD670 (8x H200-SXM-141GB, vLLM)deepseek-v4_1-flashStandardized2026-10-13 Available9.73 kW (MLC Estimated Power)—