Back previous/next report: k j
NVIDIA Vera Rubin NVL72 (4-GPU SUT)
Submitted by NVIDIAon 2026-10-17 Published on 2026-10-19
Result statement
NVIDIA ran deepseek-v4_1-flash on NVIDIA Vera Rubin NVL72 (4-GPU SUT) and achieved a provisional 103,537.0 tokens/second System TPS at 7,085.1ms TTFT P90 in the Standardized division of the Available category on October 19, 2026. Pending MLCommons Association verification, results are subject to change pending peer review. .
[1] MLPerf Endpoints v1.0 deepseek-v4_1-flash, published October 19, 2026; retrieved from endpoints.mlcommons.org on November 7, 2026. Result —, run 10: concurrency = 3,072 queries, TTFT P90 = 7,085.1ms, System TPS = 103,537.0 tokens/sec, interactivity = 33.7 tokens/sec per user and total provisioned system power = 12.61 kW. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.
NVIDIA
618254e9-a8b8-47e6-8851-d21dd6b4af51
Submission Status REVIEW_PENDING
2026-10-17
2026-10-19
Division Standardized
Deployment Client on Premises
Model Name deepseek-v4_1-flash
Division Standardized
Model Precision —
Model Link —
Calibration Link —
Model Notes —
Dataset Name mlperf_agentic_inference_dataset
Dataset Type performance
Average Input Tokens —
Dataset Link —
System TPS vs. Tokens/s per User
System TPS vs. Concurrency
Time to First Token P90 vs. Concurrency
Tokens/s per User vs. Concurrency
NVIDIA Vera Rubin NVL72 (4-GPU SUT)
Available
System Category — · —
4 accelerators
1
1
Max Supported Concurrency 3,072
Node Type ID 0
Node Count 1
Framework vLLM 0.20.2rc1.dev6106+g7cf5542c2 behind Dynamo 1.6.0.dev20260924
vLLM 0.20.2rc1.dev6106+g7cf5542c2 behind Dynamo 1.6.0.dev20260924
vLLM behind a Dynamo frontend
620
Link to Container registry.gitlab.com/nvidia/mlperf-inference-partner/nv-mlpinf-partner/centml-rubin-vllm:dsv41-rubin-7cf5542c25-r1
Operating System unspecified
Filesystem unspecified
Other Software unspecified
Software Notes Each vLLM worker runs on 12 dedicated physical cores of its GPU's CPU socket with its memory bound there (numactl --physcpubind/--membind); the Dynamo frontend and the endpoint client run on the remaining CPUs. Dynamo frontend tokenizer prefix cache 16 GB (C2048) / 32 GB (C3072) with DYN_TOKENIZER_CACHE_EXTEND=1. Rubin kernels: FlashMLA sm_107a, DeepGEMM MegaMoE SM107 with per-die routing (branch dsv41-rubin-megamoe-sm107), Rubin MXFP8 dense GEMM dispatch.
Processor
NVIDIA Vera CPU
2
176
VCPUs Per Processor 352
Accelerator
—
0
—
—
—
—
Host / Storage
unspecified
Memory Configuration unspecified
Storage Capacity unspecified
Storage Type unspecified
Networking
unspecified
NVLink + InfiniBand
Network Topology —
Other
Other Hardware —
Cooling Liquid-cooled
Hardware Notes —
| Field Name | Run 1 | Run 2 | Run 3 | Run 4 | Run 5 | Run 6 | Run 7 | Run 8 | Run 9 | Run 10 |
|---|---|---|---|---|---|---|---|---|---|---|
| Run Date | 10/17/2026 | 10/17/2026 | 10/17/2026 | 10/17/2026 | 10/17/2026 | 10/17/2026 | 10/17/2026 | 10/17/2026 | 10/17/2026 | 10/17/2026 |
| Concurrency | 16 | 24 | 32 | 64 | 128 | 256 | 512 | 1,024 | 2,048 | 3,072 |
| System Tokens/Second | 3,741.6 | 5,266.2 | 6,643.8 | 12,000.3 | 20,302.7 | 33,555.1 | 53,781.8 | 75,011.7 | 93,065.5 | 103,537.0 |
| Tokens/Second per User | 233.9 | 219.4 | 207.6 | 187.5 | 158.6 | 131.1 | 105.0 | 73.3 | 45.4 | 33.7 |
| TTFT P90 (ms) | 112.8 | 114.9 | 118.8 | 100.9 | 145.2 | 196.6 | 327.7 | 2561.7 | 2424.8 | 7085.1 |
| Utilization | 3.6% | 5.1% | 6.4% | 11.6% | 19.6% | 32.4% | 51.9% | 72.4% | 89.9% | 100.0% |
| Configuration Summary | deepseek-v4.1-flash-nvflow · online · c16 | deepseek-v4.1-flash-nvflow · online · c24 | deepseek-v4.1-flash-nvflow · online · c32 | deepseek-v4.1-flash-nvflow · online · c64 | deepseek-v4.1-flash-nvflow · online · c128 | deepseek-v4.1-flash-nvflow · online · c256 | deepseek-v4.1-flash-nvflow · online · c512 | deepseek-v4.1-flash-nvflow · online · c1024 | deepseek-v4.1-flash-nvflow · online · c2048 | deepseek-v4.1-flash-nvflow · online · c3072 |
| Config Type | online | online | online | online | online | online | online | online | online | online |
| Config Region | — | — | — | — | — | — | — | — | — | — |
| Target QPS | — | — | — | — | — | — | — | — | — | — |
| Result State | complete | complete | complete | complete | complete | complete | complete | complete | complete | complete |
| Measured Accuracy Score | — | — | — | — | — | — | — | — | — | — |
| Benchmark Version | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Result Version | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 |
| Archive URI | gs://mlperf-endpoints-blob/runs/f8a89260-f03d-4299-8b13-1f777af24d6c/archive.tar.gz | gs://mlperf-endpoints-blob/runs/6bb91092-fc26-4848-aaf0-157461dbe715/archive.tar.gz | gs://mlperf-endpoints-blob/runs/4cd88602-aed8-4aa3-b21c-81d280d13f37/archive.tar.gz | gs://mlperf-endpoints-blob/runs/6c0a4110-464c-4310-8b9d-3e48b80f68e8/archive.tar.gz | gs://mlperf-endpoints-blob/runs/860a87b6-20d8-4f59-9209-0b49f04998a5/archive.tar.gz | gs://mlperf-endpoints-blob/runs/fe4b1e32-04c3-4213-b58e-ca8bedd2f999/archive.tar.gz | gs://mlperf-endpoints-blob/runs/d0b5df69-5a34-4fd4-b0a1-49e297fc604f/archive.tar.gz | gs://mlperf-endpoints-blob/runs/75bb614f-c593-4488-8c53-c3a8de63a789/archive.tar.gz | gs://mlperf-endpoints-blob/runs/1a5e5737-f9c8-46b4-a1c8-e906500e8217/archive.tar.gz | gs://mlperf-endpoints-blob/runs/285def9f-29a8-4c67-91b2-c6a8b02add04/archive.tar.gz |
| QPS | 4.0704 | 5.8007 | 7.3865 | 12.9448 | 22.3441 | 36.5167 | 58.5040 | 80.6855 | 98.8827 | 107.5036 |
| Total Output Tokens | 18,030,856 | 17,507,245 | 17,109,488 | 36,297,046 | 71,524,422 | 105,208,158 | 136,691,236 | 196,761,109 | 246,104,851 | 360,235,195 |
| Average Output Tokens | 919.24 | 907.86 | 899.46 | 927.03 | 908.64 | 918.9 | 919.28 | 929.68 | 941.17 | 963.1 |
| Run Duration (s) | 4,818.98 | 3,324.43 | 2,575.25 | 3,024.69 | 3,522.90 | 3,135.38 | 2,541.59 | 2,623.07 | 2,644.43 | 3,479.29 |
| Total Queries | 19,615 | 19,284 | 19,022 | 39,154 | 78,716 | 114,494 | 148,693 | 211,644 | 261,488 | 374,036 |
| Total Queries Issued | 19,615 | 19,284 | 19,022 | 39,154 | 78,716 | 114,494 | 148,693 | 211,644 | 261,488 | 374,036 |
| Total Queries Failed | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Time to First Token (TTFT) (ms) | ||||||||||
| Minimum | 28.2 | 28.5 | 28.9 | 30.6 | 33.7 | 43.9 | 49.9 | 81.6 | 114.5 | 778.4 |
| Average | 80.3 | 71.8 | 96.3 | 70.1 | 98.7 | 150.0 | 301.5 | 1316.0 | 1584.7 | 5459.5 |
| P50 | 57.8 | 58.7 | 59.7 | 60.0 | 79.8 | 100.0 | 139.1 | 871.1 | 1075.5 | 4847.8 |
| P90 | 112.8 | 114.9 | 118.8 | 100.9 | 145.2 | 196.6 | 327.7 | 2561.7 | 2424.8 | 7085.1 |
| P95 | 150.0 | 148.4 | 154.3 | 124.6 | 174.5 | 241.8 | 439.5 | 3094.8 | 2890.1 | 7776.0 |
| P99 | 240.7 | 230.8 | 284.1 | 188.5 | 293.2 | 555.3 | 1068.3 | 4725.2 | 19150.0 | 27747.7 |
| P999 | 4243.3 | 940.5 | 10984.2 | 1166.2 | 3473.8 | 10627.7 | 27670.3 | 43363.8 | 40455.1 | 56189.3 |
| Maximum | 13483.2 | 1920.9 | 19071.2 | 2449.7 | 4707.5 | 13168.3 | 36462.8 | 48733.4 | 48501.9 | 64178.6 |
| Time Per Output Token (TPOT) (ms) | ||||||||||
| Minimum | 0.83 | 0.90 | 0.96 | 0.93 | 1.38 | 0.07 | 0.01 | 0.01 | 0.01 | -0.09 |
| Average | 2.01 | 2.17 | 2.38 | 2.85 | 3.59 | 4.54 | 5.85 | 8.06 | 13.1 | 15.8 |
| P50 | 1.83 | 2.01 | 2.15 | 2.64 | 3.33 | 4.17 | 5.34 | 7.45 | 12.1 | 15.1 |
| P90 | 2.86 | 3.15 | 3.41 | 4.16 | 5.24 | 6.73 | 8.66 | 12.0 | 21.1 | 24.5 |
| P95 | 3.12 | 3.43 | 3.73 | 4.52 | 5.69 | 7.34 | 9.42 | 13.0 | 23.4 | 27.2 |
| P99 | 3.75 | 4.08 | 4.54 | 5.38 | 6.74 | 8.62 | 11.0 | 15.3 | 28.0 | 32.9 |
| P999 | 17.4 | 6.31 | 27.7 | 12.0 | 11.5 | 17.5 | 27.3 | 25.5 | 39.2 | 49.4 |
| Maximum | 70.9 | 35.3 | 89.9 | 45.6 | 43.4 | 73.8 | 186.4 | 192.2 | 2125.7 | 493.1 |
| Query Latency (ms) | ||||||||||
| Minimum | 63.2 | 68.8 | 75.4 | 73.8 | 90.5 | 94.4 | 158.2 | 201.7 | 390.8 | 1632.8 |
| Average | 2314.1 | 2489.4 | 2681.3 | 3315.0 | 4106.0 | 5316.6 | 6953.2 | 10632.7 | 17803.7 | 24979.8 |
| P50 | 413.9 | 443.3 | 480.1 | 555.5 | 710.4 | 902.1 | 1206.5 | 2867.5 | 3752.1 | 8352.1 |
| P90 | 5411.6 | 5740.7 | 6226.8 | 7889.1 | 9529.0 | 12494.1 | 16895.2 | 24949.6 | 41393.2 | 55179.4 |
| P95 | 16502.7 | 16951.4 | 17801.8 | 23487.6 | 27370.1 | 36402.3 | 49132.5 | 71902.0 | 132437.9 | 164465.1 |
| P99 | 24345.5 | 26798.3 | 28910.8 | 35268.9 | 44406.4 | 56978.2 | 73428.0 | 102736.3 | 185125.7 | 219413.4 |
| P999 | 26812.6 | 28868.1 | 31742.0 | 37741.5 | 47276.5 | 60853.0 | 79011.7 | 110122.8 | 202695.6 | 240529.5 |
| Maximum | 58181.6 | 32531.1 | 59813.9 | 46012.4 | 53482.6 | 74823.9 | 115068.1 | 142374.8 | 225866.8 | 276666.1 |