Submitter NVIDIA
Submission ID 503d593f-d5a3-4a7d-a08e-17228ef09f2f
Submission Status PUBLISHED
Submission Date 2026-06-25
Publication Date —
Division Standardized
Model Name DeepSeek R1
Division Standardized
Model Precision —
Model Link —
Calibration Link —
Model Notes —
Dataset Name dsr1-perf
Dataset Type performance
Average Input Tokens —
Dataset Link —
System TPS vs. Tokens/s per User
System TPS vs. Concurrency
Time to First Token P95 vs. Concurrency
Tokens/s per User vs. Concurrency
System Name NVIDIA GB200 NVL72
System Availability Available
System Category Datacenter · GB200 NVL72 rack, disaggregated TRT-LLM, NVFP4 weights
System Size —
Node Types 1
Total Node Count 18
Max Supported Concurrency 16,384
Node Type ID 0
Node Count 18
Framework TensorRT-LLM
Serving Framework TensorRT-LLM
Inference Backend TensorRT-LLM
Driver —
Link to Container —
Operating System unspecified
Filesystem —
Other Software —
Software Notes —
Processor
Processor Model Name NVIDIA Grace (ARM Neoverse V2)
Processors per Node 1
Cores Per Processor 72
VCPUs Per Processor —
Accelerator
Accelerator Model Name NVIDIA GB200
Accelerators per Node 4
Memory Type —
Memory Capacity 192 GB HBM3e
Accelerator Interconnect NVLink 5 + NVSwitch (NVL72)
Host-Accelerator Interconnect NVLink-C2C (Grace-Blackwell)
Host / Storage
Host Memory Capacity unspecified
Memory Configuration —
Storage Capacity unspecified
Storage Type unspecified
Networking
Network Interfaces —
Network Details NVLink + InfiniBand
Network Topology NVL72 single-rack NVLink domain
Other
Other Hardware —
Cooling Liquid-cooled
Hardware Notes —
| Field Name | Run 1 | Run 2 | Run 3 | Run 4 | Run 5 | Run 6 | Run 7 | Run 8 | Run 9 | Run 10 | Run 11 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Run Date | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 |
| Concurrency | 16 | 32 | 48 | 128 | 256 | 512 | 1,024 | 2,048 | 4,096 | 8,192 | 16,384 |
| System Tokens/Second | 4,433.1 | 7,109.7 | 10,640.4 | 27,342.8 | 46,563.4 | 78,501.1 | 125,726.9 | 190,731.6 | 303,719.7 | 373,313.6 | 408,585.4 |
| Tokens/Second per User | 277.1 | 222.2 | 221.7 | 213.6 | 181.9 | 153.3 | 122.8 | 93.1 | 74.2 | 45.6 | 24.9 |
| TTFT P95 (ms) | 349.8 | 245.5 | 276.0 | 323.4 | 2250.4 | 1881.5 | 2229.9 | 2881.0 | 5416.7 | 5601.1 | 12684.8 |
| Utilization | 1.1% | 1.7% | 2.6% | 6.7% | 11.4% | 19.2% | 30.8% | 46.7% | 74.3% | 91.4% | 100.0% |
| Configuration Summary | dsr1-gb200x72-pareto-nvl72-c16 · submission | dsr1-gb200x72-pareto-nvl72-c32 · submission | dsr1-gb200x72-pareto-nvl72-c48 · submission | dsr1-gb200x72-pareto-nvl72-c128 · submission | dsr1-gb200x72-pareto-nvl72-c256 · submission | dsr1-gb200x72-pareto-nvl72-c512 · submission | dsr1-gb200x72-pareto-nvl72-c1024 · submission | dsr1-gb200x72-pareto-nvl72-c2048 · submission | dsr1-gb200x72-pareto-nvl72-c4096 · submission | dsr1-gb200x72-pareto-nvl72-c8192 · online | dsr1-gb200x72-pareto-nvl72-c16384 · online |
| Config Type | submission | submission | submission | submission | submission | submission | submission | submission | submission | online | online |
| Config Region | — | — | — | — | — | — | — | — | — | — | — |
| Target QPS | — | — | — | — | — | — | — | — | — | — | — |
| Result State | complete | complete | complete | complete | complete | complete | complete | complete | complete | complete | complete |
| Measured Accuracy Score | — | — | — | — | — | — | — | — | — | — | — |
| Benchmark Version | d093d67 | 067de9b | e042086 | 4cfc43e | 4687511 | c00be8e | unknown | c00be8e | c00be8e | 87a9d1c | b71f8e9 |
| Result Version | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 |
| Archive URI | gs://mlperf-endpoints-blob/runs/8528d1f1-602d-452e-ad7a-ef8b8727e53b/archive.tar.gz | gs://mlperf-endpoints-blob/runs/d2b792bf-81ae-46a8-a363-2b7808d107cd/archive.tar.gz | gs://mlperf-endpoints-blob/runs/bcc431a5-240e-4271-977e-371bd6ce8f20/archive.tar.gz | gs://mlperf-endpoints-blob/runs/0caf637b-c6e9-416b-bd6a-32483c3135c8/archive.tar.gz | gs://mlperf-endpoints-blob/runs/561a6d4c-0813-4b7f-8ff7-82a493e79177/archive.tar.gz | gs://mlperf-endpoints-blob/runs/905f9bc4-fdcb-42be-b090-3159a494f52e/archive.tar.gz | gs://mlperf-endpoints-blob/runs/fc7bde94-556e-4612-a206-0f4d54c83be1/archive.tar.gz | gs://mlperf-endpoints-blob/runs/db93541b-51d2-4f78-a9aa-50d49d8b54f7/archive.tar.gz | gs://mlperf-endpoints-blob/runs/241c9c6c-4683-498c-9221-652588d95652/archive.tar.gz | gs://mlperf-endpoints-blob/runs/2542562e-f00c-457d-8044-5f48a5721148/archive.tar.gz | gs://mlperf-endpoints-blob/runs/a59e2194-6710-4f70-8fd8-74a11488229d/archive.tar.gz |
| QPS | 1.2179 | 1.8775 | 2.8233 | 7.2515 | 12.3176 | 20.7697 | 33.2430 | 50.9243 | 83.7246 | 120.4745 | 108.6147 |
| Total Output Tokens | 15,972,388 | 28,779,872 | 42,964,150 | 105,954,385 | 185,610,169 | 313,328,255 | 500,365,864 | 755,446,120 | 1,105,693,062 | 1,170,066,807 | 1,704,841,539 |
| Average Output Tokens | 3,640.02 | 3,786.83 | 3,768.79 | 3,770.62 | 3,780.25 | 3,779.59 | 3,782.05 | 3,745.39 | 3,627.6 | 3,196.72 | 3,761.79 |
| Run Duration (s) | 3,602.99 | 4,047.95 | 4,037.83 | 3,875.04 | 3,986.18 | 3,991.39 | 3,979.79 | 3,960.78 | 3,640.50 | 3,134.27 | 4,172.55 |
| Total Queries | 4,388 | 7,600 | 11,400 | 28,100 | 49,100 | 82,900 | 132,300 | 201,700 | 304,800 | 377,600 | 453,200 |
| Total Queries Issued | 4,388 | 7,600 | 11,400 | 28,100 | 49,100 | 82,900 | 132,300 | 201,700 | 304,800 | 377,600 | 453,200 |
| Total Queries Failed | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Time to First Token (TTFT) P95 (ms) | |||||||||||
| Minimum | 159.8 | 139.9 | 137.5 | 141.3 | 139.2 | 135.2 | 148.2 | 159.3 | 166.2 | 191.5 | 164.6 |
| Average | 241.4 | 172.4 | 187.1 | 232.3 | 632.1 | 583.9 | 716.1 | 909.9 | 1382.1 | 1620.1 | 3508.9 |
| P50 | 216.4 | 153.7 | 161.7 | 200.0 | 294.7 | 272.7 | 383.3 | 376.3 | 381.1 | 494.5 | 452.5 |
| P90 | 291.4 | 208.2 | 248.2 | 306.0 | 1610.4 | 1452.0 | 1787.6 | 2403.5 | 4493.7 | 3340.6 | 7793.8 |
| P95 | 349.8 | 245.5 | 276.0 | 323.4 | 2250.4 | 1881.5 | 2229.9 | 2881.0 | 5416.7 | 5601.1 | 12684.8 |
| P99 | 419.5 | 285.5 | 319.2 | 373.7 | 3396.4 | 3039.0 | 3395.6 | 4013.6 | 7535.0 | 21605.4 | 48314.2 |
| P999 | 4224.9 | 2197.1 | 2224.9 | 3056.3 | 4817.8 | 3643.4 | 5857.1 | 8475.0 | 16459.5 | 33212.0 | 62517.8 |
| Maximum | 9308.8 | 2204.7 | 2243.1 | 3060.7 | 6563.3 | 4418.7 | 6263.9 | 9664.2 | 18064.3 | 35321.6 | 64880.0 |
| Time Per Output Token (TPOT) (ms) | |||||||||||
| Minimum | 2.44 | 1.87 | 1.32 | 1.71 | 1.76 | 2.61 | 2.48 | 2.52 | 3.04 | 3.85 | 11.9 |
| Average | 3.64 | 4.52 | 4.55 | 4.68 | 5.37 | 6.40 | 7.98 | 10.5 | 13.0 | 19.9 | 35.8 |
| P50 | 3.60 | 4.47 | 4.51 | 4.62 | 5.32 | 6.34 | 7.93 | 10.4 | 12.9 | 19.9 | 36.8 |
| P90 | 4.26 | 4.99 | 5.03 | 5.16 | 5.94 | 7.10 | 8.88 | 11.7 | 14.4 | 22.3 | 37.3 |
| P95 | 4.45 | 5.09 | 5.13 | 5.27 | 6.06 | 7.24 | 9.06 | 12.0 | 14.8 | 22.7 | 37.4 |
| P99 | 4.82 | 5.29 | 5.30 | 5.45 | 6.27 | 7.51 | 9.40 | 12.4 | 15.4 | 23.6 | 37.7 |
| P999 | 5.02 | 5.44 | 5.52 | 5.68 | 6.52 | 7.82 | 9.78 | 12.9 | 16.0 | 24.7 | 37.9 |
| Maximum | 5.08 | 5.77 | 5.73 | 5.93 | 6.98 | 8.35 | 10.5 | 14.5 | 18.0 | 208.0 | 38.4 |
| Query Latency (ms) | |||||||||||
| Minimum | 906.3 | 1064.0 | 997.8 | 890.4 | 983.6 | 1041.2 | 1422.9 | 1796.9 | 2075.3 | 3234.5 | 4535.5 |
| Average | 13024.9 | 16825.1 | 16891.9 | 17404.0 | 20412.0 | 24167.6 | 30148.7 | 39181.2 | 47293.9 | 65170.9 | 138263.4 |
| P50 | 7515.9 | 9724.0 | 9621.9 | 9963.1 | 11967.4 | 14073.5 | 17536.7 | 22919.7 | 28599.6 | 42994.9 | 79191.0 |
| P90 | 31124.5 | 41347.2 | 41400.6 | 43024.1 | 49435.3 | 59266.6 | 73984.7 | 96538.1 | 119391.7 | 180287.8 | 342153.8 |
| P95 | 46506.1 | 59273.0 | 59844.9 | 60953.9 | 70118.3 | 83906.2 | 104894.6 | 137745.1 | 169572.0 | 180893.6 | 482314.5 |
| P99 | 65783.3 | 83037.5 | 84546.8 | 87269.5 | 99598.1 | 118569.2 | 147865.9 | 180698.8 | 181042.3 | 181270.6 | 729728.1 |
| P999 | 74680.6 | 89515.8 | 92010.6 | 95836.0 | 108046.2 | 129310.4 | 161702.3 | 181199.9 | 181345.8 | 202043.3 | 748546.2 |
| Maximum | 80212.3 | 94733.6 | 99769.4 | 118757.1 | 123393.5 | 156836.0 | 181140.1 | 188480.6 | 198294.5 | 215971.3 | 758154.0 |