Submitter NVIDIA
Submission ID 27168b87-e0f9-4488-ae93-88a45c1c81c9
Submission Status PUBLISHED
Submission Date 2026-06-25
Publication Date —
Division Standardized
Model Name DeepSeek R1
Division Standardized
Model Precision —
Model Link —
Calibration Link —
Model Notes —
Dataset Name dsr1-perf
Dataset Type performance
Average Input Tokens —
Dataset Link —
System TPS vs. Tokens/s per User
System TPS vs. Concurrency
Time to First Token P95 vs. Concurrency
Tokens/s per User vs. Concurrency
System Name NVIDIA GB300 NVL72
System Availability Available
System Category Datacenter · GB300 NVL72 rack, disaggregated TRT-LLM, NVFP4 weights
System Size —
Node Types 1
Total Node Count 18
Max Supported Concurrency 28,080
Node Type ID 0
Node Count 18
Framework TensorRT-LLM
Serving Framework TensorRT-LLM
Inference Backend TensorRT-LLM
Driver —
Link to Container —
Operating System unspecified
Filesystem —
Other Software —
Software Notes —
Processor
Processor Model Name NVIDIA Grace (ARM Neoverse V2)
Processors per Node 1
Cores Per Processor 72
VCPUs Per Processor —
Accelerator
Accelerator Model Name NVIDIA GB300
Accelerators per Node 4
Memory Type —
Memory Capacity 288 GB HBM3e
Accelerator Interconnect NVLink 5 + NVSwitch (NVL72)
Host-Accelerator Interconnect NVLink-C2C (Grace-Blackwell)
Host / Storage
Host Memory Capacity unspecified
Memory Configuration —
Storage Capacity unspecified
Storage Type unspecified
Networking
Network Interfaces —
Network Details NVLink + InfiniBand
Network Topology NVL72 single-rack NVLink domain
Other
Other Hardware —
Cooling Liquid-cooled
Hardware Notes —
| Field Name | Run 1 | Run 2 | Run 3 | Run 4 | Run 5 | Run 6 | Run 7 | Run 8 | Run 9 | Run 10 | Run 11 | Run 12 | Run 13 | Run 14 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Run Date | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 |
| Concurrency | 1 | 2 | 4 | 48 | 64 | 128 | 256 | 512 | 1,024 | 2,048 | 4,096 | 8,192 | 16,384 | 28,080 |
| System Tokens/Second | 338.8 | 640.9 | 1,225.1 | 10,556.5 | 14,161.0 | 27,256.6 | 45,293.7 | 81,627.8 | 130,376.4 | 198,881.9 | 306,707.1 | 395,958.1 | 469,668.6 | 546,980.1 |
| Tokens/Second per User | 338.8 | 320.5 | 306.3 | 219.9 | 221.3 | 212.9 | 176.9 | 159.4 | 127.3 | 97.1 | 74.9 | 48.3 | 28.7 | 19.5 |
| TTFT P95 (ms) | 1193.0 | 1065.0 | 317.5 | 467.2 | 451.2 | 521.6 | 2679.3 | 520.1 | 572.1 | 1306.4 | 4550.5 | 7084.9 | 19877.0 | 36763.2 |
| Utilization | 0.1% | 0.1% | 0.2% | 1.9% | 2.6% | 5.0% | 8.3% | 14.9% | 23.8% | 36.4% | 56.1% | 72.4% | 85.9% | 100.0% |
| Configuration Summary | dsr1-gb300x72-pareto-nvl72-c1 · submission | dsr1-gb300x72-pareto-nvl72-c2 · submission | dsr1-gb300x72-pareto-nvl72-c4 · submission | dsr1-gb300x72-pareto-nvl72-c48 · submission | dsr1-gb300x72-pareto-nvl72-c64 · submission | dsr1-gb300x72-pareto-nvl72-c128 · submission | dsr1-gb300x72-pareto-nvl72-c256 · submission | dsr1-gb300x72-pareto-nvl72-c512 · submission | dsr1-gb300x72-pareto-nvl72-c1024 · submission | dsr1-gb300x72-pareto-nvl72-c2048 · submission | dsr1-gb300x72-pareto-nvl72-c4096 · submission | dsr1-gb300x72-pareto-nvl72-c8192 · submission | dsr1-gb300x72-pareto-nvl72-c16384 · submission | dsr1-gb300x72-pareto-nvl72-c28080-numa · submission |
| Config Type | submission | submission | submission | submission | submission | submission | submission | submission | submission | submission | submission | submission | submission | submission |
| Config Region | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| Target QPS | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| Result State | complete | complete | complete | complete | complete | complete | complete | complete | complete | complete | complete | complete | complete | complete |
| Measured Accuracy Score | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| Benchmark Version | 05ce400 | 05ce400 | 05ce400 | ced1dcd | ed0c53d | 9019316 | cd477d4 | 429c5b8 | 9cbfefe | af6bf98 | cd477d4 | 0470dab | 7265704 | e072ac2 |
| Result Version | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 |
| Archive URI | gs://mlperf-endpoints-blob/runs/49655a2c-3d3d-45f4-916e-54c439080875/archive.tar.gz | gs://mlperf-endpoints-blob/runs/00747f95-5433-4718-99a6-17c91e6000bb/archive.tar.gz | gs://mlperf-endpoints-blob/runs/cbe204a9-7436-4ff0-85b0-a7b37b4b785c/archive.tar.gz | gs://mlperf-endpoints-blob/runs/6ae0b5b8-33d9-4640-af10-5adfa595b66f/archive.tar.gz | gs://mlperf-endpoints-blob/runs/4f76a1f0-e4a7-4fc1-ab15-f767674462b2/archive.tar.gz | gs://mlperf-endpoints-blob/runs/1d17bdfe-f13c-44c3-beda-3041f3cfd2ae/archive.tar.gz | gs://mlperf-endpoints-blob/runs/a913ac1d-7f38-453a-ba89-e0cbd1ab5250/archive.tar.gz | gs://mlperf-endpoints-blob/runs/aa5cc47c-c121-41c2-99d8-ccd4bbd607b9/archive.tar.gz | gs://mlperf-endpoints-blob/runs/a9c5e6a2-71f1-4d0d-8302-f1cc94a43ab4/archive.tar.gz | gs://mlperf-endpoints-blob/runs/9784c932-acdd-46de-9096-fbb6a1e4a723/archive.tar.gz | gs://mlperf-endpoints-blob/runs/523ba87c-7040-4147-bb76-360ae7979a56/archive.tar.gz | gs://mlperf-endpoints-blob/runs/ec5cf744-36e8-4e9d-8356-618204ec2339/archive.tar.gz | gs://mlperf-endpoints-blob/runs/164c6f8d-96d1-4aab-b7e1-5fbcf7d1ff40/archive.tar.gz | gs://mlperf-endpoints-blob/runs/68478c69-8d3c-4be8-82d9-8f79a3673c63/archive.tar.gz |
| QPS | 0.0887 | 0.1568 | 0.3305 | 2.8298 | 3.7759 | 7.2743 | 11.9376 | 21.7036 | 34.7665 | 52.9578 | 81.3117 | 105.4162 | 125.2644 | 145.1861 |
| Total Output Tokens | 954,745 | 2,043,724 | 3,706,833 | 24,553,924 | 38,253,793 | 82,208,882 | 83,244,526 | 280,557,875 | 394,925,643 | 725,078,131 | 1,075,847,720 | 1,318,555,925 | 1,349,790,344 | 2,479,730,589 |
| Average Output Tokens | 3,818.98 | 4,087.45 | 3,706.83 | 3,730.47 | 3,750.37 | 3,746.99 | 3,794.19 | 3,761.03 | 3,750.05 | 3,755.48 | 3,771.99 | 3,756.17 | 3,749.42 | 3,767.44 |
| Run Duration (s) | 2,817.61 | 3,188.72 | 3,025.62 | 2,325.96 | 2,701.36 | 3,016.11 | 1,837.88 | 3,437.04 | 3,029.12 | 3,645.77 | 3,507.74 | 3,330.04 | 2,873.92 | 4,533.49 |
| Total Queries | 250 | 500 | 1,000 | 6,582 | 10,200 | 21,940 | 21,940 | 74,596 | 105,312 | 193,072 | 285,220 | 351,040 | 360,000 | 658,200 |
| Total Queries Issued | 250 | 500 | 1,000 | 6,582 | 10,200 | 21,940 | 21,940 | 74,596 | 105,312 | 193,072 | 285,220 | 351,040 | 360,000 | 658,200 |
| Total Queries Failed | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Time to First Token (TTFT) P95 (ms) | ||||||||||||||
| Minimum | 143.7 | 140.7 | 201.4 | 164.1 | 165.0 | 168.8 | 142.8 | 179.5 | 157.9 | 215.6 | 174.1 | 180.1 | 217.2 | 176.2 |
| Average | 325.9 | 276.7 | 274.5 | 312.6 | 307.6 | 368.6 | 776.8 | 397.3 | 388.9 | 608.0 | 1109.7 | 1857.6 | 4698.1 | 6033.8 |
| P50 | 229.9 | 233.3 | 239.5 | 268.1 | 266.3 | 342.7 | 329.7 | 391.8 | 329.2 | 483.6 | 430.9 | 454.4 | 535.7 | 356.4 |
| P90 | 1123.1 | 251.8 | 259.1 | 432.8 | 423.9 | 492.7 | 2051.5 | 491.5 | 541.0 | 989.2 | 2851.9 | 5612.9 | 15454.0 | 15410.8 |
| P95 | 1193.0 | 1065.0 | 317.5 | 467.2 | 451.2 | 521.6 | 2679.3 | 520.1 | 572.1 | 1306.4 | 4550.5 | 7084.9 | 19877.0 | 36763.2 |
| P99 | 1312.3 | 1269.9 | 1284.1 | 534.2 | 522.0 | 579.4 | 3902.9 | 599.4 | 726.4 | 2272.3 | 8526.0 | 18516.0 | 46295.0 | 82744.4 |
| P999 | 1331.2 | 1340.4 | 1873.2 | 1536.6 | 1464.4 | 1959.1 | 5126.2 | 2912.5 | 4622.4 | 6870.2 | 15467.2 | 27366.3 | 55336.3 | 99115.9 |
| Maximum | 1332.3 | 1395.6 | 1875.5 | 1537.3 | 1465.2 | 1994.5 | 5791.3 | 2984.8 | 4636.5 | 7897.7 | 16710.5 | 28908.9 | 57636.7 | 101763.9 |
| Time Per Output Token (TPOT) (ms) | ||||||||||||||
| Minimum | 0.00 | 0.10 | 0.67 | 2.94 | 2.93 | 3.10 | 1.73 | 4.09 | 4.72 | 4.54 | 2.78 | 5.13 | 5.45 | 12.6 |
| Average | 2.86 | 3.09 | 3.30 | 4.49 | 4.49 | 4.63 | 5.35 | 6.21 | 7.74 | 10.1 | 12.9 | 19.2 | 31.8 | 45.5 |
| P50 | 2.90 | 3.08 | 3.26 | 4.42 | 4.43 | 4.57 | 5.30 | 6.14 | 7.65 | 10.0 | 12.8 | 19.1 | 32.1 | 47.3 |
| P90 | 3.22 | 3.54 | 3.80 | 5.00 | 4.99 | 5.13 | 5.92 | 6.87 | 8.56 | 11.3 | 14.4 | 21.5 | 36.5 | 47.9 |
| P95 | 3.29 | 3.62 | 4.02 | 5.26 | 5.16 | 5.28 | 6.03 | 7.04 | 8.76 | 11.5 | 14.7 | 22.0 | 37.3 | 48.1 |
| P99 | 3.39 | 3.80 | 4.25 | 5.90 | 5.69 | 5.83 | 6.24 | 7.50 | 9.23 | 11.9 | 15.2 | 22.8 | 38.8 | 48.5 |
| P999 | 3.41 | 3.87 | 4.39 | 6.61 | 6.13 | 6.44 | 6.45 | 8.44 | 10.7 | 12.4 | 15.8 | 24.0 | 40.6 | 49.1 |
| Maximum | 3.46 | 3.90 | 4.41 | 8.20 | 6.46 | 6.93 | 6.85 | 9.82 | 12.4 | 13.5 | 17.3 | 431.8 | 44.0 | 51.5 |
| Query Latency (ms) | ||||||||||||||
| Minimum | 1047.5 | 1073.3 | 875.7 | 1056.9 | 846.9 | 1028.1 | 1214.2 | 1173.7 | 1446.5 | 1795.7 | 2097.4 | 2405.4 | 3453.9 | 6473.4 |
| Average | 11270.3 | 12666.9 | 12075.1 | 16637.7 | 16696.3 | 17258.4 | 20562.9 | 23107.9 | 28604.7 | 37600.8 | 48620.9 | 72122.0 | 120823.5 | 177589.8 |
| P50 | 6101.3 | 7498.0 | 7388.8 | 9763.9 | 9667.0 | 9991.9 | 12080.7 | 13400.7 | 16651.7 | 21644.8 | 28269.4 | 41994.9 | 71387.1 | 103139.0 |
| P90 | 26705.5 | 34241.3 | 29495.5 | 40164.0 | 41694.6 | 42185.7 | 49759.9 | 56725.0 | 70369.8 | 92578.4 | 118723.6 | 176638.6 | 292831.2 | 435537.8 |
| P95 | 37603.6 | 45757.6 | 41506.1 | 58236.5 | 58608.2 | 60123.5 | 71469.2 | 80895.8 | 99463.5 | 131976.2 | 169503.8 | 250693.8 | 411163.5 | 613731.5 |
| P99 | 54126.9 | 58665.1 | 59404.1 | 81313.8 | 82159.6 | 84936.9 | 99759.1 | 114054.1 | 141703.1 | 186543.1 | 240107.9 | 356590.0 | 594623.3 | 937055.8 |
| P999 | 57437.5 | 62077.8 | 63359.1 | 93735.6 | 90663.4 | 93502.3 | 107668.1 | 124391.9 | 154678.5 | 203888.7 | 262579.8 | 391337.1 | 663067.2 | 955828.1 |
| Maximum | 57773.8 | 63195.9 | 63650.6 | 111624.6 | 104139.5 | 110061.7 | 122968.0 | 146297.9 | 173809.6 | 257970.0 | 346067.0 | 502808.5 | 833431.1 | 971749.4 |