GB300-NVL72_GB300-288GB_aarch64x72_TRTSubmitter Google
Submission ID 60a63420-d1d1-41d7-b807-61921f313e8f
Submission Status PUBLISHED
Submission Date 2026-06-26
Publication Date —
Division Standardized
Model Name DeepSeek R1
Division Standardized
Model Precision —
Model Link —
Calibration Link —
Model Notes —
Dataset Name dsr1-perf
Dataset Type performance
Average Input Tokens —
Dataset Link —
System TPS vs. Tokens/s per User
System TPS vs. Concurrency
Time to First Token P95 vs. Concurrency
Tokens/s per User vs. Concurrency
System Name GB300-NVL72_GB300-288GB_aarch64x72_TRT
System Availability Available
System Category Datacenter · GB300 NVL72 rack, disaggregated TRT-LLM, NVFP4 weights, fp8 KV cache; MTP-3 speculative decoding
System Size —
Node Types 1
Total Node Count 18
Max Supported Concurrency 8,192
Node Type ID 0
Node Count 18
Framework TensorRT-LLM
Serving Framework TensorRT-LLM
Inference Backend TensorRT-LLM
Driver —
Link to Container —
Operating System —
Filesystem —
Other Software —
Software Notes —
Processor
Processor Model Name NVIDIA Grace (ARM Neoverse V2)
Processors per Node 1
Cores Per Processor 72
VCPUs Per Processor —
Accelerator
Accelerator Model Name NVIDIA GB300
Accelerators per Node 4
Memory Type —
Memory Capacity 288 GB HBM3e
Accelerator Interconnect NVLink 5 + NVSwitch (NVL72)
Host-Accelerator Interconnect NVLink-C2C (Grace-Blackwell)
Host / Storage
Host Memory Capacity unspecified
Memory Configuration —
Storage Capacity 960 GB Instance Memory, Attached 12,000 GB LSSD
Storage Type —
Networking
Network Interfaces —
Network Details NVLink + RoCEv2
Network Topology NVL72 single-rack NVLink domain
Other
Other Hardware —
Cooling Liquid-cooled
Hardware Notes —
| Field Name | Run 1 | Run 2 | Run 3 | Run 4 | Run 5 | Run 6 | Run 7 | Run 8 | Run 9 |
|---|---|---|---|---|---|---|---|---|---|
| Run Date | 06/26/2026 | 06/26/2026 | 06/26/2026 | 06/26/2026 | 06/26/2026 | 06/26/2026 | 06/26/2026 | 06/26/2026 | 06/26/2026 |
| Concurrency | 4 | 48 | 64 | 256 | 512 | 1,024 | 2,048 | 4,096 | 8,192 |
| System Tokens/Second | 1,234.1 | 10,864.6 | 14,294.5 | 44,857.8 | 82,717.2 | 132,474.9 | 200,860.2 | 325,843.1 | 408,369.4 |
| Tokens/Second per User | 308.5 | 226.3 | 223.4 | 175.2 | 161.6 | 129.4 | 98.1 | 79.6 | 49.8 |
| TTFT P95 (ms) | 192.6 | 281.8 | 292.0 | 2465.6 | 309.2 | 322.6 | 1428.5 | 2908.7 | 6380.1 |
| Utilization | 0.3% | 2.7% | 3.5% | 11.0% | 20.3% | 32.4% | 49.2% | 79.8% | 100.0% |
| Configuration Summary | dsr1-gb300x72-pareto-nvl72-c4 · submission | dsr1-gb300x72-pareto-nvl72-c48 · submission | dsr1-gb300x72-pareto-nvl72-c64 · submission | dsr1-gb300x72-pareto-nvl72-c256 · submission | dsr1-gb300x72-pareto-nvl72-c512 · submission | dsr1-gb300x72-pareto-nvl72-c1024 · submission | dsr1-gb300x72-pareto-nvl72-c2048 · submission | dsr1-gb300x72-pareto-nvl72-c4096 · submission | dsr1-gb300x72-pareto-nvl72-c8192 · submission |
| Config Type | submission | submission | submission | submission | submission | submission | submission | submission | submission |
| Config Region | — | — | — | — | — | — | — | — | — |
| Target QPS | — | — | — | — | — | — | — | — | — |
| Result State | complete | complete | complete | complete | complete | complete | complete | complete | complete |
| Measured Accuracy Score | — | — | — | — | — | — | — | — | — |
| Benchmark Version | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Result Version | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 |
| Archive URI | gs://mlperf-endpoints-blob/runs/4c107b18-9410-4f80-90ce-88f9e75417d4/archive.tar.gz | gs://mlperf-endpoints-blob/runs/0d2b7b67-4ea7-43ee-9e37-54ff64cc5941/archive.tar.gz | gs://mlperf-endpoints-blob/runs/ee9d8ca8-ec42-4daa-b44b-8de26e089410/archive.tar.gz | gs://mlperf-endpoints-blob/runs/03c2178d-6e91-4a2f-b69e-5fae60578750/archive.tar.gz | gs://mlperf-endpoints-blob/runs/afd13761-b8ad-476d-9da5-ade649716249/archive.tar.gz | gs://mlperf-endpoints-blob/runs/06cccf32-a58a-447a-b76e-be5ff31e3f87/archive.tar.gz | gs://mlperf-endpoints-blob/runs/cecfc4d8-b47d-4190-9516-df3ee8f2591a/archive.tar.gz | gs://mlperf-endpoints-blob/runs/a817ab19-0489-41a5-b6ab-22745a8fb213/archive.tar.gz | gs://mlperf-endpoints-blob/runs/d17716a0-752a-4757-b484-4848c0038050/archive.tar.gz |
| QPS | 0.3296 | 2.9008 | 3.8139 | 11.9251 | 22.0091 | 35.1975 | 53.7055 | 88.7754 | 108.7729 |
| Total Output Tokens | 3,744,634 | 24,652,151 | 38,229,521 | 82,529,856 | 280,355,359 | 396,368,631 | 722,095,590 | 1,046,877,185 | 1,317,920,127 |
| Average Output Tokens | 3,744.63 | 3,745.39 | 3,747.99 | 3,761.62 | 3,758.32 | 3,763.76 | 3,740.03 | 3,670.42 | 3,754.33 |
| Run Duration (s) | 3,034.34 | 2,269.04 | 2,674.43 | 1,839.81 | 3,389.32 | 2,992.03 | 3,595.02 | 3,212.83 | 3,227.27 |
| Total Queries | 1,000 | 6,582 | 10,200 | 21,940 | 74,596 | 105,312 | 193,072 | 285,220 | 351,040 |
| Total Queries Issued | 1,000 | 6,582 | 10,200 | 21,940 | 74,596 | 105,312 | 193,072 | 285,220 | 351,040 |
| Total Queries Failed | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Time to First Token (TTFT) P95 (ms) | |||||||||
| Minimum | 139.8 | 147.5 | 149.5 | 148.3 | 150.3 | 151.9 | 162.7 | 174.3 | 189.5 |
| Average | 206.6 | 199.1 | 214.8 | 758.1 | 328.0 | 368.3 | 497.2 | 838.3 | 1547.7 |
| P50 | 160.8 | 163.7 | 164.1 | 296.6 | 233.9 | 261.3 | 298.9 | 337.3 | 345.1 |
| P90 | 176.8 | 257.8 | 268.6 | 1943.7 | 297.3 | 308.5 | 1054.4 | 2064.9 | 4171.9 |
| P95 | 192.6 | 281.8 | 292.0 | 2465.6 | 309.2 | 322.6 | 1428.5 | 2908.7 | 6380.1 |
| P99 | 2136.2 | 337.3 | 511.7 | 3977.7 | 327.7 | 1284.4 | 2345.6 | 7045.2 | 18417.6 |
| P999 | 2371.8 | 2055.4 | 4166.0 | 4306.1 | 16029.1 | 15314.8 | 6665.3 | 15484.2 | 27085.2 |
| Maximum | 2436.1 | 2056.0 | 4167.0 | 4999.2 | 17199.2 | 15483.9 | 7667.5 | 16800.9 | 28647.5 |
| Time Per Output Token (TPOT) (ms) | |||||||||
| Minimum | 0.41 | 2.94 | 2.92 | 1.97 | 4.08 | 4.35 | 4.66 | 3.35 | 4.81 |
| Average | 3.28 | 4.39 | 4.45 | 5.42 | 6.12 | 7.62 | 10.0 | 12.2 | 19.1 |
| P50 | 3.27 | 4.34 | 4.37 | 5.37 | 6.05 | 7.55 | 9.94 | 12.1 | 19.0 |
| P90 | 3.77 | 4.85 | 4.90 | 6.01 | 6.77 | 8.46 | 11.2 | 13.5 | 21.5 |
| P95 | 3.95 | 4.95 | 5.06 | 6.12 | 6.90 | 8.62 | 11.4 | 13.8 | 21.9 |
| P99 | 4.20 | 5.24 | 6.02 | 6.34 | 7.14 | 8.92 | 11.8 | 14.3 | 22.7 |
| P999 | 4.36 | 5.81 | 7.47 | 6.62 | 7.44 | 9.27 | 12.3 | 14.9 | 23.6 |
| Maximum | 4.37 | 6.20 | 9.32 | 6.83 | 7.88 | 14.9 | 13.2 | 16.3 | 25.7 |
| Query Latency (ms) | |||||||||
| Minimum | 790.0 | 819.4 | 790.3 | 1060.3 | 1062.7 | 1323.9 | 1741.0 | 1967.6 | 2410.9 |
| Average | 12125.8 | 16222.8 | 16412.7 | 20639.6 | 22731.9 | 28290.6 | 36986.2 | 44431.7 | 71347.8 |
| P50 | 7189.0 | 9490.7 | 9368.6 | 12191.5 | 13182.7 | 16371.9 | 21314.7 | 26406.0 | 41386.9 |
| P90 | 28816.6 | 40134.1 | 40504.1 | 49976.5 | 55804.9 | 69551.4 | 91463.5 | 111636.5 | 175435.5 |
| P95 | 42836.3 | 56322.0 | 57573.2 | 70677.1 | 79668.1 | 99637.5 | 130027.8 | 158720.9 | 248289.1 |
| P99 | 59022.0 | 79539.8 | 80420.6 | 100511.6 | 112812.6 | 140778.9 | 180455.0 | 180980.1 | 354392.3 |
| P999 | 67050.1 | 87533.5 | 88910.9 | 109949.5 | 122736.7 | 153037.5 | 181135.6 | 181285.6 | 388915.5 |
| Maximum | 75437.5 | 93245.3 | 100102.6 | 134660.7 | 136603.6 | 180878.3 | 184593.9 | 196969.6 | 500784.6 |