Submitter NVIDIA
Submission ID 1907bca4-63a0-4b60-ba97-ef47b657d195
Submission Status PUBLISHED
Submission Date 2026-06-25
Publication Date —
Division Standardized
Model Name GPT-OSS 120B
Division Standardized
Model Precision —
Model Link —
Calibration Link —
Model Notes —
Dataset Name perf
Dataset Type performance
Average Input Tokens —
Dataset Link —
System TPS vs. Tokens/s per User
System TPS vs. Concurrency
Time to First Token P95 vs. Concurrency
Tokens/s per User vs. Concurrency
System Name NVIDIA GB300 NVL72
System Availability Available
System Category Datacenter · GB300 NVL72 rack, disaggregated TRT-LLM, MXFP4 weights
System Size —
Node Types 1
Total Node Count 18
Max Supported Concurrency 22,528
Node Type ID 0
Node Count 18
Framework TensorRT-LLM
Serving Framework TensorRT-LLM
Inference Backend TensorRT-LLM
Driver —
Link to Container —
Operating System unspecified
Filesystem —
Other Software —
Software Notes —
Processor
Processor Model Name NVIDIA Grace (ARM Neoverse V2)
Processors per Node 1
Cores Per Processor 72
VCPUs Per Processor —
Accelerator
Accelerator Model Name NVIDIA GB300
Accelerators per Node 4
Memory Type —
Memory Capacity 288 GB HBM3e
Accelerator Interconnect NVLink 5 + NVSwitch (NVL72)
Host-Accelerator Interconnect NVLink-C2C (Grace-Blackwell)
Host / Storage
Host Memory Capacity unspecified
Memory Configuration —
Storage Capacity unspecified
Storage Type unspecified
Networking
Network Interfaces —
Network Details NVLink + InfiniBand
Network Topology NVL72 single-rack NVLink domain
Other
Other Hardware —
Cooling Liquid-cooled
Hardware Notes —
| Field Name | Run 1 | Run 2 | Run 3 | Run 4 | Run 5 | Run 6 | Run 7 | Run 8 | Run 9 | Run 10 | Run 11 | Run 12 | Run 13 | Run 14 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Run Date | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 | 06/25/2026 |
| Concurrency | 8 | 17 | 35 | 70 | 140 | 272 | 544 | 1,024 | 2,048 | 3,840 | 7,168 | 12,288 | 12,800 | 22,528 |
| System Tokens/Second | 3,226.6 | 6,684.7 | 12,934.2 | 23,078.1 | 41,092.7 | 71,693.1 | 125,323.6 | 192,455.6 | 303,515.8 | 441,207.9 | 628,875.2 | 778,099.7 | 797,819.5 | 954,230.2 |
| Tokens/Second per User | 403.3 | 393.2 | 369.5 | 329.7 | 293.5 | 263.6 | 230.4 | 187.9 | 148.2 | 114.9 | 87.7 | 63.3 | 62.3 | 42.4 |
| TTFT P95 (ms) | 156.7 | 112.8 | 119.5 | 128.1 | 190.1 | 885.2 | 568.4 | 877.3 | 782.3 | 799.2 | 6115.8 | 752.4 | 1025.8 | 2472.0 |
| Utilization | 0.3% | 0.7% | 1.4% | 2.4% | 4.3% | 7.5% | 13.1% | 20.2% | 31.8% | 46.2% | 65.9% | 81.5% | 83.6% | 100.0% |
| Configuration Summary | gpt-oss-120b-sil_tep8_sys8_ng8_nc8_g1_c1_fe1 · online · c8 | gpt-oss-120b-sil_tep4_sys17_ng17_nc4_g1_c1_fe1 · online · c17 | gpt-oss-120b-sil_tep2_sys35_ng35_nc2_g1_c1_fe1 · online · c35 | gpt-oss-120b-sil_tep2_sys70_ng35_nc2_g2_c1_fe1 · online · c70 | gpt-oss-120b-sil_tep2_sys140_ng35_nc2_g4_c2_fe1 · online · c140 | gpt-oss-120b-sil_tep4_sys272_ng17_nc4_g16_c2_fe1 · online · c272 | gpt-oss-120b-sil_tep4_sys544_ng17_nc4_g32_c4_fe1 · online · c544 | gpt-oss-120b-sil_tep4_sys1024_ng16_nc8_g64_c2_fe16 · online · c1024 | gpt-oss-120b-sil_dep8_sys2048_ng8_nc8_g256_c16_fe8 · online · c2048 | gpt-oss-120b-sil_dep4_sys3840_ng15_nc12_g256_c8_fe15 · online · c3840 | gpt-oss-120b-sil_dep8_sys7168_ng7_nc16_g1024_c32_fe7 · online · c7168 | gpt-oss-120b-sil_dep2_sys12288_ng25_nc22_g512_c8_fe25 · online · c12288 | gpt-oss-120b-sil_dep2_sys12800_ng25_nc22_g512_c8_fe25 · online · c12800 | gpt-oss-120b-sil_tep1_sys22528_ng48_nc24_g512_c32_fe48 · online · c22528 |
| Config Type | online | online | online | online | online | online | online | online | online | online | online | online | online | online |
| Config Region | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| Target QPS | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| Result State | complete | complete | complete | complete | complete | complete | complete | complete | complete | complete | complete | complete | complete | complete |
| Measured Accuracy Score | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| Benchmark Version | 629c1bc | 629c1bc | 629c1bc | 629c1bc | 629c1bc | 629c1bc | 629c1bc | 629c1bc | 629c1bc | 629c1bc | 629c1bc | 629c1bc | 629c1bc | 629c1bc |
| Result Version | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 | 0.1.0 |
| Archive URI | gs://mlperf-endpoints-blob/runs/cad766b3-9ddc-43f4-b3b7-d3e7fb77682b/archive.tar.gz | gs://mlperf-endpoints-blob/runs/f33b91e1-bc12-4ad7-924d-040f209814a1/archive.tar.gz | gs://mlperf-endpoints-blob/runs/4abe5c9c-6f73-4932-8bc9-18bb33783b10/archive.tar.gz | gs://mlperf-endpoints-blob/runs/88c4775c-259c-4702-a325-2099bbd217b6/archive.tar.gz | gs://mlperf-endpoints-blob/runs/d18a2c56-620e-4e13-8789-2ac6e18b1cba/archive.tar.gz | gs://mlperf-endpoints-blob/runs/8acf867c-9a7c-4430-8839-1ddf462d057c/archive.tar.gz | gs://mlperf-endpoints-blob/runs/8666872a-2d6a-481c-b3eb-d22fa6b39a0b/archive.tar.gz | gs://mlperf-endpoints-blob/runs/736ea21d-18f6-437c-9aa1-5777550e2ee4/archive.tar.gz | gs://mlperf-endpoints-blob/runs/655970e0-1055-419f-be6b-ccbb11ff87b7/archive.tar.gz | gs://mlperf-endpoints-blob/runs/e77820f2-0af4-41da-977e-d6685bbe23d0/archive.tar.gz | gs://mlperf-endpoints-blob/runs/4d0be1c5-ead7-47d4-9ce0-8e1fa40bdfcd/archive.tar.gz | gs://mlperf-endpoints-blob/runs/20e46720-5c96-4cf0-bd03-a39ca7b985bb/archive.tar.gz | gs://mlperf-endpoints-blob/runs/6d2df9ef-ccd1-4331-ab57-37c7058ab993/archive.tar.gz | gs://mlperf-endpoints-blob/runs/1c6cf0dd-dcda-4835-9ab3-54475b13d533/archive.tar.gz |
| QPS | 2.6376 | 5.4442 | 10.5227 | 18.7475 | 33.3943 | 58.2258 | 101.8297 | 156.4203 | 246.5849 | 358.7057 | 511.1075 | 632.2840 | 648.4210 | 775.5894 |
| Total Output Tokens | 2,723,113 | 5,513,088 | 10,798,227 | 19,469,442 | 36,811,323 | 64,381,935 | 112,256,121 | 173,690,797 | 266,528,001 | 394,994,669 | 548,465,566 | 659,982,592 | 666,236,494 | 796,780,859 |
| Average Output Tokens | 1,223.32 | 1,227.86 | 1,229.17 | 1,231 | 1,230.53 | 1,231.29 | 1,230.72 | 1,230.37 | 1,230.88 | 1,230 | 1,230.42 | 1,230.62 | 1,230.4 | 1,230.33 |
| Run Duration (s) | 843.96 | 824.73 | 834.86 | 843.63 | 895.81 | 898.02 | 895.73 | 902.50 | 878.14 | 895.26 | 872.14 | 848.20 | 835.07 | 835.00 |
| Total Queries | 2,226 | 4,490 | 8,785 | 15,816 | 29,915 | 52,288 | 91,212 | 141,169 | 216,535 | 321,134 | 445,756 | 536,302 | 541,478 | 647,616 |
| Total Queries Issued | 2,226 | 4,490 | 8,785 | 15,816 | 29,915 | 52,288 | 91,212 | 141,169 | 216,535 | 321,134 | 445,756 | 536,302 | 541,478 | 647,616 |
| Total Queries Failed | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Time to First Token (TTFT) P95 (ms) | ||||||||||||||
| Minimum | 73.9 | 27.4 | 27.7 | 28.5 | 28.4 | 30.1 | 38.4 | 36.9 | 36.9 | 39.2 | 52.9 | 52.7 | 54.9 | 61.1 |
| Average | 137.7 | 87.4 | 89.9 | 89.9 | 111.6 | 244.1 | 288.4 | 398.4 | 457.6 | 515.5 | 2053.4 | 639.6 | 723.4 | 945.8 |
| P50 | 105.4 | 62.3 | 62.8 | 66.6 | 86.6 | 101.9 | 237.5 | 333.8 | 390.5 | 458.9 | 1303.4 | 471.1 | 493.9 | 371.0 |
| P90 | 135.8 | 98.3 | 101.8 | 110.2 | 159.6 | 614.2 | 462.5 | 710.5 | 658.6 | 693.3 | 4545.9 | 646.7 | 806.2 | 1170.4 |
| P95 | 156.7 | 112.8 | 119.5 | 128.1 | 190.1 | 885.2 | 568.4 | 877.3 | 782.3 | 799.2 | 6115.8 | 752.4 | 1025.8 | 2472.0 |
| P99 | 1929.7 | 1003.9 | 1039.9 | 197.4 | 293.6 | 1568.4 | 893.1 | 1282.9 | 1198.5 | 2897.5 | 9945.8 | 9441.6 | 10023.1 | 15715.4 |
| P999 | 2176.2 | 2346.4 | 2472.5 | 2752.0 | 3122.1 | 3782.1 | 4688.5 | 4140.6 | 7052.2 | 9497.2 | 15657.1 | 15539.7 | 16258.0 | 27075.1 |
| Maximum | 2262.0 | 2435.9 | 2546.2 | 2875.5 | 3798.5 | 4022.2 | 6165.1 | 4936.4 | 7892.7 | 10793.5 | 16244.2 | 17152.8 | 17855.3 | 28043.3 |
| Time Per Output Token (TPOT) (ms) | ||||||||||||||
| Minimum | 1.43 | 1.37 | 1.30 | 1.65 | 1.69 | 2.16 | 2.14 | 3.02 | 4.30 | 4.92 | 4.56 | 7.39 | 7.31 | 8.56 |
| Average | 2.39 | 2.46 | 2.62 | 2.95 | 3.30 | 3.58 | 4.09 | 4.98 | 6.32 | 8.22 | 9.58 | 15.1 | 15.2 | 22.4 |
| P50 | 2.39 | 2.47 | 2.62 | 2.95 | 3.29 | 3.56 | 4.08 | 4.98 | 6.31 | 8.26 | 9.58 | 15.1 | 15.3 | 22.5 |
| P90 | 2.43 | 2.53 | 2.73 | 3.10 | 3.44 | 3.73 | 4.17 | 5.04 | 6.45 | 8.37 | 9.79 | 15.2 | 15.4 | 23.0 |
| P95 | 2.44 | 2.56 | 2.76 | 3.14 | 3.48 | 3.74 | 4.20 | 5.05 | 6.49 | 8.39 | 9.84 | 15.3 | 15.4 | 23.1 |
| P99 | 2.46 | 2.61 | 2.84 | 3.24 | 3.55 | 3.77 | 4.24 | 5.08 | 6.56 | 8.45 | 9.95 | 15.4 | 15.5 | 23.3 |
| P999 | 2.50 | 2.66 | 2.94 | 3.36 | 3.66 | 3.78 | 4.29 | 5.11 | 6.80 | 8.48 | 9.99 | 15.4 | 15.6 | 23.5 |
| Maximum | 2.79 | 2.68 | 3.16 | 4.26 | 5.27 | 4.37 | 4.37 | 5.95 | 7.36 | 13.6 | 18.5 | 23.9 | 31.4 | 36.5 |
| Query Latency (ms) | ||||||||||||||
| Minimum | 1276.0 | 1289.0 | 1254.7 | 1437.9 | 1526.9 | 1753.5 | 2077.3 | 2557.7 | 674.1 | 3869.4 | 4718.6 | 6827.5 | 1216.9 | 7716.8 |
| Average | 3025.8 | 3114.4 | 3319.3 | 3723.2 | 4175.2 | 4654.5 | 5318.5 | 6514.9 | 8226.3 | 10616.0 | 13827.2 | 19147.9 | 19424.3 | 28436.1 |
| P50 | 2996.9 | 3058.5 | 3261.6 | 3663.0 | 4111.3 | 4572.7 | 5245.4 | 6437.4 | 8118.6 | 10463.1 | 13467.7 | 18845.5 | 19111.0 | 27928.3 |
| P90 | 3806.4 | 3907.6 | 4200.2 | 4727.2 | 5230.0 | 5863.8 | 6621.0 | 8109.4 | 10226.6 | 13213.7 | 17780.6 | 23958.1 | 24305.8 | 35742.1 |
| P95 | 4063.2 | 4212.5 | 4531.4 | 5067.6 | 5628.3 | 6317.0 | 7088.5 | 8671.5 | 10945.4 | 14165.2 | 19425.9 | 25804.1 | 26156.9 | 38610.5 |
| P99 | 4691.1 | 4970.4 | 5194.4 | 5847.5 | 6483.1 | 7365.5 | 8211.0 | 9919.6 | 12728.8 | 16614.7 | 23100.0 | 30546.3 | 31064.8 | 46507.8 |
| P999 | 5543.1 | 5828.4 | 6168.6 | 7236.8 | 7879.3 | 8578.6 | 10185.7 | 11864.8 | 15836.8 | 20800.8 | 28044.7 | 37447.3 | 38081.6 | 57631.6 |
| Maximum | 6144.3 | 6781.1 | 7099.6 | 8216.9 | 9591.8 | 10540.3 | 12881.9 | 16847.3 | 25519.3 | 28471.9 | 37424.7 | 66361.9 | 60524.8 | 93108.6 |