1 Node

System 1

Submitted by Submitter 1on 2026-02-19. Published on 2026-03-16

SUT Summary

System Name System 1

System Availability Available

System Category Datacenter · On-premise

System Size 8x Accelerator 1

Model

Model Name GPT-OSS 120B

Division Closed

Model Precision mxfp4

Model Link —

Transformation Link —

Model Notes —

Datasets

Dataset Name MLPerf Inference GPT-OSS Performance Dataset

Dataset Type Performance

Average Input Tokens 5010.64

Average Output Tokens 1300

Dataset Link Link

Measured Accuracy Score —

Throughput vs Interactivity

Throughput vs Concurrency

Time to First Token vs Concurrency

Interactivity vs Concurrency

Hardware

Processor

Processor Model Name Processor 1

Processors per Node 2

Cores Per Processor 32

VCPUs Per Processor —

Accelerator

Accelerator Model Name Accelerator 1

Accelerators per Node 8

Memory Type

Memory Capacity 256 GB

Accelerator Interconnect

Host-Accelerator Interconnect

Host / Storage

Host Memory Capacity 1TB

Memory Configuration

Storage Capacity

Storage Type

Cooling Passive OAM

Hardware Notes

Software

Framework vLLM

Operating System Linux

Other Software —

Software Notes —

Run Data

Field Name	Run 1	Run 2	Run 3	Run 4	Run 5	Run 6	Run 7
Run Date	03/06/2026	03/06/2026	03/06/2026	03/06/2026	03/06/2026	03/06/2026	03/06/2026
Concurrency	0.17	0.69	2.76	5.52	22.09	44.18	88.36
System Tokens/Second	33.5	81.5	185.9	245.4	396.6	420.0	421.0
Tokens/Second per User	193.9	118.0	67.3	44.4	18.0	9.5	4.8
TTFT P99 (ms)	6827.8	4477.1	8742.4	104930.0	495260.4	2639222.7	13065074.1
Utilization	7.9%	19.4%	44.1%	58.3%	94.2%	99.8%	100.0%
Configuration Summary	TP=4, PP=1, precision=FP4, cont_batch	TP=4, PP=1, precision=FP4, cont_batch	TP=4, PP=1, precision=FP4, cont_batch	TP=4, PP=1, precision=FP4, cont_batch	TP=4, PP=1, precision=FP4, cont_batch	TP=4, PP=1, precision=FP4, cont_batch	TP=4, PP=1, precision=FP4, cont_batch
QPS	0.0095	0.0233	0.0534	0.0700	0.1137	0.1198	0.1203
Total Output Tokens	22,427,551	22,387,661	22,246,057	22,407,098	22,315,593	22,417,180	22,387,231
Run Duration (s)	670,138.61	274,800.66	119,680.23	91,317.49	56,266.66	53,371.01	53,173.75
Total Requests	6,396	6,396	6,396	6,396	6,396	6,396	6,396
Time To First Token (TTFT) (ms)
Minimum	1102.4	1465.5	2238.5	3123.6	6984.3	67289.0	170792.0
Average	2202.5	2341.8	3721.9	6111.5	33358.1	1801413.8	8584735.1
P50	1891.0	2172.4	2950.7	3928.8	8866.9	1997308.6	10312446.2
P90	2852.0	2935.0	3957.3	5275.7	12947.1	2251361.2	10597636.7
P95	4255.8	3379.6	4482.1	6069.8	249806.5	2334694.8	10717307.7
P99	6827.8	4477.1	8742.4	104930.0	495260.4	2639222.7	13065074.1
P999	19545.2	16497.0	65084.4	128672.6	515076.7	3665757.0	13464134.1
Maximum	77085.8	16635.5	65091.9	128682.3	515994.7	4736661.4	13494291.7
Time Per Output Token (TPOT) (ms)
Minimum	145.5	352.0	432.5	702.5	2211.6	1897.7	3549.9
Average	2137.7	3553.5	6202.2	9444.5	22406.8	32341.4	32421.9
P50	2001.6	3304.2	5823.3	8870.4	21061.5	30083.7	30179.2
P90	3220.7	5309.1	9254.0	14217.3	33700.6	48990.8	49132.2
P95	3743.8	6191.8	10568.7	16546.4	38556.7	56705.5	56656.6
P99	4887.4	8190.9	13739.7	22185.0	51386.5	78101.7	74276.5
P999	7580.5	15704.9	18697.7	30332.4	76476.4	111552.4	101017.4
Maximum	17399.3	25161.0	34471.8	35229.3	177460.1	147740.0	178619.6
Request Latency (ms)
Minimum	31834.7	66850.1	52051.7	87834.0	356598.9	1431147.1	1782787.1
Average	418971.3	686824.5	1191955.6	1808173.1	4370744.0	8050099.1	14864139.3
P50	400901.5	658928.1	1147500.6	1728456.9	4199659.4	7802771.3	15639162.0
P90	590815.2	968200.8	1673188.9	2551220.0	6126371.9	10714231.9	18714110.3
P95	662359.8	1075910.8	1852093.8	2841350.8	6838981.1	11793383.9	19785209.6
P99	816140.3	1370790.7	2319437.0	3602457.0	8686452.7	14407677.9	22351816.5
P999	1214458.2	1977796.9	2903047.0	5180534.7	11359379.7	19005547.7	25715757.2
Maximum	1263405.5	2068099.4	3528129.1	5417045.0	12910347.2	23026557.6	28913186.6