1 Node

System 2

Submitted by Submitter 2on 2026-02-19. Published on 2026-03-16

SUT Summary

System Name System 2

System Availability Available

System Category Datacenter · On-premise

System Size 8x Accelerator 2

Model

Model Name GPT-OSS 120B

Division Closed

Model Precision mxfp4

Model Link —

Transformation Link —

Model Notes —

Datasets

Dataset Name MLPerf Inference GPT-OSS Performance Dataset

Dataset Type Performance

Average Input Tokens 5010.64

Average Output Tokens 1300

Dataset Link Link

Measured Accuracy Score —

Throughput vs Interactivity

Throughput vs Concurrency

Time to First Token vs Concurrency

Interactivity vs Concurrency

Hardware

Processor

Processor Model Name Processor 2

Processors per Node 2

Cores Per Processor 32

VCPUs Per Processor —

Accelerator

Accelerator Model Name Accelerator 2

Accelerators per Node 8

Memory Type

Memory Capacity 256 GB

Accelerator Interconnect

Host-Accelerator Interconnect

Host / Storage

Host Memory Capacity 1TB

Memory Configuration

Storage Capacity

Storage Type

Cooling Passive & Active

Hardware Notes

Software

Framework Inference Framework

Operating System Linux

Other Software Inference Backend v1.0

Software Notes —

Run Data

Field Name	Run 1	Run 2	Run 3	Run 4	Run 5	Run 6	Run 7
Run Date	03/07/2026	03/07/2026	03/07/2026	03/07/2026	03/07/2026	03/06/2026	03/06/2026
Concurrency	0.17	0.69	2.76	5.52	22.09	44.18	88.36
System Tokens/Second	31.7	73.4	151.7	210.7	350.8	376.2	372.9
Tokens/Second per User	183.6	106.4	54.9	38.2	15.9	8.5	4.2
TTFT P99 (ms)	7376.1	8202.7	11045.0	117445.0	566319.4	2744948.4	15132101.2
Utilization	8.4%	19.5%	40.3%	56.0%	93.2%	100.0%	99.1%
Configuration Summary	TP=4, PP=1, precision=FP4, cont_batch	TP=4, PP=1, precision=FP4, cont_batch	TP=4, PP=1, precision=FP4, cont_batch	TP=4, PP=1, precision=FP4, cont_batch	TP=4, PP=1, precision=FP4, cont_batch	TP=4, PP=1, precision=FP4, cont_batch	TP=4, PP=1, precision=FP4, cont_batch
QPS	0.0091	0.0210	0.0433	0.0605	0.1002	0.1080	0.1067
Total Output Tokens	22,330,923	22,403,343	22,395,866	22,291,611	22,400,639	22,287,872	22,359,894
Run Duration (s)	704,769.16	305,025.16	147,636.54	105,795.26	63,854.21	59,237.60	59,959.01
Total Requests	6,396	6,396	6,396	6,396	6,396	6,396	6,396
Time To First Token (TTFT) (ms)
Minimum	1207.9	1712.3	2670.7	3615.5	7406.9	45461.3	89405.5
Average	2412.8	3037.6	4465.2	7085.5	36925.8	1982966.9	9690425.2
P50	2037.9	2563.8	3602.7	4626.0	10193.6	2202151.1	11589523.0
P90	3280.8	3806.0	4900.9	6260.9	14862.2	2509911.0	11942149.5
P95	4566.0	5113.7	5500.6	7108.0	264765.3	2620285.3	12637373.6
P99	7376.1	8202.7	11045.0	117445.0	566319.4	2744948.4	15132101.2
P999	20430.0	62938.6	71212.7	146420.0	587808.0	4306026.5	15466318.2
Maximum	80479.0	63080.0	71216.6	146426.4	588829.6	4856981.9	15499261.8
Time Per Output Token (TPOT) (ms)
Minimum	208.6	285.4	580.5	1024.9	2462.4	3112.3	3618.8
Average	2235.1	3848.5	7447.6	10666.3	25096.6	35986.3	37346.0
P50	2087.1	3619.0	7029.1	9968.7	23597.8	33468.7	34524.2
P90	3347.2	5749.7	11140.2	15912.8	37342.6	54886.8	57178.2
P95	3861.2	6629.2	12782.2	18420.7	42956.1	63297.9	66992.4
P99	5205.9	8807.8	16761.7	24646.9	57081.6	86215.9	89453.9
P999	7781.7	14330.1	25677.3	38501.1	86461.2	124100.5	127450.4
Maximum	11687.0	23560.8	31628.6	99617.7	220982.8	267943.5	198467.8
Request Latency (ms)
Minimum	37316.7	73242.8	116075.6	205669.1	532022.7	1639480.2	1859259.4
Average	440623.1	762347.8	1470411.1	2098297.8	4955298.9	8922224.6	16807261.7
P50	421480.3	732175.5	1407452.8	2011210.3	4754899.8	8700339.4	17643330.6
P90	624313.2	1073267.1	2069158.6	2964609.7	6960485.7	11852587.0	21171967.4
P95	694314.8	1196354.9	2318143.5	3282161.1	7835440.8	13119819.9	22380592.9
P99	865105.2	1493647.6	2900401.8	4066207.2	9849525.8	15826930.2	24866288.1
P999	1082802.2	2095815.0	3530041.7	5529029.4	13079076.1	20137359.4	28998216.2
Maximum	1302289.9	2284929.8	4351186.1	6185412.0	14710440.2	24253892.7	32271645.3