1 Node

System 3

Submitted by Submitter 3on 2026-02-27. Published on 2026-03-16

SUT Summary

System Name System 3

System Availability Available

System Category Datacenter · Cloud

System Size 8x Accelerator 3

Model

Model Name QWEN3 CODER 480B

Division Open

Model Precision FP8

Model Link —

Transformation Link —

Model Notes —

Datasets

Dataset Name OpenOrca

Dataset Type Performance

Average Input Tokens 190.362

Average Output Tokens 292.545

Dataset Link —

Measured Accuracy Score —

Throughput vs Interactivity

Throughput vs Concurrency

Time to First Token vs Concurrency

Interactivity vs Concurrency

Hardware

Processor

Processor Model Name Processor 3

Processors per Node 2

Cores Per Processor 32

VCPUs Per Processor —

Accelerator

Accelerator Model Name Accelerator 3

Accelerators per Node 8

Memory Type

Memory Capacity 256 GB

Accelerator Interconnect

Host-Accelerator Interconnect

Host / Storage

Host Memory Capacity 1TB

Memory Configuration

Storage Capacity

Storage Type

Cooling Liquid-cooled

Hardware Notes

Software

Framework vLLM

Operating System Linux

Other Software Inference Backend v1.0

Software Notes —

Run Data

Field Name	Run 1	Run 2	Run 3	Run 4	Run 5	Run 6	Run 7
Run Date	02/21/2026	02/21/2026	02/21/2026	02/21/2026	02/22/2026	02/22/2026	02/20/2026
Concurrency	0.54	1.09	2.18	4.36	8.71	17.42	34.85
System Tokens/Second	28.6	50.6	67.9	99.0	132.1	159.9	192.4
Tokens/Second per User	52.5	46.4	31.2	22.7	15.2	9.2	5.5
TTFT P99 (ms)	2865.5	2946.4	3612.2	4613.7	5748.2	37859.1	139725.9
Utilization	14.9%	26.3%	35.3%	51.5%	68.7%	83.1%	100.0%
Configuration Summary	TP=4, PP=1, batch=256, precision=FP8, kv_cache=FP16	TP=4, PP=1, batch=256, precision=FP8, kv_cache=FP16	TP=4, PP=1, batch=256, precision=FP8, kv_cache=FP16	TP=4, PP=1, batch=256, precision=FP8, kv_cache=FP16	TP=4, PP=1, batch=256, precision=FP8, kv_cache=FP16	TP=4, PP=1, batch=256, precision=FP8, kv_cache=FP16	TP=4, PP=1, batch=256, precision=FP8, kv_cache=FP16
QPS	0.0977	0.1728	0.2318	0.3381	0.4516	0.5475	0.6559
Total Output Tokens	7,185,824	7,193,577	7,193,551	7,194,216	7,189,576	7,178,996	7,208,408
Run Duration (s)	251,485.39	142,242.92	106,012.52	72,682.01	54,422.82	44,890.06	37,471.23
Total Requests	24,576	24,576	24,576	24,576	24,576	24,576	24,576
Time To First Token (TTFT) (ms)
Minimum	1001.8	1026.3	1306.2	1568.7	2240.4	3738.1	5850.5
Average	1555.8	1691.9	2109.7	2642.6	3701.2	5922.6	11041.3
P50	1360.7	1413.7	2126.7	2382.9	3392.3	5250.5	8178.2
P90	2289.0	2346.5	2651.8	3296.1	4489.8	6254.1	9907.1
P95	2313.4	2418.0	3111.0	3748.2	4854.5	6723.7	12367.7
P99	2865.5	2946.4	3612.2	4613.7	5748.2	37859.1	139725.9
P999	3820.8	3893.4	30632.3	41814.2	64078.2	107027.8	218317.6
Maximum	46460.1	62353.9	30642.7	41832.4	64094.9	109879.6	221595.2
Time Per Output Token (TPOT) (ms)
Minimum	251.0	268.2	380.4	404.1	443.0	557.3	740.9
Average	275.9	311.9	465.9	638.8	956.2	1577.9	2608.6
P50	275.4	311.4	465.3	637.9	956.5	1579.5	2616.0
P90	284.0	323.4	480.6	658.1	981.9	1611.5	2669.7
P95	287.0	327.7	486.4	666.3	991.3	1624.0	2687.5
P99	296.1	339.6	503.2	692.2	1020.2	1665.3	2763.8
P999	346.1	397.8	578.8	796.6	1183.7	1927.4	3158.6
Maximum	908.4	1078.4	981.9	1407.3	2162.1	2862.1	6114.9
Request Latency (ms)
Minimum	1602.3	2122.7	2075.1	2833.5	3429.6	5403.8	9214.5
Average	81843.3	92557.4	137833.4	188675.6	281970.4	464204.9	770972.4
P50	77034.7	86764.7	129219.5	176393.1	264120.3	435826.5	723171.0
P90	122711.6	139499.9	206452.1	283520.8	422165.1	696626.9	1157073.1
P95	146902.1	167684.8	247319.9	340069.0	508749.4	834686.4	1397625.8
P99	236594.8	266247.5	409381.0	561709.5	820841.1	1338484.2	2235110.6
P999	286668.0	324777.3	480694.4	659355.3	987455.5	1625664.1	2695036.9
Maximum	293986.8	373253.6	488764.1	668775.9	1000196.8	1672530.6	2801331.5