Submitter Google
Submission ID google-Ironwood-192GBx4_vLLM
Submission Status —
Submission Date 2026-02-27
Publication Date 2026-03-16
Division Open
Model Name QWEN3 CODER 480B
Division Open
Model Precision FP8
Model Link Google Cloud
Calibration Link —
Model Notes —
Dataset Name OpenOrca
Dataset Type Performance
Average Input Tokens -1
Dataset Link —
System TPS vs. Tokens/s per User
System TPS vs. Concurrency
Time to First Token P95 vs. Concurrency
Tokens/s per User vs. Concurrency
System Name Google Ironwood-8 (4x Ironwood-192GB, vLLM)
System Availability Available
System Category Datacenter · Cloud
System Size 4x Ironwood
Node Types 1
Total Node Count 1
Max Supported Concurrency —
Node Type ID 1
Node Count 1
Framework vllm/vllm-tpu:nightly-ironwood-20260217-9d37d31-d00df62
Serving Framework vllm/vllm-tpu:nightly-ironwood-20260217-9d37d31-d00df62
Inference Backend —
Driver —
Link to Container —
Operating System Ubuntu 24.04.3 LTS
Filesystem —
Other Software Jax[tpu] 0.8.1
Software Notes —
Processor
Processor Model Name —
Processors per Node 2
Cores Per Processor 56
VCPUs Per Processor —
Accelerator
Accelerator Model Name Google TPU7x
Accelerators per Node 4
Memory Type HBM3E
Memory Capacity 192 GB
Accelerator Interconnect 6x ICI Links, 1200 GB/s bidirectional bandwidth
Host-Accelerator Interconnect PCIe Gen5 x16
Host / Storage
Host Memory Capacity 960 GB
Memory Configuration —
Storage Capacity —
Storage Type —
Networking
Network Interfaces 2
Network Details Ethernet; Data bandwidth per TPU chip: 100 Gbps
Network Topology —
Other
Other Hardware —
Cooling Liquid-cooled
Hardware Notes —
| Field Name | Run 1 | Run 2 | Run 3 | Run 4 | Run 5 | Run 6 | Run 7 |
|---|---|---|---|---|---|---|---|
| Run Date | 02/21/2026 | 02/21/2026 | 02/21/2026 | 02/21/2026 | 02/22/2026 | 02/22/2026 | 02/20/2026 |
| Concurrency | 1 | 1 | 2 | 4 | 9 | 17 | 35 |
| System Tokens/Second | 28.6 | 50.6 | 67.9 | 99.0 | 132.1 | 159.9 | 192.4 |
| Tokens/Second per User | 52.5 | 46.4 | 31.2 | 22.7 | 15.2 | 9.2 | 5.5 |
| TTFT P95 (ms) | 2313.4 | 2418.0 | 3111.0 | 3748.2 | 4854.5 | 6723.7 | 12367.7 |
| Utilization | 14.9% | 26.3% | 35.3% | 51.5% | 68.7% | 83.1% | 100.0% |
| Configuration Summary | TP=8, PP=1, batch=512, precision=FP8, kv_cache=FP8 | TP=8, PP=1, batch=512, precision=FP8, kv_cache=FP8 | TP=8, PP=1, batch=512, precision=FP8, kv_cache=FP8 | TP=8, PP=1, batch=512, precision=FP8, kv_cache=FP8 | TP=8, PP=1, batch=512, precision=FP8, kv_cache=FP8 | TP=8, PP=1, batch=512, precision=FP8, kv_cache=FP8 | TP=8, PP=1, batch=512, precision=FP8, kv_cache=FP8 |
| Config Type | — | — | — | — | — | — | — |
| Config Region | — | — | — | — | — | — | — |
| Target QPS | — | — | — | — | — | — | — |
| Result State | — | — | — | — | — | — | — |
| Measured Accuracy Score | — | — | — | — | — | — | — |
| Benchmark Version | — | — | — | — | — | — | — |
| Result Version | — | — | — | — | — | — | — |
| Archive URI | — | — | — | — | — | — | — |
| QPS | 0.0977 | 0.1728 | 0.2318 | 0.3381 | 0.4516 | 0.5475 | 0.6559 |
| Total Output Tokens | 7,185,824 | 7,193,577 | 7,193,551 | 7,194,216 | 7,189,576 | 7,178,996 | 7,208,408 |
| Average Output Tokens | — | — | — | — | — | — | — |
| Run Duration (s) | 17,117.17 | 9,681.66 | 7,215.66 | 4,947.05 | 3,704.25 | 3,055.41 | 2,550.45 |
| Total Queries | 24,576 | 24,576 | 24,576 | 24,576 | 24,576 | 24,576 | 24,576 |
| Total Queries Issued | — | — | — | — | — | — | — |
| Total Queries Failed | — | — | — | — | — | — | — |
| Time to First Token (TTFT) P95 (ms) | |||||||
| Minimum | 1001.8 | 1026.3 | 1306.2 | 1568.7 | 2240.4 | 3738.1 | 5850.5 |
| Average | 1555.8 | 1691.9 | 2109.7 | 2642.6 | 3701.2 | 5922.6 | 11041.3 |
| P50 | 1360.7 | 1413.7 | 2126.7 | 2382.9 | 3392.3 | 5250.5 | 8178.2 |
| P90 | 2289.0 | 2346.5 | 2651.8 | 3296.1 | 4489.8 | 6254.1 | 9907.1 |
| P95 | 2313.4 | 2418.0 | 3111.0 | 3748.2 | 4854.5 | 6723.7 | 12367.7 |
| P99 | 2865.5 | 2946.4 | 3612.2 | 4613.7 | 5748.2 | 37859.1 | 139725.9 |
| P999 | 3820.8 | 3893.4 | 30632.3 | 41814.2 | 64078.2 | 107027.8 | 218317.6 |
| Maximum | 46460.1 | 62353.9 | 30642.7 | 41832.4 | 64094.9 | 109879.6 | 221595.2 |
| Time Per Output Token (TPOT) (ms) | |||||||
| Minimum | 251.0 | 268.2 | 380.4 | 404.1 | 443.0 | 557.3 | 740.9 |
| Average | 275.9 | 311.9 | 465.9 | 638.8 | 956.2 | 1577.9 | 2608.6 |
| P50 | 275.4 | 311.4 | 465.3 | 637.9 | 956.5 | 1579.5 | 2616.0 |
| P90 | 284.0 | 323.4 | 480.6 | 658.1 | 981.9 | 1611.5 | 2669.7 |
| P95 | 287.0 | 327.7 | 486.4 | 666.3 | 991.3 | 1624.0 | 2687.5 |
| P99 | 296.1 | 339.6 | 503.2 | 692.2 | 1020.2 | 1665.3 | 2763.8 |
| P999 | 346.1 | 397.8 | 578.8 | 796.6 | 1183.7 | 1927.4 | 3158.6 |
| Maximum | 908.4 | 1078.4 | 981.9 | 1407.3 | 2162.1 | 2862.1 | 6114.9 |
| Query Latency (ms) | |||||||
| Minimum | 1602.3 | 2122.7 | 2075.1 | 2833.5 | 3429.6 | 5403.8 | 9214.5 |
| Average | 81843.3 | 92557.4 | 137833.4 | 188675.6 | 281970.4 | 464204.9 | 770972.4 |
| P50 | 77034.7 | 86764.7 | 129219.5 | 176393.1 | 264120.3 | 435826.5 | 723171.0 |
| P90 | 122711.6 | 139499.9 | 206452.1 | 283520.8 | 422165.1 | 696626.9 | 1157073.1 |
| P95 | 146902.1 | 167684.8 | 247319.9 | 340069.0 | 508749.4 | 834686.4 | 1397625.8 |
| P99 | 236594.8 | 266247.5 | 409381.0 | 561709.5 | 820841.1 | 1338484.2 | 2235110.6 |
| P999 | 286668.0 | 324777.3 | 480694.4 | 659355.3 | 987455.5 | 1625664.1 | 2695036.9 |
| Maximum | 293986.8 | 373253.6 | 488764.1 | 668775.9 | 1000196.8 | 1672530.6 | 2801331.5 |