Submitter Google
Submission ID google-Ironwood-192GBx8_vLLM
Submission Status —
Submission Date 2026-02-27
Publication Date 2026-03-16
Division Open
Model Name QWEN3 CODER 480B
Division Open
Model Precision FP8
Model Link Google Cloud
Calibration Link —
Model Notes —
Dataset Name OpenOrca
Dataset Type Performance
Average Input Tokens -1
Dataset Link —
System TPS vs. Tokens/s per User
System TPS vs. Concurrency
Time to First Token P95 vs. Concurrency
Tokens/s per User vs. Concurrency
System Name Google Ironwood-16 (8x Ironwood-192GB, vLLM)
System Availability Available
System Category Datacenter · Cloud
System Size 8x Ironwood
Node Types 1
Total Node Count 2
Max Supported Concurrency —
Node Type ID 1
Node Count 2
Framework vllm/vllm-tpu:nightly-ironwood-20260217-9d37d31-d00df62
Serving Framework vllm/vllm-tpu:nightly-ironwood-20260217-9d37d31-d00df62
Inference Backend —
Driver —
Link to Container —
Operating System Ubuntu 24.04.3 LTS
Filesystem —
Other Software Jax[tpu] 0.8.1
Software Notes —
Processor
Processor Model Name —
Processors per Node 2
Cores Per Processor 56
VCPUs Per Processor —
Accelerator
Accelerator Model Name Google TPU7x
Accelerators per Node 4
Memory Type HBM3E
Memory Capacity 192 GB
Accelerator Interconnect 6x ICI Links, 1200 GB/s bidirectional bandwidth
Host-Accelerator Interconnect PCIe Gen5 x16
Host / Storage
Host Memory Capacity 960 GB
Memory Configuration —
Storage Capacity —
Storage Type —
Networking
Network Interfaces 2
Network Details Ethernet; Data bandwidth per TPU chip: 100 Gbps
Network Topology —
Other
Other Hardware —
Cooling Liquid-cooled
Hardware Notes —
| Field Name | Run 1 | Run 2 | Run 3 | Run 4 | Run 5 | Run 6 | Run 7 | Run 8 |
|---|---|---|---|---|---|---|---|---|
| Run Date | 02/25/2026 | 02/25/2026 | 02/25/2026 | 02/25/2026 | 02/25/2026 | 02/24/2026 | 02/24/2026 | 02/28/2026 |
| Concurrency | 1 | 1 | 2 | 4 | 9 | 17 | 35 | 70 |
| System Tokens/Second | 12.0 | 22.9 | 38.8 | 64.4 | 98.5 | 131.4 | 171.6 | 199.8 |
| Tokens/Second per User | 22.0 | 21.0 | 17.8 | 14.8 | 11.3 | 7.5 | 4.9 | 2.9 |
| TTFT P95 (ms) | 2627.7 | 2663.1 | 2848.8 | 3367.6 | 4035.8 | 5442.3 | 9498.4 | 15134.0 |
| Utilization | 6.0% | 11.5% | 19.4% | 32.2% | 49.3% | 65.8% | 85.9% | 100.0% |
| Configuration Summary | TP=16, PP=1, batch=512, precision=FP8, kv_cache=FP8 | TP=16, PP=1, batch=512, precision=FP8, kv_cache=FP8 | TP=16, PP=1, batch=512, precision=FP8, kv_cache=FP8 | TP=16, PP=1, batch=512, precision=FP8, kv_cache=FP8 | TP=16, PP=1, batch=512, precision=FP8, kv_cache=FP8 | TP=16, PP=1, batch=512, precision=FP8, kv_cache=FP8 | TP=16, PP=1, batch=512, precision=FP8, kv_cache=FP8 | TP=16, PP=1, batch=1024, precision=FP8, kv_cache=FP8 |
| Config Type | — | — | — | — | — | — | — | — |
| Config Region | — | — | — | — | — | — | — | — |
| Target QPS | — | — | — | — | — | — | — | — |
| Result State | — | — | — | — | — | — | — | — |
| Measured Accuracy Score | — | — | — | — | — | — | — | — |
| Benchmark Version | — | — | — | — | — | — | — | — |
| Result Version | — | — | — | — | — | — | — | — |
| Archive URI | — | — | — | — | — | — | — | — |
| QPS | 0.0411 | 0.0780 | 0.1326 | 0.2199 | 0.3358 | 0.4478 | 0.5863 | 0.6833 |
| Total Output Tokens | 7,176,538 | 7,214,114 | 7,189,678 | 7,198,828 | 7,208,951 | 7,211,649 | 7,193,097 | 7,186,180 |
| Average Output Tokens | — | — | — | — | — | — | — | — |
| Run Duration (s) | 40,699.33 | 21,458.44 | 12,616.63 | 7,605.87 | 4,981.65 | 3,735.86 | 2,852.96 | 2,448.21 |
| Total Queries | 24,576 | 24,576 | 24,576 | 24,576 | 24,576 | 24,576 | 24,576 | 24,576 |
| Total Queries Issued | — | — | — | — | — | — | — | — |
| Total Queries Failed | — | — | — | — | — | — | — | — |
| Time to First Token (TTFT) P95 (ms) | ||||||||
| Minimum | 1163.2 | 1297.8 | 1511.0 | 1747.7 | 1953.1 | 3331.7 | 4600.8 | 6622.7 |
| Average | 1940.9 | 2015.4 | 2257.4 | 2587.5 | 3256.1 | 4728.4 | 9048.3 | 23312.8 |
| P50 | 1802.2 | 1834.4 | 2003.0 | 2518.9 | 3109.1 | 4298.8 | 6331.7 | 11657.2 |
| P90 | 2598.0 | 2633.7 | 2796.1 | 2994.6 | 3913.8 | 5061.3 | 7152.1 | 14908.6 |
| P95 | 2627.7 | 2663.1 | 2848.8 | 3367.6 | 4035.8 | 5442.3 | 9498.4 | 15134.0 |
| P99 | 2736.7 | 3136.5 | 3651.6 | 4022.0 | 4986.4 | 12923.1 | 156250.0 | 408064.0 |
| P999 | 3543.2 | 3612.9 | 11091.1 | 22970.7 | 40832.6 | 81758.8 | 222715.0 | 483273.0 |
| Maximum | 3966.8 | 16722.5 | 11100.9 | 22986.0 | 43035.1 | 84065.5 | 234174.1 | 493677.7 |
| Time Per Output Token (TPOT) (ms) | ||||||||
| Minimum | 635.8 | 660.5 | 696.5 | 727.7 | 772.9 | 881.4 | 1054.2 | 1450.0 |
| Average | 663.0 | 695.3 | 819.6 | 985.2 | 1285.4 | 1921.7 | 2920.8 | 4955.7 |
| P50 | 661.9 | 694.2 | 818.4 | 983.9 | 1284.7 | 1922.4 | 2925.7 | 5002.4 |
| P90 | 671.3 | 706.2 | 833.4 | 1004.5 | 1311.0 | 1952.7 | 2971.5 | 5117.6 |
| P95 | 675.1 | 710.7 | 839.2 | 1012.1 | 1322.5 | 1966.4 | 2989.4 | 5153.0 |
| P99 | 688.3 | 727.3 | 861.9 | 1040.1 | 1365.6 | 2023.1 | 3105.7 | 5294.5 |
| P999 | 795.2 | 863.5 | 992.3 | 1172.9 | 1506.0 | 2306.1 | 3630.0 | 6425.4 |
| Maximum | 1423.2 | 1319.2 | 1562.2 | 1900.5 | 2315.8 | 4659.9 | 5767.2 | 10909.1 |
| Query Latency (ms) | ||||||||
| Minimum | 2466.5 | 2518.6 | 3101.7 | 3327.1 | 3771.5 | 5382.7 | 8257.9 | 12619.4 |
| Average | 194604.9 | 205125.7 | 240867.9 | 289764.2 | 378403.3 | 565472.2 | 858459.6 | 1460604.1 |
| P50 | 183004.6 | 192299.8 | 225788.1 | 271819.4 | 354544.8 | 528939.8 | 805844.7 | 1373089.6 |
| P90 | 292188.1 | 308934.3 | 361530.2 | 435060.4 | 569214.8 | 848320.2 | 1288795.1 | 2196319.0 |
| P95 | 351626.5 | 371746.5 | 432422.3 | 524562.4 | 684345.1 | 1030600.4 | 1539162.7 | 2604095.7 |
| P99 | 555123.0 | 604472.1 | 719448.1 | 858295.7 | 1112878.5 | 1643599.7 | 2476371.9 | 4150556.0 |
| P999 | 681548.8 | 716491.2 | 842323.4 | 1011852.9 | 1321871.6 | 1972664.0 | 3007519.3 | 5166714.8 |
| Maximum | 705785.9 | 742232.3 | 872021.2 | 1026426.2 | 1342664.0 | 1995493.6 | 3158377.3 | 5468721.5 |