Menu
Back previous/next report: k j

Google GB300-NVL72_GB300-288GB_aarch64x4_vLLM-Dynamo

Submitted by Googleon 2026-10-16 Published on 2026-11-04

Result statement

Google ran Qwen3.6 35B-A3B on GB300-NVL72_GB300-288GB_aarch64x4_vLLM-Dynamo and achieved a verified 35,238.5 tokens/second System TPS at 387.3ms TTFT P90 in the Standardized division of the Available category on November 4, 2026 .

[1] MLPerf Endpoints v1.0 Qwen3.6 35B-A3B, published November 4, 2026; retrieved from endpoints.mlcommons.org on November 16, 2026. Result —, run 9: concurrency = 768 queries, TTFT P90 = 387.3ms, System TPS = 35,238.5 tokens/sec, interactivity = 45.9 tokens/sec per user and total provisioned system power = 8.43 kW (MLC Estimated Power). Result verified by MLCommons Association. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.

Submission Info
Google
0308b5f3-58c5-4243-8bb3-797b68f79abd
Submission Status PUBLISHED
2026-10-16
2026-11-04
Division Standardized
Deployment Client on Premises
Model
Model Name Qwen3.6 35B-A3B
Division Standardized
Model Precision —
Model Link —
Calibration Link —
Model Notes —
Datasets
Dataset Name agentic_coding
Dataset Type performance
Average Input Tokens —
Dataset Link —

System TPS vs. Tokens/s per User

10k20k30kSystem TPS50100150Tokens/s per User

System TPS vs. Concurrency

10k20k30kSystem TPS0200400600800# of queries

Time to First Token P90 vs. Concurrency

100200300400TTFT P90 (ms)0200400600800# of queries

Tokens/s per User vs. Concurrency

50100150Tokens/s per User0200400600800# of queries
SUT Summary
NVIDIA
GB300-NVL72_GB300-288GB_aarch64x4_vLLM-Dynamo
Available
System Category Datacenter · —
1 node x 4 NVIDIA GB300 (4 GPUs)
1
1
Max Supported Concurrency 768
Node Type ID 0
Node Count 1
Software
Framework vLLM Qwen-latest 7cc4a74c3427 behind Dynamo 1.4.2 (4 x TP1 aggregated workers)
vLLM Qwen-latest 7cc4a74c3427 behind Dynamo 1.4.2 (4 x TP1 aggregated workers)
vLLM 0.1.dev1+g7cc4a74c3.d20261007.cu134
NVIDIA 580.173.02 (GKE 'default' COS driver; CUDA 13.0 driver API)
Link to Container registry.gitlab.com/nvidia/mlperf-inference-partner/nv-mlpinf-partner/centml-gb300-rubin@sha256:cb0b77df93c0f9d89b8bca7f59a2520e582b9dbebdd606f7d9f36f57632a8b71
Operating System Container-Optimized OS, kernel 6.12.94+, GKE 1.35.8-gke.1036000
Filesystem ext4 on node-local NVMe SSD (/mnt/stateful_partition/kube-ephemeral-ssd)
Other Software MLCommons endpoints client ghcr.io/mlcommons/endpoints@sha256:1523051b141437b25083924d6428082f435713516eee71c54ae207b393900b2d (tag d3c57f2); Dynamo 1.4.2; containerd 2.2.7
Software Notes —
Hardware

Processor

NVIDIA Grace (Arm Neoverse V2)
0
—
VCPUs Per Processor 144

Accelerator

—
0
—
—
—
—

Host / Storage

~943 GiB per node (kubectl capacity 988634624Ki)
Memory Configuration 960 GB LPDDR5X (2x NVIDIA Grace CPU integrated memory)
Storage Capacity ~12.5 TB local SSD per node (kubectl ephemeral-storage)
Storage Type —

Networking

4x NVIDIA ConnectX-7 + 1x Google Cloud Titanium NIC (a4x-maxgpu-4g-metal)
Google Cloud Titanium / RoCE + NVLink 5 (NVL72)
Network Topology —

Other

Other Hardware —
Cooling Liquid-cooled
Hardware Notes —
Run Data
Field NameRun 1Run 2Run 3Run 4Run 5Run 6Run 7Run 8Run 9
Run Date10/16/202610/16/202610/16/202610/16/202610/16/202610/16/202610/16/202610/16/202610/16/2026
Concurrency323664128144192256512768
System Tokens/Second5,463.36,054.69,784.016,084.517,276.320,478.923,941.631,911.635,238.5
Tokens/Second per User170.7168.2152.9125.7120.0106.793.562.345.9
TTFT P90 (ms)64.666.270.783.887.196.7111.2234.5387.3
Utilization15.5%17.2%27.8%45.6%49.0%58.1%67.9%90.6%100.0%
Configuration Summaryqwen-agentic-benchmark · online · c32qwen-agentic-benchmark · online · c36qwen-agentic-benchmark · online · c64qwen-agentic-benchmark · online · c128qwen-agentic-benchmark · online · c144qwen-agentic-benchmark · online · c192qwen-agentic-benchmark · online · c256qwen-agentic-benchmark · online · c512qwen-agentic-benchmark · online · c768
Config Typeonlineonlineonlineonlineonlineonlineonlineonlineonline
Config Region—————————
Target QPS—————————
Result Statecompletecompletecompletecompletecompletecompletecompletecompletecomplete
Measured Accuracy Score—————————
Benchmark Versionunknownunknownunknownunknownunknownunknownunknownunknownunknown
Result Version0.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.0
Archive URIgs://mlperf-endpoints-blob/runs/233794da-1335-4f5d-b250-314be29f8e47/archive.tar.gzgs://mlperf-endpoints-blob/runs/fff9b96a-142f-451e-b1cf-10143978aa26/archive.tar.gzgs://mlperf-endpoints-blob/runs/a21fea77-21fa-4c13-a3f0-2c72c8d24e96/archive.tar.gzgs://mlperf-endpoints-blob/runs/99c3fce6-b30b-4434-946d-c41b2b36b501/archive.tar.gzgs://mlperf-endpoints-blob/runs/5b4a9d02-5ab1-493b-8a87-d89906d0b53b/archive.tar.gzgs://mlperf-endpoints-blob/runs/72a8b36a-2471-43bc-a7fe-aca1eaeb4628/archive.tar.gzgs://mlperf-endpoints-blob/runs/2a7f19b2-5558-461a-9742-a75ad6c9c28b/archive.tar.gzgs://mlperf-endpoints-blob/runs/ee7b5fc3-c18b-4d7a-9c1d-a4a29f4a07b7/archive.tar.gzgs://mlperf-endpoints-blob/runs/45d9856d-1cd5-445b-84ea-0566a4cd1000/archive.tar.gz
QPS14.566916.155926.263942.856246.099254.833564.039385.326694.4473
Total Output Tokens30,359,96130,311,71644,828,51875,051,67474,579,39481,197,45380,168,72498,042,141168,584,353
Average Output Tokens375.05374.76372.53375.31374.76373.47373.86373.99373.1
Run Duration (s)5,557.125,006.354,581.814,666.104,316.863,964.933,348.513,072.304,784.10
Total Queries80,95080,882120,336199,971199,004217,411214,436262,149451,845
Total Queries Issued80,95080,882120,336199,971199,004217,411214,436262,149451,845
Total Queries Failed000000000
Time to First Token (TTFT) (ms)
Minimum17.718.119.321.821.022.125.246.159.5
Average46.247.750.461.564.471.485.2178.1274.9
P5039.740.743.452.455.161.671.0151.1219.7
P9064.666.270.783.887.196.7111.2234.5387.3
P9578.079.585.4101.3105.0116.4133.3291.0581.6
P99128.8133.8144.0169.5174.9193.0222.7726.81360.6
P999337.6371.9471.0627.4731.6957.03995.52850.23199.7
Maximum5620.56016.26167.85491.96263.26436.06793.87679.67497.6
Time Per Output Token (TPOT) (ms)
Minimum0.040.030.020.020.00-0.000.040.040.00
Average1.651.692.283.704.025.076.3011.316.9
P501.521.602.133.503.854.806.0110.816.1
P902.142.263.024.825.276.468.0614.521.6
P952.362.503.325.265.737.018.7415.723.5
P992.863.014.016.236.768.1810.218.830.3
P9994.264.685.768.979.5412.715.028.549.7
Maximum1175.6310.3856.61937.91167.81978.62078.39227.36875.9
Query Latency (ms)
Minimum28.641.642.650.943.079.969.1169.2142.5
Average630.7660.4860.51393.51522.51889.12356.34230.46311.7
P50219.1228.3290.2452.6496.6604.3749.01388.72112.9
P901519.91602.82107.73463.43794.24711.15915.410465.215687.7
P952495.22649.53462.05681.96224.17709.19633.317172.725519.1
P995776.86013.17920.912702.813966.917412.821832.138650.857605.3
P99911731.011981.915860.826096.027862.435251.343266.480485.9117072.0
Maximum20497.920049.528187.945783.945010.857098.572633.7126934.2243888.1