Why Most Published Bandwidth Tables Are Wrong
Bus bandwidth is the number everyone quotes when comparing GPU clusters, and two structural traps make most published figures meaningless. We are not going to publish a table. Here is why, and what to measure instead.
Trap one: two nodes do not test a fabric
NCCL selects its algorithm based on topology. On two nodes it defaults to Tree, and the NCCL maintainer is direct about what that means in nccl-tests #106:
"By default, on two nodes, we use the Tree algorithm, which does not directly reflect the GPU-NIC speed, and that makes it harder to 'see' issues through the BW numbers."
A two-node all-reduce is dominated by intra-node NVLink. That is why a cloud vendor can publish a two-node figure well above the per-node network bandwidth those instances actually have — the number is real, it just is not measuring what the reader assumes.
Trap two: SHARP breaks the arithmetic
Bus bandwidth is a derived figure. It takes measured algorithm bandwidth and multiplies by a factor that assumes a point-to-point ring. In-network reduction — NVLink SHARP, or its switch-side equivalent — violates that assumption, because data is combined in the fabric rather than passed around it. From the same maintainer, in nccl-tests #153:
"Some algorithms now use network-accelerated techniques (SHARP) which break the 'point-to-point based' rule. So indeed with SHARP enabled and depending on what is the bottleneck, AlgBW may make more sense to interpret the numbers, and BusBw may not represent any 'Bus' bandwidth."
This is how you get published figures that exceed the physical wire rate of the links involved. They are not fraudulent; the formula simply stopped applying.
The figures worth trusting
For 8× H100 SXM in a single node, from the NCCL maintainer directly:
| Measurement | Figure |
|---|---|
| NVLink line rate | 450 GB/s |
| Effective, after ~20% protocol overhead — observed as busbw with NCCL_ALGO=RING | 370 GB/s |
| Peak with NVLink SHARP (the all-reduce default) | ~480 GB/s |
One caveat that invalidates a lot of amateur benchmarking: those peaks need large messages. 256 MB is too small to reach them, so a run that stops there under-reports and the reader concludes the hardware is faulty.
Note also that the two headline numbers differ by 30% purely on algorithm choice. Any table that does not state which algorithm produced its figures is not comparable with any other table.
Measure a ratio, not an absolute
The useful acceptance test is not "did we hit X GB/s." It is what fraction of this cluster's own theoretical ceiling are we reaching, and is that fraction stable across node pairs.
Compute the ceiling from per-GPU NIC bandwidth — 200 Gb/s is 25 GB/s per GPU, 400 Gb NDR is 50, 800 Gb is 100 — because large-message ring all-reduce bus bandwidth tracks it. Then measure the achieved fraction, and measure it per node pair.
The pair-to-pair variance is the diagnostic. A uniform 85% is a healthy cluster. An average of 85% made of half the pairs at full rate and half at 50% is a routing or ACS problem that an aggregate number would have hidden completely.
Related
- Why your GPU fabric runs at half speed — what pair-to-pair variance means
- AI cluster networking — the full symptom index