Part II · Chapter 03
Switches, Optics and Cabling
Chapter 2 of Part I described a copper spine chosen over optics because transceivers would have cost ~20 kWPower an all-optical spine would have cost NVIDIA’s stated rationale for copper. SemiAnalysis independently computed ~19.4 kW from 648 × 1.6T transceivers at ~30 W. SemiAnalysis . That argument does not stop at the rack wall - it reappears here, one tier out, with the distances multiplied by a hundred and the answer reversed.
Two generations, and why the distinction matters
InfiniBand comes in generations, each roughly twice as fast as the last, and each with a three-letter name. The two in service here are NDR and XDR: NDR carries 400 Gb/sNDR per-port rate Four lanes at 100 Gb/s. Quantum-2 generation, which NVIDIA positions "for Hopper-generation or cost-optimized deployments" - and which is what the DGX GB200 SuperPOD reference architecture actually specifies, with ConnectX-7. NVIDIA · NVIDIA Docs down one port, XDR carries 800 Gb/sXDR per-port rate Four lanes at 200 Gb/s. Quantum-X800 generation. NVIDIA · NVIDIA .
This matters more than a spec-sheet detail, because the two are often talked about as if they were the same thing. The racks in a shipping SuperPOD today are usually joined by the older, cheaper generation - not the newest one in the press release.
A port is four lanes. NDR runs them at 100 Gb/s each for 400 Gb/sNDR per-port rate Four lanes at 100 Gb/s. Quantum-2 generation, which NVIDIA positions "for Hopper-generation or cost-optimized deployments" - and which is what the DGX GB200 SuperPOD reference architecture actually specifies, with ConnectX-7. NVIDIA · NVIDIA Docs a port; XDR runs them at 200 Gb/s for 800 Gb/sXDR per-port rate Four lanes at 200 Gb/s. Quantum-X800 generation. NVIDIA · NVIDIA . The switch families are Quantum-2 and Quantum-X800 respectively, and NVIDIA is explicit that Quantum-2 is positioned "for Hopper-generation or cost-optimized deployments".NVIDIA
Which is exactly what a GB200 SuperPOD uses. NVIDIA's own reference architecture specifies QM9700 switches, with each compute tray carrying "four ConnectX-7 (CX-7) NICs to support InfiniBand NDR (400Gbps) connectivity".NVIDIA Docs Quantum-X800 and ConnectX-8 are the newer pairing, and they arrive with GB300. Both are real; they are different design points, not different marketing for the same one.
The radix is where the generations diverge in a way that changes topology rather than throughput. A 144 × 800Quantum-X800 Q3400 switch XDR, over 72 OSFP cages - two ports per cage. Around 115 Tb/s of switching capacity in one box. NVIDIA Networking Docs · NVIDIA Gb/s Q3400 presents 144 ports over 72 OSFP cages - two ports per cage - which is roughly 115 Tb/s of switching in one box.NVIDIA Networking Docs,NVIDIA There is also a 36 × 800Quantum-X800 Q3200 switch 2U air-cooled, 18 OSFP cages. NVIDIA positions it for large or mixed-generation environments. NVIDIA Networking Docs · NVIDIA Gb/s Q3200 in a 2U air-cooled chassis, which NVIDIA positions for large or mixed-generation environments.
Radix is the lever that decides how many tiers a fabric needs, and therefore how many hops and how much optics it buys. The next chapter but one works that arithmetic; the number to carry there is 144.
Reach is the real constraint
A switch port does not care what is plugged into it. The floor plan does.
This figure is interactive and needs JavaScript. The prose around it states every number it shows.
Copper at these rates reaches a few metres. NVIDIA specifies its 1.6 Tb/s active copper cable at 1.1 – 3 mActive copper reach at 1.6 Tb/s OSFP active copper cable. Enough to cross a rack or reach the one beside it, and nothing further. NVIDIA Networking Docs , which is enough to cross a rack or reach the one beside it and nothing further.NVIDIA Networking Docs Optics reach up to 500 mSingle-mode optical reach at 1.6 Tb/s Twin-port 2×DR4 transceiver, 1310 nm single-mode fibre. Two orders of magnitude more reach than copper, at a power and cost premium per link. NVIDIA Networking Docs over single-mode fibre - two orders of magnitude more, at a cost and a power draw per link that copper does not have.
That gap is why a fabric's physical design is mostly an exercise in keeping links short. Every metre past three is a transceiver, and every transceiver is watts and dollars multiplied by a port count in the thousands.
Run the multiplication and the in-rack argument reappears at cluster scale. The spine inside one NVL72 avoided roughly ~20 kWPower an all-optical spine would have cost NVIDIA’s stated rationale for copper. SemiAnalysis independently computed ~19.4 kW from 648 × 1.6T transceivers at ~30 W. SemiAnalysis of transceiver power by staying copper over a metre and a half. A fabric joining 128 racks cannot make that choice: the distances are tens to hundreds of metres, so the optics are not optional and the power is simply a cost of doing business.
The consequence is a hierarchy that shows up in every real floor plan. Rail links from a compute tray to its leaf switch are kept inside the copper envelope wherever the layout allows; leaf-to-spine hops go optical because they must cross rows. Which is also why rail alignment is worth so much - it converts the traffic that dominates a collective into the hop that is cheapest to build.
Switch and cabling figures
| XDR per-port rate | 800 Gb/s | Four lanes at 200 Gb/s. Quantum-X800 generation. |
|---|---|---|
| NDR per-port rate | 400 Gb/s | Four lanes at 100 Gb/s. Quantum-2 generation, which NVIDIA positions "for Hopper-generation or cost-optimized deployments" - and which is what the DGX GB200 SuperPOD reference architecture actually specifies, with ConnectX-7. |
| Quantum-X800 Q3400 switch | 144 × 800 Gb/s | XDR, over 72 OSFP cages - two ports per cage. Around 115 Tb/s of switching capacity in one box. |
| Quantum-X800 Q3200 switch | 36 × 800 Gb/s | 2U air-cooled, 18 OSFP cages. NVIDIA positions it for large or mixed-generation environments. |
| Active copper reach at 1.6 Tb/s | 1.1 – 3 m | OSFP active copper cable. Enough to cross a rack or reach the one beside it, and nothing further. |
| Single-mode optical reach at 1.6 Tb/s | up to 500 m | Twin-port 2×DR4 transceiver, 1310 nm single-mode fibre. Two orders of magnitude more reach than copper, at a power and cost premium per link. |
| SHARP version on Quantum-X800 | v4 | In-network reduction on the scale-out switch. Distinct from, and composable with, the SHARP engines inside the in-rack NVSwitch. |
What the generation ladder looks like, and how firmly
Each InfiniBand generation has doubled the per-lane rate: HDR at 50 Gb/s a lane and 200 Gb/s a port, NDR at 100 and 400, XDR at 200 and 800. The rates for the two generations in service here are confirmed against NVIDIA's own switch documentation and appear as specs above.NVIDIA,NVIDIA Networking Docs
The older rungs - SDR through EDR - are widely published but the IBTA's own roadmap presents them only as an image, so this site does not state them as sourced figures. That is a smaller point than it sounds: nothing in this section depends on the historical ladder, only on the two generations actually deployed. It is noted on the sources page rather than quietly rounded over.