Sources & Method

Pooled HBM3e per rack ~13.5 TB

13.4 – 13.8 TB

Sources differ on rounding, on physical versus usable-after-ECC capacity, and on SKU. NVIDIA’s 192 GB per GPU gives 13.82 TB physical and 12.96 TB after ECC; Supermicro’s datasheet quotes up to 372 GB per Superchip - 186 GB per GPU - which is where the 13.4 TB figure comes from.

Copper cables in the NVLink spine >5,000

5,000 – 5,184

NVIDIA’s OCP contribution says "over 5,000"; other sources cite 5,184. Jensen Huang described it as "5,000 NVLink cables. In total, 2 miles."

Rack power ~120 kW

120 kW nominal · 125–135 kW operating (Supermicro) · 132 kW fully loaded (Schneider Electric)

Supermicro’s datasheet states an operating power of 125–135 kW and 132 kW of installed power-shelf capacity. Steven Carlini, writing for Schneider Electric: "When fully loaded into a rack, the latest NVIDIA-based GPU servers require 132 kW of power." The commonly quoted ~120 kW is the nominal design figure, not a measured ceiling.

Coolant flow ~2–3 L/min per module

2–3 L/min per module · 30–40 L/min per rack (NVIDIA ACS) · up to ~130 L/min per rack (QCT)

Figures vary by whether they are quoted per cold plate or per rack, and by the assumed ΔT. Always state the basis.

Cost of liquid cooling per MW ~$2M retrofit

~$2M per MW to retrofit · upwards of $11M per MW for a new greenfield liquid-cooled build

STL Partners, May 2026. A widely repeated "$5–10M per MW" retrofit figure is often attributed to Schneider Electric; it does not appear in the Schneider article this site cites, and no primary source for it could be found - so it is not used here.

Leaf switches at maximum scale 128

128 (GB200 product page) vs 64 leaf and 384 spine (DGX GB200 SuperPOD reference architecture at 16 SUs)

Almost certainly two different reference designs rather than a genuine contradiction - the product page and the SuperPOD architecture do not describe the same build. Do not mix the two figures inside one calculation.

HBM3e per Blackwell Ultra GPU 288 GB

279 GB vs 288 GB

SKU and ECC accounting. Note that NVIDIA’s own MLPerf Training v5.1 blog quotes 279 GB, so this is not simply a case of secondary sources getting it wrong.

ConnectX-9 SuperNIC 1,600 Gb/s

Per GPU, via four 200 Gb/s SerDes. Listed on NVIDIA’s Spectrum-X platform page; treat as announced rather than measured.

Vera Rubin NVL144 FP4 ~3.6 EF

Announced for 2H 2026. Note that "144" counts dies, not packages - there are 72 Rubin packages.

Next-generation rack power 240 kW

Schneider Electric, forward-looking: "The next generation, expected in under a year, will require 240 kW per rack."

GB200 NVL72 product page NVIDIA

Source of the "acts as a single, massive GPU" framing and the 30× / 10× inference claims, with their measured configuration.

GB300 NVL72 product page NVIDIA

Blackwell architecture overview NVIDIA

Grace CPU Superchip NVIDIA

Describes the standalone Grace Superchip. The GB200 Grace is configured differently - do not capacity-plan from this page.

NVLink and NVLink Switch NVIDIA

NVIDIA GB200 NVL72 Delivers Trillion-Parameter LLM Training and Real-Time Inference NVIDIA Technical Blog

Confirms 900 GB/s NVLink-C2C, 30 TB unified memory, 130 TB/s fabric, and the 30× GPT-MoE-1.8T figure.

NVIDIA Blackwell Architecture Sweeps MLPerf Training v5.1 Benchmarks NVIDIA Technical Blog

GB300 NVL72 debut in MLPerf Training; NVFP4 used in training for the first time; the 10-minute Llama 3.1 405B record on more than 5,000 Blackwell GPUs.

How NVIDIA GB200 NVL72 and NVIDIA Dynamo Boost Inference Performance for MoE Models NVIDIA Technical Blog

Scaling Large MoE Models with Wide Expert Parallelism on NVL72 Rack-Scale Systems NVIDIA Technical Blog

How New GB300 NVL72 Features Provide Steady Power for AI NVIDIA Technical Blog

The power-smoothing source: programmable power caps, integrated electrolytic capacitors, a hardware power burner, and a measured 30% reduction in peak grid demand training Megatron.

Inside NVIDIA Blackwell Ultra: The Chip Powering the AI Factory Era NVIDIA Technical Blog

MNNVL User Guide — Overview NVIDIA Docs

NVIDIA IMEX Service for NVLink Networks — Overview NVIDIA Docs

IMEX brokers GPU memory export/import across OS domains over NVLink; it does not depend on CUDA and communicates over TCP and gRPC.

GB200 NVL Multi-Node Tuning Guide — Power and Thermals NVIDIA Docs

Names Power Smoothing as implemented for bulk synchronous workloads, and covers power balancing within a provisioned rack limit.

NVIDIA Dynamo — disaggregated prefill/decode serving NVIDIA

MLPerf Inference & Training results MLCommons

NVIDIA Quantum-X800 InfiniBand platform NVIDIA

144 ports of 800 Gb/s; SHARP v4; adaptive routing; telemetry-based congestion control.

NVIDIA Quantum-X800 (XDR) clusters — switch systems and cabling NVIDIA Networking Docs

Q3400-RA "144 XDR Ports over 72 OSFP Cages"; Q3200-RA "36 XDR Ports over 18 OSFP cages"; twin-port 1.6 Tb/s transceiver "1310nm SMF, up to 500m"; active copper 1.1–3 m.

NVIDIA Quantum InfiniBand switches NVIDIA

Separates the generations the site previously conflated: Quantum-X800 is XDR at 800 Gb/s and supports "over 10,000 nodes in two-level fat tree topology", while Quantum-2 is NDR at 400 Gb/s "for Hopper-generation or cost-optimized deployments".

DGX SuperPOD reference architecture (GB200) — Network Fabrics NVIDIA Docs

"Traffic per rail of each compute tray is always one hop away from other compute trays in the same Scalable Unit"; rail-aligned racks; spine-leaf-group design to 16 SUs with "8 leaf switches (one for each compute rack) and 6 spine switches"; fully non-blocking fat tree; fat-tree, Dragonfly and Torus all supported; SHIELD.

DGX SuperPOD reference architecture (GB200) — Key Components NVIDIA Docs

Compute tray = two GB200 Superchips = four B200 and two Grace. "The compute tray integrates four ConnectX-7 (CX-7) NICs to support InfiniBand NDR (400Gbps) connectivity" and "two BlueFiled-3 (BF3) NICs to support 2x200Gbps connectivity for the In-band Management and Storage networks". SU = 8 racks; compute fabric on QM9700.

NVIDIA UFM Enterprise — Subnet Manager and Adaptive Routing NVIDIA Docs

The subnet manager is "a centralized entity running on the server that discovers and configures all the InfiniBand fabric devices to enable traffic flow throughout the fabric". As of UFM v6.4 adaptive routing is part of the core subnet manager and no longer needs a plugin for AR or SHIELD configuration.

NVIDIA Spectrum-X Ethernet platform NVIDIA

SN6600 "128 ports of 800 G OSFP"; SN6800 "512 ports of 800 G in a 5U form factor"; ConnectX-8 "800 Gb/s total throughput via 2×400 G"; ConnectX-9 "1,600 Gb/s per GPU via 4×200 G SerDes"; lossless networking; adaptive routing by switch and SuperNIC "in tight coordination"; telemetry-based congestion control. Its 1.6× and 1.9× figures are vendor claims published without a measured configuration.

GPUDirect RDMA NVIDIA CUDA Docs

GPU memory reachable by the NIC "without needing to copy data to host memory", over both InfiniBand and RoCE. Queue pairs comprise a send and a receive queue; verbs give OS-bypass, zero-copy and offload. `nvidia-peermem` registers GPU memory with the InfiniBand subsystem.

The Llama 3 Herd of Models Meta AI

The best public failure data at this scale. Section 3.3.4 reports 466 interruptions over a 54-day snapshot of a 16K-H100 run, of which 419 were unexpected, with a component breakdown. Also the source for the 38-43% BF16 MFU figure and the 4D [TP, CP, PP, DP] parallelism split. An H100 cluster, not an NVL72 - cited here for what failure and utilisation look like at scale, never as a Blackwell measurement.

Orca: A Distributed Serving System for Transformer-Based Generative Models Yu et al., OSDI ’22

Introduced iteration-level scheduling - the technique the industry now calls continuous or in-flight batching - together with selective batching. The paper is the origin of the observation that with request-level batching, finished requests cannot return until the whole batch finishes.

Workers Static Assets Cloudflare Docs

Workers pricing Cloudflare Docs

R2 pricing Cloudflare Docs

GB200 Hardware Architecture — Component Supply Chain & BOM SemiAnalysis

Nvidia's Optical Boogeyman — NVL72, InfiniBand Scale Out, 800G & 1.6T Ramp SemiAnalysis

Running the NVLink spine over optics would add roughly 20 kW for transceivers and retimers alone. Also the origin of the 5,184 cable count.

This is the NVIDIA DGX GB200 NVL72 ServeTheHome

Teardown coverage: half-width nodes two-abreast in 1U, nine switch trays of two chips each, four ports and 18 links per chip, power shelves and the CDU below the compute nodes.

Global Data Center Survey 2025 — mean rack density 7.6 kW Uptime Institute

NVLink and Grace reference notes glennklockwood.com

Why Liquid Cooling For AI Data Centers Is Harder Than It Looks Steven Carlini, Schneider Electric - Forbes Technology Council

Confirmed verbatim: "When fully loaded into a rack, the latest NVIDIA-based GPU servers require 132 kW of power" and "The next generation, expected in under a year, will require 240 kW per rack." This article does NOT contain the retrofit-cost figure that is often attributed to it.

The Retrofitting Roadmap: An Evolution of Liquid Cooling STL Partners (supported by Airedale)

Liquid cooling retrofits at "around USD2 million per MW" against "upwards of USD11 million per MW" for new greenfield liquid-cooled builds.

RDMA over Commodity Ethernet at Scale Guo et al., Microsoft - ACM SIGCOMM

The production account of what goes wrong when RoCEv2 is deployed at scale. Its abstract states outright: "We have addressed the safety challenges brought by PFC-induced deadlock (yes, it happened!), RDMA transport livelock, and the NIC PFC pause frame storm problem." Cited here so the site names the failure mode rather than alluding to one.

CoreWeave, NVIDIA and IBM Set MLPerf Record with the Largest GB200 Blackwell Cluster CoreWeave

Confirmed verbatim: "2,496 NVIDIA Blackwell GPUs across 39 racks, each containing 64 active GPUs", Llama 3.1 405B "completed in 27.33 minutes", against "around 156 racks" for an equivalent H100 setup at 32 GPUs per rack.

Supermicro NVIDIA GB200 NVL72 SuperCluster datasheet (PDF) Supermicro

Read directly. Rack 2236 × 600 × 1068 mm; 8 × 1U 33 kW power shelves totalling 132 kW; operating power 125–135 kW; 10 + 8 compute trays around 9 NVLink switch trays; up to 372 GB HBM3e and 480 GB LPDDR5X per Superchip; in-rack 250 kW CDU.

QCT QoolRack Stand-Alone — Advanced Liquid Cooling for NVIDIA GB200 NVL72 Systems (PDF) QCT unconfirmed

The document exists and resolves, but its text is embedded as CID-encoded fonts and could not be extracted, so the 45 °C maximum inlet, 65 °C maximum return and ~130 L/min per-rack figures attributed to it are still second-hand here. Several secondary sources repeat exactly these numbers and credit QCT. Confirm against the readable document before treating them as primary.

Achieving Peak System and Workload Efficiency on NVIDIA GB200 NVL72 with Slurm Block Scheduling NVIDIA

Read directly. "The new topology/block plugin was introduced in the Slurm 23.11 release to support rack-scale architectures such as GB200 NVL72." One NVL72 domain is 18 nodes, and the article recommends "--segment=16 instead" of 18 for jobs above that size. On the older tree plugin: "A job split across multiple leaf switches might run slightly slower than under one single leaf switch, but the tradeoff between start time and performance is generally considered acceptable." Also quantifies the penalty for leaving the domain: "Communication crossing domain boundaries faces a steep performance drop, typically 50 GB/s (400 Gb/s) through InfiniBand or Ethernet."

Lowest token cost with GB300 NVL72 NVIDIA unconfirmed

NOT independently located. The 2.7x-tokens and >60%-cheaper-per-token claims are widely repeated and no permanent URL for the originating post could be confirmed, so only the blog index is linked. This matters less than it usually would: the chapter cites the claim as an example of a ratio published without a denominator, and that characterisation would survive even if the exact wording differs.

NVIDIA Contributes NVIDIA GB200 NVL72 Designs to the Open Compute Project NVIDIA Technical Blog

Specifies four NVLink cartridges with over 5,000 copper cables delivering 260 TB/s AllReduce bandwidth, a 1,400 A busbar, and over 100 lb of rack reinforcement steel.

InfiniBand Credit-Based Link-Layer Flow-Control (802.1 DCB TG tutorial) Mellanox Technologies, presented to IEEE 802.1

Read directly. The clearest public account of why InfiniBand does not drop: "In-band Delivery of Flow Control Credits — vs. pause/xon-xoff schemes", credits counted absolutely as the "total allowed since initialization of the link", and resiliency - losing a credit update - named as the primary challenge. Points at IBTA Volume 1, section 7.9.

InfiniBand Architecture Specification, Volume 1 — section 7.9, flow control InfiniBand Trade Association unconfirmed

The normative source, but it is not openly downloadable - access requires IBTA registration, so it has not been read here. The section reference and the per-virtual-lane credit behaviour are corroborated by the IEEE tutorial above, which cites it directly. Confirm against the specification itself before treating any wording as normative.

Explorable explainers (Gears, Cameras and Lenses, Internal Combustion Engine) Bartosz Ciechanowski

Explorable Explanations Bret Victor

Distill — interactive machine learning research Distill

A higher order estimate of the optimum checkpoint interval for restart dumps Daly, Future Generation Computer Systems

Refines Young’s 1974 first-order result. Young gives an optimum interval of about sqrt(2 * checkpoint_cost * MTBF); Daly adds restart cost, giving sqrt(2*delta*(M+R)) - delta. The site uses the first-order form, which is the one that makes the square-root behaviour visible.