Inside a Compute Tray

Each tray holds two boards, and each board holds one CPU and two GPUs. Multiply out: two boards × (1 + 2) chips, eighteen trays, and you get 36Grace CPUs per rack NVIDIA Grace CPUs and 72Blackwell GPUs per rack NVIDIA Blackwell GPUs in the rack.

In an ordinary computer, the CPU and the graphics card talk over a slot on the motherboard, and they each keep their own memory. Sending work to the GPU means copying data across. Here, the CPU and its two GPUs are joined by a much faster link and share one pool of memory, so there is no copy — they can both simply use the same data where it lies.

A GB200 "Bianca" board is one Grace CPU and two Blackwell GPUs. Two boards per tray gives the tray 2 CPUs and 4 GPUs; eighteen trays give the rack 36Grace CPUs per rack NVIDIA and 72Blackwell GPUs per rack NVIDIA respectively, arranged as 36GB200 Superchips per rack NVIDIA GB200 Superchips.^{NVIDIA,ServeTheHome}

The link that matters is NVLink-C2CNVLink-C2CThe 900 GB/s coherent link between a Grace CPU and its Blackwell GPUs, replacing PCIe. at 900 GB/sNVLink-C2C, Grace ↔ Blackwell Coherent, roughly 7× PCIe Gen5. NVIDIA — roughly seven times PCIe Gen5 — and, more importantly, it is coherent. The Grace CPU's up to 480 GBLPDDR5X per Grace CPU The standalone Grace superchip datasheet quotes 512 GB at 546 GB/s; the GB200 Grace is configured differently and uses a reduced 1 MB L2 per core. NVIDIA of LPDDR5X and the GPUs' HBM3e sit in one address space with hardware-maintained coherence. A GPU kernel can dereference a pointer into host memory; the host can touch GPU-resident data without an explicit transfer.

Practically, this is what removes the staging step that dominates so much accelerator code: no pinned host buffers, no explicit cudaMemcpy on the critical path, no double-buffering to hide a PCIe hop. For workloads that spill past HBM — long-context KV caches, large embedding tables, optimiser state — LPDDR5X becomes a slower tier of the same memory rather than a different machine's memory.

Two details in this tray are easy to skim past and both have operational consequences.

First, the Grace part in GB200 is not the same configuration as the standalone Grace Superchip. NVIDIA's standalone datasheet quotes 512 GB of LPDDR5X at 546 GB/s; the GB200 variant is configured differently and uses a reduced 1 MB of L2 per core.^NVIDIA Capacity-planning from the standalone datasheet will overstate what is available per board.

Second, the tray has no fans and no air path worth speaking of. Every significant package is under a cold plate, and the tray's thermal design point — roughly ~2,700 WGB200 Superchip power SemiAnalysis per Superchip, so around 5.4 kW per tray — is only removable by liquid. There is no degraded air-cooled mode. A coolant fault is not a performance event, it is an immediate thermal event, which is why the CDUCDUCoolant Distribution Unit — the pump-and-heat-exchanger that isolates the rack’s coolant loop from the facility’s. and the facility loop get the redundancy budget that fans used to get.