Lossless by Design

Ordinary networks are allowed to throw packets away. If a switch runs out of room, it discards what it cannot hold, and software at each end eventually notices something is missing and asks again. That works well when the traffic is thousands of unrelated conversations, because only the unlucky conversation waits.

The traffic here is not unrelated. When 9,216 GPUs combine their results, the operation is not finished until the last one has contributed. So a single packet thrown away does not delay one conversation - it delays every GPU in the job, all of which sit idle until the missing piece is asked for again and arrives.

That is why this network is built so the situation cannot arise. Instead of sending and hoping, a sender is told in advance exactly how much room the receiver has, and is not permitted to exceed it.

Ethernet was designed to drop. A switch with a full buffer discards the frame, and TCP infers the loss from missing acknowledgements and retransmits. The recovery is measured in round trips at best and in retransmission timeouts at worst, and the design is sound because it assumes statistically independent flows: the cost is paid by one flow while the rest keep moving.

A collective breaks that assumption completely. An all-reduce is a synchronising barrier - its completion time is the completion time of its slowest participant - so loss anywhere stalls everywhere. Worse, the traffic is maximally correlated: every GPU transmits at the same instant on the same layer boundary, which is precisely the pattern that fills buffers.

InfiniBand removes the failure mode rather than the recovery path. Credit-based flow controlcredit-based flow controlThe receiver tells the sender how much buffer it has; the sender may not exceed it. Congestion becomes a stall, never a drop. has the receiver advertise how much buffer space it has, and the sender may not exceed it. Congestion therefore becomes a stall at the sender, never a discard in the fabric.^{Mellanox Technologies, presented to IEEE 802.1}

Two details of the mechanism are worth having exactly, because they are what make it survivable rather than merely elegant.

First, credits are delivered in band - carried in the link's own traffic rather than as a separate signalling channel, which is what the tutorial contrasts against "pause/xon-xoff schemes".^{Mellanox Technologies, presented to IEEE 802.1} Second, and more importantly, they are absolute: a credit advertisement is a running total of everything permitted since the link initialised, not an increment. That choice is a resiliency decision. An incremental scheme desynchronises permanently if one update is lost; an absolute one self-corrects on the very next advertisement, because the next number supersedes the last. The named primary challenge in this design is not the accounting - it is the loss of a flow-control update itself.

Credits are issued per virtual lanevirtual laneOne of several independently buffered and independently credited channels sharing a physical link., so the link is not a single credited channel but several independently buffered ones. A lane whose receive buffers are full ceases transmission while the other lanes continue.^{InfiniBand Trade Association} That is the mechanism underneath InfiniBand's quality of service, and it is also the first line of defence against the price discussed below.

When a receiver stops returning credits, its upstream sender stops transmitting. If that sender is a switch, its own buffers begin to fill, and it in turn stops returning credits to its upstream. Backpressure walks the fabric in the opposite direction to the traffic that caused it, and it can reach senders that had nothing to do with the congested link. This is head-of-line blockinghead-of-line blockingA packet that cannot move prevents the packets queued behind it from moving either, even where those could proceed.: a packet that cannot move holds up packets queued behind it that could have.

A lossy network never has this problem, because it resolves congestion by destroying the blocked packet. Losslessness is the decision to hold the packet instead - and to accept that holding it has consequences elsewhere.

The practical upshot is that a lossless fabric needs two further mechanisms that a lossy one can do without, and both appear later in this section. Virtual lanes contain backpressure within a traffic class, so a saturated class cannot freeze the others. Adaptive routing moves flows off the congested path entirely, which is the only way to stop backpressure rather than merely bound it.

This is also the reason the equivalent Ethernet story is harder rather than simply different. RoCERoCERDMA over Converged Ethernet - the same verbs and queue pairs, carried by Ethernet instead of InfiniBand. runs the same RDMA semantics over Ethernet and therefore needs the same lossless guarantee, but it has to build it from PFCPFCPriority Flow Control - Ethernet’s pause frame, per traffic class: "stop sending" rather than "here is how much you may send". and ECNECNExplicit Congestion Notification - marking packets rather than dropping them, so the sender slows before anything is lost. - a reactive pause and an end-to-end congestion signal - rather than getting it from the link layer by construction. Chapter 7 is about how close that gets.