Memory Bandwidth Console

Memory bandwidth is one of the most consequential numbers in a system and one of the simplest to compute: channels times bus width times data rate, divided by eight to get bytes. Everything else is a trade-off in how you reach a target. There are only two levers — make the bus wider or make the pins faster — and the memory technologies split sharply on which they choose.

HBM goes wide: a thousand-and-twenty-four-bit interface per stack at a moderate rate, stacked in-package on a silicon interposer, reaching terabytes per second. GDDR goes fast: a narrow thirty-two-bit interface driven at blistering per-pin speeds, with many channels, for graphics. DDR5 sits in the middle as mainstream system memory, and LPDDR optimizes the same width-and-speed balance for power. Same goal, opposite strategies, each with its own cost, capacity, and power profile.

The reason this matters so much is that performance is frequently limited by bandwidth, not compute. When a workload needs to move more data per second than the memory can supply, the processor stalls — its cores idle, waiting — and adding FLOPS does nothing. Only more bandwidth helps. This is the defining characteristic of AI inference and many data-intensive workloads, and it's why matching memory bandwidth to the workload is as important a design decision as choosing the processor itself.

It's also why AI forced the industry to HBM. Accelerators need terabytes per second to feed their compute, an order of magnitude beyond what DDR provides, so every high-end AI chip pays HBM's cost and packaging complexity for that bandwidth. Size your memory here against the requirement; derive that requirement from the workload's arithmetic intensity in the HBM Bandwidth roofline console, and price the HBM in the HBM Cost console.