Roofline Model

Definition

The roofline model (Williams, Waterman & Patterson, 2009) is a back-of-envelope upper bound on kernel performance derived from two hardware numbers (peak FLOP/s, peak bandwidth) and one kernel number (arithmetic intensity). It plots attainable FLOP/s vs intensity on log-log axes and tells you — before profiling — whether a kernel is memory-bound or compute-bound, hence which knob to turn.

Key math

$P_{\text{attainable}}(I)=\min (\underset{\text{compute roof}}{\underbrace{P_{\text{peak}}}},\underset{\text{memory roof}}{\underbrace{\beta \cdot I}})$

• Memory-bound (left): $P=\beta I$, rises with slope $\beta$. More FLOP/s capability is useless; raise $I$ (fusion, reuse) or move fewer bytes (lower precision).
• Compute-bound (right): $P=P_{\text{peak}}$, flat ceiling. Use faster math units (Tensor Cores, lower precision).

The lines cross at the ridge point:
$\boxed{I^{\ast }=\frac{P_{\text{peak}}}{\beta }}$
$I<I^{\ast }\Rightarrow$ memory-bound; $I>I^{\ast }\Rightarrow$ compute-bound.

H100 SXM5 ridge points ($\beta =3.35$ TB/s):
$I_{\text{BF16}}^{\ast }=\frac{9.9\times 10^{14}}{3.35\times 10^{12}}\approx 296,I_{\text{FP8}}^{\ast }=\frac{1.98\times 10^{15}}{3.35\times 10^{12}}\approx 591\frac{\text{FLOP}}{\text{byte}}$

Peak vs achievable. Vendors quote the most flattering peak (with 2:4 sparsity, lowest precision). Build your roofline from dense peaks in the precision you actually use; a great GEMM reaches ~80–90% of dense peak, most kernels far less. Lower precision raises $I^{\ast }$ (memory wall), pushing more ops memory-bound.

Why it matters

“Is this kernel memory- or compute-bound?” organizes nearly every LLM optimization decision — fusion, precision, tiling, recompute vs store. Nsight Compute plots the measured $(I,\text{FLOP/s})$ dot against both roofs: on the memory roof → bandwidth-bound; on the flat roof → compute-bound; below both → occupancy/latency/overhead problem. Horace He’s framing (compute vs bandwidth vs overhead) is the canonical mental model, and the roofline is its formalization.

Taught in

• 03-performance-modeling-roofline — §4 the roofline model, ridge point, classification.

See also

• arithmetic-intensity — the x-axis
• memory-wall — ridge point over generations
• tensor-core — the compute roof
• gpu-memory-hierarchy — the memory roof ($\beta$)
• mfu — utilization at the run level
• gemm — the canonical compute-bound kernel
• gpu-systems-for-llms