MFU (Model FLOPs Utilization)

Definition

MFU (Model FLOPs Utilization) is the fraction of aggregate peak FLOP/s spent on the useful model FLOPs (the $6ND$ math that defines the model), ignoring redundant work. It is the hardware- and recompute-agnostic efficiency number that maps directly to tokens-per-dollar — distinct from HFU (Hardware FLOPs Utilization), which counts all FLOPs executed including activation recomputation. Popularized by PaLM (Chowdhery et al., 2022).

Key math

$\text{MFU}=\frac{6N\cdot (\text{tokens/s})}{G\cdot P_{\text{peak}}}$
$N$ = non-embedding params, $G$ = number of GPUs, $P_{\text{peak}}$ = dense peak per GPU in the training precision.

The $6ND$ rule (Kaplan 2020; Chinchilla 2022): total training compute of a dense transformer is
$\boxed{C\approx 6ND}$
$N$ = params, $D$ = tokens. The 6 = 2 (forward: one multiply-add per param per token) + 4 (backward: grad w.r.t. input + w.r.t. weight, ~2× forward). Ignores attention’s $O({\text{seq}}^2)$ term (small unless sequences are very long).

$\text{HFU}=\frac{\text{hardware FLOPs/s (incl. recompute)}}{G\cdot P_{\text{peak}}}\ge \text{MFU}$

Deriving cost: effective per-GPU throughput $=\text{MFU}\cdot P_{\text{peak}}$, so
$\text{GPU-hours}=\frac{6ND}{\text{MFU}\cdot P_{\text{peak}}\cdot 3600},\text{wall-clock}=\frac{\text{GPU-hours}}{G}$

Typical MFU: ~35–55% on well-tuned large-scale training (PaLM 46% @ 540B; strong Megatron/H100 runs in the 40s–low 50s). >60% is a red flag — usually a wrong $N$, double-counted embeddings, or a sparse-peak $P_{\text{peak}}$. What drags it down: memory-bound ops (softmax, LayerNorm, elementwise), un-overlapped communication, pipeline bubbles, small GEMMs, launch overhead, and recompute (the MFU/HFU gap).

Why it matters

Don’t confuse MFU and HFU — HFU ≥ MFU always. With activation recomputation you might see HFU 60% but MFU 45%; the extra 15 points are wasted forward passes that trade compute for activation memory. MFU is the number for training efficiency and cost; reporting HFU as MFU flatters your run.

MFU + $6ND$ is the calculation for planning a training run: cost, time, cluster sizing, and carbon all derive from it, and it’s a 10-second sanity check on any training log.

Taught in

• 03-performance-modeling-roofline — §6 MFU/HFU, §7 the 6ND rule, §9 cost estimation.

See also

• gemm — the compute-bound FLOPs MFU measures
• roofline-model — per-kernel version of the same question
• arithmetic-intensity — what drags MFU down
• mixed-precision-training — precision sets $P_{\text{peak}}$
• memory-wall — why MFU rarely nears 100%
• gpu-systems-for-llms