Mixed-Precision Training

Definition

Mixed-precision training runs the forward/backward GEMMs in low precision (BF16/FP16/FP8) to get Tensor-Core speed while keeping numerically sensitive state in higher precision (FP32 master weights, FP32 accumulation) to preserve convergence. Standardized as AMP (Micikevicius et al., 2017). It is why the industry can march down the precision ladder — each step ~2× compute and ~2× less HBM/network traffic.

Key math — the formats

A float is sign · mantissa · 2^exp; exponent bits set range, mantissa bits set precision.

Format	Bits	Exp	Mantissa	Range	Notes
FP32	32	8	23	$10^{\pm 38}$	reference; master weights
TF32	19*	8	10	$10^{\pm 38}$	FP32 range, FP16 precision; TC input
FP16	16	5	10	$10^{\pm 5}$	narrow range → needs loss scaling
BF16	16	8	7	$10^{\pm 38}$	FP32 range → won for training
FP8 E4M3	8	4	3	$\pm 448$	fwd (weights/acts)
FP8 E5M2	8	5	2	$\pm 57344$	grads (more range)

*TF32 uses 19 significant bits but occupies a 32-bit slot. BF16 beat FP16 because it keeps all 8 of FP32’s exponent bits (literally FP32 with 16 mantissa bits chopped) → gradients don’t underflow, usually no loss scaling, trivial FP32↔BF16 conversion.

How it works — the recipe

1. Master weights in FP32. The optimizer keeps a high-precision copy. Update $w\leftarrow w-\eta g$: if $\eta g$ (~$10^{-4}$) is tiny relative to $w$ (~1.0), adding in BF16 (relative resolution $2^{-8}\approx 0.4\%$) rounds to a no-op — update swamping. FP32 (23 mantissa bits) preserves it; you round to BF16 only for the math, never for accumulating updates. (Adam moments also FP32.)
2. Compute in low precision. Cast weights/activations to BF16 (or FP8) for the forward/backward GEMMs — where the speedup lives.
3. Loss scaling (FP16; usually unneeded for BF16). Multiply loss by $S$ before backward so small grads land in range; divide grads by $S$ before the step. Dynamic loss scaling adapts $S$.
4. FP32 accumulation inside the Tensor-Core MMA and for reductions (softmax, LayerNorm, loss).

Why it matters

Precision choice directly sets the MFU peak and the ridge point: BF16 → ~990 TFLOP/s, FP8 → ~1979, so FP8 can roughly halve GPU-hours for matmul-bound training and halve HBM/NVLink bytes. The catch: FP8’s tiny mantissa needs per-tensor/per-block scaling (NVIDIA Transformer Engine), and its higher ridge (~591) pushes more ops memory-bound — which is why stable FP8 training is a frontier-lab capability, not a free switch.

Taught in

• 03-performance-modeling-roofline — §5 precision formats, tensor cores, mixed-precision recipe.

See also

• tensor-core — the units low precision feeds
• mfu — precision sets peak FLOP/s
• roofline-model — precision moves the ridge point
• memory-wall — lower precision cuts bytes moved
• gemm — the GEMMs run in low precision
• gpu-systems-for-llms