Batch normalization
Normalization computed across a batch of examples — huge for vision, awkward for language.
Think of it like
Curving a test using the whole class’s scores that day: great when the class is big and stable, shaky when it’s tiny.
Example
ResNets and image models lean on batch norm; transformers mostly skipped it for layer norm.
How it actually works
BatchNorm normalizes each feature using statistics across the current batch. It accelerated deep vision training dramatically, but its batch-dependence causes trouble with small batches, variable sequence lengths, and inference-time mismatch. Language models largely moved on.
For product teams
Mostly a vision-era tool; if you’re in language land you’ll rarely touch it.
For engineers
Normalizes over the batch dimension per feature; batch-dependent, dominant in CNNs, rare in transformers.
Related
- Normalization — A flavor of normalization.
- Layer normalization — Supplanted in NLP by layer normalization.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome