Batch size
How many examples the model looks at before making one weight update.
Think of it like
Like how many quiz answers you grade before adjusting your teaching — a few gives quick but jumpy feedback, many gives a steadier read.
Example
A batch size of 256 means the model averages the error over 256 examples, then takes one optimization step.
How it actually works
Batch size trades gradient quality against speed and memory: larger batches give smoother, more reliable gradient estimates and use hardware efficiently, but need more memory and can generalize slightly worse. Smaller batches are noisier but sometimes find better minima. It interacts tightly with the learning rate — change one and you often must retune the other.
For product teams
A core cost-and-quality knob; it’s bounded by your GPU memory and coupled to the learning rate.
For engineers
Examples per gradient update; larger = smoother gradients + more memory, coupled to learning rate; extendable via accumulation.
Related
- Stochastic gradient descent — Defines each stochastic gradient descent update.
- Gradient accumulation — Stretched by gradient accumulation.
- Learning rate — Interacts with learning rate.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome