Label smoothing
Also called Label Smoothing
Tell the model the right answer is "very likely" rather than "100% certain," so it stays humble and generalizes better.
Think of it like
A teacher who marks the answer correct but reminds you nothing is ever quite certain, keeping you from overconfidence.
Example
Instead of a target of exactly 1.0 for the correct class, training uses 0.9, spreading the remaining 0.1 across the others.
How it actually works
Standard cross-entropy pushes the model toward assigning all probability to the correct token, which encourages overconfidence and sharp, brittle outputs. Label smoothing softens the target — a little probability mass is spread to the wrong classes — which regularizes the model, improves calibration, and often generalizes better. Too much smoothing, though, blurs genuine distinctions.
For product teams
A small tweak that makes a model's confidence more trustworthy and less brittle.
For engineers
Replace one-hot targets with a softened distribution; regularizes and improves calibration under cross-entropy.
Related
- Cross-entropy — The loss it modifies.
- Overfitting — The problem it helps guard against.
- Regularization — The broader family it belongs to.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome