Regularization
Any technique that keeps a model from over-memorizing so it generalizes to new data.
Think of it like
Like a coach who stops you from over-optimizing your form for one practice court, so your game holds up anywhere.
Example
Dropout, weight decay, and data augmentation are all regularization — each fights overfitting a different way.
How it actually works
Regularization is the umbrella for methods that constrain a model’s effective complexity to improve generalization: penalties on weights, dropout, early stopping, augmentation, and more. The goal is to shift the balance from fitting the training set toward performing on unseen data. Too much and the model underfits; the art is finding the sweet spot for your data size and task.
For product teams
The family of tricks that make a model work in the real world, not just on its training set.
For engineers
Techniques constraining effective capacity (weight decay, dropout, early stopping, augmentation) to improve generalization.
Related
- Weight decay — Includes weight decay.
- Dropout — Also includes dropout.
- Generalization — Improves generalization.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome