Hyperparameter tuning
Also called Hyperparameter Optimization
The trial-and-error hunt for the training settings that make a model work best.
Think of it like
Dialing in a new espresso machine — adjust grind, dose, and time, taste, repeat until it is right.
Example
A team sweeps learning rates from 1e-5 to 1e-3 and batch sizes of 16, 32, 64, then keeps the combo with the best validation loss.
How it actually works
Because hyperparameters are not learned by gradients, you search: try configurations, measure each on validation data, keep the winner. Strategies range from brute-force grid search to random search to smarter Bayesian methods that spend trials where they seem promising. The cost is that each trial is a full training run, so it gets expensive fast at scale.
For product teams
Often the difference between a mediocre and a strong model — and a real line item in the compute budget.
For engineers
Search over hyperparameters (grid, random, or Bayesian) using validation performance as the objective.
Related
- Hyperparameter — The settings being searched over.
- Grid search — The brute-force version of the search.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome