Hyperparameter
Also called Hyperparameter
A setting you choose before training that shapes how learning happens, as opposed to a weight the model learns on its own.
Think of it like
The oven temperature and bake time — you set them; the cake does not decide them for you.
Example
Learning rate, batch size, and number of layers are all hyperparameters picked before the run starts.
How it actually works
Parameters (weights) are learned by gradient descent; hyperparameters are the dials outside that loop — learning rate, batch size, dropout, architecture size. They are not differentiable with respect to the loss, so you cannot train them directly; you search over them by trying values and comparing on validation data.
For product teams
The knobs that decide whether a training run succeeds, wastes money, or quietly underperforms.
For engineers
Configuration set outside the optimization loop; tuned via search rather than gradients.
Related
- Hyperparameter tuning — The process of searching over these settings.
- Learning rate — The single most sensitive one.
- Parameters — What the model learns instead.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome