Loss function
The formula that scores how wrong the model is, giving training something to minimize.
Think of it like
Like a golf score — the lower the number, the better you’re doing, and every swing aims to bring it down.
Example
A model’s loss on a batch is high when its predictions miss the targets; training keeps pushing that number lower.
How it actually works
The loss function turns the gap between predictions and targets into a single number that optimization minimizes. Its choice defines what “good” means — cross-entropy for classification, mean-squared error for regression, custom losses for special goals. Because the model optimizes exactly what you measure, a poorly chosen loss produces a model that’s technically great at the wrong thing.
For product teams
You get the behavior you measure — the loss encodes your definition of success, so choose it carefully.
For engineers
Scalar objective quantifying prediction error; its gradient drives all weight updates. Choice defines the target behavior.
Related
- Cross-entropy — A common one is cross-entropy.
- Gradient descent — What gradient descent minimizes.
- Objective function — The broader term is objective function.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome