Cold start
The delay when a service boots up and hasn't finished loading the model into memory yet.
Think of it like
The time it takes to warm up a car engine on a freezing morning.
Example
A Lambda function gets invoked for the first time and spends 5 seconds loading the model weights before it can process the first request.
How it actually works
Cold starts happen when no instance is running and a new one has to spin up, load the model from disk or network, and prepare it for inference. For large models, loading itself can take seconds. Keeping warm instances running costs money; letting them sleep saves money but introduces latency.
For product teams
Cold starts hurt user experience on low-traffic services — trade-off between cost and responsiveness.
For engineers
Minimize loading time: quantize models, use fast storage (NVMe vs network), parallelize loading, or keep instances warm.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome