Spot instances
Cheap cloud compute that can be interrupted at any time — you save money in exchange for the risk of losing the instance.
Think of it like
Buying concert tickets at the last minute for 90% off, but the show might be cancelled.
Example
A spot GPU costs 80% less than an on-demand GPU, but cloud providers can reclaim it with 30 seconds notice.
How it actually works
Spot instances are good for fault-tolerant workloads. If one instance dies, you spin up another. Inference services can handle spot instances: if a server dies, requests fail over to another replica. Training is risky on spot (you might lose 8 hours of progress). Spot pricing varies by region and demand.
For product teams
Spot instances can cut compute costs dramatically if your workload is fault-tolerant.
For engineers
Use spot for inference (stateless), not training; build recovery into your orchestration.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome