Canary Deployment
Rolling out a new model to a small fraction of traffic first, then gradually increasing if metrics look good.
Think of it like
Coal miners brought canaries into mines to detect poisonous gas early; new model is your canary.
Example
Route 5% of requests to the new model v1.1, monitor latency and accuracy, then increment to 50% if all looks well.
How it actually works
Mitigates blast radius of buggy or degraded models. Requires ability to split traffic, capture metrics, and rollback automatically. Drift detection rules trigger automated rollback. Can run multiple versions in parallel to compare. Good for high-stakes production systems.
For product teams
Dramatically reduces risk of model updates while maintaining continuous improvement velocity.
For engineers
Traffic router (Envoy, istio, or application code) weights requests; observability pipeline monitors canary metrics vs baseline; automated rollback if error rate or latency exceeds threshold.
Related
- Variant of.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome