Cascade
A chain of fallbacks — try model A, if it fails try B, if that fails try C.
Think of it like
Calling your boss, then their boss, then the CEO if each person is unavailable.
Example
Route to vLLM (fast), fall back to TensorRT (slower), fall back to ONNX Runtime (slowest).
How it actually works
Cascading adds redundancy and flexibility. You get graceful degradation — the system is still responsive even if preferred options fail. The tradeoff: increased complexity, higher costs (you pay for unused backups), and potential cascading failures if not designed carefully.
For product teams
Cascades provide safety but are expensive — use when reliability matters more than cost.
For engineers
Implement cascades with clear failure policies; avoid cascades that degrade UX (slow fallback = bad UX).
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome