Fallback model
A secondary model you route to if the primary fails or gets overloaded.
Think of it like
Calling your backup driver if the main driver is sick.
Example
Primary model is Claude; it's overloaded; requests fall through to GPT-4 as a fallback.
How it actually works
Fallback models add resilience but cost money (you pay for both). You need to decide when to fallback — timeout, error rate spike, or queue depth. Fallbacks should be clearly logged so you know when they're happening.
For product teams
Fallbacks are insurance against outages at the cost of higher per-request spend.
For engineers
Implement fallback logic with timeouts and error rates; monitor fallback rate closely.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome