Your AI provider will go down. Then what?
Every AI feature depends on a provider that will have a bad day. Three fallbacks to build before it does: degrade, switch and time out smart.
Abhijit Chatterjee Chattopadhyay

Your AI provider will go down. The question is what your product does when it does.
The weakest link you did not build
Sam Newman, in his new book Building Resilient Distributed Systems, is blunt about it. In an excerpt published by The Pragmatic Engineer (opens in a new tab), he writes that OpenAI and Anthropic "don't have a great track record in terms of providing high levels of availability for their services, at least when compared to other managed service providers, like public cloud vendors."
Add rate limits, token budgets running out and slow responses, and your shiny AI feature now has a dependency that fails more often than the rest of your stack.
Three fallbacks
1. Degrade, do not die. Decide now what the product does without the model. Newman calls this graceful degradation. Limit the feature, do not break the page. A search box can fall back to plain search. A drafting tool can fall back to a blank form. Users forgive a smaller feature. They do not forgive an error screen.
2. Have a second route. The same models are often sold by more than one provider. Newman gives two examples, OpenAI models on Azure and Anthropic models through Google's platform. A backup route may lag a little on the newest models, but most products do not need the newest model to stay up.
3. Time out smart. AI calls vary wildly in how long they take. Newman suggests setting timeouts based on how much output you expect, or watching for the first streamed words instead of waiting for the whole answer. A fixed 30-second timeout is either too short or far too long.
One trap
Do not chain three vendors in one flow to be safe. Newman points out that a flow using models from several vendors needs all of them up at once. More vendors in one path means more ways to fail, not fewer.
Do this today
- List every feature that calls an AI model.
- For each, write one line: what the user sees if the model is down.
- Check whether your main model is available from a second provider, and test one call through it.
- Replace any fixed timeout on AI calls with one based on expected output or first response.
Your next step
If you want fallbacks built into your AI features, book a free 30-minute 1:1 assessment. I look at your setup with you and tell you what the work would take.
Drafted with AI help, edited and approved by me.