Skip to content
← Writing
Writing · October 2026 · 2 min read

Your AI provider will go down. Then what?

Every AI feature depends on a provider that will have a bad day. Three fallbacks to build before it does: degrade, switch and time out smart.

Abhijit Chatterjee Chattopadhyay

The Logic Atelier logo above the words: Your AI provider will go down. Then what?

Your AI provider will go down. The question is what your product does when it does.

The weakest link you did not build

Sam Newman, in his new book Building Resilient Distributed Systems, is blunt about it. In an excerpt published by The Pragmatic Engineer (opens in a new tab), he writes that OpenAI and Anthropic "don't have a great track record in terms of providing high levels of availability for their services, at least when compared to other managed service providers, like public cloud vendors."

Add rate limits, token budgets running out and slow responses, and your shiny AI feature now has a dependency that fails more often than the rest of your stack.

Three fallbacks

1. Degrade, do not die. Decide now what the product does without the model. Newman calls this graceful degradation. Limit the feature, do not break the page. A search box can fall back to plain search. A drafting tool can fall back to a blank form. Users forgive a smaller feature. They do not forgive an error screen.

2. Have a second route. The same models are often sold by more than one provider. Newman gives two examples, OpenAI models on Azure and Anthropic models through Google's platform. A backup route may lag a little on the newest models, but most products do not need the newest model to stay up.

3. Time out smart. AI calls vary wildly in how long they take. Newman suggests setting timeouts based on how much output you expect, or watching for the first streamed words instead of waiting for the whole answer. A fixed 30-second timeout is either too short or far too long.

One trap

Do not chain three vendors in one flow to be safe. Newman points out that a flow using models from several vendors needs all of them up at once. More vendors in one path means more ways to fail, not fewer.

Do this today

  • List every feature that calls an AI model.
  • For each, write one line: what the user sees if the model is down.
  • Check whether your main model is available from a second provider, and test one call through it.
  • Replace any fixed timeout on AI calls with one based on expected output or first response.

Your next step

If you want fallbacks built into your AI features, book a free 30-minute 1:1 assessment. I look at your setup with you and tell you what the work would take.

Drafted with AI help, edited and approved by me.

New articles by email

I will never sell your details, for any reason, and I never send spam.

How I use your details
  • I, Abhijit Chatterjee Chattopadhyay of The Logic Atelier, hold them, in my own newsletter system hosted on Fly.io. Emails go out through Resend.
  • Your name is for greeting you, your email is for sending new articles, and your phone number, if you give it, is so I can reach you directly about my work.
  • If you do not confirm your email within 7 days, I delete the sign-up. If you unsubscribe, I delete your details 30 days later.
  • If Google Analytics is running on your visit, it is told that a sign-up was made, and nothing you typed.
  • To see or delete what I hold about you at any time, email hello@thelogicatelier.com.
  • The full detail is on the privacy page.
NextYour AI agent has more access than you think
← cd ../writing

Your AI provider will go down. Then what?

ai-provider-down.mdfront matter
title: Your AI provider will go down. Then what?
date: 2026-10-22
reading_time: 2 min
author: Abhijit Chatterjee Chattopadhyay
ai_provider_down_cover_869f1ad7_752fa59e1b.pngcover
The Logic Atelier logo above the words: Your AI provider will go down. Then what?

Your AI provider will go down. The question is what your product does when it does.

Sam Newman, in his new book Building Resilient Distributed Systems, is blunt about it. In an excerpt published by The Pragmatic Engineer (opens in a new tab), he writes that OpenAI and Anthropic "don't have a great track record in terms of providing high levels of availability for their services, at least when compared to other managed service providers, like public cloud vendors."

Add rate limits, token budgets running out and slow responses, and your shiny AI feature now has a dependency that fails more often than the rest of your stack.

Three fallbacks

1. Degrade, do not die. Decide now what the product does without the model. Newman calls this graceful degradation. Limit the feature, do not break the page. A search box can fall back to plain search. A drafting tool can fall back to a blank form. Users forgive a smaller feature. They do not forgive an error screen.

2. Have a second route. The same models are often sold by more than one provider. Newman gives two examples, OpenAI models on Azure and Anthropic models through Google's platform. A backup route may lag a little on the newest models, but most products do not need the newest model to stay up.

3. Time out smart. AI calls vary wildly in how long they take. Newman suggests setting timeouts based on how much output you expect, or watching for the first streamed words instead of waiting for the whole answer. A fixed 30-second timeout is either too short or far too long.

One trap

Do not chain three vendors in one flow to be safe. Newman points out that a flow using models from several vendors needs all of them up at once. More vendors in one path means more ways to fail, not fewer.

Do this today

  • List every feature that calls an AI model.
  • For each, write one line: what the user sees if the model is down.
  • Check whether your main model is available from a second provider, and test one call through it.
  • Replace any fixed timeout on AI calls with one based on expected output or first response.

Your next step

If you want fallbacks built into your AI features, book a free 30-minute 1:1 assessment. I look at your setup with you and tell you what the work would take.

Drafted with AI help, edited and approved by me.

subscribeemail

// I will never sell your details, for any reason, and I never send spam.

How I use your details
  • I, Abhijit Chatterjee Chattopadhyay of The Logic Atelier, hold them, in my own newsletter system hosted on Fly.io. Emails go out through Resend.
  • Your name is for greeting you, your email is for sending new articles, and your phone number, if you give it, is so I can reach you directly about my work.
  • If you do not confirm your email within 7 days, I delete the sign-up. If you unsubscribe, I delete your details 30 days later.
  • If Google Analytics is running on your visit, it is told that a sign-up was made, and nothing you typed.
  • To see or delete what I hold about you at any time, email hello@thelogicatelier.com.
  • The full detail is on the privacy page.

vi.Contact

06 · contact

Talk to me.

Talk to me.

Production AI and software architecture. Based in Dubai, working with teams in the Emirates and Ireland.

I work Dubai hours, Monday to Friday, the same week you do. Dubai is three hours ahead of Dublin in summer and four in winter, so my afternoon is your morning.

I will never sell your details, for any reason, and I never send spam.

How I use your enquiry
  • Your enquiry comes to me, Abhijit Chatterjee Chattopadhyay of The Logic Atelier. The form is handled by Formspree, which passes it to my inbox and keeps a copy in my Formspree account.
  • I use it to read your enquiry and reply. If no work follows, I delete it within 12 months, and sooner whenever you ask.
  • If Google Analytics is running on your visit, it is told that an enquiry was sent, and nothing you typed.
  • To see or delete what I hold about you, email hello@thelogicatelier.com.
  • The full detail is on the privacy page.