Skip to content
← Writing
Writing · October 2026 · 2 min read

Your AI's answer can be valid and still wrong

AI models can now be forced to answer in the exact format your code expects. That ends one class of bugs. Here is what it does not check.

Abhijit Chatterjee Chattopadhyay

The Logic Atelier logo above the words: Your AI's answer can be valid and still wrong.

For years, the first bug in almost every AI feature was the same: the model sent back something your code could not read. A missing field, a stray sentence, a bracket in the wrong place.

That bug is going away. The next one is quieter.

What changed

On 19 October, AWS announced structured outputs for all Anthropic Claude models on Amazon Bedrock (opens in a new tab), where the model is available. You give the model a schema, a description of the exact fields and types you expect, and it is held to it while it writes.

In AWS's words, this "changes output formatting from probabilistic to deterministic". Every response is valid JSON, field types are enforced, and required fields are always present. Other providers offer similar features.

For anyone who has written retry loops around AI output, that is real progress.

What it does not check

The same post is honest about two of the limits:

  1. It can still fail. AWS lists two cases that produce responses that do not match the schema: the model refusing, and the model running out of tokens before it finishes. They advise checking the stop reason in every response.
  2. Ranges are not enforced. Number limits such as minimum and maximum, and text limits such as minimum and maximum length, are listed as not supported. An amount will always be a number. Not necessarily a sensible one.

The third limit is the one it cannot tell you. Valid is not right. A schema guarantees the field is there and has the right type. It cannot know whether the model put the right customer, date or amount in it.

AWS is right that you no longer need code to check the format. You still need code to check the facts.

Keep the business rules in your code

The checks that protect your business still belong after the model, in your own code:

  • Is the amount within the range you allow?
  • Is the date in the future, if it has to be?
  • Does the customer or order ID actually exist?
  • Do the fields agree with each other?

A schema makes those checks easier to write, because you always know what you are checking. It does not replace them.

Do this today

  • Find every place your code retries or patches AI output because of format problems. Those are your first candidates for a schema.
  • For any field with a fixed list of values, define that list in the schema.
  • Write down the business rules the schema cannot enforce, and check them in code.
  • Handle the two failure cases: a refusal, and an answer cut off by a token limit.

Your next step

I write one practical note like this every day. Sign up just below to get them by email.

Drafted with AI help, edited and approved by me.

New articles by email

I will never sell your details, for any reason, and I never send spam.

How I use your details
  • I, Abhijit Chatterjee Chattopadhyay of The Logic Atelier, hold them, in my own newsletter system hosted on Fly.io. Emails go out through Resend.
  • Your name is for greeting you, your email is for sending new articles, and your phone number, if you give it, is so I can reach you directly about my work.
  • If you do not confirm your email within 7 days, I delete the sign-up. If you unsubscribe, I delete your details 30 days later.
  • If Google Analytics is running on your visit, it is told that a sign-up was made, and nothing you typed.
  • To see or delete what I hold about you at any time, email hello@thelogicatelier.com.
  • The full detail is on the privacy page.
NextEight rules from eight days of AI in production
← cd ../writing

Your AI's answer can be valid and still wrong

valid-not-correct.mdfront matter
title: Your AI's answer can be valid and still wrong
date: 2026-10-26
reading_time: 2 min
author: Abhijit Chatterjee Chattopadhyay
valid_not_correct_cover_e2c48e1a_3ac24518ca.pngcover
The Logic Atelier logo above the words: Your AI's answer can be valid and still wrong.

For years, the first bug in almost every AI feature was the same: the model sent back something your code could not read. A missing field, a stray sentence, a bracket in the wrong place.

That bug is going away. The next one is quieter.

What changed

On 19 October, AWS announced structured outputs for all Anthropic Claude models on Amazon Bedrock (opens in a new tab), where the model is available. You give the model a schema, a description of the exact fields and types you expect, and it is held to it while it writes.

In AWS's words, this "changes output formatting from probabilistic to deterministic". Every response is valid JSON, field types are enforced, and required fields are always present. Other providers offer similar features.

For anyone who has written retry loops around AI output, that is real progress.

What it does not check

The same post is honest about two of the limits:

  1. It can still fail. AWS lists two cases that produce responses that do not match the schema: the model refusing, and the model running out of tokens before it finishes. They advise checking the stop reason in every response.
  2. Ranges are not enforced. Number limits such as minimum and maximum, and text limits such as minimum and maximum length, are listed as not supported. An amount will always be a number. Not necessarily a sensible one.

The third limit is the one it cannot tell you. Valid is not right. A schema guarantees the field is there and has the right type. It cannot know whether the model put the right customer, date or amount in it.

AWS is right that you no longer need code to check the format. You still need code to check the facts.

Keep the business rules in your code

The checks that protect your business still belong after the model, in your own code:

  • Is the amount within the range you allow?
  • Is the date in the future, if it has to be?
  • Does the customer or order ID actually exist?
  • Do the fields agree with each other?

A schema makes those checks easier to write, because you always know what you are checking. It does not replace them.

Do this today

  • Find every place your code retries or patches AI output because of format problems. Those are your first candidates for a schema.
  • For any field with a fixed list of values, define that list in the schema.
  • Write down the business rules the schema cannot enforce, and check them in code.
  • Handle the two failure cases: a refusal, and an answer cut off by a token limit.

Your next step

I write one practical note like this every day. Sign up just below to get them by email.

Drafted with AI help, edited and approved by me.

subscribeemail

// I will never sell your details, for any reason, and I never send spam.

How I use your details
  • I, Abhijit Chatterjee Chattopadhyay of The Logic Atelier, hold them, in my own newsletter system hosted on Fly.io. Emails go out through Resend.
  • Your name is for greeting you, your email is for sending new articles, and your phone number, if you give it, is so I can reach you directly about my work.
  • If you do not confirm your email within 7 days, I delete the sign-up. If you unsubscribe, I delete your details 30 days later.
  • If Google Analytics is running on your visit, it is told that a sign-up was made, and nothing you typed.
  • To see or delete what I hold about you at any time, email hello@thelogicatelier.com.
  • The full detail is on the privacy page.

vi.Contact

06 · contact

Talk to me.

Talk to me.

Production AI and software architecture. Based in Dubai, working with teams in the Emirates and Ireland.

I work Dubai hours, Monday to Friday, the same week you do. Dubai is three hours ahead of Dublin in summer and four in winter, so my afternoon is your morning.

I will never sell your details, for any reason, and I never send spam.

How I use your enquiry
  • Your enquiry comes to me, Abhijit Chatterjee Chattopadhyay of The Logic Atelier. The form is handled by Formspree, which passes it to my inbox and keeps a copy in my Formspree account.
  • I use it to read your enquiry and reply. If no work follows, I delete it within 12 months, and sooner whenever you ask.
  • If Google Analytics is running on your visit, it is told that an enquiry was sent, and nothing you typed.
  • To see or delete what I hold about you, email hello@thelogicatelier.com.
  • The full detail is on the privacy page.