Your AI's answer can be valid and still wrong
AI models can now be forced to answer in the exact format your code expects. That ends one class of bugs. Here is what it does not check.
Abhijit Chatterjee Chattopadhyay

For years, the first bug in almost every AI feature was the same: the model sent back something your code could not read. A missing field, a stray sentence, a bracket in the wrong place.
That bug is going away. The next one is quieter.
What changed
On 19 October, AWS announced structured outputs for all Anthropic Claude models on Amazon Bedrock (opens in a new tab), where the model is available. You give the model a schema, a description of the exact fields and types you expect, and it is held to it while it writes.
In AWS's words, this "changes output formatting from probabilistic to deterministic". Every response is valid JSON, field types are enforced, and required fields are always present. Other providers offer similar features.
For anyone who has written retry loops around AI output, that is real progress.
What it does not check
The same post is honest about two of the limits:
- It can still fail. AWS lists two cases that produce responses that do not match the schema: the model refusing, and the model running out of tokens before it finishes. They advise checking the stop reason in every response.
- Ranges are not enforced. Number limits such as minimum and maximum, and text limits such as minimum and maximum length, are listed as not supported. An amount will always be a number. Not necessarily a sensible one.
The third limit is the one it cannot tell you. Valid is not right. A schema guarantees the field is there and has the right type. It cannot know whether the model put the right customer, date or amount in it.
AWS is right that you no longer need code to check the format. You still need code to check the facts.
Keep the business rules in your code
The checks that protect your business still belong after the model, in your own code:
- Is the amount within the range you allow?
- Is the date in the future, if it has to be?
- Does the customer or order ID actually exist?
- Do the fields agree with each other?
A schema makes those checks easier to write, because you always know what you are checking. It does not replace them.
Do this today
- Find every place your code retries or patches AI output because of format problems. Those are your first candidates for a schema.
- For any field with a fixed list of values, define that list in the schema.
- Write down the business rules the schema cannot enforce, and check them in code.
- Handle the two failure cases: a refusal, and an answer cut off by a token limit.
Your next step
I write one practical note like this every day. Sign up just below to get them by email.
Drafted with AI help, edited and approved by me.