How do you know your AI works?
Most AI features ship after a few prompts and a look at the answers. Google's engineers explain why that misses failures. Four habits that replace it.
Abhijit Chatterjee Chattopadhyay

Here is how most AI features get tested. You run three or four prompts, read the answers, and if they look reasonable, you ship.
That is close to how Google's engineers describe it in their post on evaluating AI agents (opens in a new tab), published on 15 October. Their title calls it the vibe check. Their point is that it misses the failures that matter.
Why looking at the answer is not enough
An AI agent can go wrong and still sound right. In the post's words, an agent "can get stuck in a retry loop, fire off multiple tool calls, yet still return a polite, plausible-sounding final answer."
The answer you read looks fine. Behind it, the bill went up, a system was called several times instead of once, and the customer waited. None of that shows in the text.
The good news in the post is that the fix does not need to be big: "any repeatable evaluation system (no matter how simple) is better than relying on ad-hoc spot checks or no system at all."
Four habits that make testing real
- Write down what success means, in the user's terms. Who is using it, what they expect, and how you would check it. Every part of that sentence becomes a test.
- Read the runs before you build scores. The post suggests reading about twenty full runs and marking every place the agent hesitated, made something up or picked the wrong tool. That list tells you what to test.
- Ask yes-or-no questions. When a model grades the answers, a 1 to 5 scale moves between runs, because the line between a 3 and a 4 is fuzzy. "Did the agent verify the account ID before calling the refund tool?" does not move.
- Keep the test set fixed while you tune. Every time you add or remove a test, you can no longer tell whether the AI changed or the test did. I made the same case for model updates in Your model changed and nobody told you.
Whatever you test, measure time, cost and the number of tool calls alongside the answer. A correct answer that took far too long is still a failure.
After launch, test on every change
The post's rule is simple: whenever you change a prompt, swap a model or edit a tool, run the same tests again and compare. Passing once proves very little. Passing the same tests after every change is what tells you it still works.
Do this today
- Pick one AI feature and write one sentence on what a good result looks like for its user.
- Read twenty real runs from start to finish, not just the final answers.
- Turn the three most common problems into yes-or-no checks you can run again.
Your next step
I write one practical note like this every day. Sign up just below to get them by email.
Drafted with AI help, edited and approved by me.