Can your AI receptionist get a phone number right?
A new test of AI voice agents found they often get names, numbers and emails wrong, and the errors add up on longer calls. A five-minute test to run on yours.
Abhijit Chatterjee Chattopadhyay

An AI receptionist, the AI phone agent that answers your calls, can sound perfect and still book your customer under the wrong name, with the wrong phone number. You would not hear the mistake on the call. You would find it when the confirmation never arrives.
That is the problem a new benchmark from Sierra, a company that builds AI voice agents, set out to measure.
What the test found
On 22 October, Sierra published a benchmark for voice agents (opens in a new tab) that tests one simple job: capturing a detail correctly. It covers "200 tasks across ten entity types, including names, phone numbers, addresses, and emails", with a cooperative caller "who spells accurately when asked."
Its conclusion: "Even in this simple setting, today's voice agents struggle to get the details right."
The errors also add up. Sierra points to a separate multi-detail study in the research paper behind the benchmark: when a call needed three details, the agents got all three right on only about half of calls, even though each single detail was right about three times out of four. I made the same point about agents in general in Right three times out of four is not reliable.
The post adds that asking again does not always help: sometimes the agent "repeats the same mistake", and sometimes it swaps the first wrong value for a new wrong one.
What made it better
The same post reports that checking each detail and letting the agent try again raised success on multi-detail calls. That matches what works on a person's phone line too: read it back, spell it out, confirm it in writing.
A detail that looks right is not the same as a detail that is right. More on that in Your AI's answer can be valid and still wrong.
How to test an AI receptionist in five minutes
- Call it yourself. Give a name that is hard to spell, an email address with a dot and a number in it, and a local postcode such as an Eircode.
- Listen for the read-back. Does it repeat the details and ask you to confirm? If not, that is the first fix.
- Check what was stored. Open the booking or the CRM record and compare it, character by character, with what you said.
- Try it in a noisy room and with a second caller. Accents, background noise and fast speakers are where errors grow.
- Make it confirm in writing. A text or email with the details lets the customer catch what the AI missed.
Do this today
- Make one test call to your AI line with three hard details.
- Compare the stored record with what you said.
- If any detail is wrong, add a read-back and a written confirmation before more customers call.
Your next step
I write one practical note like this every day. Sign up just below to get them by email.
Drafted with AI help, edited and approved by me.