Skip to content
← Writing
Writing · November 2026 · 2 min read

Can your AI receptionist get a phone number right?

A new test of AI voice agents found they often get names, numbers and emails wrong, and the errors add up on longer calls. A five-minute test to run on yours.

Abhijit Chatterjee Chattopadhyay

The Logic Atelier logo above the words: Can your AI receptionist get a phone number right?

An AI receptionist, the AI phone agent that answers your calls, can sound perfect and still book your customer under the wrong name, with the wrong phone number. You would not hear the mistake on the call. You would find it when the confirmation never arrives.

That is the problem a new benchmark from Sierra, a company that builds AI voice agents, set out to measure.

What the test found

On 22 October, Sierra published a benchmark for voice agents (opens in a new tab) that tests one simple job: capturing a detail correctly. It covers "200 tasks across ten entity types, including names, phone numbers, addresses, and emails", with a cooperative caller "who spells accurately when asked."

Its conclusion: "Even in this simple setting, today's voice agents struggle to get the details right."

The errors also add up. Sierra points to a separate multi-detail study in the research paper behind the benchmark: when a call needed three details, the agents got all three right on only about half of calls, even though each single detail was right about three times out of four. I made the same point about agents in general in Right three times out of four is not reliable.

The post adds that asking again does not always help: sometimes the agent "repeats the same mistake", and sometimes it swaps the first wrong value for a new wrong one.

What made it better

The same post reports that checking each detail and letting the agent try again raised success on multi-detail calls. That matches what works on a person's phone line too: read it back, spell it out, confirm it in writing.

A detail that looks right is not the same as a detail that is right. More on that in Your AI's answer can be valid and still wrong.

How to test an AI receptionist in five minutes

  1. Call it yourself. Give a name that is hard to spell, an email address with a dot and a number in it, and a local postcode such as an Eircode.
  2. Listen for the read-back. Does it repeat the details and ask you to confirm? If not, that is the first fix.
  3. Check what was stored. Open the booking or the CRM record and compare it, character by character, with what you said.
  4. Try it in a noisy room and with a second caller. Accents, background noise and fast speakers are where errors grow.
  5. Make it confirm in writing. A text or email with the details lets the customer catch what the AI missed.

Do this today

  • Make one test call to your AI line with three hard details.
  • Compare the stored record with what you said.
  • If any detail is wrong, add a read-back and a written confirmation before more customers call.

Your next step

I write one practical note like this every day. Sign up just below to get them by email.

Drafted with AI help, edited and approved by me.

New articles by email

I will never sell your details, for any reason, and I never send spam.

How I use your details
  • I, Abhijit Chatterjee Chattopadhyay of The Logic Atelier, hold them, in my own newsletter system hosted on Fly.io. Emails go out through Resend.
  • Your name is for greeting you, your email is for sending new articles, and your phone number, if you give it, is so I can reach you directly about my work.
  • If you do not confirm your email within 7 days, I delete the sign-up. If you unsubscribe, I delete your details 30 days later.
  • If Google Analytics is running on your visit, it is told that a sign-up was made, and nothing you typed.
  • To see or delete what I hold about you at any time, email hello@thelogicatelier.com.
  • The full detail is on the privacy page.
NextYour staff's AI chats may be kept for three years
← cd ../writing

Can your AI receptionist get a phone number right?

ai-receptionist.mdfront matter
title: Can your AI receptionist get a phone number right?
date: 2026-11-07
reading_time: 2 min
author: Abhijit Chatterjee Chattopadhyay
ai_receptionist_cover_a7fa5854_8708737852.pngcover
The Logic Atelier logo above the words: Can your AI receptionist get a phone number right?

An AI receptionist, the AI phone agent that answers your calls, can sound perfect and still book your customer under the wrong name, with the wrong phone number. You would not hear the mistake on the call. You would find it when the confirmation never arrives.

That is the problem a new benchmark from Sierra, a company that builds AI voice agents, set out to measure.

What the test found

On 22 October, Sierra published a benchmark for voice agents (opens in a new tab) that tests one simple job: capturing a detail correctly. It covers "200 tasks across ten entity types, including names, phone numbers, addresses, and emails", with a cooperative caller "who spells accurately when asked."

Its conclusion: "Even in this simple setting, today's voice agents struggle to get the details right."

The errors also add up. Sierra points to a separate multi-detail study in the research paper behind the benchmark: when a call needed three details, the agents got all three right on only about half of calls, even though each single detail was right about three times out of four. I made the same point about agents in general in Right three times out of four is not reliable.

The post adds that asking again does not always help: sometimes the agent "repeats the same mistake", and sometimes it swaps the first wrong value for a new wrong one.

What made it better

The same post reports that checking each detail and letting the agent try again raised success on multi-detail calls. That matches what works on a person's phone line too: read it back, spell it out, confirm it in writing.

A detail that looks right is not the same as a detail that is right. More on that in Your AI's answer can be valid and still wrong.

How to test an AI receptionist in five minutes

  1. Call it yourself. Give a name that is hard to spell, an email address with a dot and a number in it, and a local postcode such as an Eircode.
  2. Listen for the read-back. Does it repeat the details and ask you to confirm? If not, that is the first fix.
  3. Check what was stored. Open the booking or the CRM record and compare it, character by character, with what you said.
  4. Try it in a noisy room and with a second caller. Accents, background noise and fast speakers are where errors grow.
  5. Make it confirm in writing. A text or email with the details lets the customer catch what the AI missed.

Do this today

  • Make one test call to your AI line with three hard details.
  • Compare the stored record with what you said.
  • If any detail is wrong, add a read-back and a written confirmation before more customers call.

Your next step

I write one practical note like this every day. Sign up just below to get them by email.

Drafted with AI help, edited and approved by me.

subscribeemail

// I will never sell your details, for any reason, and I never send spam.

How I use your details
  • I, Abhijit Chatterjee Chattopadhyay of The Logic Atelier, hold them, in my own newsletter system hosted on Fly.io. Emails go out through Resend.
  • Your name is for greeting you, your email is for sending new articles, and your phone number, if you give it, is so I can reach you directly about my work.
  • If you do not confirm your email within 7 days, I delete the sign-up. If you unsubscribe, I delete your details 30 days later.
  • If Google Analytics is running on your visit, it is told that a sign-up was made, and nothing you typed.
  • To see or delete what I hold about you at any time, email hello@thelogicatelier.com.
  • The full detail is on the privacy page.

vi.Contact

06 · contact

Talk to me.

Talk to me.

Production AI and software architecture. Based in Dubai, working with teams in the Emirates and Ireland.

I work Dubai hours, Monday to Friday, the same week you do. Dubai is three hours ahead of Dublin in summer and four in winter, so my afternoon is your morning.

I will never sell your details, for any reason, and I never send spam.

How I use your enquiry
  • Your enquiry comes to me, Abhijit Chatterjee Chattopadhyay of The Logic Atelier. The form is handled by Formspree, which passes it to my inbox and keeps a copy in my Formspree account.
  • I use it to read your enquiry and reply. If no work follows, I delete it within 12 months, and sooner whenever you ask.
  • If Google Analytics is running on your visit, it is told that an enquiry was sent, and nothing you typed.
  • To see or delete what I hold about you, email hello@thelogicatelier.com.
  • The full detail is on the privacy page.