Skip to content
← Writing
Writing · November 2026 · 2 min read

When an AI shopper fails, it often still buys

Visa tested AI models on realistic shopping tasks. Many failed, and about half of the failures still ended in a purchase. What sellers can put on a product page so an AI buyer gets it right.

Abhijit Chatterjee Chattopadhyay

The Logic Atelier logo above the words: When an AI shopper fails, it often still buys

Ask an AI assistant to buy you a TV and it has to work out what you actually need. Size, budget, the features you forgot to mention. A new test shows how often it gets that wrong, and what it does next.

What Visa tested

On 5 November, Visa launched the Visa Agentic Commerce Benchmark (opens in a new tab). The first round put 22 AI models from 12 labs through 15 shopping challenges, such as buying a TV, a laptop or a personal care appliance. The AI had to find out what the customer needed through conversation, and not buy if nothing fitted.

The results:

  • "Across testing, selected AI models achieved an overall benchmark pass rate of about half (57.2%)."
  • "The best performing models passed 82.2% of attempts, the worst 28.9%."
  • "About 1-in-2 (54%) of failed runs still ended in the AI making a purchase."
  • Each model had three attempts per scenario, and all three passed "just 34.2% of the time."

This is Visa's own benchmark, and Visa has an interest in AI shopping. It also tested choosing, not paying. But the pattern is worth a seller's attention.

The finding that matters for sellers

The models that asked better questions did better. In Visa's words, the five models "that uncovered the most hidden buying factors" averaged a 70.2% pass rate. The five that uncovered the least passed 44.4% of the time.

A hidden buying factor is the thing the customer cares about but did not say. Will it work with my setup? Does it fit the space? What is in the box? If the answer is buried in an image, a PDF or a vague sentence, an AI buyer may never find it, and buy anyway.

That is a return, a complaint or a chargeback with your name on it. More on getting ready for these buyers in Is your checkout ready for an AI buyer?

What to put on your product pages

1. State the deal-breakers in plain text. Size, compatibility, power, what is included, what is not. As words on the page, not only in a picture.

2. Answer the questions customers ask before they buy. Your support inbox already tells you what they are.

3. Make return terms easy to find. An AI that is unsure should be able to see what happens if it is wrong.

4. Let the right agents in. None of this helps if your site blocks the assistant your customer sent. See Some of the bots on your site are customers.

Do this today

  • Pick your best-selling product and list the three questions customers ask most before buying.
  • Check that each answer is written on the product page as text.
  • Ask an AI assistant to choose that product for a made-up customer, and see what it misses.

Your next step

If you want help getting your site ready for AI buyers, book a free 30-minute 1:1 assessment. I look at your setup with you and tell you what the work would take.

New articles by email

I will never sell your details, for any reason, and I never send spam.

How I use your details
  • I, Abhijit Chatterjee Chattopadhyay of The Logic Atelier, hold them, in my own newsletter system hosted on Fly.io. Emails go out through Resend.
  • Your name is for greeting you, your email is for sending new articles, and your phone number, if you give it, is so I can reach you directly about my work.
  • If you do not confirm your email within 7 days, I delete the sign-up. If you unsubscribe, I delete your details 30 days later.
  • If Google Analytics is running on your visit, it is told that a sign-up was made, and nothing you typed.
  • To see or delete what I hold about you at any time, email hello@thelogicatelier.com.
  • The full detail is on the privacy page.
NextYour Copilot licence may come with a meter
← cd ../writing

When an AI shopper fails, it often still buys

ai-shopping-agents-fail.mdfront matter
title: When an AI shopper fails, it often still buys
date: 2026-11-13
reading_time: 2 min
author: Abhijit Chatterjee Chattopadhyay
ai_shopping_agents_fail_cover_923101a6_28cdd71ce5.pngcover
The Logic Atelier logo above the words: When an AI shopper fails, it often still buys

Ask an AI assistant to buy you a TV and it has to work out what you actually need. Size, budget, the features you forgot to mention. A new test shows how often it gets that wrong, and what it does next.

What Visa tested

On 5 November, Visa launched the Visa Agentic Commerce Benchmark (opens in a new tab). The first round put 22 AI models from 12 labs through 15 shopping challenges, such as buying a TV, a laptop or a personal care appliance. The AI had to find out what the customer needed through conversation, and not buy if nothing fitted.

The results:

  • "Across testing, selected AI models achieved an overall benchmark pass rate of about half (57.2%)."
  • "The best performing models passed 82.2% of attempts, the worst 28.9%."
  • "About 1-in-2 (54%) of failed runs still ended in the AI making a purchase."
  • Each model had three attempts per scenario, and all three passed "just 34.2% of the time."

This is Visa's own benchmark, and Visa has an interest in AI shopping. It also tested choosing, not paying. But the pattern is worth a seller's attention.

The finding that matters for sellers

The models that asked better questions did better. In Visa's words, the five models "that uncovered the most hidden buying factors" averaged a 70.2% pass rate. The five that uncovered the least passed 44.4% of the time.

A hidden buying factor is the thing the customer cares about but did not say. Will it work with my setup? Does it fit the space? What is in the box? If the answer is buried in an image, a PDF or a vague sentence, an AI buyer may never find it, and buy anyway.

That is a return, a complaint or a chargeback with your name on it. More on getting ready for these buyers in Is your checkout ready for an AI buyer?

What to put on your product pages

1. State the deal-breakers in plain text. Size, compatibility, power, what is included, what is not. As words on the page, not only in a picture.

2. Answer the questions customers ask before they buy. Your support inbox already tells you what they are.

3. Make return terms easy to find. An AI that is unsure should be able to see what happens if it is wrong.

4. Let the right agents in. None of this helps if your site blocks the assistant your customer sent. See Some of the bots on your site are customers.

Do this today

  • Pick your best-selling product and list the three questions customers ask most before buying.
  • Check that each answer is written on the product page as text.
  • Ask an AI assistant to choose that product for a made-up customer, and see what it misses.

Your next step

If you want help getting your site ready for AI buyers, book a free 30-minute 1:1 assessment. I look at your setup with you and tell you what the work would take.

subscribeemail

// I will never sell your details, for any reason, and I never send spam.

How I use your details
  • I, Abhijit Chatterjee Chattopadhyay of The Logic Atelier, hold them, in my own newsletter system hosted on Fly.io. Emails go out through Resend.
  • Your name is for greeting you, your email is for sending new articles, and your phone number, if you give it, is so I can reach you directly about my work.
  • If you do not confirm your email within 7 days, I delete the sign-up. If you unsubscribe, I delete your details 30 days later.
  • If Google Analytics is running on your visit, it is told that a sign-up was made, and nothing you typed.
  • To see or delete what I hold about you at any time, email hello@thelogicatelier.com.
  • The full detail is on the privacy page.

vi.Contact

06 · contact

Talk to me.

Talk to me.

Production AI and software architecture. Based in Dubai, working with teams in the Emirates and Ireland.

I work Dubai hours, Monday to Friday, the same week you do. Dubai is three hours ahead of Dublin in summer and four in winter, so my afternoon is your morning.

I will never sell your details, for any reason, and I never send spam.

How I use your enquiry
  • Your enquiry comes to me, Abhijit Chatterjee Chattopadhyay of The Logic Atelier. The form is handled by Formspree, which passes it to my inbox and keeps a copy in my Formspree account.
  • I use it to read your enquiry and reply. If no work follows, I delete it within 12 months, and sooner whenever you ask.
  • If Google Analytics is running on your visit, it is told that an enquiry was sent, and nothing you typed.
  • To see or delete what I hold about you, email hello@thelogicatelier.com.
  • The full detail is on the privacy page.