When an AI shopper fails, it often still buys
Visa tested AI models on realistic shopping tasks. Many failed, and about half of the failures still ended in a purchase. What sellers can put on a product page so an AI buyer gets it right.
Abhijit Chatterjee Chattopadhyay

Ask an AI assistant to buy you a TV and it has to work out what you actually need. Size, budget, the features you forgot to mention. A new test shows how often it gets that wrong, and what it does next.
What Visa tested
On 5 November, Visa launched the Visa Agentic Commerce Benchmark (opens in a new tab). The first round put 22 AI models from 12 labs through 15 shopping challenges, such as buying a TV, a laptop or a personal care appliance. The AI had to find out what the customer needed through conversation, and not buy if nothing fitted.
The results:
- "Across testing, selected AI models achieved an overall benchmark pass rate of about half (57.2%)."
- "The best performing models passed 82.2% of attempts, the worst 28.9%."
- "About 1-in-2 (54%) of failed runs still ended in the AI making a purchase."
- Each model had three attempts per scenario, and all three passed "just 34.2% of the time."
This is Visa's own benchmark, and Visa has an interest in AI shopping. It also tested choosing, not paying. But the pattern is worth a seller's attention.
The finding that matters for sellers
The models that asked better questions did better. In Visa's words, the five models "that uncovered the most hidden buying factors" averaged a 70.2% pass rate. The five that uncovered the least passed 44.4% of the time.
A hidden buying factor is the thing the customer cares about but did not say. Will it work with my setup? Does it fit the space? What is in the box? If the answer is buried in an image, a PDF or a vague sentence, an AI buyer may never find it, and buy anyway.
That is a return, a complaint or a chargeback with your name on it. More on getting ready for these buyers in Is your checkout ready for an AI buyer?
What to put on your product pages
1. State the deal-breakers in plain text. Size, compatibility, power, what is included, what is not. As words on the page, not only in a picture.
2. Answer the questions customers ask before they buy. Your support inbox already tells you what they are.
3. Make return terms easy to find. An AI that is unsure should be able to see what happens if it is wrong.
4. Let the right agents in. None of this helps if your site blocks the assistant your customer sent. See Some of the bots on your site are customers.
Do this today
- Pick your best-selling product and list the three questions customers ask most before buying.
- Check that each answer is written on the product page as text.
- Ask an AI assistant to choose that product for a made-up customer, and see what it misses.
Your next step
If you want help getting your site ready for AI buyers, book a free 30-minute 1:1 assessment. I look at your setup with you and tell you what the work would take.