Some of the bots on your site are customers
Customers now send AI assistants to compare, read and buy on their behalf. To your site they look like bots. Block them all and you may block buyers.
Abhijit Chatterjee Chattopadhyay

Most websites treat automated traffic the same way: as bots to block. But some of those bots now work for your customers. A person asks their AI assistant to compare prices, find an answer or book a table, and the assistant visits your site to do it.
Block it, and you have turned away a customer without ever knowing they were there.
What Cloudflare said this week
Cloudflare, whose bot protection many websites use, wrote on 4 November (opens in a new tab) about exactly this. "A customer might ask a personal agent to compare products across your site, find support articles, or complete a purchase." And: "To a site owner, all three interactions show up as automated traffic."
Its advice is the part worth keeping: "Blocking agents altogether can feel like the safest choice, but it can mean losing customers."
These are not the crawlers that collect pages to train AI models, which you can ask to stay away in your robots.txt file, and the well-behaved ones do. A personal agent is visiting for one customer, right now.
Cloudflare is launching tools to tell these agents apart. You do not need its tools to take the main lesson: decide on purpose, page by page, what an AI agent may do on your site.
Allow, limit or block
- Pages that sell: allow. Product pages, prices, opening hours, menus and booking pages are what a customer's assistant needs to send business your way. If it cannot read them, it recommends someone else.
- Actions that cost money: limit. Checkout, bookings and sign-ups can go through, but the customer should confirm. I covered this from the shop's side in Is your checkout ready for an AI buyer?
- Accounts and private areas: block unless you know the agent. Logins and account pages are where fraud happens. An agent the customer has used for weeks is different from one that appeared today.
The mistake is not choosing either way. The mistake is leaving it to a default setting nobody has looked at.
Do this today
- Open your website host's or firewall's bot settings and read what they currently do to AI agents and to crawlers. They are not the same decision: an agent is acting for a person who is on your site now, a crawler is collecting pages for later.
- Ask an AI assistant to find your opening hours or a product price on your site, and see whether it can.
- Write down which pages you want agents to read, which actions need the customer to confirm, and which areas stay closed.
Your next step
I write one practical note like this every day. Sign up just below to get them by email.
Drafted with AI help, edited and approved by me.