Does AI Recommend Your Company?

A growing share of buying questions never reaches a search engine. A customer types something like “recommend a good company that does this kind of work near me, one that publishes prices” into ChatGPT or Perplexity, and gets three names with reasons. No ten blue links, no comparing them personally, no visits to ten websites. Just a finished answer. If the company’s name is not in it, then as far as that customer is concerned, the company does not exist.

This is not a forecast for next year. It is already happening, and it shows up in server logs.

Bots read, but they do not send anyone back

The most telling number in this discussion is a simple one: how many times a crawler has to fetch content from websites before a single person lands on one of those sites from a model answer.

In July 2026 Anthropic crawlers fetched roughly 1,900 pages for every single visit they sent back, and OpenAI crawlers roughly 250.¹ A year earlier the same ratio for Anthropic ran into the tens of thousands, so the direction is improvement, from a very high starting point. Two things are worth knowing about these numbers: they move from month to month, and Cloudflare itself notes they are probably overstated, because visits arriving from the Claude app carry no information about where they came from and cannot be counted. The order of magnitude stays the same and that is the conclusion: the content is read at scale and the visitor does not come back. Classic search operated for two decades on an exchange where both sides gained something: our content to index, our traffic in return. When the answer is assembled inside a chat window, that exchange stops being symmetrical. The content is still needed. The click is not.

The conclusion for a business owner is uncomfortable but simple. Visibility can no longer be measured by organic sessions alone. Part of the buying decision now happens in a place no on-site analytics tool can see.

• 1,900:1crawls per referral, Anthropic bots, July 2026
• 250:1the same ratio for OpenAI bots
• 137ksites studied that carry an llms.txt file
• 97%of them logged not one request for it

What Cloudflare announced, and why

There is a fight about money going on in the background, and it is worth understanding so you do not buy something too early.

On 1 July 2025 Cloudflare, one of the largest companies handling internet traffic, launched pay-per-crawl: a mechanism letting a site owner charge an AI bot for fetching content by responding with HTTP status 402, meaning “payment required.” It sounded like the start of a new revenue stream for publishers.

Exactly one year later, on 1 July 2026, Cloudflare set that idea aside itself and announced the Monetization Gateway: billing not for fetching a page, but for actual use of the content in an answer. For now it is a waitlist and two partners, Ceramic.ai and You.com. On 4 August 2026 Cloudflare Wallets followed, stablecoin wallets from which AI agents are meant to pay for access to services.

The direction makes sense, but the scale is still negligible. The whole market for these micropayments is smaller today than the annual turnover of one mid-sized wholesaler, and a single transaction is worth a few cents. If anyone is currently pitching “earning money from AI crawlers” as a service for an ordinary business, they are selling an announcement, not a product.

There is, however, a second side to the same announcement, available right now at no charge. AI Crawl Control works on every Cloudflare plan, including the entry-level one, and shows which AI bots visit a site, how often and for what. That is the first measurement worth taking, because it takes fifteen minutes and requires no purchase.

It is also worth knowing what not to buy. The past year brought a fashion for llms.txt, a short set of instructions for language models placed on the site. Across 137,000 sites that had added the file, 97% never logged a single request for it, and among the few that logged anything, most requests came from SEO tools rather than from models.² That does not mean the idea is wrong in principle. It means it does not change the outcome today and does not deserve its own line on an invoice.

The other side of the same question is more interesting. A website can tell crawlers directly what they may and may not do, and only a few percent of companies do so today. Most have made no decision at all, because they do not know there is one to make.

What decides whether a model names you

The boring truth: the same things that have shaped the quality of a company’s information online for years, only now the stakes are higher.

Structured data. Schema.org markup that states plainly what a page is: organisation, service, price, opening hours, location, review. The model does not have to infer it from the visual layout, it is handed the facts. For a single-location service business this is a few hours of work.

A readable pricing page. Prices as text, not as an image or a downloadable PDF. A range is better than nothing, “contact us” is worst. Buying questions almost always involve price, and a model cannot quote a number it cannot see.

Consistency. The same company name, address, phone number and service scope on the website, in the Google business profile, in industry directories and on social profiles. Contradictory data is the most common reason a model describes a company cautiously, or skips it in favour of a competitor whose details line up.

A page that reads without JavaScript. If content only appears after scripts run in the browser, some bots will see an empty page. Simple test: turn JavaScript off and see what is left.

A technical readiness scan. The tool isitagentready.com scores a site across four categories and points to specific fixes. It is not an oracle and it does not replace measuring actual visibility, but as a starting point it needs neither payment nor signup.

The 30-question method

A technical score tells you whether a site can be read. It does not tell you whether models actually name the company. That has to be checked separately, and it can be done in-house.

The method is simple to describe and laborious to run.

Step one: write out 30 questions a customer genuinely asks before buying, in your own industry. Not SEO keywords, but full sentences with intent, built around three patterns: what it costs, who does this in a given city or region, who has a good reputation for it. In practice that sounds like “how much does an order management system cost for a wholesaler,” “who runs AI training for senior managers in a specific region” or “which law firm knows its way around IT contracts.” Two thirds should carry no company name, one third should include it, to see what the model says about the company when asked directly.

Step two: ask each question in at least three places. ChatGPT, Perplexity and Google’s AI overview are a sensible minimum, because they draw on different sources. Start a fresh conversation each time, with no logged-in history skewing the result.

Step three: record three things for every answer. Whether the company was named. In which position. Which source the model cited. The last one is the most revealing, because it usually turns out models cite an industry directory or a review portal rather than the company’s own site.

Step four: count the share. How many of the 30 questions produced a mention, how many mentions were factually correct, how many times competitors appeared. That first number is your baseline. Without it, every later change to the site is guesswork.

Step five: repeat after six to eight weeks with the same set of questions. Model answers vary by nature, so a single measurement says little, while the difference between two measurements says a great deal.

A full run for one company takes about a day of work if done honestly, with everything written down. It can be shortened, but the comparison loses credibility when it is.

What to do in the coming month

There is no revolution here to miss. There is a shift in proportion: a growing share of buying decisions is made in conversation with a model rather than on a list of results, and most companies do not know what the model says about them. Two things in the coming month take time rather than budget: turn on a view of AI bot traffic and work through 30 questions your own customers ask.

If it is easier to have this measured and written up from the outside, we run such measurements as part of our services. The result is usually less dramatic and more specific than owners expect before the first run.

Footnotes

1. Cloudflare Radar, crawl-to-refer ratios for AI crawlers: https://blog.cloudflare.com/ai-search-crawl-refer-ratio-on-radar/. ↩

2. Ahrefs, analysis of 137,000 sites carrying an llms.txt file: https://ahrefs.com/blog/llmstxt-study/. ↩