Hostinger analysis of 66.7B bot interactions finds AI assistants expanding rapidly, even as companies block training bots and lose control of brand context.
Companies racing to block artificial intelligence bots from scraping their websites may be undermining their own influence over how customers encounter their brands.
That is the conclusion of a new analysis by Hostinger, a Vilnius-based provider of web hosting and AI-driven business tools. Drawing on 66.7 billion verified bot interactions across five million websites, the company found that AI assistant crawlers — the systems used by tools such as OpenAI’s ChatGPT and Apple’s Siri — are expanding their reach even as companies restrict the use of bots for training AI models.
For decades, search engines indexed websites and directed users back to them, allowing brands to control pricing context, messaging and attribution within their own channels. That model is shifting. AI assistants increasingly answer questions directly, offering summaries and recommendations that may end a customer journey before a website is ever visited.
Hostinger’s data shows that OpenAI’s SearchBot expanded from covering 52 percent of websites to 68 percent over a five-month period. Applebot doubled its reach from 17 percent to 34 percent. By contrast, traditional search crawlers remained broadly stable — suggesting AI is layering itself atop search rather than replacing it.
At the same time, access by AI model-training crawlers has fallen sharply. OpenAI’s GPTBot dropped from 84 percent website coverage in August to 12 percent by November, while Meta’s ExternalAgent declined from 60 percent to 41 percent.
Hostinger argues that many companies conflate two distinct functions. Training crawlers collect data to improve models over time. Assistant crawlers retrieve content in real time to answer user queries. Blocking the former does little to prevent the latter from summarizing, ranking or recommending products and services.
Also Read: From AI Vision to Retail Personalization at Scale
The result, the company says, is a growing disconnect: AI assistants mediate more customer decisions, while companies have fewer signals shaping how those systems interpret their content.
The risks extend beyond traffic declines. Pricing nuances may be compressed into simplified summaries. Brand messaging can be reframed outside approved guidelines. Advertising effectiveness may erode as paid acquisition loses upstream visibility. And e-commerce attribution becomes murkier when customer journeys conclude inside AI interfaces rather than on company websites.
Some businesses are shifting from blanket blocking to more selective AI governance. Tools such as llms.txt — a machine-readable file designed to guide AI assistants toward authoritative pages — and structured, AI-ready content frameworks are emerging as ways to maintain influence without exposing proprietary data.
“With AI assistants increasingly answering questions directly, the web is shifting from a click-driven model to an agent-mediated one,” said Tomas Rasymas, head of AI at Hostinger. “The real risk for businesses isn’t AI access itself, but losing control over how pricing, positioning and value are presented when decisions are made.”
Also Read: CMOs Should Turn Calendars Into a Marketing Channel
The analysis examined anonymized log data collected during three six-day periods in June, August and November 2025. Only verified crawler traffic was included, classified using documented user agents and behavioral patterns.
As AI systems increasingly become the intermediary between brands and customers, the question facing companies is no longer whether to allow AI access — but how to retain control in a marketplace where the first impression may be generated by a machine.
