Tollbit Q2 2025 report revealed a staggering statistic: there has been a 336% increase in websites blocking AI crawlers over the past year. This is not just a trend - it is a fundamental shift in how the internet operates.
The Data Behind the Shift
- 336% increase in sites adding AI crawler blocks year-over-year
- 5.6 million sites now block GPTBot (up from 3.3 million in July 2025)
- 5.8 million sites block ClaudeBot (up from 3.2 million in July 2025)
- 13.26% of AI bot requests now ignore robots.txt directives (up from 3.3%)
- 300% increase in AI crawler bot traffic (Akamai report)
What This Means for Traditional SEO
The rise of AI search is not replacing Google - it is creating a parallel discovery channel. Smart SEO strategy in 2026 must address both:
- Google Search: Still drives significant traffic, still matters
- AI Assistants: ChatGPT, Claude, Perplexity, Google AI Overviews - a growing discovery channel
Businesses that optimize only for Google are leaving money on the table. Businesses that optimize only for AI are ignoring proven traffic sources. The answer is both.
The New SEO Framework: SEO + GEO + AIVO
Modern search optimization requires three layers:
- SEO (Search Engine Optimization): Rankings in traditional search results
- GEO (Generative Engine Optimization): Visibility in AI-generated responses
- AIVO (AI Visibility Optimization): Technical optimization for AI discoverability
Each layer builds on the others. Good SEO supports GEO. Good AIVO supports both.
Practical Steps for 2026
- Audit your AI accessibility: Check if your hosting provider blocks AI crawlers
- Implement comprehensive schema: FAQPage, HowTo, Article, LocalBusiness
- Create AI-consumable content: Clear, well-structured, factual information
- Deploy llms.txt: Machine-readable business summary
- Build authority: Reviews, citations, earned media
- Monitor AI mentions: Track when and where AI recommends your business
The Competitive Advantage
With 336% more sites blocking AI crawlers, the businesses that remain accessible gain disproportionate visibility. It is the same principle that made early SEO adopters successful: when fewer businesses compete in a channel, each visible business gets more exposure.
At AppWT, we have been helping Michigan businesses navigate this shift since AI search became mainstream. Learn about our GEO optimization services.
Tags
Frequently Asked Questions
How many websites now block AI crawlers?
As of late 2025, approximately 5.6 million websites block OpenAI GPTBot and 5.8 million block ClaudeBot. Tollbit reported a 336% year-over-year increase in sites adding AI crawler blocks.
Is traditional SEO dead because of AI?
Traditional SEO is not dead, but it is evolving. SEO now needs to include AI visibility optimization (AIVO) and Generative Engine Optimization (GEO) alongside traditional search ranking strategies.
What is the difference between GEO and SEO?
SEO focuses on ranking in traditional search results. GEO focuses on getting your business recommended by AI assistants like ChatGPT, Claude, and Perplexity. Both are important for a complete digital strategy in 2026.
Enjoyed this article?
Share it with your network
