The web is now machine-majority
In 2026, the web crossed a threshold. According to Imperva's Bad Bot Report (2026 edition, using 2025 data), 53% of all web traffic is automated. Cloudflare's Matthew Prince confirmed in June 2026 that bots generate 57.5% of HTML web traffic specifically. AI crawlers alone account for 20.3% of verified bot traffic. This is no longer an edge case or a stat to cite — it is the environment. If you run a website and your analytics tool does not filter bots, the majority of what you are looking at is machine traffic. You are optimizing for bots.
What changed in 2026
Three things shifted the bot landscape. First, AI crawlers exploded. GPTBot, ClaudeBot, PerplexityBot, Bytespider, and dozens of new AI crawlers now scrape the web continuously for training data and search results. Second, scraping-as-a-service went mainstream. Tools like Bright Data and ScrapingBee made it trivial for anyone to run thousands of concurrent requests. Third, AI agents started browsing. With the rise of AI agents that visit sites to extract information, we are seeing a new category of automated traffic that is not a crawler and not a human — it is a bot pretending to be a human. Your analytics tool was not built for this.
Why this destroys your metrics
If 53% of your traffic is bots and your analytics tool does not filter them, every metric is wrong. Your visitor count is inflated by 50%+. Your bounce rate is meaningless — bots load a page and leave instantly, which looks like a 100% bounce. Your conversion rate is halved — the denominator includes bots that will never convert. Your traffic source data is polluted — bots often show up as Direct or with fake referrers. Your average session duration is skewed — bots spend 0 seconds. If you are making business decisions based on unfiltered analytics in 2026, you are reading tea leaves.
See which channels actually make money. Connect Stripe, filter bots, and track AI search traffic — all cookieless.
Try Dashly free →How different tools handle bots (or do not)
GA4 has a "Bot Filtering" toggle in Admin settings, but it only excludes hits from known data center IP ranges. It does not detect behavioral bots, AI crawlers, or spoofed user-agents. In our testing, GA4 over-reports visitors by 30-40% compared to bot-filtered analytics. Plausible and Fathom do not filter bots by default — they count everything. PostHog has partial filtering but it is opt-in and requires configuration. Dashly filters bots at collection time using three layers: user-agent matching against 2,000+ known bots, behavioral analysis (render time, interaction signals, request patterns), and server-side IP filtering. Bots never enter your dashboard data.
The AI crawler dilemma: block or allow
AI crawlers (GPTBot, ClaudeBot, PerplexityBot) are a new category. They scrape your content to train models or answer queries. You have a choice: allow them if you want your content indexed by AI search engines (Perplexity, ChatGPT search), or block them via robots.txt if you do not want your content used for AI training. Either way, you should filter them from your analytics. A GPTBot visit is not a human reading your content — it is a machine scraping it. If you do not filter AI crawlers, they inflate your visitor count and pollute your traffic source data. Dashly filters AI crawlers automatically. If you use another tool, check whether GPTBot and ClaudeBot are in their bot list.
How to verify your bot filtering is working in 2026
After enabling bot filtering, check four things. First, your visitor count should drop by 30-50% — if it does not, your filter is too loose. Second, check for AI crawler user-agents in your raw logs — if you see GPTBot, ClaudeBot, or PerplexityBot, they should be filtered. Third, your conversion rate should improve — the denominator shrank while conversions stayed the same. Fourth, your traffic sources should look different — bots often show up as Direct from data center IPs. If you see a referrer like semrush.com or ahrefs.com sending hundreds of visits, those are SEO crawlers, not humans. In 2026, bot filtering is not optional — it is the difference between accurate data and noise.
FAQ
What percentage of web traffic is bots in 2026?
53% of all web traffic is automated, according to Imperva's 2026 Bad Bot Report. Cloudflare reports 57.5% of HTML web traffic is bots. AI crawlers alone account for 20.3% of verified bot traffic.
Does GA4 filter AI crawlers?
No. GA4's bot filtering only excludes hits from known data center IP ranges. It does not detect GPTBot, ClaudeBot, PerplexityBot, or other AI crawlers. In testing, GA4 over-reports visitors by 30-40% compared to bot-filtered analytics.
Should I block GPTBot and ClaudeBot?
It depends. If you want your content indexed by AI search engines (Perplexity, ChatGPT search), allow them. If you do not want your content used for AI training, block them via robots.txt. Either way, filter them from your analytics so they do not inflate your visitor counts.
How does Dashly filter bots in 2026?
Dashly uses three layers: user-agent matching against 2,000+ known bots (including all AI crawlers), behavioral analysis (render time, interaction signals, request patterns), and server-side IP filtering. Bots are excluded at collection time, so they never enter your dashboard data.
Why is my conversion rate so low?
If you are not filtering bots, your conversion rate is artificially low because bots inflate your visitor count (the denominator) while never converting. After enabling bot filtering, your visitor count drops 30-50% and your conversion rate improves proportionally.