99% of My Website Traffic Is Bots | PatronView<br>PUBLISHED AUG 07, 2026<br>99% of My Website Traffic Is Bots<br>A year of fighting scrapers on my 1.5 million-page website. What I tried, how I failed, and what's working now.<br>August 7, 2026 · 27 MIN READ · GENERAL
That's me waving to you because blue is one of my favorite colors. If you're trying to steal my data: start here! I'll tell you exactly what firewall rules I have in place so you can work around them more easily.
I'm sort of joking. I don't even know if a web crawler reads blog posts to get tips like this.
But I've spent a year fighting bots on my philanthropy donor database, PatronView. Nobody teaches you this stuff when you're vibe coding, so I'll share what I've learned.
Some of what's below:
My site got 3.6 million requests from Chinese bots in just one day.
Cloudflare says Anthropic's crawlers run about 3,000 crawls per visitor referred. I measured mine at 35,000:1 .
Two days ago I blocked Amazon's AI search crawler. It was reading 117,000 pages a day and has never sent me a single visitor.
My CAPTCHA solve rate is 0.24% . The bots don't even try.
And every firewall rule I run is listed at the bottom.
214 bot page loads for every 1 human
In the week I published this post, my server answered 2.5 million requests from the outside world and served 1.28 million full pages . But my visitor stats only recorded 5,977 pageviews.
So for every page load I see, about 214 more happen that I never see.
The blue square is the traffic I can see. The gray squares are what actually pounds my server. I use self-hosted Plausible Community Edition for analytics, and it only counts visitors who run JavaScript. Almost no bot does.<br>My visitor stats show a tidy little site, about 500 visitors a day. But my server answers millions of real requests every week.
If you only look at visitor stats from a JavaScript tool like Plausible or Fathom or Google Analytics , you have no idea what's hitting your server. I didn't.
The first botnet
PatronView is a custom database of American philanthropists that I cobbled together with various levels of intelligence. It has 1.5 million individual profile pages built from IRS 990 forms, public donor walls, and annual reports.
And yes, my site gets its data by scraping those public documents. So I'm a scraper writing a blog post complaining about scrapers. I'm aware of how that sounds.
From the beginning, SEO crawlers like SemrushBot, AhrefsBot, MJ12bot, and DataForSEOBot pounded the site, crawling every single page over and over. Blocking them in robots.txt and then at the Cloudflare level with security rules was my gateway drug into all of this.
Then in November 2025, four thousand "visitors" showed up over a few days. Each visited exactly one page with a bounce rate of 99%. More telling was that they had no referrer. That's usually the easiest way I spot a bot.
And they were only crawling my fund pages (like this one, this one, and this one), which only 10% of real visitors ever touch.
But those 4,000 bots were just the warm-up.
The day China arrived
On April 22, my site took 3.6 million requests in a single day . They came from 361,844 unique IP addresses, and nearly all of them were in China.
I tried to imagine what all those Chinese bots looked like. With apologies for the cliche red dragon!!
Cloudflare's Managed Challenge (their invisible CAPTCHA) absorbed 1.18 million of those in the first ten hours. And yet an alarming number of these Chinese bots were passing the challenge.
01M2M3MApr 10Apr 15Apr 20Apr 22Apr 25Apr 30Apr 22: 3.62M requests from 361K unique IPsRequests per day in April 2026. I implemented the China-wide block on April 23rd and immediately saw relief.<br>Cloudflare security events during the flood: 1.18M managed challenges in about 10 hours.<br>So I did the thing you're not supposed to do: I blocked the entire country of China at the edge. A few days later, a copycat wave from Vietnam started, so I blocked Vietnam, too. And then Singapore.
My site is a database of American museum donors, in English. My real search traffic is 95.9% United States and 1.3% Canada.
And frankly, I'm not interested in visitors from China or Vietnam being able to access this data.
When I tweeted about the flood of Chinese web traffic, the replies told me everyone is fighting this:
Matt Paulson from MarketBeat: "Don't forget to add Russia to the list."
Jack Ellis from Fathom Analytics: "We saw a ton of spam traffic from China over the last 6 months. I will say that customers have seen China drop completely now."
Jeremy Brandt : "Our country block list on Cloudflare for all of our sites is... extensive."
Chris Lewicki : "I decided to stop fighting it. My counter-measures were introducing too much complexity in my life."
Rodrigo Rocco : "It's been crazy lately, they are using thousands of residential IPs making only 1 call each, so hard to stop it."
Remember that last one from Rodrigo, because it comes back to bite me...