Beehiiv adds Cloudflare AI Crawl Control so writers can block or allow bots - SiliconANGLE
You are using an outdated browser. Please upgrade your browser to improve your experience.
SHARE
UPDATED 09:00 EDT / JUNE 23 2026
AI
Beehiiv adds Cloudflare AI Crawl Control so writers can block or allow bots
by<br>Duncan Riley
Cloudflare Inc. and newsletter platform beehiiv Inc. today launched an integration that hands independent publishers a single toggle to decide whether artificial intelligence crawlers can reach their work.
The integration bakes Cloudflare’s AI Crawl Control into the beehiiv dashboard. Writers face a choice. They can open their archives to AI search engines and agents and chase the distribution that brings, or shut AI scraping off and hold their content back for licensing deals later. Beehiiv is turning the option on for its users starting today.
Behind the launch is a problem that has gotten worse fast. AI systems now eat up a large and rising share of web traffic and fending them off has meant fiddling with robots.txt files or writing firewall rules. That is work most solo creators were never going to do. Putting the controls in the dashboard takes the code out of it.
Flip the controls on and a publisher gets a dashboard, built on Cloudflare’s application programming interfaces, that names the AI crawlers knocking on the door. It shows what is getting blocked.
It also shows how much referral traffic those crawlers are sending back, the trade-off at the heart of the open-or-block decision. Want to block one model and wave another through? That is a click. Cloudflare says the list keeps itself current as fresh crawlers turn up.
The deal is the latest in a year-long campaign. Cloudflare flipped on default AI-scraper blocking for new customers in 2025 and stood up a Pay Per Crawl marketplace where publishers can charge AI firms for access. In January, it bought AI data marketplace startup Human Native Ltd. to flesh out licensed-content tooling for rights holders.
Cloudflare has framed the campaign as a fight for content creators "from independent bloggers to the world’s largest publishers," in the words of co-founder and Chief Executive Matthew Prince. He called the beehiiv partnership the next step in giving newsletter operators the transparency and control to set their own terms with AI companies, whether they want discovery or want to hold their work back.
Tyler Denk, beehiiv’s co-founder and CEO, sees the integration as a bargaining chip for writers. AI is changing how readers stumble onto content, he said and creators ought to have a hand on that lever.
Some 135,000 publishers run newsletters, websites and podcasts on beehiiv. It takes nothing from their subscription revenue and the audiences stay theirs. For Cloudflare, the integration buys reach into the creator economy and another on-ramp for the Pay Per Crawl model it is wagering will change how AI companies pay for what they take.
Image: SiliconANGLE/Ideogram
A message from John Furrier, co-founder of SiliconANGLE:
Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network , where technology leaders connect, share intelligence and create opportunities.
15M+ viewers of theCUBE videos , powering conversations across AI, cloud, cybersecurity and more
11.4k+ theCUBE alumni — Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network.
About SiliconANGLE Media
SiliconANGLE Media is a recognized leader in digital media innovation, uniting breakthrough technology, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE, theCUBE Network, theCUBE Research, CUBE365, theCUBE AI and theCUBE SuperStudios — with flagship locations in Silicon Valley and the New York Stock Exchange — SiliconANGLE Media operates at the intersection of media, technology and AI.
Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.
Like Free Content? Subscribe to follow.
Subscribe
-->
LATEST STORIES
Exabeam launches Praxen, an open-source tool to verify AI agent behavior
Lama AI raises $10M to accelerate automated loan originations
Beehiiv adds Cloudflare AI Crawl Control so writers can block or allow bots
Linux Foundation extends DNS to AI agents with new Agent Name Service
Virtue AI pulls the rug out from under the feet of shadow AI agents
Partly raises $50M at a $500M valuation to crack the US auto parts market
LATEST STORIES
Exabeam launches Praxen, an open-source tool to verify AI agent behavior<br>SECURITY - BY...