Cloudflare sets AI crawler deadline: separate search or be blocked

IE 11 is not supported. For an optimal experience visit our site on another browser.<br>Skip to Content

The Cloudflare office in San Francisco in March 2025.Smith Collection / Gado via Getty Images

Share<br>Add NBC News to Google

July 1, 2026, 9:00 AM EDT<br>By Samantha Elkins

Cloudflare, a company that oversees much of the internet’s web traffic, is digging its heels in over AI web crawlers.

Limited time: Save 25% on NBC News subscription<br>Get exclusive reporting, live Q&As and ad-free reading.

Just a moment.<br>We are getting your experience ready.

&copy; 2026 NBCUniversal Media, LLC<br>NBC News LogoToday Logo