Key facts
- Cloudflare will block AI crawlers from accessing ad-supported content by default.
- The policy change takes effect on September 15, 2026.
- The aim is to ensure publishers are compensated for content used by AI models.
- Site owners can adjust settings to control AI crawler access.
- The move addresses concerns about AI using web data without compensation.
Cloudflare announced a significant policy change that will block AI crawlers from accessing ad-supported content by default. This new policy is set to take effect on September 15, 2026. The primary goal of this initiative is to ensure that publishers are adequately compensated when their content is utilized by AI models and agents for training or other purposes. Site owners will have the ability to customize these settings, choosing whether to permit or deny AI crawler access to their ad-supported materials. This decision by Cloudflare addresses a growing concern within the digital publishing industry regarding the uncompensated use of web content by artificial intelligence systems. The change is expected to impact how AI models gather data and could lead to new models of content licensing and revenue sharing.