Key facts
- Patreon is now actively blocking AI training bots from scraping its content.
- The company is partnering with Cloudflare to implement these stricter measures.
- Patreon's previous method involved asking bots not to scrape via robots.txt files.
- The new approach uses Cloudflare's AI Crawl Control technology for direct blocking.
- Patreon aims to give creators more control over the use of their work by AI companies.
Patreon has begun actively blocking AI bots from scraping creator content, a significant shift from its previous approach of requesting they refrain. The membership platform for creators is collaborating with internet infrastructure provider Cloudflare to implement these enhanced measures.
Previously, Patreon relied on robots.txt files to instruct AI crawlers not to access its site. However, the company found these requests were often ignored, with AI training bots continuing to scrape content. This led to the decision to move towards direct blocking, utilizing Cloudflare's AI Crawl Control technology.
The strengthened measures are a response to the increasing sophistication of AI scraping and the growing trend of online publishers and creators grappling with AI models ingesting their work. Patreon's paywalled content has historically been less accessible to crawlers, but new discovery tools could expose more material.
Cloudflare offers various tools to combat AI scraping, including a 'Pay Per Crawl' marketplace and default blocking of 'mixed-use' crawlers on pages with ads. Patreon's updated policies will allow bots that index pages and organize information to return users to the platform, but will prevent training on creator content without consent.
Drew Rowny, Patreon's product chief, emphasized that creators should have a meaningful say in how their work is utilized by AI companies, contrasting Patreon's vision with the broader internet where creators often have to accept AI training to grow their audience.