Upgrade to SI Premium - Free Trial

Cloudflare and GoDaddy partner to control AI web crawlers

April 20, 2026 10:32 AM

Cloudflare Inc. (NYSE: NET) and GoDaddy Inc. (NYSE: GDDY) announced a partnership to provide website owners with tools to manage AI crawler access to their content, according to a press release statement.

GoDaddy will integrate Cloudflare's AI Crawl Control into its website hosting platform, allowing website owners to control which AI-powered crawlers can access their content. The tool enables users to allow, block, or require payment for AI access to their websites.

The companies are supporting the Agent Name Service (ANS), an open standard introduced by GoDaddy that uses domain name system and public key infrastructure to verify AI agent identities. Cloudflare previously introduced Web Bot Auth in 2025 as a cryptographic method to verify bot traffic.

"The Internet is evolving into a high-velocity, AI-driven ecosystem, and that requires a new kind of transparent infrastructure," said Stephanie Cohen, Chief Strategy Officer at Cloudflare. "By putting tools like AI Crawl Control and open standards into the hands of website owners, we are providing essential underpinnings for a new Internet business model."

GoDaddy Chief Strategy Officer Jared Sine stated the partnership aims to provide customers "the transparency they need to thrive in an AI-first world."

The integration allows website owners to set permissions for AI crawlers while providing audit logs to track which agents access their content. The companies position this as a response to the shift from traditional search-based web traffic to AI-generated answers that may bypass original content sources.

Categories

Corporate News

Next Articles