No. 7 of the most-visited websites in Germany · As of November 15, 2026
Can ChatGPT, Claude and Gemini read WEB.DE?
Every morning, we open the web.de homepage with the same fetch tools ChatGPT, Claude and Gemini use. This page shows which AI can read it, which gets turned away and why.
The short answer
ChatGPT and Claude can read web.de. Their fetch tools opened the homepage. Gemini uses web.de as a source, but through Google Search. Google’s own fetch tool rejected the homepage. Meta AI: robots.txt allows Meta’s fetcher. We can’t test it, because Meta offers no public fetch tool.
Result per AI
What web.de’s robots.txt says
User-agent: * Disallow: /deals/ Disallow: /test/ User-agent: OAI-SearchBot User-agent: ChatGPT-User User-agent: Claude-SearchBot User-agent: Claude-User User-agent: PerplexityBot User-agent: Perplexity-User Disallow: /deals/ Disallow: /test/ Allow: /magazine/ Allow: / User-agent: AI2Bot User-agent: Ai2Bot User-agent: Amazonbot User-agent: Applebot-Extended User-agent: CCBot User-agent: Cincraw User-agent: ClaudeBot User-agent: cohere-ai User-agent: Diffbot User-agent: FriendlyCrawler User-agent: GPTBot User-agent: Google-Extended User-agent: ImagesiftBot User-agent: img2dataset User-agent: Meta-ExternalAgent User-agent: PetalBot User-agent: Semanticbot User-agent: TimpiBot User-agent: VelenPublicWebCrawler User-agent: Yandex Disallow: /deals/ Disallow: /test/ Disallow: /magazine/ Allow: /magazine/in-eigener-sache/ Allow: /magazine/unicef/ Allow: /magazine/so-arbeitet-die-redaktion/
Excerpt showing only the rules for AI agents and for all bots (*). robots.txt is a request from the website, not a technical lock. Anthropic says its bots honor it; for the ChatGPT-User fetcher, OpenAI says the rules may not apply.
Bot protection
When we loaded it in a regular Chrome browser, web.de showed no bot protection.
Changes
Nothing has changed for web.de since daily checks began on November 14, 2026.
Frequently asked questions about WEB.DE
Can ChatGPT read web.de?
Yes. ChatGPT can read web.de: its fetch tool opened the homepage. (As of November 15, 2026)
Can Claude read web.de?
Yes. Claude can read web.de: its fetch tool opened the homepage. (As of November 15, 2026)
Can Gemini read web.de?
Yes. Gemini uses web.de as a source, but through Google Search. Google’s own fetch tool rejected the homepage. (As of November 15, 2026)
Does WEB.DE block AI crawlers in robots.txt?
No. Its robots.txt blocks none of the AI names used by OpenAI, Anthropic, Google and Meta.
How to cite this data
Source: i4u AI Agent Radar, i4u.business/en/ki-agent-radar/web-de/
Data as of: November 15, 2026 · Licence: CC BY 4.0, credit “i4u” with a link. · Press and questions: info@i4u.business
Download the data: CSV, Top 50 · JSON, Top 50