I4UDeutsch
← Back to the top 50
No. 7 of the most-visited websites in Germany · As of November 15, 2026

Can ChatGPT, Claude and Gemini read WEB.DE?

Every morning, we open the web.de homepage with the same fetch tools ChatGPT, Claude and Gemini use. This page shows which AI can read it, which gets turned away and why.

The short answer

ChatGPT and Claude can read web.de. Their fetch tools opened the homepage. Gemini uses web.de as a source, but through Google Search. Google’s own fetch tool rejected the homepage. Meta AI: robots.txt allows Meta’s fetcher. We can’t test it, because Meta offers no public fetch tool.

Result per AI

ChatGPTreads the page
Claudereads the page
Geminireads the page
Meta AI*robots.txt allows

What web.de’s robots.txt says

User-agent: *
Disallow: /deals/
Disallow: /test/

User-agent: OAI-SearchBot
User-agent: ChatGPT-User
User-agent: Claude-SearchBot
User-agent: Claude-User
User-agent: PerplexityBot
User-agent: Perplexity-User
Disallow: /deals/
Disallow: /test/
Allow: /magazine/
Allow: /

User-agent: AI2Bot
User-agent: Ai2Bot
User-agent: Amazonbot
User-agent: Applebot-Extended
User-agent: CCBot
User-agent: Cincraw
User-agent: ClaudeBot
User-agent: cohere-ai
User-agent: Diffbot
User-agent: FriendlyCrawler
User-agent: GPTBot
User-agent: Google-Extended
User-agent: ImagesiftBot
User-agent: img2dataset
User-agent: Meta-ExternalAgent
User-agent: PetalBot
User-agent: Semanticbot
User-agent: TimpiBot
User-agent: VelenPublicWebCrawler
User-agent: Yandex
Disallow: /deals/
Disallow: /test/
Disallow: /magazine/
Allow: /magazine/in-eigener-sache/
Allow: /magazine/unicef/
Allow: /magazine/so-arbeitet-die-redaktion/

Excerpt showing only the rules for AI agents and for all bots (*). robots.txt is a request from the website, not a technical lock. Anthropic says its bots honor it; for the ChatGPT-User fetcher, OpenAI says the rules may not apply.

Bot protection

When we loaded it in a regular Chrome browser, web.de showed no bot protection.

Changes

Nothing has changed for web.de since daily checks began on November 14, 2026.

Frequently asked questions about WEB.DE

Can ChatGPT read web.de?

Yes. ChatGPT can read web.de: its fetch tool opened the homepage. (As of November 15, 2026)

Can Claude read web.de?

Yes. Claude can read web.de: its fetch tool opened the homepage. (As of November 15, 2026)

Can Gemini read web.de?

Yes. Gemini uses web.de as a source, but through Google Search. Google’s own fetch tool rejected the homepage. (As of November 15, 2026)

Does WEB.DE block AI crawlers in robots.txt?

No. Its robots.txt blocks none of the AI names used by OpenAI, Anthropic, Google and Meta.

Badge for web.de

The badge shows the latest check and updates daily.

web.deweb.de

<a href="https://i4u.business/en/ki-agent-radar/web-de/"><img src="https://i4u.business/en/ki-agent-radar/badge/web.de-light.svg" alt="AI Agent Radar: which AI agents can read web.de" height="42"></a>

Check another website

How to cite this data

Source: i4u AI Agent Radar, i4u.business/en/ki-agent-radar/web-de/

Data as of: November 15, 2026 · Licence: CC BY 4.0, credit “i4u” with a link. · Press and questions: info@i4u.business

Download the data: CSV, Top 50 · JSON, Top 50