Claude-User
Claude-User is Anthropic's user-triggered fetch agent. Anthropic says that when individuals ask Claude questions, it may access websites using a Claude-User agent. It honors robots.txt under the Claude-User token. Anthropic publishes a shared IP list for all its bots at claude.com/crawling/bots.json, with no reverse DNS suffix and no per-bot breakdown.
What Claude-User does
In logs, Claude-User requests identify themselves with the Claude-User token. Anthropic publishes no full example string, so the rest of the header is undocumented. Anthropic describes the visits as support for Claude users; when a person asks Claude a question, Claude may access a website using this agent. The Claude Code tools reference adds that its WebFetch tool sets a User-Agent header beginning with Claude-User and an Accept header that prefers Markdown over HTML, so servers that support content negotiation can return Markdown directly.
Blocking Claude-User in robots.txt stops Anthropic retrieving your pages in response to a user query. Anthropic's own wording is that this may reduce your site's visibility for user-directed web search. Allowing it means Claude can access your pages when a person's question leads it there. Anthropic says the token lets site owners control which sites can be accessed through these user-initiated requests. The robots.txt statement in Anthropic's help article is made for its bots collectively, and it covers Claude-User.
Verification relies on IP ranges alone. Anthropic publishes one list at claude.com/crawling/bots.json covering all of its bots. The list is not split per bot, so an address match confirms the request came from Anthropic infrastructure but does not tell you which agent sent it. Anthropic documents no reverse DNS suffix for Claude-User.
Operator note. Anthropic's help article states: "Claude-User allows site owners to control which sites can be accessed through these user-initiated requests." Its robots.txt statement ("Anthropic's Bots respect 'do not crawl' signals by honoring industry standard directives in robots.txt") is made for all three bots collectively, so it covers Claude-User. The Claude Code tools reference states: "WebFetch sets a User-Agent header beginning with Claude-User, and an Accept header that prefers Markdown over HTML so servers that support content negotiation can return Markdown directly." The Claude Platform web fetch tool documentation notes that the tool "currently does not support websites dynamically rendered with JavaScript" and lists a url_not_allowed error for URLs blocked by "Anthropic-side restrictions, such as private addresses and robots.txt", but that page does not name the Claude-User user-agent, so those statements cannot be firmly attributed to this bot; no full user-agent string is documented.
Controlling Claude-User with robots.txt
Use the token Claude-User in robots.txt. Anthropic documents that Claude-User honors robots.txt directives.
User-agent: Claude-User
Disallow: /User-agent: Claude-User
Allow: /Verifying a request is really Claude-User
Anyone can put Claude-User in a User-Agent header. Anthropic publishes a single shared IP range list for all of its bots at https://claude.com/crawling/bots.json. The list is not broken down per bot, and no reverse-DNS suffix is documented.
Anthropic publishes a machine-readable list. Our probe read it on 2026-09-06; the file's own creation time was 2026-08-18.
- Prefixes
- 26
- IPv4
- 26
- IPv6
- 0
First entries at fetch time: 216.73.216.0/22, 34.162.230.222/32, 34.162.244.71/32
SHA-256 of the fetched body: 8b2b107cbf0a92b3e2efecbb549a7a7d61b262fa18994af5f432c2d282c3ff94
Common questions
Should I block Claude-User?
That depends on whether you want Claude to access your pages when a person asks it to. Anthropic says disabling Claude-User prevents its system from retrieving your content in response to a user query, which may reduce your site's visibility for user-directed web search. Anthropic says the token lets site owners control which sites can be accessed through these user-initiated requests.
Does Claude-User run JavaScript?
Anthropic's Claude-User documentation does not say. The Claude Platform web fetch tool docs state that tool currently does not support websites dynamically rendered with JavaScript, but that page never names the Claude-User agent, so the statement cannot be firmly tied to this bot. Treat JavaScript execution as undocumented.
How do I verify Claude-User?
Check the source IP against Anthropic's published list at https://claude.com/crawling/bots.json. The list covers every Anthropic bot in one shared set, so a match confirms Anthropic origin rather than the specific agent. Anthropic documents no reverse DNS suffix, so a hostname lookup is not a documented method.
Sources
Every fact on this page was checked against Anthropic's own documentation, listed below, and re-checked by a second reviewer before publication. Reviewed September 2026.
- https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler
- https://privacy.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler
- https://claude.com/crawling/bots.json
- https://code.claude.com/docs/en/tools-reference
- https://platform.claude.com/docs/en/agents-and-tools/tool-use/web-fetch-tool
Related crawlers
This registry documents how operators describe their own bots so site owners can identify and control them. It does not publish third-party IP lists or guess at undocumented behaviour. To see how your own site responds to automated visitors, the bot detection scanner reads a URL's live response and names the protection it finds.