ClaudeBot
Anthropic operates ClaudeBot, which collects web content that could contribute to training its generative AI models. Site owners control it with the robots.txt token ClaudeBot, which Anthropic says it honors, placed on every subdomain. Verification is by published IP ranges only, from a shared list at claude.com/crawling/bots.json; Anthropic documents no reverse-DNS suffix.
What ClaudeBot does
In access logs, requests from this crawler carry the user-agent token ClaudeBot. Anthropic does not publish a full user-agent string, so the text around the token is not documented. The stated purpose is to collect web content that could contribute to training Anthropic's generative AI models. Anthropic also says its bots respect anti-circumvention technologies and will not attempt to bypass CAPTCHAs on the sites they crawl. Anthropic does not document whether the crawler executes JavaScript.
Allowing ClaudeBot leaves the site's pages available for possible inclusion in Anthropic's training datasets. Blocking it carries the meaning Anthropic assigns, which is that a robots.txt restriction "signals that the site's future materials should be excluded from our AI model training datasets." Anthropic says the opt-out has to be made in robots.txt on every subdomain. Anthropic advises against relying on IP blocking for opt-out, because a firewall rule impedes the crawler's reading of robots.txt and so "may not work correctly or persistently guarantee an opt-out."
The documented robots.txt examples are "User-agent: ClaudeBot / Disallow: /" to block it and "User-agent: ClaudeBot / Crawl-delay: 1" to slow it, since Anthropic supports the non-standard Crawl-delay extension. To confirm a request is from Anthropic, compare the source IP against the shared list at claude.com/crawling/bots.json. Anthropic gives a contact address in its help article for reporting malfunctions, and the email must come from the affected domain.
Operator note. Anthropic's documentation states that its bots "respect 'do not crawl' signals by honoring industry standard directives in robots.txt", "respect anti-circumvention technologies (e.g., we will not attempt to bypass CAPTCHAs for the sites we crawl)", and support the non-standard Crawl-delay extension, with the examples "User-agent: ClaudeBot / Crawl-delay: 1" and "User-agent: ClaudeBot / Disallow: /". Opt-out must be done via robots.txt on every subdomain; IP blocking "may not work correctly or persistently guarantee an opt-out" because it impedes reading robots.txt. No full user-agent string and no JavaScript-execution statement are documented. A contact address for malfunction reports is given in the help article, and the email must come from the affected domain.
Controlling ClaudeBot with robots.txt
Use the token ClaudeBot in robots.txt. Anthropic documents that ClaudeBot honors robots.txt directives.
User-agent: ClaudeBot
Disallow: /User-agent: ClaudeBot
Allow: /Verifying a request is really ClaudeBot
Anyone can put ClaudeBot in a User-Agent header. Anthropic publishes a single shared IP range list for all of its bots at https://claude.com/crawling/bots.json and states: "If a crawler has a source IP address on this list, it indicates that the crawler is coming from Anthropic." The list is not broken down per bot, and no reverse-DNS suffix is documented.
Anthropic publishes a machine-readable list. Our probe read it on 2026-09-06; the file's own creation time was 2026-08-18.
- Prefixes
- 26
- IPv4
- 26
- IPv6
- 0
First entries at fetch time: 216.73.216.0/22, 34.162.230.222/32, 34.162.244.71/32
SHA-256 of the fetched body: 8b2b107cbf0a92b3e2efecbb549a7a7d61b262fa18994af5f432c2d282c3ff94
Common questions
Should I block ClaudeBot?
That depends on whether you want your content used for AI training. Anthropic states that restricting ClaudeBot signals the site's future materials should be excluded from its training datasets. If you block it, use robots.txt on every subdomain rather than IP rules, because Anthropic says IP blocking may not reliably register as an opt-out.
How do I verify ClaudeBot?
Compare the request's source IP with the list Anthropic publishes at claude.com/crawling/bots.json. Anthropic says a source IP on that list indicates the crawler is coming from Anthropic. The list is shared across all Anthropic bots, and Anthropic documents no reverse-DNS suffix.
Does ClaudeBot run JavaScript?
The operator's documentation does not say. Anthropic's help article describes robots.txt handling, Crawl-delay support and a policy of not bypassing CAPTCHAs, but it makes no statement about JavaScript execution, and it publishes no full user-agent string beyond the ClaudeBot token.
Sources
Every fact on this page was checked against Anthropic's own documentation, listed below, and re-checked by a second reviewer before publication. Reviewed September 2026.
Related crawlers
This registry documents how operators describe their own bots so site owners can identify and control them. It does not publish third-party IP lists or guess at undocumented behaviour. To see how your own site responds to automated visitors, the bot detection scanner reads a URL's live response and names the protection it finds.