Claude-SearchBot
Anthropic operates Claude-SearchBot, which the operator says navigates the web to improve search result quality and analyzes online content to enhance the relevance and accuracy of search responses. It honors robots.txt under the token Claude-SearchBot and supports Crawl-delay. Site owners verify it against the shared Anthropic IP list at claude.com/crawling/bots.json; no reverse-DNS suffix is documented.
What Claude-SearchBot does
In server logs the bot identifies itself with the user-agent token Claude-SearchBot. Anthropic does not publish a full user-agent string for it, so the exact header format is undocumented. According to the operator, the bot analyzes online content to enhance the relevance and accuracy of search responses, and this registry files it as an AI search index crawler. The operator publishes no crawl-rate figures and makes no statement about whether the bot executes JavaScript.
Blocking changes how the site appears in the search results the operator describes. In the operator's words, disabling Claude-SearchBot prevents its system from indexing your content for search optimization, which may reduce your site's visibility and accuracy in user search results. Allowing it keeps the site eligible for that index. A Disallow rule addressed to User-agent Claude-SearchBot in robots.txt is the documented control. Anthropic states that its crawlers honor robots.txt directives and support the Crawl-delay directive, so a site can slow the bot without excluding it.
Anthropic also states that its crawlers respect anti-circumvention measures and do not bypass CAPTCHAs. Verification relies on a single shared IP list covering every Anthropic bot, published at claude.com/crawling/bots.json with 26 IPv4 prefixes. The list is not broken down per bot, so a matching address confirms Anthropic ownership but not which Anthropic bot made the request. Pair the IP match with the user-agent token to tell them apart.
Operator note. Anthropic's crawler documentation applies to all of its bots collectively: they honor robots.txt directives, support Crawl-delay, and respect anti-circumvention technologies, with no CAPTCHA bypass. No full user-agent string, JavaScript-execution statement, or crawl-rate figures are documented.
Controlling Claude-SearchBot with robots.txt
Use the token Claude-SearchBot in robots.txt. Anthropic documents that Claude-SearchBot honors robots.txt directives.
User-agent: Claude-SearchBot
Disallow: /User-agent: Claude-SearchBot
Allow: /Verifying a request is really Claude-SearchBot
Anyone can put Claude-SearchBot in a User-Agent header. Anthropic publishes a single shared IP range list for all of its bots at https://claude.com/crawling/bots.json. The list is not broken down per bot, and no reverse-DNS suffix is documented.
Anthropic publishes a machine-readable list. Our probe read it on 2026-09-06; the file's own creation time was 2026-08-18.
- Prefixes
- 26
- IPv4
- 26
- IPv6
- 0
First entries at fetch time: 216.73.216.0/22, 34.162.230.222/32, 34.162.244.71/32
SHA-256 of the fetched body: 8b2b107cbf0a92b3e2efecbb549a7a7d61b262fa18994af5f432c2d282c3ff94
Common questions
Should I block Claude-SearchBot?
That depends on whether you want your content indexed for the search responses Anthropic describes. The operator says blocking it prevents indexing for search optimization and may reduce your site's visibility and accuracy in user search results. If you want that visibility, allow it; if not, add a Disallow rule for User-agent Claude-SearchBot in robots.txt.
Does Claude-SearchBot run JavaScript?
The operator's documentation does not say. Anthropic's crawler article makes no statement about JavaScript execution for Claude-SearchBot, and it publishes no crawl-rate figures for the bot either. Treat rendering behaviour as undocumented rather than assuming the bot does or does not run scripts.
How do I verify Claude-SearchBot?
Compare the request's source IP against the shared Anthropic list at https://claude.com/crawling/bots.json, which holds 26 IPv4 prefixes for all Anthropic bots. No reverse-DNS suffix is documented. Because the list is not split per bot, an IP match shows the request came from Anthropic; the Claude-SearchBot user-agent token tells you which bot.
Sources
Every fact on this page was checked against Anthropic's own documentation, listed below, and re-checked by a second reviewer before publication. Reviewed September 2026.
Related crawlers
This registry documents how operators describe their own bots so site owners can identify and control them. It does not publish third-party IP lists or guess at undocumented behaviour. To see how your own site responds to automated visitors, the bot detection scanner reads a URL's live response and names the protection it finds.