Google-Safety
Google operates Google-Safety, a user agent that handles abuse-specific crawling, such as malware discovery for publicly posted links on Google properties. Google states it ignores robots.txt and is unaffected by crawling preferences, and documents no robots token. Site owners verify requests by reverse DNS to a rate-limited-proxy hostname under google.com with matching forward DNS, or against Google's special-crawlers.json IP file.
Google-SafetyWhat Google-Safety does
The user-agent string is the bare token Google-Safety with nothing else attached, which makes it easy to spot in logs. Google lists it among its special-case crawlers and describes the work as abuse-specific crawling, giving malware discovery for publicly posted links on Google properties as the example. Google's wording ties the fetches to links posted publicly on Google properties; it does not describe a broader crawl of the site. Nothing in Google's documentation for this agent describes crawl rate or JavaScript execution.
Google states plainly that Google-Safety ignores robots.txt rules and is unaffected by crawling preferences, and it publishes no robots.txt token for it. There is therefore no documented way to opt out through robots.txt. Google documents no indexing role for this agent, so allowing it has no documented effect on search indexing. Blocking it at the server level would stop Google from checking links to the site for malware, and Google does not document what follows from that.
Verification uses reverse DNS with a matching forward lookup. A reverse DNS lookup on the requesting IP should return a hostname of the form rate-limited-proxy-***-***-***-***.google.com, and a forward lookup on that hostname should return the same IP. Google also publishes special-crawlers.json, an IP range file that covers this agent and can be matched against directly. Either method confirms the request is Google's; a user-agent match alone does not.
Operator note. 'The Google-Safety user agent ignores robots.txt rules.' No robots.txt token is given.
Controlling Google-Safety with robots.txt
Use the token undocumented in robots.txt. Google documents that this fetcher does not consult robots.txt, so a robots rule will not stop it; see the operator note.
User-agent: undocumented
Disallow: /User-agent: undocumented
Allow: /Verifying a request is really Google-Safety
Anyone can put Google-Safety in a User-Agent header. Reverse DNS to rate-limited-proxy-***-***-***-***.google.com with matching forward DNS, or match the source IP against special-crawlers.json.
host <request-ip>
# expect a hostname ending in google.com (rate-limited-proxy-***-***-***-***.google.com)
host <that-hostname>
# expect it to resolve back to <request-ip>Google publishes a machine-readable list. Our probe read it on 2026-09-06; the file's own creation time was 2026-09-04.
- Prefixes
- 272
- IPv4
- 136
- IPv6
- 136
First entries at fetch time: 2001:4860:4801:2008::/64, 2001:4860:4801:200c::/64, 2001:4860:4801:200d::/64
SHA-256 of the fetched body: 9da99f8a169f35bc80153ec102d0be5674975861f3705614ef6ae51fd32ea5a1
Common questions
Should I block Google-Safety?
Google describes it as abuse-specific crawling, such as checking publicly posted links for malware. It ignores robots.txt and has no robots token, so the only option is a server-level block using the verified hostname or the special-crawlers.json ranges. Google does not document what happens to a site that blocks it.
How do I verify Google-Safety?
Do a reverse DNS lookup on the IP and confirm it returns a hostname matching rate-limited-proxy-***-***-***-***.google.com, then do a forward lookup on that hostname and confirm it returns the same IP. You can also match the IP against Google's special-crawlers.json file.
Does Google-Safety respect robots.txt?
No. Google's documentation says the Google-Safety user agent ignores robots.txt rules and is unaffected by crawling preferences. No robots.txt token is documented, so there is no directive you can write for it.
Sources
Every fact on this page was checked against Google's own documentation, listed below, and re-checked by a second reviewer before publication. Reviewed September 2026.
Related crawlers
This registry documents how operators describe their own bots so site owners can identify and control them. It does not publish third-party IP lists or guess at undocumented behaviour. To see how your own site responds to automated visitors, the bot detection scanner reads a URL's live response and names the protection it finds.