Google Site Verifier
Google operates Google Site Verifier, a user-triggered fetcher that retrieves Search Console verification tokens on a user's request. It generally ignores robots.txt and Google documents no robots.txt token for it. Site owners verify requests by reverse DNS to gae.googleusercontent.com or google.com with a matching forward lookup, or against Google's published user-triggered-fetchers-google.json IP ranges.
Mozilla/5.0 (compatible; Google-Site-Verification/1.0)What Google Site Verifier does
The request shows up in logs with the user agent Mozilla/5.0 (compatible; Google-Site-Verification/1.0). Google classes it as a user-triggered fetcher, meaning it acts on the request of a user, and the documentation uses this fetcher as its canonical example of that category. Its only stated job is fetching Search Console verification tokens, so a site owner should expect to see it when someone attempts to verify ownership of the property in Search Console.
Blocking this fetcher breaks Search Console verification for the property, because Google cannot retrieve the token. Google states that user-triggered fetchers generally ignore robots.txt, and it documents no robots.txt token for Google Site Verifier, so a Disallow rule is not the way to control it. If you need to stop it, a firewall or server-level rule on the user-agent string or the published IP ranges is the remaining option, and verification will then fail for that property. Allowing it has no documented effect beyond completing verification.
Google publishes a dedicated IP list for its own user-triggered fetchers at developers.google.com/static/crawling/ipranges/user-triggered-fetchers-google.json. Reverse DNS on a Google Site Verifier address resolves to a hostname of the form ***-***-***-***.gae.googleusercontent.com or google-proxy-***-***-***-***.google.com, and a forward lookup on that hostname should return the original IP. Either check confirms the request came from Google, which separates a real verification fetch from a request that only copies the user-agent string.
Operator note. User-triggered fetcher; generally ignores robots.txt. The docs use it as the canonical example of a fetcher that 'acts on the request of a user.'
Controlling Google Site Verifier with robots.txt
Use the token undocumented in robots.txt. Google documents that this fetcher does not consult robots.txt, so a robots rule will not stop it; see the operator note.
User-agent: undocumented
Disallow: /User-agent: undocumented
Allow: /Verifying a request is really Google Site Verifier
Anyone can put Google-Site-Verification in a User-Agent header. Reverse DNS with matching forward DNS, or match the source IP against user-triggered-fetchers.json or user-triggered-fetchers-google.json.
host <request-ip>
# expect a hostname ending in gae.googleusercontent.com (***-***-***-***.gae.googleusercontent.com) or google.com (google-proxy-***-***-***-***.google.com)
host <that-hostname>
# expect it to resolve back to <request-ip>Google publishes a machine-readable list. Our probe read it on 2026-09-06; the file's own creation time was 2026-09-04.
- Prefixes
- 496
- IPv4
- 248
- IPv6
- 248
First entries at fetch time: 2001:4860:4801:4004::/64, 2001:4860:4801:4008::/64, 2001:4860:4801:4009::/64
SHA-256 of the fetched body: 5590a1a5d2fbe83872191aa2d8d45e9c854c56c49b742df9d33b7612f5482a5a
Common questions
Should I block Google Site Verifier?
No, unless you never intend to verify the site in Search Console. Its only documented purpose is fetching verification tokens, and the fetch happens because a user asked for it. Blocking it at the firewall stops verification from completing. It generally ignores robots.txt, so a Disallow rule would not affect it in any case.
Does Google Site Verifier respect robots.txt?
Generally not. Google documents it as a user-triggered fetcher, and states that fetchers in that category generally ignore robots.txt rules because they act on the request of a user. Google also lists no robots.txt token for it. Control, if needed, has to happen at the server or firewall level.
How do I verify Google Site Verifier?
Run a reverse DNS lookup on the requesting IP. A real request resolves to a hostname ending in gae.googleusercontent.com or google.com, and a forward lookup on that hostname returns the same IP. Or match the IP against user-triggered-fetchers-google.json (or user-triggered-fetchers.json), Google's published range files for user-triggered fetchers.
Sources
Every fact on this page was checked against Google's own documentation, listed below, and re-checked by a second reviewer before publication. Reviewed September 2026.
Related crawlers
This registry documents how operators describe their own bots so site owners can identify and control them. It does not publish third-party IP lists or guess at undocumented behaviour. To see how your own site responds to automated visitors, the bot detection scanner reads a URL's live response and names the protection it finds.