Google-Extended
Google operates Google-Extended, a robots.txt product token rather than a crawler. Publishers use it to manage whether content Google crawls may train future Gemini models or be used for grounding in Gemini Apps and Vertex AI. It has no user agent string of its own and no IP range file; verify the underlying Google crawler, such as Googlebot, instead.
What Google-Extended does
Site owners will not see Google-Extended in their access logs. Google states that the token has no separate HTTP request user agent string; crawling happens under existing Google user agents such as Googlebot, and the Google-Extended name works only inside robots.txt in a control capacity. The token tells Google whether content crawled from the site may be used for training future generations of Gemini models behind Gemini Apps and the Vertex AI API for Gemini, and for grounding, which Google describes as providing content from the Google Search index to the model at prompt time to improve factuality and relevancy.
Disallowing Google-Extended withdraws the site's content from Gemini training and from grounding in Gemini Apps and Grounding with Google Search on Vertex AI. Google says the token does not affect a site's inclusion in Google Search and is not a ranking signal, so a disallow rule leaves normal search indexing untouched. Google's documentation gives this example group: user-agent: Google-Extended, allow: /archive/1Q84, disallow: /archive/. Google's changelog records an update to the token's description on 2025-04-25, based on publisher feedback.
Operator note. Policy-only token with no crawler of its own and no IP range file. Example robots.txt group in the docs: 'user-agent: Google-Extended / allow: /archive/1Q84 / disallow: /archive/'. Changelog 2025-04-25: 'Updated the description of the Google-Extended product token' based on publisher feedback.
Controlling Google-Extended with robots.txt
Google-Extended is a policy token. It has no crawler of its own; it tells Google how content already fetched by its other crawlers may be used. Put it in robots.txt like any other token.
User-agent: Google-Extended
Disallow: /User-agent: Google-Extended
Allow: /Verifying a request is really Google-Extended
Anyone can put Google-Extended in a User-Agent header. Google-Extended has no crawler of its own to verify: 'Google-Extended doesn't have a separate HTTP request user agent string. Crawling is done with existing Google user agent strings; the robots.txt user-agent token is used in a control capacity.' Verify the underlying Google crawler (for example Googlebot) instead.
Common questions
Should I block Google-Extended?
Block it if you do not want Google to use your content for Gemini training or grounding. Google states that a disallow rule for this token does not change your inclusion in Google Search and is not a ranking signal, so the effect is limited to Gemini products rather than search visibility.
How do I verify Google-Extended requests?
You cannot verify it directly because it never appears as a user agent. Google explains that crawling is done with existing Google user agent strings and the Google-Extended token works only in a control capacity inside robots.txt. Verify the crawler that did appear in your logs, such as Googlebot, using Google's published method for that crawler.
Does blocking Google-Extended remove my site from Google Search?
No. Google's documentation says Google-Extended does not impact a site's inclusion in Google Search and is not used as a ranking signal. The token only controls whether crawled content feeds Gemini model training and grounding in Gemini Apps and Vertex AI.
Sources
Every fact on this page was checked against Google's own documentation, listed below, and re-checked by a second reviewer before publication. Reviewed September 2026.
Related crawlers
This registry documents how operators describe their own bots so site owners can identify and control them. It does not publish third-party IP lists or guess at undocumented behaviour. To see how your own site responds to automated visitors, the bot detection scanner reads a URL's live response and names the protection it finds.