Googlebot News
Google uses the Googlebot-News token so site owners can set crawling preferences for the Google News product, including news.google.com and the Google News app. It is a robots.txt control token only; Google crawls with various Googlebot user agent strings, and it honors robots.txt. Verify visits by reverse DNS to googlebot.com, google.com or googleusercontent.com with forward confirmation, or against common-crawlers.json.
What Googlebot News does
Site owners will not see Googlebot-News in their logs. Google states that Googlebot-News does not have a separate HTTP request user agent string and that the crawling is done with various Googlebot user agent strings. The fetches that feed Google News therefore look like ordinary Googlebot traffic, with the same reverse-DNS hostnames such as crawl-***-***-***-***.googlebot.com or geo-crawl-***-***-***-***.geo.googlebot.com, and the same published common-crawlers.json ranges.
The token exists for robots.txt. Google states that crawling preferences addressed to User-agent: Googlebot-News affect the Google News product, which covers news.google.com and the Google News app. Disallowing Googlebot-News removes pages from Google News, and leaving it allowed keeps them eligible. Google also notes that the token matches the Googlebot token, so rules written for Googlebot are relevant to news crawling as well, and a site owner who wants different treatment for news should write a separate Googlebot-News group.
Operator note. 'Googlebot-news doesn't have a separate HTTP request user agent string. Crawling is done with various Googlebot user agent strings.' The token is therefore a robots.txt control token only, and the crawler also matches the Googlebot token.
Controlling Googlebot News with robots.txt
Googlebot-News is a policy token. It has no crawler of its own; it tells Google how content already fetched by its other crawlers may be used. Put it in robots.txt like any other token.
User-agent: Googlebot-News
Disallow: /User-agent: Googlebot-News
Allow: /Verifying a request is really Googlebot News
Anyone can put Googlebot-News in a User-Agent header. Reverse DNS to googlebot.com, google.com, or googleusercontent.com with matching forward DNS, or match the source IP against the published common-crawlers.json ranges.
host <request-ip>
# expect a hostname ending in googlebot.com (crawl-***-***-***-***.googlebot.com or geo-crawl-***-***-***-***.geo.googlebot.com)
host <that-hostname>
# expect it to resolve back to <request-ip>Google publishes a machine-readable list. Our probe read it on 2026-09-06; the file's own creation time was 2026-09-04.
- Prefixes
- 317
- IPv4
- 170
- IPv6
- 147
First entries at fetch time: 2001:4860:4801:10::/64, 2001:4860:4801:11::/64, 2001:4860:4801:12::/64
SHA-256 of the fetched body: 241e712f9c902b33aa4c0cc79268c34cb00c025edbc27298547be298763ddade
Common questions
Should I block Googlebot-News?
Only if you do not want your pages in Google News, news.google.com or the Google News app. Google says preferences addressed to this token affect the Google News product, so a Disallow for Googlebot-News removes the site from that product. Google does not describe any other effect of the rule.
How do I verify Googlebot-News?
You cannot verify it by user agent, because Google says it has no separate request string and crawls with various Googlebot strings. Verify the request as Googlebot instead, by reverse DNS to googlebot.com, google.com or googleusercontent.com with a matching forward lookup, or by matching the IP against common-crawlers.json.
Why does Googlebot-News never appear in my logs?
Google states that Googlebot-News does not have a separate HTTP request user agent string. The crawling that feeds Google News is done with the ordinary Googlebot user agent strings, so the token is for robots.txt and does not show up as a user agent in an access log.
Sources
Every fact on this page was checked against Google's own documentation, listed below, and re-checked by a second reviewer before publication. Reviewed September 2026.
Related crawlers
This registry documents how operators describe their own bots so site owners can identify and control them. It does not publish third-party IP lists or guess at undocumented behaviour. To see how your own site responds to automated visitors, the bot detection scanner reads a URL's live response and names the protection it finds.