Other · Google
AdSense user agent, IP ranges and robots.txt
AdSense is Google's other. It identifies itself with the Mediapartners-Google user-agent token, applies robots.txt only in part, per its operator, and runs from 272 published IP prefixes.
- Operator
- Reviewed
- September 2026
- Sources
- 2
- Operator
- User-agent token
- Mediapartners-Google
- Purpose
- Other
- robots.txt token
- Mediapartners-Google
- Honors robots.txt
- Partially, see notes
- Published IP ranges
- Yes, 272 prefixes (136 IPv4, 136 IPv6)
- Verification
- Reverse DNS to rate-limited-proxy-***-***-***-***.google.com with matching forward DNS, or match the source IP against special-crawlers.json.
- Reverse DNS suffix
- google.com (rate-limited-proxy-***-***-***-***.google.com)
- Full user-agent example
- Mediapartners-Google
Block or allow AdSense in robots.txt
User-agent: Mediapartners-Google
Disallow: /User-agent: Mediapartners-Google
Allow: /Verify a request is really AdSense
Anyone can put Mediapartners-Google in a User-Agent header. Reverse DNS to rate-limited-proxy-***-***-***-***.google.com with matching forward DNS, or match the source IP against special-crawlers.json.
host <request-ip>
# expect a hostname ending in google.com (rate-limited-proxy-***-***-***-***.google.com)
host <that-hostname>
# expect it to resolve back to <request-ip>Published IP ranges
Google publishes a machine-readable list. Our probe read it on 2026-09-06; the file's own creation time was 2026-09-04.
- Prefixes
- 272
- IPv4
- 136
- IPv6
- 136
First entries at fetch time: 2001:4860:4801:2008::/64, 2001:4860:4801:200c::/64, 2001:4860:4801:200d::/64
SHA-256 of the fetched body: 9da99f8a169f35bc80153ec102d0be5674975861f3705614ef6ae51fd32ea5a1
What AdSense does
Mediapartners-Google is the user-agent of Google's AdSense crawler. Google says it visits participating sites so it can provide them with relevant ads, and that robots.txt rules addressed to Mediapartners-Google control it while the global user agent (*) is ignored. Verify it by reverse DNS to rate-limited-proxy-***-***-***-***.google.com with matching forward DNS, or against Google's special-crawlers.json ranges.
This crawler shows up in logs with the bare user-agent string Mediapartners-Google, or on mobile devices with a device prefix followed by (compatible; Mediapartners-Google/2.1; +http://www.google.com/bot.html). Google says the AdSense crawler visits participating sites so that it can provide them with relevant ads. Google files it under special-case crawlers, and its verification page ties it to hostnames of the form rate-limited-proxy-***-***-***-***.google.com. Google also publishes the IP ranges for its special-case crawlers in special-crawlers.json.
The robots.txt behaviour is the part to get right. Google states that this crawler ignores the global user agent (*), so a site-wide Disallow written for every bot does not stop Mediapartners-Google. A group addressed to the Mediapartners-Google token does. Blocking it means the AdSense crawler cannot visit your pages, and Google's documented purpose for those visits is providing relevant ads, so a block works against ad relevance on an AdSense site. Google's page for this crawler does not describe any effect on Google Search.
Operator note. Mediapartners is a special-case crawler; 'The global user agent (*) is ignored.' Mobile agent: '(Various mobile device types) (compatible; Mediapartners-Google/2.1; +http://www.google.com/bot.html)'.
Common questions
- Should I block Mediapartners-Google?
- If your site runs AdSense, blocking it works against you, since Google says the crawler visits participating sites so it can provide them with relevant ads. If you do want to block it, address the Mediapartners-Google token directly; Google states this crawler ignores the global user agent (*).
- How do I verify Mediapartners-Google?
- Reverse-resolve the requesting IP; Google says the hostname takes the form rate-limited-proxy-***-***-***-***.google.com. Then resolve that hostname forward and confirm it returns the same IP. As an alternative, match the IP against Google's published special-crawlers.json at https://developers.google.com/static/crawling/ipranges/special-crawlers.json.
- Does Mediapartners-Google honor robots.txt?
- Partially. Google says rules addressed to the Mediapartners-Google user agent control the AdSense crawler, but the global user agent (*) is ignored. A robots.txt that blocks all bots with User-agent: * therefore does not block this crawler; you need a separate group naming Mediapartners-Google.
Sources
The operator documentation used for this entry. Recorded review date September 2026; consult the linked pages for current policy and compare them with this dated snapshot.
This registry documents how operators describe their own bots so site owners can identify and control them. It does not publish third-party IP lists or guess at undocumented behaviour. To see how your own site responds to automated visitors, the bot detection scanner reads a URL's live response and names the protection it finds.