AdSense
Mediapartners-Google is the user-agent of Google's AdSense crawler. Google says it visits participating sites so it can provide them with relevant ads, and that robots.txt rules addressed to Mediapartners-Google control it while the global user agent (*) is ignored. Verify it by reverse DNS to rate-limited-proxy-***-***-***-***.google.com with matching forward DNS, or against Google's special-crawlers.json ranges.
Mediapartners-GoogleWhat AdSense does
This crawler shows up in logs with the bare user-agent string Mediapartners-Google, or on mobile devices with a device prefix followed by (compatible; Mediapartners-Google/2.1; +http://www.google.com/bot.html). Google says the AdSense crawler visits participating sites so that it can provide them with relevant ads. Google files it under special-case crawlers, and its verification page ties it to hostnames of the form rate-limited-proxy-***-***-***-***.google.com. Google also publishes the IP ranges for its special-case crawlers in special-crawlers.json.
The robots.txt behaviour is the part to get right. Google states that this crawler ignores the global user agent (*), so a site-wide Disallow written for every bot does not stop Mediapartners-Google. A group addressed to the Mediapartners-Google token does. Blocking it means the AdSense crawler cannot visit your pages, and Google's documented purpose for those visits is providing relevant ads, so a block works against ad relevance on an AdSense site. Google's page for this crawler does not describe any effect on Google Search.
Operator note. Mediapartners is a special-case crawler; 'The global user agent (*) is ignored.' Mobile agent: '(Various mobile device types) (compatible; Mediapartners-Google/2.1; +http://www.google.com/bot.html)'.
Controlling AdSense with robots.txt
Use the token Mediapartners-Google in robots.txt. Google documents limits on how this bot applies robots.txt; see the operator note.
User-agent: Mediapartners-Google
Disallow: /User-agent: Mediapartners-Google
Allow: /Verifying a request is really AdSense
Anyone can put Mediapartners-Google in a User-Agent header. Reverse DNS to rate-limited-proxy-***-***-***-***.google.com with matching forward DNS, or match the source IP against special-crawlers.json.
host <request-ip>
# expect a hostname ending in google.com (rate-limited-proxy-***-***-***-***.google.com)
host <that-hostname>
# expect it to resolve back to <request-ip>Google publishes a machine-readable list. Our probe read it on 2026-09-06; the file's own creation time was 2026-09-04.
- Prefixes
- 272
- IPv4
- 136
- IPv6
- 136
First entries at fetch time: 2001:4860:4801:2008::/64, 2001:4860:4801:200c::/64, 2001:4860:4801:200d::/64
SHA-256 of the fetched body: 9da99f8a169f35bc80153ec102d0be5674975861f3705614ef6ae51fd32ea5a1
Common questions
Should I block Mediapartners-Google?
If your site runs AdSense, blocking it works against you, since Google says the crawler visits participating sites so it can provide them with relevant ads. If you do want to block it, address the Mediapartners-Google token directly; Google states this crawler ignores the global user agent (*).
How do I verify Mediapartners-Google?
Reverse-resolve the requesting IP; Google says the hostname takes the form rate-limited-proxy-***-***-***-***.google.com. Then resolve that hostname forward and confirm it returns the same IP. As an alternative, match the IP against Google's published special-crawlers.json at https://developers.google.com/static/crawling/ipranges/special-crawlers.json.
Does Mediapartners-Google honor robots.txt?
Partially. Google says rules addressed to the Mediapartners-Google user agent control the AdSense crawler, but the global user agent (*) is ignored. A robots.txt that blocks all bots with User-agent: * therefore does not block this crawler; you need a separate group naming Mediapartners-Google.
Sources
Every fact on this page was checked against Google's own documentation, listed below, and re-checked by a second reviewer before publication. Reviewed September 2026.
Related crawlers
This registry documents how operators describe their own bots so site owners can identify and control them. It does not publish third-party IP lists or guess at undocumented behaviour. To see how your own site responds to automated visitors, the bot detection scanner reads a URL's live response and names the protection it finds.