YandexFeed
Yandex operates YandexFeed, which downloads structured data through links that website owners submitted to Yandex services; it makes no indexing requests. Its robots.txt token is YandexFeed, and Yandex says it does not follow general robots.txt rules, so directives must name it. Verification is by reverse DNS to yandex.ru, yandex.net or yandex.com with forward confirmation; Yandex publishes no IP range list.
Mozilla/5.0 (compatible; YandexFeed/1.0; +http://yandex.com/bots)What YandexFeed does
In logs, this bot appears as "Mozilla/5.0 (compatible; YandexFeed/1.0; +http://yandex.com/bots)". Yandex says its visits download structured data via links that site owners themselves submitted to Yandex services, so the requests go to links you supplied. Yandex states that YandexFeed does not make indexing requests. The documentation gives no crawl frequency, and Yandex does not document whether the bot executes JavaScript. The user-agent string carries the operator link http://yandex.com/bots.
Yandex describes the requests as downloads of links a site owner submitted, so allowing YandexFeed keeps that submitted data flowing to the Yandex service it was registered with, and blocking it stops that delivery. Yandex documents that YandexFeed does not follow general robots.txt rules, so a rule under "User-agent: *" has no effect on it; a Disallow must be placed under "User-agent: YandexFeed" to apply. Yandex records this as "General robots.txt rules: No", and it is the reason this registry lists robots.txt support as partial.
To confirm a request is from Yandex, run a reverse DNS lookup on the source IP, check that the hostname ends in yandex.ru, yandex.net or yandex.com, and then resolve that hostname forward to confirm it returns the same IP. Yandex also provides an IP address check tool in Yandex Webmaster. Yandex publishes no IP range list for this bot, so a user-agent match on its own does not establish origin.
Operator note. It does not take into account the general rules in robots.txt.
Controlling YandexFeed with robots.txt
Use the token YandexFeed in robots.txt. Yandex documents limits on how this bot applies robots.txt; see the operator note.
User-agent: YandexFeed
Disallow: /User-agent: YandexFeed
Allow: /Verifying a request is really YandexFeed
Anyone can put YandexFeed in a User-Agent header. Reverse DNS lookup on the requesting IP must return a host ending in yandex.ru, yandex.net or yandex.com, and a forward DNS lookup of that host must return the same IP. Yandex Webmaster also provides an IP address check tool.
host <request-ip>
# expect a hostname ending in yandex.ru, yandex.net, yandex.com
host <that-hostname>
# expect it to resolve back to <request-ip>Common questions
Should I block YandexFeed?
Yandex says YandexFeed only downloads structured data from links that site owners submitted to Yandex services and makes no indexing requests. If you submitted a feed, blocking the bot stops that feed being fetched. If you did not, the operator's documentation does not describe any other reason for it to visit.
Does YandexFeed obey robots.txt?
Partially. Yandex documents that YandexFeed does not follow general robots.txt rules, meaning directives under "User-agent: *" do not apply to it. To restrict it you need a section addressed to "User-agent: YandexFeed". The documentation does not describe any further exceptions.
How do I verify YandexFeed?
Reverse-resolve the source IP and check that the hostname ends in yandex.ru, yandex.net or yandex.com, then do a forward DNS lookup on that hostname and confirm it returns the same IP. Yandex Webmaster also has an IP address check tool. Yandex publishes no IP range list for this bot.
Sources
Every fact on this page was checked against Yandex's own documentation, listed below, and re-checked by a second reviewer before publication. Reviewed September 2026.
Related crawlers
This registry documents how operators describe their own bots so site owners can identify and control them. It does not publish third-party IP lists or guess at undocumented behaviour. To see how your own site responds to automated visitors, the bot detection scanner reads a URL's live response and names the protection it finds.