FacebookBot
Meta operated FacebookBot. Its retired documentation said the crawler "crawls public web pages to improve language models for our speech recognition technology." It honored robots.txt under the FacebookBot token. Site owners identified it by user-agent string or by IP addresses from a whois query against AS32934. Meta documents no reverse-DNS method, publishes no range file and no longer lists it.
Mozilla/5.0 (compatible; FacebookBot/1.0; +https://developers.facebook.com/docs/sharing/webmasters/facebookbot/)What FacebookBot does
In access logs the request identifies itself as "Mozilla/5.0 (compatible; FacebookBot/1.0; +https://developers.facebook.com/docs/sharing/webmasters/facebookbot/)". The link in that string now redirects to Meta's general web-crawlers page, which does not mention FacebookBot. Everything known about the bot comes from a Wayback Machine capture of the retired page, archived 2023-05-20, which said the crawler fetched public pages to improve the language models behind Meta's speech recognition technology. Meta's current documentation does not say whether the bot still runs.
The retired page described polite crawling that respects standard robots.txt rules. FacebookBot honored any rule directed at its own user-agent token, and if robots.txt did not mention FacebookBot but did mention Googlebot, it followed the Googlebot rules instead. The default rate was one page per second, and the bot supported a Crawl-delay directive; the documented example "Crawl-delay: 5" meant one page every five seconds. It did not crawl content that was not publicly accessible, such as password-protected pages that require a login.
Because the documented purpose is AI training, blocking the FacebookBot token in robots.txt withholds a site's pages from the speech recognition language models the retired page described, and allowing it contributes pages to that training data. The archived page gave facebookbot@fb.com as the contact address. Meta does not document whether the bot is still active, so a site owner who sees the token in logs has only the retired page to go on.
Operator note. FacebookBot is not listed on Meta's current web crawlers page; the facts here come from Meta's retired FacebookBot documentation, which states: "We use polite web crawling protocols that respect standard robots.txt rules." That documentation also states: "FacebookBot will honor all rules directed at it via the User-Agent string documented above. If these rules don't mention FacebookBot but do mention Googlebot, FacebookBot will follow Googlebot's instructions." and "FacebookBot will crawl 1 page per second by default.", supports a Crawl-delay directive (example "Crawl-delay: 5 # 1 page per 5 seconds"), and says the bot "will not crawl any content that is not publicly accessible on the web, such as password-protected pages that require a login." The contact given on that page is facebookbot@fb.com. Whether the bot is still operating is undocumented.
Controlling FacebookBot with robots.txt
Use the token FacebookBot in robots.txt. Meta documents that FacebookBot honors robots.txt directives.
User-agent: FacebookBot
Disallow: /User-agent: FacebookBot
Allow: /Verifying a request is really FacebookBot
Anyone can put FacebookBot in a User-Agent header. Meta's retired FacebookBot documentation states: "Webmasters can easily identify traffic originating from the FacebookBot, either via its User-Agent string, or its originating IP addresses" and "You may also obtain the current list of IP addresses used by FacebookBot by running the following command: whois -h whois.radb.net -- '-i origin AS32934' | grep ^route". No reverse-DNS method is documented.
Common questions
Should I block FacebookBot?
That depends on whether you want your public pages used to improve Meta's speech recognition language models, which is the only purpose the retired documentation gave. Blocking the FacebookBot token in robots.txt removed pages from that use. Meta's current documentation does not say whether the bot is still active.
How do I verify FacebookBot?
The retired page said site owners could identify traffic by the user-agent string or by originating IP, and gave a whois command that lists routes announced by AS32934: whois -h whois.radb.net -- '-i origin AS32934' | grep ^route. Meta documents no reverse-DNS method and publishes no IP range file for this bot.
Does FacebookBot follow Crawl-delay?
The retired documentation said yes. FacebookBot crawled one page per second by default and accepted a Crawl-delay directive to slow that down; the documented example "Crawl-delay: 5" set one page every five seconds. Whether current traffic still behaves this way is undocumented, since Meta no longer lists the bot.
Sources
Every fact on this page was checked against Meta's own documentation, listed below, and re-checked by a second reviewer before publication. Reviewed September 2026.
Related crawlers
This registry documents how operators describe their own bots so site owners can identify and control them. It does not publish third-party IP lists or guess at undocumented behaviour. To see how your own site responds to automated visitors, the bot detection scanner reads a URL's live response and names the protection it finds.