DuckDuckBot
DuckDuckGo operates DuckDuckBot, the web crawler behind its search results, which the operator says exists to improve results and support its own indexes. DuckDuckGo says the bot respects WWW::RobotRules but names no robots.txt token for it. Site owners verify requests only by matching the source IP against the published list at duckduckgo.com/duckduckbot.json; no reverse-DNS suffix is documented.
DuckDuckBot/1.1; (+http://duckduckgo.com/duckduckbot.html)What DuckDuckBot does
In logs DuckDuckBot appears as DuckDuckBot/1.1 followed by a link to http://duckduckgo.com/duckduckbot.html, which redirects to the operator's help page. DuckDuckGo describes the crawler's job as constantly improving its search results and offering users the best and most secure search experience possible, and its sources page says the company maintains this crawler together with many indexes to support results. A visit from DuckDuckBot is therefore part of building the material that DuckDuckGo draws on when it answers searches.
Blocking DuckDuckBot removes the site from the material this crawler feeds into DuckDuckGo's own indexes, which the operator lists among the sources of its results. The help page says the bot respects WWW::RobotRules, the Perl module it links to on metacpan, but it never names a User-agent token to put in robots.txt; the only identifier the page gives is the product name in the user-agent string. The documentation says nothing about crawl rate or JavaScript execution, and it gives no way to request a crawl.
Verification rests on the IP list alone. The help page prints the originating addresses inline and links to https://duckduckgo.com/duckduckbot.json. Every prefix is a /32 IPv4 host address. DuckDuckGo documents no reverse-DNS suffix.
Operator note. The help page states that the crawler "respects WWW::RobotRules", linking to the Perl module page on metacpan. The page does not name a robots.txt User-agent token for this crawler; the only token it gives is the product name in the user-agent string, DuckDuckBot/1.1. The URL in the user-agent string, http://duckduckgo.com/duckduckbot.html, leads to the help page at https://duckduckgo.com/duckduckgo-help-pages/results/duckduckbot. The documentation does not say whether the crawler executes JavaScript, what its crawl rate is, or how to request a crawl.
Controlling DuckDuckBot with robots.txt
Use the token undocumented in robots.txt. DuckDuckGo documents that DuckDuckBot honors robots.txt directives.
User-agent: undocumented
Disallow: /User-agent: undocumented
Allow: /Verifying a request is really DuckDuckBot
Anyone can put DuckDuckBot in a User-Agent header. Verification is by published IP list only. The help page lists the originating IP addresses inline and links to the machine-readable list at https://duckduckgo.com/duckduckbot.json. The published list contains IPv4 host addresses only, and no reverse-DNS suffix is documented.
DuckDuckGo publishes a machine-readable list. Our probe read it on 2026-09-06; the file's own creation time was 2026-09-01.
- Prefixes
- 486
- IPv4
- 486
- IPv6
- 0
First entries at fetch time: 104.43.54.127/32, 104.43.55.116/32, 104.43.55.117/32
SHA-256 of the fetched body: fd9039362dfaaad236d50c07ddd12bf0cc7f0d530fbff59b8feb1b22e0db2459
Common questions
How do I verify DuckDuckBot?
Compare the requesting IP with the list DuckDuckGo publishes at https://duckduckgo.com/duckduckbot.json. The file holds individual /32 IPv4 addresses. DuckDuckGo documents no reverse-DNS check.
What robots.txt User-agent should I use for DuckDuckBot?
DuckDuckGo does not name one. The help page says the crawler respects WWW::RobotRules, but it gives no User-agent token; the only identifier in the docs is the product name DuckDuckBot in the user-agent string. The operator does not confirm whether a rule addressed to that name is matched.
Does DuckDuckBot run JavaScript?
The operator's documentation does not say. The DuckDuckBot help page covers the crawler's purpose, the robots rules it follows and its IP addresses, but it says nothing about JavaScript execution or crawl rate, and it gives no way to ask for a page to be crawled.
Sources
Every fact on this page was checked against DuckDuckGo's own documentation, listed below, and re-checked by a second reviewer before publication. Reviewed September 2026.
Related crawlers
This registry documents how operators describe their own bots so site owners can identify and control them. It does not publish third-party IP lists or guess at undocumented behaviour. To see how your own site responds to automated visitors, the bot detection scanner reads a URL's live response and names the protection it finds.