Applebot-Extended
Applebot-Extended is a robots.txt token operated by Apple. It does not crawl webpages; Apple uses it only to decide whether content fetched by Applebot may train Apple's general purpose foundation models behind Apple Intelligence, Services, and Developer Tools. Apple states it honors robots.txt. Apple documents no user-agent string and no IP range, so there is no traffic to verify.
What Applebot-Extended does
Site owners will not find Applebot-Extended in their access logs. Apple says the token never sends requests, so there is no user-agent string to match and Apple publishes no example. Applebot does the crawling itself. Applebot-Extended exists so publishers can write a robots.txt rule that tells Apple how the data Applebot already collected may be used. Apple's documented example is a User-agent: Applebot-Extended group with Disallow: /private/, which opts that path out of training for Apple's generative AI models while leaving Applebot free to fetch it.
Disallowing Applebot-Extended has a single documented effect. Apple opts the affected pages out of training for its general purpose foundation models. Pages that disallow the token can still be included in search results, and Apple says site rules for Applebot-Extended are not considered in ranking for Search. Apple adds that allowing the token will help improve the capabilities and quality of its generative AI models over time. A disallow rule for this token does not stop Applebot crawling, so a site owner who wants to stop Apple fetching pages needs a rule for Applebot.
Operator note. Applebot-Extended is a policy-only token; Apple states: 'Applebot-Extended does not crawl webpages. Webpages that disallow Applebot-Extended can still be included in search results. Applebot-Extended is only used to determine how to use the data crawled by the Applebot user agent.' The documented robots.txt example is 'User-agent: Applebot-Extended / Disallow: /private/'; disallowing it opts content out of foundation-model training only, it does not stop Applebot crawling, and 'Site rules for Applebot-Extended are not considered in ranking for Search.' No user agent string example is given because it never sends requests. Apple adds: 'Allowing Applebot-Extended will help improve the capabilities and quality of Apple's generative AI models over time.'
Controlling Applebot-Extended with robots.txt
Applebot-Extended is a policy token. It has no crawler of its own; it tells Apple how content already fetched by its other crawlers may be used. Put it in robots.txt like any other token.
User-agent: Applebot-Extended
Disallow: /User-agent: Applebot-Extended
Allow: /Verifying a request is really Applebot-Extended
Anyone can put Applebot-Extended in a User-Agent header. There is no separate traffic to verify: Apple states 'Applebot-Extended does not crawl webpages', so it has no user agent string or IP range of its own. It is a robots.txt policy token only, and the crawling itself is done by Applebot.
Common questions
Should I block Applebot-Extended?
Block it if you do not want Apple to train its general purpose foundation models on your content. Apple says pages that disallow the token can still be included in search results and that the rule is not considered in ranking for Search. Blocking it does not stop Applebot crawling; Apple documents that as a separate user agent.
How do I verify Applebot-Extended requests?
You cannot, because there are none. Apple states that Applebot-Extended does not crawl webpages, so it never sends requests and has no user-agent string. Apple publishes no IP range or reverse DNS pattern for it either. Any request claiming to be Applebot-Extended is not described in Apple's documentation. Verify Applebot instead, since that is the agent that fetches pages.
Does Applebot-Extended honor robots.txt?
Yes. Apple documents it as a robots.txt token and gives the example User-agent: Applebot-Extended followed by Disallow: /private/. Apple uses these rules only to decide how data crawled by Applebot may be used for training its foundation models; the rules do not change what Applebot fetches.
Sources
Every fact on this page was checked against Apple's own documentation, listed below, and re-checked by a second reviewer before publication. Reviewed September 2026.
Related crawlers
This registry documents how operators describe their own bots so site owners can identify and control them. It does not publish third-party IP lists or guess at undocumented behaviour. To see how your own site responds to automated visitors, the bot detection scanner reads a URL's live response and names the protection it finds.