Applebot-Extended
Applebot-Extended is a robots.txt token, not a crawler. Disallowing it tells Apple not to use content crawled by Applebot to train its generative foundation models. Apple says pages that disallow it can still appear in Apple's search results, so blocking it does not affect Siri or Spotlight visibility.
What it does
Apple says Applebot-Extended does not crawl webpages. It is only used to determine how data crawled by Applebot may be used. Disallowing it opts your content out of training Apple's foundation models that power generative AI features such as Apple Intelligence.
Pages that disallow Applebot-Extended can still be included in search results, and Apple says the rule is not considered in search ranking. Crawling itself is controlled by the Applebot token.
In our 2026-11 study, 18.0% of popular sites with a robots.txt blocked it. →
robots.txt token
Applebot-Extended
How to allow it
User-agent: Applebot-Extended
Allow: /How to block it
User-agent: Applebot-Extended
Disallow: /How to verify real visits
There is nothing to verify in your logs: Applebot-Extended never appears as a user agent. The underlying requests come from Applebot, which you can verify by reverse DNS (applebot.apple.com) or against Apple's published IP ranges.