Index My Site

Applebot: Apple's crawler for Siri, Spotlight and Safari

Applebot is the crawler behind search in Siri, Spotlight and Safari. Apple says it may also help train Apple’s foundation models, and gives publishers a separate token, Applebot-Extended, to opt out of that training.

What Applebot does

Apple says Applebot powers search features across its devices, including Spotlight, Siri and Safari, and that allowing Applebot lets your content appear in search results for Apple users.

If robots.txt does not mention Applebot but does mention Googlebot, Applebot follows the Googlebot rules. A Googlebot block can therefore block Apple’s search too.

Applebot-Extended is a robots.txt token, not a crawler. Disallowing it opts your content out of training Apple’s foundation models, and Apple says those pages can still appear in search. Applebot does not follow Crawl-delay.

OperatorApple
robots.txt tokenApplebot
User agentMozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15 (Applebot/0.1; +http://www.apple.com/go/applebot)Mozilla/5.0 (iPhone; CPU iPhone OS 17_4_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4.1 Mobile/15E148 Safari/604.1 (Applebot/0.1; +http://www.apple.com/go/applebot)
Used forSiri, Spotlight and Safari
If you block itApple cannot crawl the page for search across its devices.
Check a visit is realRun a reverse DNS lookup on the IP address and check the name is in the applebot.apple.com domain. Apple also publishes Applebot’s IP ranges at search.developer.apple.com/applebot.json.

Source: Apple: about Applebot

Check whether your robots.txt allows Applebot

We read the site's robots.txt file and nothing else. Crawl rules say who may visit. They do not show whether a page is indexed or cited.

Results

View robots.txt

How to block or allow Applebot in robots.txt

Use the Applebot token for search, and Applebot-Extended for training. If you write Googlebot rules and no Applebot group, Applebot follows the Googlebot rules.

To block Applebot from the whole site:

User-agent: Applebot
Disallow: /

To allow it everywhere, even when another group blocks all crawlers:

User-agent: Applebot
Allow: /

A crawler named in a group follows only that group, so repeat any shared Disallow lines in its group. See how robots.txt blocks work for the details.

The Googlebot rules Applebot may follow

CrawlerUsed forIf you block it
GooglebotGoogle Search, AI Overviews and AI ModeGoogle cannot crawl the page for Search, including AI Overviews and AI Mode.

Allowed Applebot? Get the page indexed

Apple has no submission form for web search, so Applebot finds pages through links. Apple says its AI answers in Siri and Search respect the nosnippet meta tag rather than robots.txt, so use nosnippet if you want a page found but not quoted.

Submit your pages to send them to Google, and add Bing to send them to Bing as well. It works for any public URL, including reviews and posts on other sites that mention you. See what each AI answer needs before it can cite a page.

Frequently asked questions

Apple says that when robots.txt does not mention Applebot but mentions Googlebot, Applebot follows the Googlebot instructions. Add an Applebot group if you want different rules.

A robots.txt token that controls whether content Applebot crawls can train Apple’s foundation models. It does not crawl, and Apple says blocking it does not remove pages from search or affect ranking.

No. Apple says Applebot does not follow Crawl-delay, and slows down by itself when a site slows down or returns errors.

Apple says publishers can opt out of those answers with the nosnippet meta tag. Blocking Applebot-Extended alone keeps your content in search and out of training.