What PerplexityBot does
Perplexity says PerplexityBot is designed to surface and link websites in search results on Perplexity, and is not used to crawl content for AI foundation models.
Perplexity-User is a separate fetcher that visits a page when a person asks Perplexity a question that needs it. Perplexity says it generally ignores robots.txt, because a person requested the fetch.
Perplexity says changes to its crawler settings can take up to 24 hours to apply.
| Operator | Perplexity |
|---|---|
| robots.txt token | PerplexityBot |
| User agent | Mozilla/5.0 (compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot) |
| Used for | Perplexity search |
| If you block it | Perplexity does not surface and link the page in its search results. |
| Check a visit is real | Perplexity publishes PerplexityBot’s IP addresses at perplexity.ai/perplexitybot.json and recommends firewall rules that match both the user agent and the IP address. |
Source: Perplexity: Perplexity crawlers