What OAI-SearchBot does
OpenAI says OAI-SearchBot is used to surface websites in search results in ChatGPT’s search features, and that sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though they can still appear as navigational links.
OpenAI’s help page for ChatGPT search says that to make a website eligible, you allow OAI-SearchBot to crawl it and make sure your host or CDN allows traffic from OpenAI’s published search bot IP addresses. Placement is not guaranteed.
OpenAI says a robots.txt change can take about 24 hours to reach ChatGPT search results.
| Operator | OpenAI |
|---|---|
| robots.txt token | OAI-SearchBot |
| User agent | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot |
| Used for | ChatGPT search |
| If you block it | The page is not shown in ChatGPT search answers, apart from navigational links. |
| Check a visit is real | Match the request’s IP address against openai.com/searchbot.json, and allow those addresses at your firewall or CDN. When it fetches robots.txt, the user agent adds a robots.txt marker. |