What Bingbot does
Microsoft says Bingbot is its standard crawler and handles most of Bing’s crawling each day. It renders pages with a recent version of Microsoft Edge, which is why its user agents include the Edge version.
Bing’s robots.txt guide says a page that is not crawled will not be indexed. Bing also supports Crawl-delay, and Bing Webmaster Tools has a crawl control setting.
When Bingbot finds a group in robots.txt that names it, it ignores the generic group, so rules under User-agent: * have to be repeated in a Bingbot group.
| Operator | Microsoft |
|---|---|
| robots.txt token | Bingbot |
| User agent | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/W.X.Y.Z Safari/537.36Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) |
| Used for | Bing and Copilot |
| If you block it | Bing cannot crawl the page for its index. |
| Check a visit is real | Run a reverse DNS lookup on the IP address and check the name ends in search.msn.com, then a forward lookup to confirm it resolves back to the same address. Bing also publishes its IP ranges at bing.com/toolbox/bingbot.json and a Verify Bingbot tool. |
Source: Bing: which crawlers Bing uses