Index My Site

AI answers can only cite pages that are indexed

ChatGPT, Google AI Overviews, Copilot and Perplexity search before they answer, and they cite what their search finds. To be cited, a page has to be open to their crawler and in the index they search. Index My Site gets your pages into Google and Bing, the two indexes most of these answers draw on.

What each AI answer needs before it can cite a page

Pick an assistant to see the steps its maker documents. Some are changes on your site. Getting the page into Google and Bing is what Index My Site does.

Google AI Overviews and AI Mode

Google says a page must be indexed and eligible to show a snippet, and the site must not be opted out of generative AI features in Search Console. Sites are included by default. Source

  1. 1Allow GooglebotOn your siteAI Overviews and AI Mode follow your Googlebot rules. There is no separate AI crawler to allow.
  2. 2Get the page into GoogleIndex My Site does thisIndex My Site sends it to Google for 1 credit, then searches for it to confirm it is indexed.
  3. 3Allow a snippetOn your sitenosnippet, data-nosnippet and a short max-snippet limit what Google can quote from the page.
Start indexing

ChatGPT search

OpenAI says a site is eligible when it allows OAI-SearchBot. ChatGPT also queries partner search providers, and its help page names Microsoft. Source

  1. 1Allow OAI-SearchBotOn your siteIn robots.txt, and at your firewall or CDN for OpenAI’s published IP ranges. GPTBot is the separate training crawler.
  2. 2Get the page into BingIndex My Site does thisIndex My Site sends it to Bing with the Bing option, 2 credits per URL including Google.
Start indexing

Microsoft Copilot

Microsoft describes Copilot experiences that rely on Bing’s index, so a page has to be in Bing first. Source

  1. 1Allow BingbotOn your siteAnd leave out noindex, which removes a page from Bing and from Copilot answers.
  2. 2Get the page into BingIndex My Site does thisIndex My Site sends it to Bing with the Bing option, 2 credits per URL including Google.
Start indexing

Gemini

Gemini apps ground answers with content from the Google Search index. Source

  1. 1Allow Googlebot and Google-ExtendedOn your siteBlocking Google-Extended opts the site out of Gemini grounding, and Google Search keeps following your Googlebot rules.
  2. 2Get the page into GoogleIndex My Site does thisIndex My Site sends it to Google for 1 credit, then searches for it to confirm it is indexed.
Start indexing

Perplexity

PerplexityBot surfaces and links pages in Perplexity’s answers. There is no submission form. Source

  1. 1Allow PerplexityBotOn your siteIn robots.txt, and at your firewall or CDN for Perplexity’s published IP ranges.
  2. 2Link to new pagesOn your siteIts crawler finds pages through links, so link new pages from pages that are already well known.

Claude

Anthropic says blocking Claude-SearchBot or Claude-User may reduce a site’s visibility in Claude’s answers. Source

  1. 1Allow Claude-SearchBot and Claude-UserOn your siteClaudeBot is the training crawler. Claude’s search uses Claude-SearchBot.
  2. 2Link to new pagesOn your siteClaude has no submission form, so its crawler has to find the page through links.

Why indexing comes first in LLM SEO

An AI answer is written from search results. Google says AI Overviews and AI Mode may run several related searches for one question and link to the pages they find. ChatGPT can rewrite a question into searches it sends to partner providers. Better writing helps a page get chosen from those results, but a page that is not in them is never considered.

Crawled

The search crawler is allowed in and has visited the page. Robots.txt, firewalls and bot protection can stop this without you noticing.

Indexed

The search engine has stored the page and can return it for a search. New pages can wait days or weeks for this. Submitting them shortens the wait.

Cited

The assistant picks the page from its results and links to it. This is where content quality decides, and only an indexed page gets this far.

Check AI search crawlers can reach your page

Paste a page address to see whether Googlebot, Bingbot, OAI-SearchBot, PerplexityBot, Claude-SearchBot and Applebot may visit it, and which rule decides each one.

We read the site's robots.txt file and nothing else. Crawl rules say who may visit. They do not show whether a page is indexed or cited.

Results

View robots.txt

Get the pages that mention you indexed too

An answer about your product can cite any page that mentions you: a review, a roundup of tools, press coverage, a comparison, or your own LinkedIn and Reddit posts. Each one has to be indexed before it can be cited. Index My Site has no ownership check, so you can submit them all in one job.

Pages worth submitting

  • Reviews and roundups that list your product
  • Press coverage, interviews and podcast pages
  • Guest posts and partner pages
  • Directory and marketplace listings
  • Your own LinkedIn and Reddit posts

What it costs and what comes back

Each URL is checked before it is sent, and pages that cannot be indexed are skipped without charge. A URL costs 1 credit for Google, or 2 with Bing. If Google has not indexed it after the final check, the standard credit comes back.

Index the pages that mention you

Frequently asked questions

Making your pages findable and citable in answers from ChatGPT, Google AI Overviews, Copilot, Perplexity and other assistants. It is also called AEO or GEO. Most of these answers search first and then write, so the first step is that the search system behind them has crawled and indexed your page.

Allow OAI-SearchBot in robots.txt and at your firewall, and get the page into Bing, because ChatGPT queries partner search providers and OpenAI names Microsoft among them. After that, OpenAI says it ranks results by relevance and reliability, and placement is never guaranteed.

Google says a page must be indexed and eligible to show a snippet, and the site must be included in generative AI features in Search Console, which is the default. So allow Googlebot, get the page into Google’s index, and do not block snippets on the text you want quoted.

Allow PerplexityBot in robots.txt and its IP ranges at your firewall. Perplexity has no submission form, so link to new pages from pages its crawler already knows.

No. GPTBot collects content for training. ChatGPT search uses OAI-SearchBot, and OpenAI treats the two separately.

Google says AI Overviews and AI Mode need no special optimizations, and its requirements do not include llms.txt. The crawler documentation from OpenAI, Perplexity and Anthropic linked on this page asks for robots.txt access.

Only the site owner can change those pages, but you can get them indexed. Index My Site accepts any public URL, so you can submit reviews, roundups, press coverage and your own social posts that mention you.

Get your pages into the indexes AI answers use

Create an account, buy credits from $25, and send any public URL to Google and Bing. Each one reports whether it was indexed.