A Route Check Alone Isn't AI Crawler Reading Proof
If you run a WordPress site, IndexMesh for WordPress can confirm your routes and files are reachable, and, where server logs or referral data exist, identify individual crawler requests. That is not AI crawler reading proof of indexing, an answer citation, or a future visit.
AI Crawler Reading Proof, in Short
- A route check confirms your
robots.txt,llms.txt, and sitemap are servable; it does not confirm any crawler actually requested them. - Server access logs are the only evidence that names a specific crawler visit, and most hosts only retain them for days to weeks.
- Referral data confirms a person clicked from an AI answer to your site, not that the crawler itself read the page being cited.
- No AI provider currently publishes a way to confirm a specific page was used in a specific AI answer.
Most founders ask this after they've already set crawler rules in robots.txt. The rules feel like the finish line, but a rule sitting in a text file and a bot actually requesting your pages are two separate facts.
What GPTBot, ClaudeBot, and PerplexityBot actually do when they visit
Not every AI crawler that reaches your site is doing the same job. GPTBot collects training data for OpenAI's models. ClaudeBot does the same for Anthropic. PerplexityBot builds the index Perplexity's search answers draw from. None of the three confirms a citation, and none of them tells you anything about how an AI system will eventually describe your business.
A request from one of these crawlers only confirms that it fetched a page. It says nothing about whether that page will be quoted, summarised, or ignored. The full breakdown of what each AI crawler does and what blocking it costs covers the difference between training, search-index, and retrieval crawlers in more detail than fits here.
What Counts as AI Crawler Reading Proof?
Three kinds of evidence exist, and they answer different questions.
Server access logs. Your host's raw access logs record every request, including the user agent string. A GPTBot visit log is usually a matter of asking your host for raw access logs and searching them for GPTBot, ClaudeBot, or PerplexityBot. Most shared hosts keep a rolling window of a few days to a few weeks, so this only works if you check before the log rotates.
Referral data. AI referral tracking looks at your analytics for traffic where the referrer is chatgpt.com, perplexity.ai, or a similar domain. This confirms a person clicked through from an AI answer to your site. It does not confirm the crawler itself ever requested the specific page that answer cited, and many AI surfaces send no referrer at all, so an empty report does not mean no traffic happened.
Route and file checks. This is what WordPress Site Health, and the IndexMesh for WordPress plugin's own AI crawler controls diagnostics, actually do: they request your robots.txt, llms.txt, sitemap, and homepage the way a crawler would, and confirm the response is real, not a 404, a redirect, or a caching layer serving something stale. That confirms your site is servable. It is not the same as confirming a specific crawler read it.
None of these three is AI bot analytics in the dashboard sense of "here is a graph of every bot that visited this month." That kind of aggregated bot analytics requires either a log-analysis tool pointed at your raw server logs or a security plugin that already inspects every request, and it is a separate purchase from anything on this page.
What a route check does not prove
A check that your robots.txt and llms.txt return a real 200 response proves those files are being served correctly. It does not prove GPTBot, ClaudeBot, or PerplexityBot has read them, and it does not prove your content has been used to answer anyone's question. IndexMesh for WordPress makes this distinction on purpose: its diagnostics check that a route responds, using a bundled offline taxonomy to recognise known crawler user agents where they appear in requests it makes to your own site. It stores no request logs and no IP addresses, and it does not claim to track incoming bot traffic, because a route responding correctly and a crawler actually requesting it are not the same fact, and neither one is a promise that your content shows up in an AI citation.
This is a deliberate limit, not a missing feature. A plugin that claimed to prove AI citations would be promising something no one, including the AI providers themselves, currently documents as verifiable from the outside.
Checking AI crawler reading WordPress site logs and referral data
If you want request-level evidence rather than a route check, here's the order that actually works.
- Ask your host for raw access logs, not just an analytics summary. Most control panels have a raw log download under a hosting or metrics section.
- Search the log for the crawler token you care about.
GPTBot,ClaudeBot, andPerplexityBotare the three most commonly cited names; the full crawler reference lists the rest, including which ones respect robots.txt and which don't reliably. - Cross-check the requesting IP against the operator's published ranges, since a user agent string can be spoofed. OpenAI, Anthropic, and Perplexity each publish verification lists.
- Check your analytics for AI referral traffic separately, filtering for
chatgpt.com,perplexity.ai, and similar domains in the referrer field. - Confirm your routes are actually reachable first, since a bot visit log full of blank rows sometimes just means your CDN or caching layer never let the request through. WordPress Site Health and the IndexMesh for WordPress plugin's route checks catch that failure mode before you go looking for it in a log.
Why this page states limits instead of promises
IndexMesh's founders, Xavier Emerson and Leenat Rose, built the free IndexMesh for WordPress plugin around the same rule this page follows: publish accurate signals, check that they're actually being served, and never promise a ranking, a citation, or a particular AI provider's behaviour. That's a deliberate choice, not a hedge. No one outside OpenAI, Anthropic, or Perplexity can currently confirm what happens to a page after their crawler reads it, so a page or a plugin claiming otherwise would be stating something it cannot know.
AI Crawler Reading Proof FAQ
How do I know for sure if GPTBot has visited my WordPress site?
Check your host's raw access logs for the GPTBot user agent string, and cross-check the requesting IP against OpenAI's published verification range, since a user agent alone can be spoofed. A route-check tool like WordPress Site Health confirms your files are servable, not that GPTBot specifically requested them.
Can IndexMesh for WordPress tell me which AI crawlers have visited my site?
No. It checks that your robots.txt, llms.txt, sitemap, and homepage respond correctly, the way a crawler would request them, but it stores no request logs and no IP addresses. It confirms your routes are reachable, not who has read them.
What's the difference between a crawler visit and an AI citation?
A visit means a crawler requested a page. A citation means an AI system quoted or referenced that page in an answer. Logs and referral data can confirm a visit. No major AI provider publishes a way to confirm a specific page was used in a specific answer.
Does AI referral tracking actually work?
Partially. When someone clicks a link from ChatGPT, Perplexity, or a similar tool into your site, your analytics can sometimes show the referring domain. Several AI surfaces send no referrer at all, so a blank report doesn't mean no one clicked, it can just mean the referrer was stripped.
Why doesn't the IndexMesh plugin just show me a bot analytics dashboard?
Because that would require logging every request against your site, which the plugin is built specifically not to do. It makes outbound requests only to your own site, to check that your files are being served, and nothing it collects ever leaves your server.
Set your crawler rules before you go looking for evidence
Checking for a visit only matters once your robots.txt actually says what you intend. The free IndexMesh for WordPress plugin groups AI crawlers by what they do, shows you the exact output it serves after you save, and checks that your files are really being served, without collecting or storing anything about who requests them.
Every AI crawler, what it does → · What a blanket block actually costs → · A free Block AI Crawlers alternative →