Three Sources Just Answered "Does llms.txt Work?"
llms.txt is a plain text file at the root of a website that summarises what the site is and points to the pages that matter most, written for AI systems rather than for people. Does llms.txt work the way its proposers hoped? Not yet: Google says it does nothing for rankings, and the log data shows almost no AI crawler ever requests one. It was proposed in 2024, it is not an official standard, and no search engine or AI provider has committed to using it.
llms.txt, in Short
- llms.txt is an unratified, community-run proposal from September 2024, not a search or AI standard.
- Google's own guidance says the file neither helps nor hurts visibility, and Ahrefs' log study found almost no AI crawler ever requests one.
- robots.txt still controls crawler access today; llms.txt controls nothing.
- Publishing an accurate llms.txt costs little and forces a useful self-description exercise, even though it will not by itself earn AI citations.
- What correlates with AI-answer visibility is earned brand mentions elsewhere on the web, not anything published on your own domain.
What llms.txt controls, and what it does not
It is a curation format: it describes a site, and it controls nothing.
The specification, in four facts.
- Proposed
- By Jeremy Howard on 3 September 2024, and published at llmstxt.org
- Governed
- By a GitHub repository and a Discord channel
- Not ratified
- It is not an IETF standard and it is not a W3C recommendation
- Not a control
- It does not control crawler access, which is what robots.txt does, and it does not list a site's URLs for indexing, which is what a sitemap does
llms.txtdescribesWhat the site is, and which pages matter most
robots.txtcontrolsWhich crawlers may access what
sitemap.xmllistsWhich URLs exist, and when they changed
What an llms.txt file looks like
The spec asks for one required element and offers three optional ones.
# Acme Analytics
> Acme Analytics is a hosted product analytics tool for small software teams. It is self-serve and needs no data warehouse.
Pricing is usage based. Support is by email and answered within one business day.
## Docs
- [Quickstart](https://example.com/docs/quickstart): install the SDK and send a first event
- [API reference](https://example.com/docs/api): every endpoint, with request and response examples
## About
- [Pricing](https://example.com/pricing): plans, limits and what counts as an event
- [Security](https://example.com/security): data handling, retention and sub-processors
## Optional
- [Changelog](https://example.com/changelog): release notes, most recent first
- The H1. The site or project name. The only element the spec requires
- The blockquote. A short summary of what the site is, written for a reader with no other context
- The H2 sections. Lists of links, each with a line saying what is behind it
Optional. A section by that exact name, marking links that can be skipped when a shorter context is needed
Does llms.txt work? The evidence
Three sources answer this, and none of them are ambiguous.
“Google Search itself doesn’t use them”
Google’s guidance says you do not need to create machine-readable files, AI text files, markup or Markdown to appear in Google Search, including its generative AI features, and that creating them will neither harm nor help visibility or rankings.
Google Search Central, “Optimizing your website for generative AI features on Google Search”. Published 15 May 2026; the llms.txt subsection added 15 June 2026
97%
Across 137,210 domains, 97% of published llms.txt files received no requests at all in the period studied. Of the requests that did arrive, AI retrieval bots accounted for 1.1%.
Ahrefs, “llms.txt study”, Louise Linehan and Xibeijia Guan, published 15 June 2026, using May 2026 log data
8.7%
8.7% of the Tranco top 1,000 sites publish an llms.txt file, rising to 15.8% among the 549 that responded to the probe. The tracker runs a monthly HTTPS check and has done since June 2025.
Rankability llms.txt adoption tracker, June 2026 datapoint
Google states that it ignores the file, the log data shows that almost nothing requests it, and no vendor documentation confirming that any AI system reads one could be located. On the evidence so far, llms.txt effectiveness for AI search is close to zero.
Why you keep reading contradictory advice
The advice conflicts because the sources conflict, including inside Google.
- Google contradicts itself. Chrome Lighthouse 13.3 ships an experimental llms.txt audit, describing the file as reducing the time agents spend crawling a site to understand it, while Google Search says the file is unnecessary. Reported by Search Engine Journal, 20 May 2026
- The vendor confirmations are not sourced. Pages circulating since April 2026 state that Anthropic and Perplexity have publicly confirmed consuming llms.txt. No primary source could be located for either claim, and the pages asserting it cite none
- Google’s own advice points elsewhere. John Mueller of Google Search Relations said in June 2026 that whether the file does anything is “purely speculative for now”, that not blocking AI agents matters more, and named WebMCP as the more practical direction. Reported by Search Engine Journal, 2 June 2026
So is it worth publishing?
There is a real case for it, and it is worth being precise about what the case is.
It costs almost nothing
One text file, written once, updated when your positioning changes. On most sites it is a short job rather than a project
The downside is nil
If adoption arrives, a site already publishing an accurate file is ready. If it never arrives, you have lost very little
It forces a useful exercise
Writing down what your business does, which pages matter and who is behind them is worth doing whether or not a machine reads the result
It is honest infrastructure
Publishing an accurate description of yourself in a machine-readable place is reasonable on its own terms, whether or not it is currently rewarded
What it will not do is get you mentioned in AI answers, and if that is the problem, this file is not the fix.
If llms.txt is not the lever, what is
Two studies point the same way, and neither of them points at your own domain.
84%
Across more than 25 million links cited by ChatGPT, Claude and Gemini, 84% trace back to earned media rather than brand-owned content. Paid and advertorial content accounted for 0.3%. The finding has held steady across editions.
Muck Rack, “What Is AI Reading?”, May 2026 edition
0.664 vs 0.218
Across 75,000 brands, branded mentions across the web correlated with AI Overview visibility at 0.664, against 0.218 for backlinks. This is a correlation, not a demonstrated cause.
Ahrefs AI Overview brand study, 75,000 brands
AI-visibility tools optimize the website. IndexMesh builds and connects everything else.
Mesh AI, the platform that does the connecting, is in build. The sentence above is a statement about the category, not a claim about a product you can use today.
The half that the evidence above points at is what other sources say about you, and it is not on your domain.
Where llms.txt sits among the standards that shipped
The standards being ratified and shipped in 2026 are the permission and licensing ones, and llms.txt is not among them.
| Standard | Question it answers | Who reads it | Status | Enforced? |
|---|---|---|---|---|
robots.txt | Which crawlers may access what | Search and AI crawlers | Real, published as RFC 9309, universally implemented | Voluntary, and widely honoured by the major operators |
sitemap.xml | What URLs exist, and when they changed | Search engines | Real, universally implemented | Treated as a hint, never a guarantee |
llms.txt | What this site is, and what matters most on it | AI tools, in theory | Proposed September 2024. Community-governed, unratified | No |
llms-full.txt | The full text of the pages worth reading | AI tools, in theory | A de facto convention. The spec’s own name for it is llms-ctx-full.txt | No |
| WebMCP | What actions a page offers an agent | Browsers with agent support | W3C Draft Community Group Report, 10 February 2026. Early preview in Chrome 146 Canary | Not applicable. It is a browser API, not a policy file |
| RSL 1.0 | What licence your content is offered under | Crawlers that choose to honour it | Specification published 10 December 2025 | No enforcement mechanism |
| Cloudflare Content Signals | Which uses of your content you permit | Crawlers that choose to honour it | A robots.txt extension, shipped September 2025 | Advisory only |
robots.txt is the one on this list that changes crawler behaviour today, so if only one of them gets your attention, make it that one.
MCP is worth separating from all of this: it is a protocol for connecting AI clients to tools and servers, its current specification is dated 25 November 2025, and it is not a file you publish on a website.
How to add llms.txt to a WordPress site
Three routes, in rough order of effort.
Your SEO plugin may already generate a basic file
Check yours before you install anything. Compare the editorial controls, not whether /llms.txt exists.
Yoast SEO Free can generate llms.txt automatically: recent and cornerstone content, a physical file refreshed weekly, limited manual page selection.
AIOSEO Lite generates one by default, with an optional llms-full.txt.
Rank Math Free provides a configurable LLMS Txt module.
SEOPress offers an editable virtual file in PRO.
That may be enough if you want a hands-off content index.
Write it by hand
It is a text file at /llms.txt. Markdown headings, a short description, and links to the pages that matter. On a small site this beats anything generated automatically, because you know which pages actually matter.
Use a plugin
Several exist, including IndexMesh for WordPress . Reads: whichever of the four providers above is active, and whether it is already generating a physical /llms.txt file. Adds: guided sections for site identity, offerings, positioning, key pages, authority and AI guidance, with per-item pinning and exclusion, a full Markdown preview, and a check on which version is actually being served, at both /llms.txt and the .well-known convention path. Never touches: a file it did not create. It detects the physical file, names the current owner on the Status screen, and defers to it by default; taking over the route is a separate, confirmed action, not automatic.
We would rather you knew that four plugins you may already own can do this than install ours believing it is the only option.
Having an llms.txt file is not the same as controlling what it says.
Which of those plugins currently owns /llms.txt on your site, and what each one lets you change, is a longer answer than a page about the format should give. It is on the feature page , four providers row by row. If your SEO plugin already generates llms.txt for free, here is when that is enough and when it is not: see Your SEO Plugin Already Makes llms.txt. Here's When That's Not the Whole Job.. Already running a dedicated llms.txt plugin instead? See Looking for a Website LLMs.txt Alternative That Covers More Than llms.txt?.
Is your file the one being served?
On a WordPress site, more than one thing can claim the same public route.
The question is not whether you configured a file. It is whether yours is the one being served. When Yoast's llms.txt feature is enabled it writes a physical file to the web root, and a real file at the root can take precedence over the /llms.txt route WordPress generates on the fly. Without conflict detection, you could edit one version while visitors and crawlers receive another.
IndexMesh for WordPress detects that mismatch and shows which source owns /llms.txt.
When it finds one, the plugin defers to the current owner by default, and offers a documented way to take the route over with the original file backed up first. It never overwrites a file it did not create.
- Once the plugin owns the file
- Yoast's own documentation says it will not overwrite an existing llms.txt, so a scheduled refresh should not reclaim it
- What deferring does not touch
- Deferring applies to the root route. /.well-known/llms.txt and the read-only REST payload are separate surfaces and are unaffected by a root conflict
Built by people who hold their own plugin to this page's standard
IndexMesh's founders, Xavier Emerson and Leenat Rose, built IndexMesh for WordPress's llms.txt feature to the same evidence-first rule this page argues for: it generates and serves the file, detects when another plugin already owns the route, and defers rather than overwriting anything it did not create. Publishing an accurate llms.txt is worth doing on its own terms, whether or not any AI system reads it, and that is the standard the plugin itself is held to.
Common questions
Eight questions people ask before deciding whether llms.txt is worth their time.
Does Google use llms.txt?
No. Google Search Central states that creating these files "will neither harm nor help your site’s visibility or rankings in Google Search". The guide was published on 15 May 2026 and the llms.txt subsection was added on 15 June 2026.
Do AI crawlers actually fetch llms.txt?
Mostly not. Ahrefs published a log study on 15 June 2026 finding that the great majority of published llms.txt files were never requested by anything, and that requests from AI retrieval systems were a small fraction of the few that did arrive.
Have OpenAI, Anthropic or Perplexity said they use llms.txt?
No documentation confirming it could be located. Pages circulating since April 2026 assert that Anthropic and Perplexity have publicly confirmed consuming the file, and cite nothing for either claim. Anthropic does publish an llms.txt for its own documentation, which is a different thing from reading yours.
What does an llms.txt file look like, and do I need llms-full.txt too?
Markdown: an H1 with the site name, then a blockquote summarising the site, then H2 sections listing links. A section headed Optional marks skippable links. llms-full.txt carries full page content instead, under the name llms-ctx-full.txt; if the short file is not read, the long one probably is not either.
Is llms.txt the same as robots.txt?
No. robots.txt controls which crawlers may access which parts of a site, and the major operators honour it. llms.txt describes what a site is and controls nothing. Different jobs, and only one of them has an effect today.
Where should llms.txt live, and is /.well-known/llms.txt an official location?
At the root of the domain, at /llms.txt. Some tools also serve a copy at /.well-known/llms.txt, including IndexMesh for WordPress. RFC 8615 defines the /.well-known/ prefix and a registry of names within it, and llms.txt is not registered there, so that path is a convention rather than a standard location.
Will llms.txt improve my search rankings?
No. Google has stated directly that Google Search ignores the file. Anyone claiming a ranking benefit is not describing documented behaviour.
Is llms.txt dead?
Unadopted rather than dead. The spec is maintained and sites keep publishing files, but no major AI provider has documented reading one, so the accurate description is a proposal that has not been taken up yet.
Last updated
- Evidence last checked:
- Editorial guidance last reviewed:
We review this reference at least quarterly and sooner when the underlying evidence changes.
Google's guidance on llms.txt changed twice in the eight weeks before this was written, and WebMCP is still moving.
Publishing llms.txt without hand-editing a file
Everything on this page is something you can do by hand.
The IndexMesh WordPress plugin does it for you and then checks the result: it generates llms.txt from your content, serves it at the root and the .well-known path, detects when another plugin is already producing one, and tells you whether the file is genuinely served rather than only saved.
Free, and it works alongside your existing SEO plugin.
We will tell you when it is ready.
One email when it is ready. After that, only when there is something genuinely worth reading. Unsubscribe in one click.