# What is the llms.txt file? Definition and business implications

> llms.txt is a proposed web standard introduced in September 2024 by Answer.AI, providing AI models with a structured map of a website's content. Placed at the root of the domain, it is not a formal standard and its adoption by major LLMs remains uneven in 2026.

## Definition
The llms.txt format is inspired by robots.txt but plays an inverse role: instead of forbidding access to certain parts, it orients AI models towards content deemed strategic. It lists, in markdown, the URLs of main resources with a short description of each. Adoption in 2026 is uneven. On the publisher side: Anthropic, Stripe, Vercel, Cloudflare, Cursor, Hugging Face publish their llms.txt. Yoast SEO has integrated it into its WordPress plugin. On the consumer side: some developer tools, such as Cursor through its @Docs feature, can index it; consumer LLMs (ChatGPT, Claude.ai, Perplexity) do not formally commit, even if their crawlers sometimes fetch it. In June 2026, Google clarified that the file is not needed for its search engine and influences neither visibility nor ranking in Google Search. A SERanking study from November 2025 on 300,000 domains found no statistical correlation between the presence of a llms.txt and improved citations in AI answer engines.

## Example
Anthropic publishes its llms.txt on platform.claude.com (the former docs.anthropic.com address redirects there), with a standard structure: an introduction describing the documentation, then thematic sections (models and pricing, API reference, SDKs, release notes) listing each important page, often with a short description. The file is complemented by an llms-full.txt containing the full text of the documents for direct ingestion. Pattern reproduced notably by Vercel, LangChain and Cloudflare. For a mid-cap without API documentation, the stake is different: a useful llms.txt lists the strategic URLs (product pages, glossary, case studies, terms and conditions) rather than an exhaustive technical library.

## For a leader
“Should we deploy a llms.txt on our site, and what can we expect concretely?” This question, asked of your IT director or SEO provider, deserves an honest answer. Three sub-questions to chain. First, are we a company whose users query AI about our offer (technical documentation, API, product sheet)? If yes, llms.txt has immediate value. For others, it is an option, not a priority. Second, what is the opportunity cost? Deploying a llms.txt takes little work, because the file is a plain text document. The cost argument carries little weight. Third, what indicators will we track? Without a measured baseline (citations in Perplexity, AI Overviews before deployment), you will never know if the investment paid off. The WYP rule: deploy, measure, do not oversell.

## FAQ

**What is the llms.txt file?**
It is a proposed web standard introduced in September 2024 by Answer.AI that gives AI models a structured map of a site's content. Placed at the root of the domain, it lists in markdown the URLs of the main resources with a short description. It is not a formal standard, and its adoption by major LLMs remains uneven in 2026.

**How does it differ from robots.txt?**
It plays an inverse role. Where robots.txt forbids access to certain parts of the site, llms.txt instead orients AI models towards the content deemed strategic.

**Do major LLMs actually read it?**
Adoption is uneven. Some developer tools, such as Cursor through its @Docs feature, can index it, but consumer LLMs (ChatGPT, Claude.ai, Perplexity) do not formally commit to it. In June 2026, Google clarified that it is not needed for its search engine and influences neither visibility nor ranking. A SERanking study from November 2025 on 300,000 domains found no statistical correlation between its presence and improved citations.

**Should we deploy one?**
The real question is: do your users query AI about your offer (technical documentation, API, product sheet)? If so, llms.txt has immediate value. For others, it is an option, not a priority.

**What does it cost and what can we expect?**
Deploying it takes little work, as it is a plain text file: the cost argument carries little weight. But without a baseline measured before deployment (citations in Perplexity, AI Overviews), you will not know whether it paid off. The WYP rule: deploy, measure, do not oversell.

## See also

- [GEO (Generative Engine Optimization)](https://wyp.agency/en/glossary/geo/)
- [AEO (Answer Engine Optimization)](https://wyp.agency/en/glossary/aeo/)
- [Schema.org for AI](https://wyp.agency/en/glossary/schema-org-ai/)
- [Citability](https://wyp.agency/en/glossary/citability/)
- [Usage discipline](https://wyp.agency/en/glossary/usage-discipline/)

## Sources

- llms.txt proposal, maintained at llmstxt.org, introduced by Jeremy Howard, Answer.AI, September 2024 (https://llmstxt.org/)
- Anthropic documentation, llms.txt implementation example (https://platform.claude.com/llms.txt)
- Google's Guide to Optimizing for Generative AI Features on Google Search, Google Search Central, 2026 (https://developers.google.com/search/docs/fundamentals/ai-optimization-guide)
- Google Search Central, documentation updates log: FAQ rich result discontinued (7 May 2026) and llms.txt note (June 2026) (https://developers.google.com/search/updates)

---
Entry from the WYP.agency AI glossary · https://wyp.agency/en/glossary/llms-txt/ · Last updated : 2026-10-14