What Is llms.txt? A Plain Definition for Business Owners
llms.txt is a proposed file in Markdown (plain text with a few formatting marks) that a website can publish at /llms.txt to give AI tools a short summary of the site and links to its best pages.
Key points
- llms.txt is a proposal, not a web standard, and the only required part is a top heading with the site's name.
- Google says Search, including AI Overviews and AI Mode, ignores the file, so it neither helps nor hurts you there.
- The research asks three separate questions: was the file fetched, were its contents used, was the site cited. Fetches and citations have been measured, content use hasn't, and nothing shows the file earning citations.
What is llms.txt?
In Markdown, a line starting with `#` is a heading, and `[words](address)` is a link. The file is voluntary. It doesn't block or allow anything, and no AI company is obliged to read it.
Jeremy Howard published the llms.txt proposal in September 2024 and revised it as version 2 in August 2026. His reason is that an HTML page "wraps its information in navigation, ads, and JavaScript, and converting it back into clean text is difficult and imprecise." A short list of clean links spares a tool that cleanup.
He expected the file to "mainly be useful for inference rather than training, and that is how it has been used." Inference is a trained model producing an answer, and the pages a tool fetches at that point give it context to work from. Training is building the AI model in the first place.
The proposal also says where the file can live. A file at the root, /llms.txt, covers the whole site. One in a folder, such as /docs/llms.txt, covers only that folder, since "a file covers the URLs under its path." For a service business site, one root file covers every page.
What does a real llms.txt file look like?
Hypothetical example: a plumbing company choosing its essential pages asks which ones answer what a customer needs to know before calling. That gives the business name as the top heading, a one-sentence summary of what it does and where, and groups of links to its service pages, service area, any public pricing and contact. In the file's format those are the H1, the blockquote and the H2 sections. A blog post that answers a common customer question, such as what to do about a burst pipe, belongs in those main sections too. Only pages a customer could skip go under Optional.
The proposal's own example, written for a software tool called FastHTML, shows those parts. Here it is shortened, with one block of notes and two sections left out and part of the summary cut (marked ...):
```
FastHTML
FastHTML is a python library ... for creating server-rendered hypermedia applications.
Docs
- FastHTML quick start: A brief overview of many FastHTML features
```
- The H1 heading (`# FastHTML`) names the site or project. It's the only required part.
- The blockquote (the line starting with `>`) is a short summary holding the facts a reader needs to make sense of the rest.
- Notes can follow as plain paragraphs or bullet points, with no headings.
- H2 sections such as `## Docs` hold the file lists. Each line is a link, then optionally a colon and a note on what the page contains.
- The Optional section (not shown here) is, by convention, for "secondary information: links an agent can skip when a shorter context is needed."
The quick start link ends in `.md` because the proposal also suggests offering clean Markdown copies of key pages. That's a separate job, and how hard it is depends on your website platform.
How is llms.txt different from robots.txt and a sitemap?
| File | What it does | Does it control crawler access? |
|---|---|---|
| robots.txt | Tells crawlers which URLs on your site they may crawl | It gives instructions that well-behaved crawlers follow, but can't force them |
| Sitemap | Gives search engines information about your pages and files and how they relate | No |
| llms.txt | Summarizes your site and points AI tools to the pages worth reading | No |
The proposal says llms.txt "is instead used on demand, when an agent needs information about a topic while assisting a user." It can't let ChatGPT's search crawler, OAI-SearchBot, into your site or keep it out. That's done in robots.txt, and OpenAI's crawler documentation also asks sites to allow requests from its published IP ranges, so a firewall or hosting setting can still block a crawler that robots.txt allows.
Do ChatGPT, Perplexity and Google actually use llms.txt?
None of the studies below shows that they rely on it, and Google says its Search doesn't. Keep three questions apart when you read the research. Was the file fetched, were its contents used, and was the site cited?
Was it fetched?
Ahrefs' llms.txt study looked at 137,210 domains in Ahrefs Web Analytics that had traffic in May 2026.
- 28% of those domains published an llms.txt file. Ahrefs treats that as an upper bound, because its customers "skew more technical and SEO-aware than the web at large."
- 97% of those files received zero traffic that month.
- OAI-SearchBot, PerplexityBot and Claude's search crawler together made "only a couple of hundred fetches across thousands of sites."
Ahrefs also found no AI bots requesting llms.txt on sites without one, which it says "kills the assumption that AI systems actively hunt for llms.txt files."
Were the contents used?
None of the studies here measures that. As Ahrefs puts it, "a bot requesting your llms.txt isn't proof anything read or acted on it."
Was the site cited?
SE Ranking's study looked at nearly 300,000 domains. Of those, 10.13% had an llms.txt file.
It compared that with how often each site appeared as a source in sampled AI responses. Having the file "didn't make a domain more likely to be cited by AI models."
The page is dated October 2026 and doesn't say when its data was collected. SE Ranking calls its findings context-dependent, so one dataset showing no relationship doesn't rule out an effect elsewhere.
The two adoption figures count different sites, and neither tells you how many local businesses have the file.
What does Google say?
Google's guide to succeeding in generative AI features on Google Search says you don't need "AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn't use them." It adds that an llms.txt file made for other services "will neither harm nor help your site's visibility or rankings in Google Search." What Google does require is that a page be indexed and eligible to appear in Search with a snippet.
Should a local business publish an llms.txt file?
Publish one only if you have a reason, such as a tool you use that reads it. Being able to serve the file isn't a reason. If you do publish, it must pass the checks below before launch, and whoever edits your service pages keeps it current.
- Don't pay for custom work done only to win citations in ChatGPT, Perplexity, Gemini, Claude or Google AI Overviews. The evidence above doesn't support it.
- Keep one your platform already generates or a tool you use reads, as long as it passes those checks. Developer documentation sites publish it, Anthropic's among them.
- Treat it like any page that states facts about you. A stale file sends any tool that reads it to wrong or missing pages.
How do you check an llms.txt file before it goes live?
- Open your domain followed by /llms.txt in a browser. You should see plain text starting with `#` and your business name, not a styled web page, an error page or a redirect to your homepage.
- Read the summary line. It should say what you do and where, in words that match your website and your Google Business Profile.
- Copy each URL into your browser. The links may show as plain text, not clickable. Each one should open the exact page its note describes.
- Check the location. A root file covers the whole site. A file in a folder covers only that folder.
- Name an owner. Whoever edits your service pages updates the file at the same time.
Run these five checks on any file you publish. On a hosted website builder, ask whoever manages the site whether the platform can serve a plain-text file at that address before anyone writes one.
Even an accurate llms.txt won't fix a site that OAI-SearchBot can't crawl or that Google hasn't indexed. A wrong one can cost you a job if a tool that reads it sends a prospect to a service you've stopped offering. Read Vector Index's look at whether llms.txt works to see which basics to fix first (crawler access, accurate business facts, useful service pages, a clear way to inquire) and when a file is worth testing as an experiment.
FAQ
Is llms.txt required to show up in AI search?
No. It's a voluntary proposal. Google says no AI text file is needed to appear in Google Search, including AI Overviews and AI Mode, and none of the studies above shows ChatGPT, Perplexity, Gemini or Claude requiring one.
Does ChatGPT read llms.txt?
The studies above don't show that it relies on the file. In Ahrefs' data, OAI-SearchBot, PerplexityBot and Claude's search crawler combined made only a couple of hundred fetches across thousands of sites, and a fetch doesn't show the contents were used.
Can llms.txt hurt my Google rankings?
No. Google says the file will neither harm nor help your visibility or rankings in Google Search.
What is llms-full.txt?
A companion file some sites publish with much more of their content in one place. Anthropic's developer documentation links to one from its llms.txt "for more comprehensive documentation."