What llms.txt is
llms.txt is a proposed standard (llmstxt.org) for a Markdown file at the root of your site that gives AI assistants a curated map of your most useful pages. A crawler working through your HTML sees navigation, scripts and cookie banners. A model reading llms.txt sees a short summary and a list of the pages worth reading, with a note on each.
The format
# Project name
> One or two sentences on what this is.
Optional paragraphs with more context.
## Docs
- [Quick start](https://example.com/start): install and first run
- [API reference](https://example.com/api): every endpoint
## Optional
- [Changelog](https://example.com/changelog)- H1 title. The only required part. It must be the first line.
- Blockquote summary. A short
>line right after the title. - H2 sections of link lists, each item
- [name](url): note. - An Optional section for links a tool can skip when space is short.
Mistakes this catches
- A catch-all route serving your HTML homepage at /llms.txt with a 200.
- Prose or bare URLs where list items should be links.
- Several links squeezed into one list item.
- Links to pages that have moved or no longer exist.
Does it help?
No major AI company has committed to reading llms.txt, so treat it as cheap insurance rather than a ranking factor. Coding assistants and agent tools are the most common readers today, which makes it most useful for developer products, APIs and documentation sites. It takes an hour, helps the tools that do read it, and doubles as a useful index for people. Many sites also publish llms-full.txt with the full text in one file.
Write yours with the llms.txt generator, and check AI crawlers can reach your pages with the robots.txt tester.