llms.txt is a plain-text file at the root of a website (yoursite.com/llms.txt) that gives AI models a short, structured summary of what the site is, what it offers, and which pages matter most. It's written in plain Markdown, meant to be read by a language model rather than a browser, and it exists because models have a limited context window and no patience for navigation menus, cookie banners or marketing copy. Think of it as an index card left on top of the filing cabinet, so the model doesn't have to open every drawer.
Who came up with it, and is it a real standard?
The proposal comes from Jeremy Howard (Answer.AI), published in September 2024, using the model of robots.txt and sitemap.xml: both plain-text files sitting at a site's root that exist purely to talk to machines. It is not an official W3C or IETF standard, and no major AI lab has committed to crawling it on a fixed schedule. It's closer to a convention: something enough sites and enough tools have adopted that it's become worth doing, the way favicon.ico became a de facto default long before anyone standardized it.
What does one actually look like?
Short. A single H1 with the site or product name, a one-line blockquote summary, then a few H2 sections listing links with a short description each: typically "Core", "Docs" or "Key articles", and sometimes an "Optional" section for secondary material. Kiwa's own file (kiwastudio.com/llms.txt) is under 400 words: what the studio does, links to the homepage, the GEO service page, the blog, and a running list of the articles worth reading first. No fluff, because fluff is exactly what the format exists to cut.
How is it different from robots.txt and sitemap.xml?
| robots.txt | sitemap.xml | llms.txt | |
|---|---|---|---|
| Audience | Crawlers (rules) | Crawlers (URL list) | Language models (context) |
| Format | Directives | XML | Markdown, human-readable |
| Purpose | Allow/block access | List every URL that exists | Explain what matters and why |
| Content | Rules, no meaning | Locations, no meaning | Summary + curated links |
robots.txt decides who's allowed in the door. sitemap.xml lists every room in the building. llms.txt is the one-paragraph brief someone hands you before the tour, so you know what you're actually looking at.
Do AI engines actually read it and use it?
Some do, some don't, and none guarantee it. Tools built specifically to ingest documentation for coding assistants (several IDE and agent products) look for llms.txt directly, and some now expect it from libraries and APIs. General-purpose answer engines (ChatGPT, Claude, Perplexity, Gemini) have not published a commitment to fetch it as part of every answer. What's true across all of them: none of it works if the underlying pages aren't otherwise crawlable and citable. llms.txt is a shortcut on top of a well-structured site, not a substitute for one. A model that can't reach or trust your actual pages won't be rescued by a tidy index pointing at them.
Will it help my SEO or AI visibility?
Directly, no measurable ranking effect has been demonstrated: it isn't a ranking signal in the way structured data or page speed are. Indirectly, it does two things worth having: it removes ambiguity for the tools that do read it, and writing one forces you to state, in one paragraph, what your business is and does. That paragraph often turns out to be missing or buried on the homepage itself. The exercise of writing llms.txt is frequently more valuable than the file.
How do you write one?
- Write the one-line summary first. What you do, for whom, in one sentence a stranger would understand with zero context.
- List the pages that actually matter. Homepage, your core service or product pages, docs if you have them. Not every page; curation is the point.
- Add one line of description per link. Not a title restated: what a model would need to know to decide whether to cite that page.
- Keep a "key articles" or "key docs" section current. Update it when you publish something worth surfacing; delete entries for pages that no longer exist.
- Put it at the root, as plain text, at /llms.txt. No auth wall, no JavaScript rendering required to read it.
Where to start
If your site has ten pages or fewer, write it in twenty minutes and move on. If your value proposition takes you longer than a paragraph to explain, that's the more urgent problem to fix; llms.txt only exposes it, it doesn't solve it. We write one for every client site we build, alongside the crawlability and schema work that actually determines whether a model can act on what it says.
A tidy index card won't fix a messy filing cabinet.
Request a GEO audit