An llms.txt file is a short Markdown document at the root of your site (yoursite.com/llms.txt) that tells AI language models who you are and which of your pages hold the answers worth quoting. It's a 2024 proposal, still short of a standard, and Google says its Search, AI Overviews included, doesn't use it. Nothing requires one. Writing it takes minutes and can't hurt, and any system that does read it gets a clean map instead of a guess, which is why this site publishes one.
Robots.txt and sitemaps leave one gap
Robots.txt tells crawlers what they may read. A sitemap tells them what exists. Neither says what anything means, and a language model working with limited context gets more from a short summary written by a person than from a raw list of URLs. Jeremy Howard of Answer.AI proposed llms.txt in September 2024 to fill that gap. It's a Markdown file that states your site's purpose and lists its most important pages, one line each, so a model arriving at your domain can get its bearings in a single read.
Inside, it's a page of plain Markdown
The format is kept simple on purpose. An H1 with your name, a one-line summary, then sections of links, each with a short description. This is the shape, adapted from ours:
# Houtos Labs
> Evaluation-led web, app & web3 studio. We don't guess. We grade.
## Key pages
- [The Grader](https://houtoslabs.com/grader.html): free website audit across six lenses
- [What is SEO?](https://houtoslabs.com/what-is-seo.html): the complete plain-English guide
- [Methodology](https://houtoslabs.com/methodology.html): how the grading rubric works
Be choosy. A good llms.txt says who you are, then lists only the pages you'd want quoted: the ones that sell and the ones that answer. Give each a description a model could lift word for word.
What it can do, and what it can't
- It does hand a clean, curated map to any system that reads it. Which assistants read it, and how, is mostly undocumented; Google has said its Search doesn't.
- It does make you edit. Deciding which pages count as your canonical answers sharpens the site itself.
- It doesn't control anything. No crawler is obliged to read it, and no major engine has committed to using it for ranking or citation. Robots.txt is voluntary too, as Google's own documentation points out, but it's a long-established standard that reputable crawlers follow.
- It doesn't replace the fundamentals. If a model can't read the pages themselves (content that needs scripts to appear, crawlers you've blocked), a tidy index of those pages won't rescue you.
Write yours in four steps
- List your ten most important pages. Start with who you are and what you offer, then add the answers you most want engines to quote.
- Give each page one plain line. Say what question it answers, in the words a customer would use.
- Format it as Markdown. Your name as the H1, a blockquote summary, then link lists grouped under headings. Save it as plain text.
- Publish it at the root (yoursite.com/llms.txt) and update it whenever a key page changes. Treat it as the sitemap's editorial sibling.
Writing the file takes five minutes. The bigger job is building a site that AI systems can retrieve and are willing to quote. Our GEO guide and AEO guide cover that job from start to finish.
Common llms.txt questions
Do AI companies use llms.txt?
Few say so publicly. Google has said its Search, AI features included, doesn't use llms.txt, and no other major engine has committed to honoring it. Writing one is still a cheap bet: it costs minutes, whatever does read it gets a cleaner map of your site, and picking your canonical pages is worth doing for its own sake.
Is llms.txt the same as robots.txt?
No. Robots.txt states what crawlers may read; llms.txt describes what matters and what each key page answers. Neither one can force anything. Google's own documentation says robots.txt can't enforce crawler behavior, though reputable crawlers follow it. A crawler can obey your robots.txt and never open your llms.txt.
Will llms.txt improve my search rankings?
No. Google says an llms.txt file neither helps nor harms visibility or rankings in its Search, and it wasn't designed to. It exists to orient language models. File it under cheap AEO and GEO work, not under SEO tactics.
What should I put in my llms.txt file?
One line on who you are. Then your key pages, the ones that sell and the ones that answer, each with a one-line description a model could quote as written. Ten to thirty links beat three hundred. The value is in what you leave out.
An index only helps if the pages behind it can be read
Before you write an llms.txt, check the pages it will point to. That's the question the Grader's Findability lens is built for, and the grade is free. It runs in sample mode for now, so the result is a labeled estimate until live scanning reaches the lens.
Grade my site The SEO & AEO audit