Technical SEO
2026-07-23 · 7 min read
Sapun Lamichhane
Founder & CEO of Arcetis
llms.txt is a proposed convention (documented at llms.txt) for a plain markdown file, served at a site's root, that gives AI systems a direct, structured summary of what the site offers — its real services, content, and key pages — instead of leaving an AI system to infer that from parsing marketing-oriented homepage HTML.
It's the AI-era analogue of robots.txt and sitemap.xml: robots.txt tells crawlers what they're allowed to access, sitemap.xml tells them what pages exist, and llms.txt tells an AI system what those pages are actually about, in a format built for machine reading rather than human browsing.
“An AI system summarizing what a business does from its homepage HTML alone is working from marketing copy. The same system reading a structured llms.txt file is working from a direct, unambiguous inventory.”
Ours is generated directly from the same real content arrays that power the visible site — services, industries, case studies, the Learning Center, glossary, comparisons, and published posts — so it can't drift out of sync with what's actually published; there's no separate, manually maintained version to go stale.
It links out to a companion file, llms-full.txt, with expanded detail for AI systems that ingest full page content rather than just a link summary — including complete framework guide content, not just links to it, since a system doing full-content ingestion benefits from having that content directly available rather than needing to follow another link to get it.
An AI system summarizing what a business does from its homepage HTML alone is working from marketing copy — headlines, hero text, calls to action — written to persuade a human visitor, not to be parsed cleanly by a machine. The same system reading a structured llms.txt file is working from a direct, unambiguous inventory: here are the services, here's what each one actually is, here's where to find more.
That distinction matters most for anything nuanced enough that marketing copy alone would misrepresent it — the difference between an SEO service page's promotional framing and the actual scope of what's included, for instance. An llms.txt file gives an AI system a place to get that scope stated plainly rather than reconstructing it from persuasive copy.
A short, direct description of the business — what it actually does, not a tagline. A clear list of services or products with real, unpromotional one-line descriptions. Links to substantive content (case studies, guides, glossary) rather than just navigation pages. And — if it's true — an explicit statement of any verifiable credentials, since an AI system summarizing a business benefits from being told what's actually checkable, the same way a human reader would.
What we wouldn't recommend: treating it as another marketing surface. An llms.txt file padded with promotional language defeats its own purpose — the value is specifically in being a plain, structured, unpersuasive summary a machine can trust more than it trusts a homepage.
Adoption is still early and not universal — it's a proposed, community-driven convention rather than a formally required standard any AI system must support. We publish one because the cost of doing so is low and the potential upside (a cleaner, more accurate machine-readable summary) is real, not because every system currently guarantees it reads one.
No — they serve different purposes. Structured data marks up specific page content for search engines and AI systems parsing that page; llms.txt gives a sitewide, plain-text summary independent of any single page's markup. Both together are stronger than either alone.
Generated from the site's actual real content wherever possible, exactly like ours — a manually written, separately maintained file drifts out of sync with the live site over time, which defeats the purpose of giving AI systems an accurate summary.
