llms.txt is a proposed standard: a markdown file served at your website's root (yourdomain.com/llms.txt) that gives AI systems a curated index of your most important pages — a site name, a one-line summary, and sections of annotated links. It's like a sitemap written for language models instead of search crawlers.

Do Google, OpenAI or Anthropic use llms.txt?

Not officially. No major AI search engine has committed to consuming llms.txt, and Ahrefs found the vast majority of llms.txt files receive no bot requests at all. It is however read by developer tools and AI agents, costs nothing, and takes minutes to add — a cheap forward-looking bet, not a ranking lever.

What Is llms.txt? (And Should Your Website Have One?)

Q: Is llms.txt the same as robots.txt?

No. robots.txt controls which crawlers may access which paths — it's about permission. llms.txt is about guidance: it tells AI systems which pages matter most and what your site is about. They live side by side at your site root and do different jobs.

By the SiteBeat team · Updated 3 July 2026 · 5 min read

Quick answer: llms.txt is a proposed standard — a markdown file at yourdomain.com/llms.txt that gives AI systems a curated, annotated index of your most important pages. No major AI search engine officially consumes it yet, but it costs nothing, takes ten minutes, and is read by a growing set of AI agents and developer tools. Add one; just don't expect it to be a ranking lever.

What does an llms.txt file look like?

The llms.txt spec is deliberately simple: an H1 with your site name, a blockquote summary, then sections of links with one-line descriptions:

# Acme Widgets

> Acme makes industrial widgets and publishes weekly widget-maintenance guides.

## Products
- [Widget catalogue](https://acme.com/products): Full product range with specs.

## Guides
- [Widget maintenance 101](https://acme.com/guides/maintenance): The definitive maintenance guide.

The idea: language models have limited context windows, and your website is full of navigation, cookie banners and boilerplate. llms.txt hands an AI the map — what matters, where it is, what it covers — in a format models parse trivially.

Who actually reads llms.txt today?

Honest answer: fewer systems than the hype suggests.

Not (officially) the big engines. Google's John Mueller has said Google doesn't use it, and neither OpenAI nor Anthropic has committed to consuming it for search. An Ahrefs study found the overwhelming majority of published llms.txt files received zero requests from AI bots.
Yes: developer tools and agents. Coding assistants and AI agents (Cursor, Claude Code and others) fetch llms.txt when working with documentation sites, and adoption among docs-heavy products is real.
Adoption is growing but minority — surveys put it around 10% of domains, skewed heavily toward developer-facing sites.

So should you add one?

Yes — with calibrated expectations. The case for: it's free, it's ten minutes of work, it can only help AI systems that do look for it, and it signals technical care. The case against adding it instead of real fixes: none of the engines citing you today depend on it. If your content is JavaScript-rendered or your robots.txt blocks OAI-SearchBot, an llms.txt fixes nothing — do the fundamentals first.

How do you create one?

Manually: follow the format above — site name, one-line summary, 10–25 of your most important pages with descriptions. Save as llms.txt at your web root, served as plain text.
Automatically: a SiteBeat scan builds one for you from your actual crawled pages — real titles, real URLs — as part of the AI readiness Fix-it kit, along with a corrected robots.txt and prefilled JSON-LD schema.

Get your llms.txt generated from a real crawl

Free scan shows your AI readiness grade · the €29 audit includes the full Fix-it kit

Scan my site →

Frequently asked questions

Is llms.txt the same as robots.txt?

No. robots.txt is about permission (which crawlers may fetch which paths). llms.txt is about guidance (which pages matter and what they cover). They coexist at your site root.

Does llms.txt improve my ranking in ChatGPT or Perplexity?

There's no evidence of that today. Treat it as cheap future-proofing and agent-friendliness, not as GEO's main lever. Crawler access, server-rendered content and quotable structure move the needle far more.

What about llms-full.txt?

An unofficial variant that inlines your full content into one file so agents can ingest everything in a single fetch. Useful for documentation sites; overkill for most others.