llms.txt: the new robots.txt, and how to write yours

§01what it is

llms.txt is a plain-text file you place at yoursite.com/llms.txt. Like robots.txt tells search engine crawlers what to index, llms.txt tells AI models what your site is about, what content is most important, and how you want to be described.

The spec was proposed in late 2024 and has no formal RFC — it’s a community standard that AI providers have begun adopting voluntarily. The format is intentionally simple: a markdown file with three optional sections — a site description, a list of important pages with descriptions, and a list of excluded pages.

§02adoption numbers

As of Q1 2026, roughly 12% of Fortune 1000 companies have an llms.txt file. Among developer-focused SaaS companies, the number is higher — closer to 28%. Early adopters are concentrated in dev tools, APIs, and AI-native products.

The long tail is much lower. Of the 200 SaaS companies in our benchmark, only 14 had llms.txt files. That means 93% of your competitors probably don’t have one, which makes it an unusually low-competition advantage right now.

§03how each model uses it

Model behavior varies significantly. Here’s what we know:

Perplexity is the most consistent adopter. When Browse mode is active and a site has an llms.txt, Perplexity reads it before crawling and uses the page list to prioritize which URLs to index. This is the most direct citation impact we’ve measured from the file.

Claude (with web browsing enabled) reads llms.txt when it encounters a domain during a browsing session. The site description appears to influence how Claude describes your company in responses — companies with specific, well-written descriptions get paraphrased more accurately than those without.

ChatGPT behavior is inconsistent. Browse with Bing occasionally picks up llms.txt files, but there’s no documented crawling protocol. The impact on training data (which is what actually matters for most ChatGPT citations) is unclear.

Gemini has no public documentation on llms.txt support as of this writing. Anecdotal evidence suggests the file is read but not weighted heavily.

§04how to write yours

Three rules for writing an llms.txt that actually helps:

1. The description should be a definition, not a tagline. “The project management tool built for engineers” is a tagline. “Acme is issue-tracking and project planning software for software engineering teams. It uses a cycle-based workflow, has a keyboard-first interface, and integrates natively with GitHub and Linear.” — that’s a definition. Models use descriptions as entity definitions. Write it like an encyclopedia entry.
2. List your comparison pages explicitly. If you have comparison pages, they should be in the Key Pages section. They’re the most citable pages on your site and you want models to know they exist.
3. Keep it short. llms.txt files that run more than 50 lines start to look like SEO manipulation and may be treated as such. The spec authors recommend a concise file. Cover your 5-8 most important pages and stop.

What not to do: don’t stuff it with keywords, don’t list every page on your site, don’t use the description to make claims (“the best project management tool”) — that reads as self-promotion, not entity description.

§05does it actually move citation rate?

Measured impact from our 14-company sample: companies with an llms.txt file showed an average 11% higher Perplexity citation rate compared to their category peers without one. The effect on other models was smaller and noisier — 4-6% on Claude, no statistically significant effect on ChatGPT or Gemini.

Honest assessment: llms.txt is not a magic lever. The brands with the highest citation rates don’t have it (Stripe, Notion, and Linear all lack llms.txt files as of this writing). Content quality is the primary driver.

But: it takes 30 minutes to write and deploy. An 11% Perplexity lift for 30 minutes of work is a high-ROI activity relative to most content investments. Write yours, then focus on the things that matter more — your comparison pages, your docs depth, your community presence.

And then run a check six weeks later to see if it moved the number.