What is llms.txt, and does it help SEO?
Over 844,000 websites have added a file that no AI platform has ever confirmed reading.
llms.txt is a text file that lists a website’s key pages in Markdown, for AI tools to read. It does not help SEO. Google has confirmed it does not use llms.txt for rankings or AI Overviews, and no major AI provider has confirmed using it either.
Markdown is a plain-text format that uses symbols like # and * instead of visual styling, so both people and machines can read it without special software.
That doesn’t make llms.txt pointless. A small number of AI coding tools do read it, and it costs almost nothing to create. Here’s what it does, why it caught on despite the lack of evidence, and when it’s worth your time.
Quick answer
llms.txt does not help you rank in Google Search or appear in AI Overviews. Google has confirmed this directly.
- It’s not a ranking signal, and Google has no plans to make it one
- A small number of AI coding tools read it by convention
- It costs almost nothing to create, so it’s low-risk if you want one anyway
What does an llms.txt file contain?
An llms.txt file lives at yoursite.com/llms.txt. It lists a site’s most important pages in Markdown, with a short description under each link. Each link follows a fixed pattern, title, URL, then a one line description. AI agents parse this mechanically, so the format has to stay consistent.
Pages usually sit under headings like “Docs” or “Guides,” with an “Optional” section for lower priority pages an agent can skip if it’s running low on context.
Some sites also publish an llms-full.txt. This version pastes the full content of each page directly into the file, which is useful when a coding tool needs to read a whole documentation set in one fetch instead of following dozens of links one by one. Cursor and Claude Code both use this file when they’re working against a documentation site.
A lot of people assume llms.txt works like robots.txt and lets you block pages with a disallow rule. It doesn’t. There’s no standard way to disallow anything in llms.txt. If you want to keep a page out of it, you just leave it off the list.
If you’re on WordPress, Yoast SEO and Rank Math can both generate and refresh this file for you automatically, so it isn’t necessarily something you have to build by hand.
Does llms.txt help SEO?
No. Google has said directly that llms.txt does not affect rankings or visibility in Google Search, including AI Overviews and AI Mode.
On June 15, 2026, Google added a section to its official AI optimisation guide titled “Clarifying guidance on llms.txt files.” It states that these files aren’t needed to appear in Google Search, and that keeping one “will neither harm nor help your site’s visibility or rankings in Google Search, as Google Search ignores them.”
This isn’t a new position. Google’s Search team has said for over a year that it doesn’t use llms.txt, going back to comments from Gary Illyes at Search Central Live in July 2025. The June update exists because Google wanted to, in its own words, “address questions from the community” after the file kept getting sold as an AI visibility tactic by SEO tools and agencies.
“Will neither harm nor help your site’s visibility or rankings in Google Search.”
Google Search Central, AI optimisation guide
The data backs this up. Ahrefs tracked 137,210 domains with a valid llms.txt file and found 97% received zero requests for it in May 2026. Of the small number that were fetched, most requests came from GPTBot and Claude Code rather than anything tied to search ranking or citation.
SE Ranking ran a statistical model across roughly 300,000 domains and found no correlation between having an llms.txt file and getting cited in AI answers. Removing the file from their model actually made it more accurate.
No other major AI provider has confirmed using it either. OpenAI’s own crawler documentation controls GPTBot and ChatGPT-User through robots.txt and never mentions llms.txt.
Why did llms.txt become popular if it doesn’t help rankings?
Two things happened at once. A single platform update made adoption automatic for thousands of sites, and AI search made SEOs anxious enough to grab any lever that felt like control.
The mechanical trigger came first. In November 2024, Mintlify added llms.txt support across every documentation site it hosts, so sites like Anthropic and Cursor had one overnight without anyone deciding to add it.
SEO tools and WordPress plugins followed. Some started flagging a missing llms.txt the same way they’d flag a missing sitemap, turning a choice into an error to fix. By October 2025, BuiltWith counted more than 844,000 live llms.txt files, despite no major AI platform ever confirming it reads them.
The psychological trigger is simpler. Ahrefs’ Ryan Law has summed it up well – people want to influence how LLMs see their brand, and right now there aren’t many tools that let them. llms.txt gave anxious marketers something to do, whether or not it worked.
Neither reason is new. SEO chased AMP pages the same way in the early 2010s, and schema markup went through a similar rush before it settled into a smaller, sensible role. llms.txt is following the same arc.
So, who actually uses llms.txt?
AI coding tools are the main users of llms.txt. Cursor, Windsurf, Claude Code, GitHub Copilot, Cline and Aider all check for /llms.txt or /llms-full.txt when a developer points them at a documentation site, so they can find the right pages fast instead of crawling the whole site and guessing.
That’s why the file shows up mostly on developer-facing sites. Stripe, Vercel, Cloudflare, Anthropic and Supabase all publish one, mainly so their own docs are easier for coding agents to read.
The scale is still small. Ahrefs tracked 137,210 domains with a valid llms.txt file and found only 3% were ever fetched at all. Of those, most requests came from GPTBot and Claude Code. GPTBot fetching a file is routine crawling behaviour though, so it doesn’t confirm OpenAI uses llms.txt to decide what to cite. Limy.AI logged more than 500 million AI bot events over 90 days and found only 408 targeted an llms.txt file.
Chrome Lighthouse checks for the file too, as part of an audit category called Agentic Browsing, which looks at how ready a site is for browser agents to operate it. That’s a separate concern from search ranking, and it accounts for a tiny fraction of the fetches Ahrefs recorded. John Mueller summed up the real use case well when he called llms.txt a temporary crutch, perhaps to save some tokens, for AI coding tools reading developer docs.
“A temporary crutch, perhaps to save some tokens.”
John Mueller, Google Search Advocate
Should you create an llms.txt file?
For most sites, no, not as an SEO move. It costs little to make, but there’s no evidence it affects rankings, traffic or AI citations, and Google has said directly that it ignores the file.
The exception is documentation-heavy sites and developer tools. If your audience already uses AI coding assistants like Cursor or Claude Code to work with your product, an llms.txt file gives those tools a faster way to find your docs. That’s a real, if narrow, benefit, and it’s the only case backed by actual usage data rather than hope.
If you’re running an ecommerce site, a local business, a blog or most other typical sites, your time is better spent on the things that do move AI citations and rankings – clear content that answers questions, solid technical SEO, and pages that demonstrate expertise. None of that changes because llms.txt exists.
If you decide to make one anyway, it’s low risk. Keep it short, list only your best pages with honest one-line descriptions, and skip it entirely if you don’t have the time to maintain it. A stale llms.txt pointing at moved or deleted pages is worse than no file at all.
Is there a real alternative to llms.txt?
Google’s real investment in AI agents right now isn’t llms.txt. It’s a browser standard called WebMCP, currently in early testing in Chrome, which lets a website tell an AI agent exactly what it can do rather than making the agent guess. It’s early, and mostly relevant to developers today, but if you want a signal of where Google’s attention is going, it’s there, not in a text file Search has already said it ignores.
Okay, so llms.txt isn’t the answer. Now what?
Fair question. New AI standards are going to keep showing up, WebMCP is just this month’s version. Chasing each one as it lands isn’t a strategy, and it’s an expensive one if it’s eating time you could spend on things that work.
What matters most is simpler than any of this. Is your content structured in a way AI can find and trust? Are you showing up in the answers people are already getting, or is a competitor getting cited instead of you?
Those are answerable questions, and we can answer them for you.
An llms.txt file is a plain text file, written in Markdown, placed at the root of a website. It lists a site's most important pages with short descriptions, so AI tools can find relevant content without crawling the entire site. Jeremy Howard proposed the format in September 2024.
No. Google confirmed in its June 2026 AI optimisation guide that Search does not use llms.txt for rankings, AI Overviews, or AI Mode. Having one will neither help nor hurt your visibility, since Google Search ignores the file entirely.
robots.txt is an official, decades-old standard that tells crawlers which pages they can access. llms.txt is a newer, unofficial proposal that summarises a site's best pages for AI tools to read. robots.txt controls access. llms.txt works more like a curated guide, and no major search engine treats it as a directive.
Not for SEO specifically. It won't affect your rankings or AI citations, since Google has confirmed it ignores the file. It's worth creating if you run a documentation-heavy or developer-focused site, since AI coding tools like Cursor and Claude Code do read it against technical docs.
Yes. robots.txt can block specific AI crawlers by naming their user agent, such as GPTBot or Google-Extended, in a disallow rule. This is different from llms.txt, which has no blocking function at all. If you want to keep AI bots out, robots.txt is the right tool.
Create a plain text file named llms.txt, add an H1 with your site name, an optional one-line summary, then H2 sections listing your key pages as Markdown links with descriptions. Upload it to your site's root directory. WordPress users can also generate one through Yoast SEO or Rank Math.