How many sites publish an llms.txt? We checked 10,000 domains

Published

Short answer

Of the 10,000 most popular domains, 12.4% publish an llms.txt (1,240), and 16.7% of those whose home page answers with HTML. Larger sites publish more often. Sites that publish one block AI search crawlers on 1.4% of cases, against 15.0% for sites that do not. This counts publication only: it does not show whether any AI assistant reads the file.

llms.txt is one of the most discussed files in AI search, and one of the least measured. We already read the robots.txt of the top 10,000 domains, and in the same pass we asked each one for /llms.txt. This is what came back.

On this page
  1. How did we check?
  2. What did we find?
  3. Do bigger sites publish more?
  4. Who publishes one?
  5. What does this not show?
  6. Should you publish one?

How did we check?

We took the top 10,000 domains of the Tranco list (list ID JZWZY) and requested /llms.txt once from each on 21 November 2026, as AISeeWhy's own user agent, after reading the home page and following its redirect to the main host.

A domain counts as publishing an llms.txt when /llms.txt itself answers with a 200, a non-empty text body that is not an HTML page, and no redirect to some other page such as a login or the home page. We did not check the format of the file in this run.

What did we find?

Do bigger sites publish more?

Yes, by a clear margin. We split the websites (domains whose home page answered with HTML) by Tranco rank:

Who publishes one?

Among the 6,192 domains that serve a robots.txt, the ones that publish an llms.txt look different from the rest:

In other words, the sites that bother to publish an llms.txt are mostly the sites that already let AI crawlers in.

What does this not show?

It shows a link, not a cause. A site that takes AI visibility seriously may publish an llms.txt and also keep its robots.txt open; that does not mean the file made the difference. The count also measures publication only. It does not show whether any AI assistant reads the file, and the major AI search crawlers do not say they use it.

The numbers describe the home-page rules and the file at one address on one day, for the most popular domains. Smaller sites may look different.

Should you publish one?

It is cheap and harmless: a short Markdown summary of your site and a list of your most useful pages. Keep it accurate and keep its links working. Do not treat it as a substitute for letting crawlers in: an llms.txt on a site whose firewall refuses the crawler, or whose robots.txt disallows it, helps nobody.

Free tools

Next steps

Frequently asked questions

What share of websites have an llms.txt?

Among the 10,000 most popular domains we checked, 12.4% publish one. Among the domains whose home page answers with HTML the share is 16.7%, and it is higher for the largest sites: 22.6% in the top 1,000.

Do sites with an llms.txt allow AI crawlers more often?

In our data, yes: they block AI search and user crawlers on 1.4% of cases against 15.0% for sites without one. This is a correlation. It does not show that the file changes how crawlers treat a site.

Does an llms.txt help me get cited by AI assistants?

Our data cannot say. It counts which sites publish the file, not whether an assistant uses it, and the major AI search crawlers do not state that they read it. Crawlable pages and open access matter more.

How do you tell whether a site has a real llms.txt?

Request /llms.txt and check that it answers with a 200 at that address, with a text body that is not an HTML page. A redirect to the home page or a login page is not an llms.txt.

Sources

More articles