How llms.txt Works And Why It Matters
What the file is, what goes in it, and why most sites are missing it.
Source: https://contentcucumber.com/research/how-llmstxt-works-why-matters/
By Brent Peterson, CEO, Content Cucumber. Issue 43. Published 2026-04-23
What the file is, what goes in it, and why most sites are missing it.
In this issue
- What Goes in an llms.txt. The format is simple. A text file at the root of the site.
- The Allow vs Block Decision. There's a deliberate choice available in robots.txt.
- What This Looks Like In Practice. An audit of four ecommerce sites turned up every possible state.
- Most Operators Don't Know About Either File. In conversations with ecommerce operators. Platform teams, agency owners, brand managers.
- What an AI Audit Misses. The three failing sites took about an hour each to fix.
- Check Your Own Sites. Open a new tab. Type a domain followed by /llms.txt.