· 10 min read · ai search / llms.txt / ai crawlers
llms.txt explained: what it is, which AI crawlers read it, and how to write one
What llms.txt does, which AI crawlers actually read it, what to put in it, and a copy-and-adapt template with Wideweb's own file as the example.
Turns out 97% of llms.txt files across 137,000 domains got zero requests in a month, and the biggest group of bots fetching the rest were SEO audit tools, not AI engines.
Dima Ogrenich · Lead engineer at Wideweb