No, llms.txt probably won’t change how your products show up in AI search right now. It’s a proposed text file you’d place at the root of your domain, written in plain markdown, meant to give AI models a quick summary of what your site is and where the useful content lives. The idea sits next to robots.txt in concept, a simple file that tells a crawler what to look at, except nobody has confirmed that ChatGPT, Gemini, Perplexity or Google’s AI Overviews actually read it or act on it.
For an online store owner weighing up where to spend an afternoon, that matters. You could add an llms.txt file to your Shopify or WooCommerce setup this week and there’s no solid evidence it would shift whether your product pages get pulled into an AI answer. The reasons are worth understanding though, because they point at what these AI tools do rely on, and that’s a more useful place to put your time.
Table of Contents
Where Llms.txt Came From
The llms.txt proposal came from Jeremy Howard, a data scientist, in 2024. His pitch was simple: websites are messy, full of navigation, ads and scripts, and that makes it hard for an AI model to pull out what actually matters. An llms.txt file would act like a clean, short briefing document instead.
The name borrows the shape of robots.txt, which has told search crawlers what they can and can’t index since the 1990s. But the two files do different jobs entirely.
| File | Purpose | Backed by official standard |
|---|---|---|
| robots.txt | Tells crawlers what to skip | Yes, since 1994 |
| sitemap.xml | Lists pages for indexing | Yes, supported by Google and Bing |
| llms.txt | Summarises site content for AI models | No, still a proposal |
That last row is the detail worth sitting with. No search engine and no AI lab, not OpenAI, not Google, not Anthropic, has said they crawl llms.txt or give it any weight in how they answer a query. John Mueller from Google has commented on this directly, and his view is worth reading if you want it straight from someone inside the search engine, in Does llms.txt Help? What John Mueller Says.
How the File Is Supposed to Work
The file sits at yoursite.com/llms.txt, written in plain markdown so a model can scan it without parsing a full web page. Most versions list the site name, a short summary of what the business sells, and links to key pages such as docs, category pages or a product catalogue. The whole point is to give an AI model a clean text version of your store instead of making it wade through menus, pop-ups and tracking scripts.
For an online retailer, that usually means picking a handful of pages that matter most and listing them in order of importance. A typical file might include:
- A one-line description of the business and what it sells
- Links to your top category or collection pages
- A link to shipping and returns info, since AI tools often get asked about this
- A link to your FAQ page if you have one
Keep the file short. There’s no required length, but most working examples run under 200 lines, because the file is meant to be a summary, not a mirror of your sitemap.
Do AI Crawlers Even Read It
No major AI company has confirmed it parses llms.txt files. OpenAI, Google and Anthropic have all stayed quiet on the file despite it circulating for close to two years, which is the real question a retailer needs answered before investing time.
Some third-party AI tools and developer frameworks do reference llms.txt in their own setup docs, but that’s a different thing from GPTBot or Google-Extended actually fetching and using the file when they crawl your store. Those crawlers still lean on the same signals they always have.
- GPTBot reads standard HTML and respects robots.txt rules
- Google-Extended follows normal crawl paths and structured data markup
- Perplexity’s crawler behaves similarly, prioritising rendered page content
- No major crawler log has shown confirmed llms.txt fetches at scale
One analysis of over 137,000 sites found that 97% of llms.txt files go unread by the bots that matter, a figure worth sitting with before you spend an afternoon building one, as detailed in We Analyzed 137K Sites: 97% of llms.txt Files Never Get ….
What Shoppers’ AI Tools Rely On Instead
AI Overviews and similar tools pull from structured data that’s already proven to work, not from a text file sitting quietly at your root domain. Product, Review and FAQ schema markup gets read and used because it’s tied to established standards that Google and others have supported for years.
Clear product and category copy matters just as much. A page that explains what a product does, who it suits and what makes it different gets pulled into AI summaries far more often than a hidden file nobody’s confirmed is being read. Site speed and crawlability still count too, since AI crawlers behave a lot like the traditional bots that have indexed the web for two decades.
- Add Product and Review schema to every product page
- Write FAQ content in plain language, then mark it up with FAQ schema
- Fix slow-loading pages before touching anything experimental
- Check that category pages have real, readable copy, not just a grid of thumbnails
This is the work ScaleCurrent’s AI SEO service focuses on, because it’s backed by evidence rather than a proposal still waiting on adoption.
Should You Still Add One Anyway
Adding an llms.txt file costs little time and carries no known downside, so there’s no real harm in doing it. The catch is that it won’t fix crawl errors, close schema gaps or rewrite thin product pages, which are the things actually shown to affect how AI tools surface a store.
Treat it as a small, low-cost experiment rather than a plan you build a quarter around. If you’ve got twenty minutes spare after the higher-impact work is done, it’s a reasonable thing to tick off.
- Write a short site summary and list your top five pages
- Upload the file to your root domain as plain markdown
- Move on to schema markup and page speed, where the evidence actually sits
- Revisit the file every few months in case adoption changes
If you’re not sure where your store stands on the things that do matter, a free AI check or Search Audit through getting in touch will tell you faster than guessing at a proposal with no confirmed uptake.
Chasing the Trend or Fixing What Is Broken
New file-based SEO ideas move fast on social media, long before any platform confirms they do anything. A developer posts a screenshot, a thread gets shared, and suddenly it looks like everyone’s store needs one. That speed is the whole reason llms.txt has a following with no confirmed adoption behind it.
The real decision for a retailer is where dev hours go this quarter. Spend them building a speculative file, or spend them fixing crawl errors, thin category pages and missing schema, the things already shown to affect whether AI tools pull your products into an answer. One path is a bet on a proposal still waiting for a search engine to back it. The other is work with a track record.
| Choice | Time cost | Payoff if wrong |
|---|---|---|
| Build llms.txt now | An afternoon | Nothing, no known harm either |
| Fix crawl and schema gaps | Days to weeks | Measurable visibility gains |
| Guess which gaps matter | Ongoing, repeated effort | Wasted budget, no clear result |
A free AI check or local SEO style audit removes the guesswork from that last row. It tells you which gaps are actually costing you visibility before you touch anything experimental, so the dev time you do spend goes where it’s proven to count.
What To Fix First This Quarter
Pick one store, your own, and get it checked before you decide anything. A free AI check or Search Audit will tell you whether your product pages carry schema, whether category pages read as real copy, and where crawl errors are quietly costing you visibility. That’s a short, factual list, not a guess. Once you have it, the order of work is obvious: confirmed fixes first, an llms.txt file last if you still want one. ScaleCurrent runs these checks for online retailers who’d rather know than guess. Email hello@scalecurrent.com.au with your store URL and ask what’s actually affecting your AI visibility, then spend your hours there.
Want more customers from Google and AI search? Tell us about your business.
Llms.txt Questions Store Owners Still Have
Will adding llms.txt hurt my SEO in any way?
No. There’s no confirmed case of an llms.txt file causing a ranking drop, a crawl problem, or any kind of penalty. The file sits quietly at your root domain and does nothing if no bot reads it. The only real cost is the time it takes to write it, not any risk to your existing search or AI visibility.
Can I use llms.txt instead of schema markup?
No, they solve different problems and one doesn’t replace the other. Schema markup is a coding standard that Google and other platforms have supported for years, and it’s what actually gets read when an AI tool pulls product details, reviews or FAQs into an answer. Llms.txt is a plain text summary with no confirmed readership. If you only have time for one, schema wins.
How do I know if AI crawlers are even visiting my store?
Check your server logs or a log-file analyser for user agents like GPTBot, Google-Extended or PerplexityBot. Most hosting dashboards and tools like Cloudflare let you filter traffic by user agent, so you can see hits even if they don’t show up in standard analytics. If none of those names appear over a few weeks, AI crawlers simply aren’t reaching your pages yet.
Is there a plugin or app that builds llms.txt automatically?
A few third-party apps and developer tools have started generating llms.txt files for Shopify and WordPress stores, usually by pulling your sitemap into a plain text summary. None of them are official releases from Shopify, Google or any AI lab, so treat them as convenience tools, not proof the file gets used. Writing one by hand takes about the same amount of time anyway.
