Shopify's new default robots.txt: what changed in 2026

The practical consequence is the part worth reading twice: creating a robots.txt.liquid template now opts your store out of the managed file and pins it to the older rules. That inverts the advice everyone (this site included) has given for years, which was that keeping the robots.default_groups loop meant Shopify's updates flowed through to you automatically.

Shopify's own help page still documents the older "key entries", including /search and /policies/⁴, so an audit that quotes the help page will be wrong for most stores.

# Shopify storefront. Public product, collection, page, blog, policy, cart, and localized HTML is crawlable.
# Agent instructions: https://theme-dawn-demo.myshopify.com/agents.md
# UCP discovery: https://theme-dawn-demo.myshopify.com/.well-known/ucp
# UCP/MCP endpoint: https://theme-dawn-demo.myshopify.com/api/ucp/mcp
# Agents should use UCP/MCP for catalog, cart, and checkout. Payment requires buyer approval.
#
User-agent: *
Allow: /
Disallow: /admin
Disallow: /cart/
Disallow: /checkout
Disallow: /orders
Disallow: /account
Disallow: /services
Disallow: /sf_*
Disallow: /cart.js
Disallow: /recommendations/products
Disallow: /collections/*sort_by*
Disallow: /collections/*+*
Disallow: /collections/*filter*&*filter*
Disallow: /*?*preview_theme_id=*
Sitemap: https://theme-dawn-demo.myshopify.com/sitemap.xml

User-agent: adsbot-google
Disallow: /checkout

# we use Shopify as our ecommerce platform

User-agent: *
Disallow: /admin
Disallow: /cart
Disallow: /orders
Disallow: /checkouts/
Disallow: /checkout
Disallow: /collections/*sort_by*
Disallow: /collections/*+*
Disallow: /search
Disallow: /policies/
Disallow: /*/policies/
Disallow: /services/login_with_shop
Disallow: /.well-known/shopify/monorail
Sitemap: https://www.example.com/sitemap.xml

User-agent: Nutch
Disallow: /

User-agent: AhrefsBot
Crawl-delay: 10

User-agent: MJ12bot
Crawl-delay: 10

User-agent: Pinterest
Crawl-delay: 1

Rules the managed file drops

Rules the managed file adds

One detail worth a look if you rely on the filter block: in the older set the rule is written Disallow: */collections/*filter*&*filter*, with no leading slash. Google's specification describes rule values as paths beginning with /⁶, and the managed file writes the same rule correctly as /collections/*filter*&*filter* plus a localized /*/… variant. We did not test how Googlebot treats the slashless form.

The managed file also drops every third-party bot group. On the loop output, Nutch is disallowed outright and AhrefsBot, AhrefsSiteAudit, MJ12bot and Pinterest each get a crawl delay². The managed file has only * and adsbot-google.

If neither line matches, someone has rewritten the template by hand. Seven of the 38 stores we checked were in that state, with output matching between 14% and 89% of the documented default rules. You can paste the file into the robots.txt checker, which reports which default you're on and flags rules that went missing.

1. /search is crawlable, and that is how the noindex works

On every Online Store front we checked, in both classes, /search?q=shirt returned HTTP 200 with X-Robots-Tag: noindex, nofollow. A bare /search carried no such header. Google is explicit that a page must be crawlable for a noindex to be seen⁵, so on a templated store the Disallow: /search rule actively hides that noindex from Googlebot. Blocked URLs can still be indexed from links, without the noindex ever being read.

2. A template pins your rules

As long as the template exists, the loop prints the older set. Any future change Shopify makes to the managed file passes you by. That is the opposite of the old guidance.

3. The file now talks to AI agents

The managed file's comment block points agents at /agents.md, /.well-known/ucp and /api/ucp/mcp, and states that payment requires buyer approval. Those are comments, so no crawler is bound by them, but they are a clear signal of where Shopify expects agent traffic to go.

4. The bot groups are gone

If you relied on Shopify's default to slow Ahrefs or block Nutch, the managed file does neither. Add your own group if you want that behaviour, with the trade-off that adding a template pins everything else too.

On one large DTC store, three lines in the live file fuse a rule with the next directive. The pattern, with the store's hashed rule shortened:

Disallow: /*/collections/*/products/*-[a-f0-9]...-remoteDisallow: /collections/*-loop
Disallow: /*/collections/*/products/*-[a-f0-9]...-remoteSitemap: https://www.example.com/sitemap.xml

A parser reads …-remoteDisallow: /collections/*-loop as one rule with a nonsense path, and the store's Sitemap: line disappears into the rule above it. The model that fits every case we saw: the user_agent and rule objects print a line break before themselves, while the sitemap object prints one after, so a template that appends its own rules the way Shopify's documentation shows can leave the last rule and the following directive on one line³. That is inferred from output, not from documentation.

The fix is to make the template end every printed line itself. The generator writes it that way, and its checker flags a pasted file that has fused directives.

• Shopify's Dawn demo store served the old file in the Internet Archive's 2025-11-15 snapshot and the managed file in its 2026-06-23 snapshot. There is no snapshot in between⁷.
• One store in our sample (olaplex.com) has snapshots with an unchanged content digest from 2025-11-01 to 2026-03-27, all of them the old file. It serves the managed file today⁸.

So the switch reached those two stores somewhere between late March and late June 2026. Stores do not have to have moved at the same time, and we can't tell from the outside whether the rollout was gradual.

• Sample: 38 storefronts. Shopify's two demo stores plus 36 well-known merchants. Hand-picked, not random, so the class shares here do not estimate the share across all Shopify stores.
• User agent: ShopifyRankedResearch/1.0 (+https://shopifyranked.com/shopify-seo/robots-txt-managed-default/).
• Classification: managed = the managed header line and two groups. loop = output covering 100% of the documented older rule set. custom = a template whose output covers only part of it (14–89% in this sample).
• Limits: one fetch per store, from one country, on one day. Redirects were followed to the canonical host. A store can change its template at any time.

Raw files, the classification CSV and the header results are kept with this note's working data. The classification table below is published under CC BY 4.0: reuse it with a link to this page.

Header checks, three stores per class: every managed and loop store returned X-Robots-Tag: noindex, nofollow on /search?q=shirt. None of the three custom stores did; all three are custom or headless storefronts where Shopify's Online Store search isn't what answers that URL.

• Whether deleting a robots.txt.liquid template moves a live store back to the managed file. Expected, untested.
• Whether the rollout was gradual, and whether any store still receives the old file without a template.
• Whether Shopify intends the managed file to keep changing. It is "managed", so assume yes.
• The exact line-break behaviour of the Liquid objects. Our model fits every case we saw, but Shopify documents none of it.

• 2026-09-16 — First published. 38 storefronts fetched; Wayback bounds for the switch; line-fusion evidence from two stores.