What robots.txt.liquid actually is

robots.txt.liquid is a Shopify theme template that renders the /robots.txt file served at the root of every Shopify store. It lives in theme/templates/robots.txt.liquid in the theme code editor. It is not included in themes by default — Shopify auto-serves a default robots.txt when no template is present. You create the template manually when you need to add or override rules. Per Shopify Dev Docs, it cannot be a JSON template — it must remain Liquid.

The file has three components per Shopify's docs²: User agent (which crawler the rules apply to), Rules (Allow / Disallow directives, plus optional crawl-delay), and Sitemap URL (optional; you can add extra sitemaps beyond Shopify's auto-generated /sitemap.xml). The available Liquid objects are robots.default_groups, group.user_agent, group.rules, and group.sitemap. The sitemap object exposes directive (returns "Sitemap") and value (the URL).

Shopify's verbatim warnings — read these first

Two verbatim warnings from Shopify's Editing robots.txt.liquid help page are worth quoting in full: 'Shopify Support can't help with edits to the robots.txt.liquid file' and 'Incorrect use of the feature can result in loss of all traffic.' Both are literal. A misplaced Disallow rule on the wrong user agent block can de-index your entire store. This is the most powerful and most dangerous file Shopify exposes to merchants.

The verbatim default robots.txt.liquid template

Shopify's Dev Docs publish a default Liquid template for robots.txt.liquid. The block iterates over robots.default_groups and renders each group's user agent, rules, and optional sitemap. This is the starting point Shopify recommends — layer your customisations on top of this loop, not in place of it. Replacing the loop with plain text permanently disconnects the store from Shopify's automatic default-rule updates.

liquid Default robots.txt.liquid template — verbatim from Shopify Dev Docs

 {% for group in robots.default_groups %} {{- group.user_agent -}} {% for rule in group.rules %} {{- rule -}} {% endfor %} {%- if group.sitemap != blank -%} {{ group.sitemap }} {%- endif -%} {% endfor %}

Per Shopify Dev Docs²: it is "strongly recommended to use the provided Liquid objects whenever possible" because "The default rules are updated regularly to ensure that SEO best practices are always applied."

What Shopify's default robots.txt blocks (and why)

Shopify's default robots.txt blocks /admin (the merchant admin), /cart (the cart page), /checkout (the checkout flow), /collections/*+* (filtered collection URLs that produce duplicate content), /search (internal search results pages that have no canonical content value), and /policies/ (auto-generated policy pages that Shopify wants to keep tidy in the SERP). Each block exists for a reason — never unblock without understanding the consequence.

The leaf at /shopify-seo/robots-txt-default-rules/ covers each rule individually with the reasoning. The most-misunderstood block is /collections/*+*: the *+* pattern matches the filter URL signature Shopify generates when a customer combines two filters (e.g. /collections/dresses/red+xl). Each combination is a unique URL with thin, duplicate content — blocking the pattern saves crawl budget and prevents duplicate-content downgrades.

The customizations Shopify supports

Per Shopify's help doc, the supported customizations are: allow or disallow specific URLs, add crawl-delay rules for specific crawlers, add additional sitemap URLs, and block specific crawlers entirely. Changes take effect instantly, though crawlers may lag in fetching the updated file. Shopify recommends against replacing the Liquid with plain text — that prevents future automatic default-rule updates from applying.

The most common legitimate customisations: adding a Disallow rule for an uninstalled app that left crawlable URLs (cluster leaf app-leftover routes), adding an image sitemap URL from a third-party app (cluster leaf sitemap directive), and adding AI crawler rules (cluster leaf AI bots).

AI crawlers and the new robots.txt battlefield

In 2026 the active question on robots.txt.liquid is not classical SEO crawlers — it's AI crawlers. GPTBot (OpenAI's training crawler), ClaudeBot (Anthropic's training crawler), PerplexityBot, Google-Extended (Google's AI training opt-out), and Common Crawl all use robots.txt user-agent strings. The strategic decision is not 'block all AI bots' — that closes off Catalog visibility into agents that read your store, plus the OpenAI fetchers that resolve product pages from ChatGPT shopping cards. Cluster leaf 1C-2 walks the decision matrix bot by bot.

The wedge for this leaf is at /shopify-ai-search/ai-crawlers/ — the AI-side complement to this cluster. The two clusters together cover the full SEO + AI crawler decision. Shopify confirms in its crawling-your-store doc that "your store can be indexed by search engines and large language models (LLMs) without signatures" — meaning AI bots crawl by default unless you explicitly block them in robots.txt.liquid.

The ThemeKit preservation gotcha

Per Shopify's Editing robots.txt.liquid help: ThemeKit (Shopify's CLI for theme development) preserves robots.txt.liquid across pushes; admin theme uploads do not. This is the single most-missed operational detail in Shopify robots.txt management — an agency pushes a new theme version through the admin, the robots.txt.liquid file is silently dropped, and the store reverts to default rules. The fix is to use ThemeKit for any theme deployment that includes robots.txt.liquid customisation, or to manually re-add the file after each admin upload.

The leaf at /shopify-seo/robots-txt-themekit/ covers the preservation behaviour, the post-upload audit pattern, and the CI/CD setup for teams that need automated theme deploys without losing robots.txt.liquid.