Skip to content

robots.txt.liquid · ThemeKit gotcha

ThemeKit Preserves robots.txt.liquid. Admin Uploads Don't.

Per Shopify's Editing-robots-txt help page: ThemeKit (and its modern successor, Shopify CLI) preserves robots.txt.liquid across theme deploys; admin .zip theme uploads silently drop the file1. This is the single most operationally dangerous detail in Shopify robots.txt management. An agency pushes a new theme version through the admin, the custom robots.txt.liquid is gone, and the store reverts to default rules — sometimes losing customisations that took hours to validate. The fix is procedural: never use admin uploads on themes that include robots.txt.liquid customisation.

Published Verified 2026-05-22

The gotcha in one sentence

Shopify's robots.txt.liquid file lives in the theme's templates folder. When a theme is uploaded via the admin (Online Store > Themes > Add theme > Upload zip file), Shopify processes the upload through a path that does not include robots.txt.liquid — the file is silently omitted from the new theme. When the same theme is pushed via ThemeKit or Shopify CLI from a local directory, the file is included in the push and preserved. The result: agencies that use admin uploads for deploys lose their robots.txt.liquid customisations on every theme update.

Why the discrepancy exists is undocumented but consistent. The likely explanation: admin .zip upload was originally designed for full theme replacements from the Theme Store, where preserving merchant-customised files would conflict with the "this is a fresh theme" mental model. ThemeKit was built for incremental development where preserving local changes is the whole point.

Why ThemeKit and admin uploads differ

ThemeKit (and Shopify CLI, its modern successor) operate on individual theme files via API calls — pushing or pulling each file independently. A push that includes templates/robots.txt.liquid in the local directory will sync it; a push that doesn't will leave the remote copy untouched. Admin uploads operate on theme bundles — the zip file is treated as the complete theme definition, and Shopify's upload pipeline does not include robots.txt.liquid in the bundle even if it's in the source zip. The asymmetry is unique to this one file; most other custom templates survive admin uploads.

Per the Shopify CLI docs2, the modern tooling preserves the same file-by-file behaviour ThemeKit established. The CLI is Shopify's recommended replacement for ThemeKit as of 2023, and any team currently using ThemeKit should migrate to the CLI for ongoing support — the preservation behaviour is identical.

The operational pattern that survives

The pattern that survives theme deploys: (1) Keep robots.txt.liquid version-controlled in your local theme source. (2) Use Shopify CLI (or ThemeKit) for all deploys. shopify theme push from the source directory deploys every file including robots.txt.liquid. (3) Never use the admin Upload zip file path for production deploys on a theme that includes robots.txt.liquid customisation. (4) If admin uploads are unavoidable (some platforms require them), include robots.txt.liquid as a separate manual step after every upload.

The post-upload audit checklist

After every theme update (regardless of method), verify robots.txt.liquid is intact. Three checks. (1) Online Store > Themes > Edit code > Templates folder — confirm robots.txt.liquid is present in the file list. (2) curl https://yourstore.com/robots.txt and verify your custom rules appear in the output. (3) Use Google Search Console's robots.txt tester to confirm the custom rules render correctly. Do this on every theme deploy, not just the ones you suspect might have dropped the file.

The audit takes 90 seconds and catches the silent-drop failure mode every time. The cost of skipping the audit is potentially weeks of crawled-but-unintended traffic on URLs you thought were blocked, or AI bots crawling content you intended to keep out of training corpora. The asymmetry justifies the discipline.

Agency workflow implications

Shopify agencies deploying themes for clients face this asymmetry every week. Three workflow rules avoid the trap. (1) Standardise on Shopify CLI for every client deploy — no exceptions, even for one-off uploads. (2) Document robots.txt.liquid customisations in the project repo so any developer touching the theme knows what's in the file. (3) Add a post-deploy verification step to every release checklist: curl the rendered /robots.txt and confirm the custom rules are present. Teams that don't enforce these three rules eventually lose a robots.txt.liquid customisation in production.

The cost of failure is asymmetric: a dropped robots.txt.liquid that should have blocked AI bots silently allows them in for weeks until someone notices. A dropped robots.txt.liquid that should have disallowed app-leftover routes silently exposes those routes to indexing for weeks. The asymmetry justifies the workflow discipline.

For the broader robots.txt.liquid context — what goes in the file in the first place — see the cluster hub. For the AI-bot rules most commonly affected by this drop, see /shopify-seo/robots-txt-ai-bots/.

Keep reading