Shopify Rollouts A/B Testing: How to Track Which Variant Won

TL;DR

Shopify's native A/B testing shipped in June 2026 as part of Rollouts. Theme experiments report four rates on the rollout's own page; checkout experiments report exactly one number, checkout conversion rate [2]. That is the entire native readout. Nothing tells GA4, Meta, or your order data which variant a visitor saw: the web pixel event context carries no theme or experiment field [3], and Shopify documents no order-level variant marker. If you run experiments as replacement themes, you can close the gap yourself with a variant label wired into a GA4 custom dimension and a cart attribute. Theme-edit experiments allow only a partial version of that, and checkout-config experiments cannot be self-instrumented at all.

Key Takeaways

• Rollouts is available on Basic plan or higher; experiments (the A/B testing type) require Grow plan or higher [4].
• A theme experiment reports four metrics: conversion rate over time, bounce rate, reached-checkout rate, added-to-cart rate. A checkout experiment reports one: checkout conversion rate [2].
• Launching a rollout immediately collects no analytics at all, and the change becomes permanent. Never use "launch immediately" for anything you intend to measure [2].
• No pixel, app or custom, can natively distinguish variants: the Web Pixels event context contains document, navigator, and window data, and no theme, rollout, or experiment field [3].
• You cannot change Liquid templates inside a rollout. Code-level tests mean building a separate theme and running it as a replacement [5].
• For replacement-theme experiments, a hardcoded label in each theme, sent to a GA4 custom dimension and written to a cart attribute, gives you per-session and per-order variant truth.
• Rollouts supports online store checkouts only; headless and custom storefront checkouts cannot be tested [4].

What is Shopify Rollouts, and which plan do you need?

Rollouts arrived on June 5, 2026: scheduling, gradual publishing, and A/B testing for "your themes and checkout and customer accounts configurations" [1]. A rollout is a scheduled set of changes to your theme, checkout and accounts pages, or both; an experiment is "a type of rollout for testing a new strategy by comparing control and treatment groups" [4].

The plan split trips people up, because most coverage flattens it. Shopify's help documentation states rollouts are available on Basic plan or higher, while experiments require Grow plan or higher [4]. So a Basic store can schedule and gradually publish changes, but the compare-two-groups testing type is gated one tier up.

Two mechanics matter for measurement before we get to the tracking gap. First, when you create an experiment, a copy of your published theme or config is created automatically, so you keep editing the live one independently [1]. Second, traffic assignment is a single percentage of eligible visitors, set with presets or a slider [5]. The only visitor-assignment rule Shopify documents is that a visitor gets at most one rollout per resource, with overlapping rollouts sharing traffic proportionally [6]. How assignment persists, for how long, and what happens when a visitor clears storage: none of that is documented today.

What does Shopify show you natively?

Less than you probably assume. A theme experiment's results page reports four rates: conversion rate over time, bounce rate, reached-checkout rate, and added-to-cart rate. A checkout and accounts experiment reports checkout conversion rate, and that is the complete list [2]. Revenue, average order value, and orders are reported for launch-type rollouts, not for experiments. No export and no API for these numbers is documented, and Shopify publishes no statistical-significance methodology for calling a winner.

One sharp edge deserves its own warning: if you choose to launch a rollout immediately rather than schedule it, Shopify collects no analytics for it whatsoever, and the changes are permanent [2]. An experiment you meant to measure, launched the fast way, measures nothing.

Archiving has teeth too. Archiving a rollout permanently deletes its theme changes, and you must archive unended rollouts before publishing a new theme or checkout configuration [6].

The gap: nothing tells you which variant a session or order got

Here is the part Shopify's materials do not address. There is no documented way to see variant assignment anywhere outside the rollout's own results card. Not in GA4. Not in any pixel. Not on the order.

This is verifiable at the schema level. The Web Pixels API event context, the complete browser-side data a pixel receives, contains document fields (location, referrer, title, character set), navigator fields (language, user agent, cookies enabled), and window dimensions [3]. No theme id. No theme name. No rollout or experiment field. Both variants serve the same URLs, so a tracking pixel sees identical events from control and treatment. Any tracking app is in the same position: the event stream carries nothing theme-specific. And the order record is no better; Shopify documents no order-level field marking which variant produced a purchase.

The consequence: you get Shopify's four rates (or one, for checkout tests), and if you want to know whether the treatment moved revenue, AOV, returning-customer rate, or anything else in your GA4 or ad-platform data, you have to carry the variant signal yourself. Whether that gap stays open is worth watching; the feature is six weeks old, and everything above is what the documentation says as of July 17, 2026.

How do you label the variant in GA4?

For replacement-theme experiments, cleanly. Shopify closes the in-place door: "You can't change Liquid templates as part of a rollout" [5]. A test involving template changes therefore means building a separate theme and running the experiment against it as a replacement. That replacement theme is one you created, with code you control, which makes instrumentation a one-line job.

Hardcode a label. In the treatment theme's layout, set a variant marker, for example checkout-cta-v2; in your live theme, control. A hardcoded label per theme beats echoing theme.id or theme.name: Shopify's Liquid theme object exists but is deprecated, with values explicitly "subject to change" [7], and a label you chose reads better in reports anyway.

Then move it into GA4. If your GA4 tag runs in the theme (gtag or a dataLayer setup), attach the label as an event parameter and register it as an event-scoped custom dimension. The mechanics of registering one are covered in GA4 custom dimensions for Shopify. If your GA4 events come from a sandboxed pixel instead, the theme cannot reach into the sandbox directly; the documented bridge is a first-party cookie or localStorage value, which a custom pixel can read through the standard API's browser module [8]. That path works when you control the pixel code. App pixels only carry the label if the app exposes a field for it. For background on why the sandbox forces this dance, see the Web Pixels API guide.

How do you stamp the variant on the order?

This is the workaround worth doing even if you skip the GA4 one, because it survives every pixel configuration. Cart attributes ride along through checkout and show up on the resulting order, where step 3 below has you verify them on your own store before trusting anything.

The walkthrough, for a replacement-theme experiment:

1. Before creating the experiment, add a snippet to the treatment theme that runs once per session and writes the variant label to the cart: a small fetch to /cart/update.js with { "attributes": { "ab_variant": "checkout-cta-v2" } } [9].
2. Also before creating the experiment, add the same snippet to your live theme with "control" as the value, so both arms are labeled no matter which theme copy the experiment serves.
3. Place an order through each variant's preview, then check Shopify admin → Orders → order details: the attribute appears on the order as an additional detail.
4. From there it is join-ready: filter orders by attribute value over the experiment window and compare revenue, AOV, discount usage, whatever the four native rates do not answer.

One honest caveat to carry with the results: since visitor-assignment persistence is undocumented [6], treat the label as capturing the variant at the moment of carting rather than a guarantee about everything that visitor saw across sessions. For joining variant to revenue, carted-variant is the key you actually want.

Why theme-edit and checkout experiments resist instrumentation

The workarounds above assume the treatment runs different code than control. Two experiment types break that assumption.

Theme-edit experiments apply theme-editor customizations that exist only inside the rollout, while everything you did not customize stays synced to the published theme [5]. Both arms run the same Liquid, so no label can differ between them by code. If your edit happens to change a setting that a snippet reads, the signal can ride along, but Shopify does not document how rollout-scoped customizations interact with Liquid-read settings, so verify in preview before trusting it.

Checkout and accounts configuration experiments are the hard stop. There is no Liquid on checkout, no documented configuration identifier in the pixel context [3], and the native readout is a single metric [2]. If a checkout-config decision needs more evidence than one conversion rate, your options are testing the equivalent change theme-side where instrumentation works, or running the config change as a time-split comparison across periods, with the usual seasonality caveats. Checkout's script surface has been shrinking by design since extensibility replaced liquid checkout, a story covered in checkout extensibility and conversion tracking.

Also resist the tempting shortcuts. A custom pixel cannot sniff Shopify.theme from the page; the sandbox does not reach the top frame's window. And do not read Shopify's visitor counts as GA4 sessions; the two are defined differently, and purchases can even open a new GA4 session.

A practical playbook for your first measured experiment

Run code tests as replacement themes, label both arms before launch, schedule rather than launch immediately, and give the experiment a pre-registered success metric that Shopify's native rates can actually answer, with your custom dimension and order attribute carrying everything else. When Shopify's conversion rate and your GA4 numbers disagree at the margins, that is expected rather than alarming; the two systems count differently, as unpacked in Shopify Analytics vs GA4. What you are buying with the labels is the ability to ask questions Shopify's card cannot answer: did the treatment shift AOV, did it move returning customers differently than new ones, did revenue per session follow the conversion rate.

The baseline that makes any of this trustworthy is conversion data that does not wobble while you test. WeltPixel Conversion Tracking delivers your Shopify purchases server-side to GA4, Meta, TikTok, Google Ads, and more, so your experiment readout is not polluted by browser-side purchase loss, and your variant custom dimension lands on sessions whose conversions actually got recorded. Install it from the Shopify App Store before the experiment starts, and both arms get measured with the same clean ruler.

FAQ

Which Shopify plan do I need for A/B testing with Rollouts?

Rollouts itself is available on Basic plan or higher, but experiments, the type that compares control and treatment groups, require Grow plan or higher [4]. Headless and custom storefront checkouts are not supported either way [4].

Can GA4 or my pixel automatically see which theme variant a visitor got?

No. The Web Pixels event context contains no theme, rollout, or experiment field [3], and both variants serve identical URLs. GA4 only learns the variant if you send it yourself, via an event parameter registered as a custom dimension.

Does the order show which variant produced it?

Not natively; Shopify documents no order-level variant field. A cart attribute written by each theme variant carries onto the order and closes that gap; verify it once with a test order per arm.

Why does my rollout show no analytics?

Most likely it was launched immediately. Shopify states that a rollout launched immediately collects no analytics, and its changes are permanent [2]. Schedule anything you intend to measure instead.

Is checking theme.id a reliable way to detect the variant?

Only replacement themes are grounded for that, since they are separate themes you created. Even then, Shopify marks the Liquid theme object deprecated with values subject to change [7]. A hardcoded label per theme is sturdier.

Sources

1. Shopify Changelog, "Schedule, publish, and A/B test new themes and checkout and customer account configurations," changelog.shopify.com/posts/schedule-publish-and-a-b-test-new-themes-and-checkout-and-customer-account-configurations (June 5, 2026)
2. Shopify Help Center, "Rollout analytics," help.shopify.com/en/manual/markets-new/rollouts/analytics (accessed July 17, 2026)
3. Shopify.dev, "Web Pixels API standard events: page_viewed," shopify.dev/docs/api/web-pixels-api/standard-events/page_viewed (accessed July 17, 2026)
4. Shopify Help Center, "Rollouts," help.shopify.com/en/manual/markets-new/rollouts (accessed July 17, 2026)
5. Shopify Help Center, "Create a rollout," help.shopify.com/en/manual/markets-new/rollouts/create-a-rollout (accessed July 17, 2026)
6. Shopify Help Center, "Managing rollouts," help.shopify.com/en/manual/markets-new/rollouts/managing-rollouts (accessed July 17, 2026)
7. Shopify.dev, "Liquid objects: theme," shopify.dev/docs/api/liquid/objects/theme (accessed July 17, 2026)
8. Shopify.dev, "Web Pixels API standard API: browser," shopify.dev/docs/api/web-pixels-api/standard-api (accessed July 17, 2026)
9. Shopify.dev, "AJAX Cart API reference," shopify.dev/docs/api/ajax/reference/cart (accessed July 17, 2026)