Measuring a Shopify Rollouts Checkout Test with Intelligems

Overview

Shopify Rollouts lets you A/B test two checkout configurations natively — for example, one-page checkout versus three-page checkout. However, the reporting is limited: Shopify gives you a single number, checkout conversion rate. No revenue per visitor, no average order value, no statistical significance, and no way to segment.

This guide shows you how to detect which Rollouts group a visitor landed in and assign them to a matching Intelligems test group. Once that's wired up, Intelligems reports on the Shopify test the same way it reports on any Intelligems test — full analytics, full significance testing — while Shopify continues to serve the actual checkout experience.

This is an advanced setup. It relies on Shopify behavior that isn't officially documented, which means it can change without warning. Follow the verification steps, and set up the monitoring described at the end.

---

Before you begin

You'll need:

• A Shopify store on Grow plan or higher (Rollouts experiments require it)

• An active Rollouts experiment on a checkout and accounts configuration — not a theme rollout, and not a scheduled launch

• Intelligems installed and running on the store

• Theme code access in Shopify (Online Store → Themes → Edit code)

You do not need a developer, but you will be pasting code into your theme. If that's not something you're comfortable doing, hand this article to whoever manages your theme.

Theme rollouts are not supported. This guide only works for checkout and accounts configuration experiments. Theme experiments use a different mechanism.

---

How it works

When a Rollouts checkout experiment is running, Shopify assigns each visitor to a checkout configuration. That configuration has an ID, and Shopify includes that ID on every storefront page — before the visitor ever reaches checkout.

We read that ID, translate it into "control" or "treatment," and use the Intelligems JavaScript API to place the visitor in the matching test group.

The Intelligems test itself is set to not assign anyone by default. Nobody enters the test through normal random assignment — every visitor is placed explicitly, based on what Shopify already decided. That's what keeps the two systems perfectly in sync.

The result: Shopify decides who sees which checkout, and Intelligems measures the outcome.

---

Step 1 — Create the Intelligems test

In the Intelligems admin, create a new Content Test with two variations. Name them after the two checkout experiences so your reports are readable:

• Variation 1: 1 Page Checkout — set this as the control

• Variation 2: 3 Page Checkout

Set the traffic split to match your Shopify rollout (usually 50/50).

Make no changes inside either variation. No copy edits, no CSS, no JavaScript. This test exists only to bucket visitors and attribute revenue — Shopify is doing all the real work at checkout. If you add content changes, you'll be running two overlapping experiments and neither result will be trustworthy.

Save the test as a draft. Do not start it yet.

---

Step 2 — Find your store's Shopify IDs

Every store has different IDs. You need three, and you'll collect them by walking through your own checkout twice.

The two snippets you'll use

Keep these handy — you'll run them several times.

Snippet A — run on any storefront page:

(function () {
  var out = {};
  var s = document.querySelector('script[src*="/checkouts/internal/preloads.js"]');
  var m = s && (s.getAttribute('src') || '').match(/[?&]configuration_id=(\d+)/);
  out.checkoutProfileId = m ? m[1] : null;
  var scripts = document.getElementsByTagName('script');
  for (var i = 0; i < scripts.length; i++) {
    var t = scripts[i].textContent || '';
    if (t.indexOf('rolloutTreatmentIds') === -1) continue;
    var g = function (k) {
      var mm = t.match(new RegExp('"' + k + '"\\s*:\\s*\\[([^\\]]*)\\]'));
      return mm ? mm[1].split(',').map(function (v) {
        return v.trim().replace(/^"|"$/g, '');
      }) : [];
    };
    out.rolloutIds = g('rolloutIds');
    out.rolloutTreatmentIds = g('rolloutTreatmentIds');
    break;
  }
  console.log(out);
})();

Snippet B — run on a checkout page:

document.querySelector('meta[name="serialized-checkoutLayout"]').content

Opening the browser console

You'll need this repeatedly. In Chrome: right-click anywhere on the page → Inspect → click the Console tab. Paste the snippet, press Enter.

If Chrome asks you to type allow pasting before it will accept pasted code, type it and press Enter, then paste again.

Collecting session A

1. Open a new incognito window (Cmd+Shift+N / Ctrl+Shift+N)

2. Go to your storefront homepage

3. Run Snippet A. Write down checkoutProfileId — a long number

4. Add any product to cart and proceed to checkout

5. Run Snippet B. Write down the result — "one-page" or "three-page"

6. Go back to your storefront homepage in the same window

7. Run Snippet A again. Now rolloutIds and rolloutTreatmentIds are populated. Write down both

You now have one complete row.

Collecting session B

Repeat the whole sequence in a fresh incognito window. You need to land in the other arm, so check checkoutProfileId at step 3 — if it matches session A, close the window and start again with a new one.

Assignment is random, so this may take a few tries. Two or three is typical.

Your ID table

Fill this in:

	Checkout profile ID	Layout	Treatment ID
Session A
Session B

Rollout ID: the single number in rolloutIds — the same in both sessions.

You can cross-check the rollout ID in the Shopify admin: Markets → Rollouts, open your experiment, and it's the number at the end of the browser URL.

If rolloutIds contains more than one number, you have multiple rollouts running. Use the one matching your experiment from the admin URL, and take the treatment ID at the same position in rolloutTreatmentIds. First rollout ID pairs with first treatment ID, second with second.

---

Step 3 — Find your Intelligems IDs

You need three more values from Intelligems: the experiment ID and the two variation IDs.

You can find all three by opening the test you created in Step 1 > Select the three dots on the top right > Show Info. Here you'll see the Experiment ID and the Variant IDs as shown below:

Your complete ID set

You should now have six values:

Value	Where it came from
Rollout ID	Step 2
Checkout profile ID — control	Step 2
Checkout profile ID — treatment	Step 2
Treatment ID — control	Step 2
Treatment ID — treatment	Step 2
Intelligems experiment ID	Step 3
Intelligems variation ID — control	Step 3
Intelligems variation ID — treatment	Step 3

---

Step 4 — Set the test to not assign anyone + Publish

In the Intelligems admin, open your test and go to Targeting → Advanced Targeting. Create a single audience:

• Targeting type: Javascript

• Operator: evaluates to true

• Expression: true

• Action: Do not assign (re-assess on later page loads)

That's the whole targeting configuration. One rule, matching everyone, assigning nobody.

Why? Intelligems will now never bucket a visitor on its own. Every assignment comes from the script in the next step, which reads what Shopify already decided. Without this, Intelligems would randomly assign visitors before our script had a chance to read the Rollouts arm — and a visitor randomly placed in "3 Page Checkout" who actually saw one-page checkout would silently corrupt both sides of your results.

Now publish your Intelligems test. This is required to complete the verification in Step 6 & 7.

---

Step 5 — Add the script to your theme

Go to Online Store → Themes, click ⋯ next to your live theme, and choose Edit code. Open layout/theme.liquid.

Duplicate your theme first. Themes → ⋯ → Duplicate. If anything goes wrong you can revert instantly.

Find this line:

{{ content_for_header }}

Paste the block below underneath it.

<script>
(function () {
  /* ---------- EDIT THESE FOUR VALUES ---------- */
  var ROLLOUT_ID    = 'YOUR_ROLLOUT_ID';
  var EXPERIMENT_ID = 'YOUR_INTELLIGEMS_EXPERIMENT_ID';

  // Shopify checkout profile ID  ->  Shopify treatment ID
  var PROFILES = {
    'PROFILE_ID_FOR_CONTROL':   'TREATMENT_ID_FOR_CONTROL',
    'PROFILE_ID_FOR_TREATMENT': 'TREATMENT_ID_FOR_TREATMENT'
  };

  // Shopify treatment ID  ->  Intelligems variation ID
  var GROUPS = {
    'TREATMENT_ID_FOR_CONTROL':   'IG_VARIATION_ID_FOR_CONTROL',
    'TREATMENT_ID_FOR_TREATMENT': 'IG_VARIATION_ID_FOR_TREATMENT'
  };
  /* -------------------------------------------- */

  var COOKIE = 'ig_rollout_' + ROLLOUT_ID;
  var MAXAGE = 60 * 60 * 24 * 90;

  function signal(name, detail) {
    try { window.dispatchEvent(new CustomEvent('ig:' + name, { detail: detail })); } catch (e) {}
    console.warn('[ig-rollout]', name, detail);
  }

  function fromProfile() {
    var s = document.querySelector('script[src*="/checkouts/internal/preloads.js"]');
    if (!s) return null;
    var m = (s.getAttribute('src') || '').match(/[?&]configuration_id=(\d+)/);
    if (!m) return null;
    var id = PROFILES[m[1]];
    if (!id) { signal('unknown_profile', { profileId: m[1] }); return null; }
    return id;
  }

  function arr(txt, key) {
    var m = txt.match(new RegExp('"' + key + '"\\s*:\\s*\\[([^\\]]*)\\]'));
    if (!m || !m[1].trim()) return [];
    return m[1].split(',').map(function (v) { return v.trim().replace(/^"|"$/g, ''); });
  }

  function fromTrekkie() {
    var s = document.getElementsByTagName('script');
    for (var i = 0; i < s.length; i++) {
      var txt = s[i].textContent || '';
      if (txt.indexOf('rolloutTreatmentIds') === -1) continue;
      var ids  = arr(txt, 'rolloutIds');
      var tids = arr(txt, 'rolloutTreatmentIds');
      var idx  = ids.indexOf(ROLLOUT_ID);
      return (idx !== -1 && tids[idx]) ? String(tids[idx]) : null;
    }
    return null;
  }

  function cookie(v) {
    document.cookie = COOKIE + '=' + encodeURIComponent(v) +
      '; path=/; max-age=' + MAXAGE + '; SameSite=Lax; Secure';
  }

  function cached() {
    var m = document.cookie.match(new RegExp('(?:^|; )' + COOKIE + '=([^;]*)'));
    return m && m[1] ? decodeURIComponent(m[1]) : null;
  }

  function resolve() {
    var p = fromProfile();
    var t = fromTrekkie();
    if (p && t && p !== t) signal('mismatch', { profile: p, trekkie: t });
    if (t) return { treatmentId: t, source: 'trekkie' };
    if (p) return { treatmentId: p, source: 'profile' };
    var c = cached();
    return { treatmentId: c, source: c ? 'cookie' : 'unresolved' };
  }

  function stampCart(treatmentId) {
    fetch('/cart.js').then(function (r) { return r.json(); }).then(function (cart) {
      if (!cart || !cart.token) return;
      var a = cart.attributes || {};
      if (a.shopify_rollout_treatment === treatmentId) return;
      return fetch('/cart/update.js', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ attributes: {
          shopify_rollout_id: ROLLOUT_ID,
          shopify_rollout_treatment: treatmentId
        }})
      });
    }).catch(function () {});
  }

  function run() {
    var out = resolve();

    window.__igRollout = out;
    window.igRolloutIs = function (id) { return out.treatmentId === String(id); };

    if (!out.treatmentId) { signal('unresolved', { rolloutId: ROLLOUT_ID }); return; }

    cookie(out.treatmentId);
    stampCart(out.treatmentId);

    var user = window.igData && window.igData.user;
    if (!user) return;

    var groupId = GROUPS[out.treatmentId];
    if (!groupId) { signal('unmapped_treatment', { treatmentId: out.treatmentId }); return; }

    var current = user.getTestGroup(EXPERIMENT_ID);
    if (current && current.id === groupId) return;
    if (current) signal('drift', { from: current.id, to: groupId });

    user.assignTestGroup(EXPERIMENT_ID, groupId);
  }

  if (window.igData) run();
  else window.addEventListener('ig:ready', run, { once: true });
})();
</script>

Now replace the placeholder values using the IDs you collected. Keep the quote marks. A filled-in example looks like this:

  var ROLLOUT_ID    = '4685909';
  var EXPERIMENT_ID = '97aa132e-ab9c-4974-8c72-07251ac06c95';

  var PROFILES = {
    '4578672725': '9371733',
    '4578246741': '9404501'
  };

  var GROUPS = {
    '9371733': '323382f0-4614-4de2-a5de-3c6a02d707ca',
    '9404501': '104b4084-9525-4756-927f-e9c697cafb2c'
  };

Click Save.

Placement. The script waits for the ig:ready event before assigning, so it doesn't need to beat Intelligems to the page. It does need to sit below {{ content_for_header }}, because that's what renders the value it reads.

---

Step 6 — Verify the script

Open your storefront in a normal browser window, open the console, and run:

window.__igRollout

You should get something like:

{ treatmentId: "9404501", source: "profile" }

Then confirm the assignment landed, using your experiment ID: [Make sure your Intelligems test is published]

window.igData.user.getTestGroup("YOUR-EXPERIMENT-ID")

This should return the variation matching the arm above — treatment ID 9404501 should return your 3 Page Checkout variation.

What you see	What it means	What to do
Both return correctly	Working	Continue to Step 7
__igRollout is undefined	Script didn't run	Check for typos in theme.liquid; look for red console errors
source: "unresolved"	Script ran, found nothing	Make sure it's below content_for_header
unknown_profile warning	Profile ID isn't in your map	Re-check Step 2 — you likely mistyped an ID
unmapped_treatment warning	Treatment ID isn't in GROUPS	The two maps don't line up; re-check Step 5
getTestGroup returns nothing	Assignment didn't fire	Confirm your experiment ID and variation IDs are correct

Do not continue until both calls return correctly.

---

Step 7 — Test it end to end

Open a fresh incognito window and go to your storefront. Run both console checks from Step 6 and note which arm you got.

Then add a product to cart and go to checkout. Confirm the checkout you actually see matches:

• Three-page: URL ends in /information, and you step through Information → Shipping → Payment

• One-page: no /information in the URL, everything on a single screen

Repeat in three or four more fresh incognito windows until you've seen both arms match correctly. If the console ever disagrees with the checkout you're shown, stop and go back to Step 2 — your ID mapping is reversed or wrong.

---

Step 8 — Confirm the arm reaches your orders

Place one test order in each arm.

If your store uses Shopify's Bogus Gateway for testing, the card number is 1, with any future expiry date and any CVV. Otherwise use whatever test payment method your store has enabled.

Then go to Shopify admin → Orders, open each test order, and scroll to Additional details. You should see:

shopify_rollout_id: 4685909
shopify_rollout_treatment: 9404501

This is your backstop. Every order now carries its own Rollouts arm, so even if in-flight bucketing ever misses a visitor, you can rebuild the analysis from order data alone.

---

Step 9 — Launch

Your Intelligems test should already be live.

Make sure your theme with the new code is also live as well as your Shopify Rollout test. If all 3 are live, then your test is now live.

Lopsided groups is the failure mode to watch for. If one arm is near zero while the other fills up, detection is broken. Pause the test and diagnose rather than letting it collect unusable data.

---

Reading your results

One thing to be aware of when you compare Intelligems numbers to Shopify's.

Visitors are bucketed on their first pageview, so your Intelligems test includes everyone who visited the store — including people who never reached checkout and therefore never saw either checkout experience. Shopify's own results card counts only visitors who reached checkout.

That's not a problem, it's just a different denominator, and it's the more useful one for most questions: sitewide conversion rate and revenue per visitor tell you whether the checkout change moved the business, not just whether it moved checkout completion.

For a like-for-like comparison with Shopify's number, filter your Intelligems analytics to visitors who reached checkout. Bucketing early means you can always narrow down to that population — the reverse isn't possible.

---

Monitoring

Check these in the first day, then weekly.

Group balance. Should track your rollout's traffic split. A sudden drift means something changed on Shopify's side.

The canary. On your storefront, run window.__igRollout.source in the console. Normal values are "profile" or "trekkie". If you see "unresolved" on a page where it previously worked, or an ig:unknown_profile warning appears, Shopify has changed something and bucketing has stopped working correctly.

Page coverage. Run window.__igRollout on a product page, a collection page, and /cart. If it resolves on the homepage but not elsewhere, visitors who land deep in your site never get bucketed, and your results will skew toward homepage traffic.

Drift warnings. An ig:drift warning means Shopify moved a visitor to a different arm than Intelligems had them in. The script corrects it automatically, but if these are frequent, raise it with support — it would mean Rollouts assignment isn't as sticky as expected.

---

Troubleshooting

Symptom	Likely cause	Fix
window.__igRollout is undefined	Script not in the theme, or has a syntax error	Re-check theme.liquid; look for red console errors
source: "unresolved" on every page	Script placed above content_for_header	Move it below
unknown_profile warning	Rollout was edited or recreated; profile IDs changed	Redo Step 2 and update the PROFILES map
unmapped_treatment warning	PROFILES and GROUPS don't line up	Every value in PROFILES must be a key in GROUPS
Console arm doesn't match the checkout shown	Control and treatment mapped backwards	Swap the two values in your PROFILES map
Nobody is being assigned to either group	Wrong experiment or variation IDs	Re-run Step 3 and update EXPERIMENT_ID and GROUPS
Visitors appear in groups at random	Targeting isn't set to Do Not Assign	Redo Step 4
mismatch warning in console	The two detection signals disagree	Rare. Note how often it happens and raise it with support
Orders missing the rollout attributes	Cart was created before the script was installed	Only affects pre-existing carts; new sessions are fine

---

Limitations and caveats

Checkout configuration experiments only. Theme rollouts work differently and aren't covered by this guide.

Undocumented behaviour. Everything here reads Shopify internals that carry no compatibility guarantee. Shopify can change them in any release. The monitoring section is how you find out.

Re-verify after editing the rollout. If you edit, restart, or recreate the Shopify rollout, the checkout profile IDs can change. Redo Step 2 and update the PROFILES map.

Don't run other Intelligems tests on checkout at the same time. Overlapping tests on the same surface make results impossible to attribute.

Visitor assignment persistence is undocumented. Shopify doesn't publish how long a visitor stays in the same Rollouts arm or what happens when they clear their browser storage. The drift and mismatch warnings are your early signals if assignments start moving.

---

What you get

Once this is running, your Intelligems test reports on the Shopify Rollouts experiment with the full analytics suite — revenue per visitor, average order value, conversion rate, statistical significance, and audience segmentation — while Shopify continues to serve the actual checkout experiences and manage traffic allocation.