Shipping Experience QA Checklist

Let Atlas QA take the first pass. Our quality assurance agent reviews your test configuration and your live storefront and returns a report telling you what's broken, why it matters, and how to fix it — with screenshots. Run it from the QA & Preview tab of any test. Learn more.

Prerequisites: How Shipping Tests Work:

This checklist covers Shipping experiences, both Shipping Rates and Free Shipping. Each module has its own: Content, Merchandising, Pricing, Checkout, and Post-purchase.

The two Shipping types work differently, so they're QAed differently.

• A Shipping Rates experience adds Intelligems as a rate carrier in your Shopify admin and serves a different rate to each group at checkout, based on their group, cart value and item weight. Start with the pre-flight below, then follow the Shipping Rates track.

• A Free Shipping experience discounts a qualifying rate to $0 once the cart meets your requirement. Your Shopify rates stay where they are. Start with the pre-flight, then follow the Free Shipping track.

Check your setup first

Before you go anywhere near your site, a few things decide whether the experience can work at all.

Both types:

• Is the Intelligems script in your live theme? Here's where to find it.

• Using a Progress Bar? It needs a one-time snippet in your theme on top of the script.

• Selling in more than one currency? Free shipping thresholds are always measured in your store's default currency, and Shipping Rates has a separate Rate Currency field. Plan a pass per currency, and check the number a shopper actually sees against the number your threshold is set to.

• Do you have a bundle app? Bundles that get expanded into component line items when they're added to cart can strip the properties Intelligems uses to identify a shopper's group. Add a bundle to your cart, then add the same products individually at the same cart value, and compare. If the group-specific behavior only works with individual products, your bundle app is rebuilding the cart.

Shipping Rates only:

• Has Intelligems been added as a rate provider in Shopify? Check it like this:

  1. From your Shopify admin, go to Settings in the lower left and open Shipping and delivery.

  2. Select the shipping profile you chose to run your experience in.

  3. Scroll down to the zone you chose.

  4. Under Carrier and app rates you should see "Intelligems Shipping (Rates provided by app)", like the screenshot below.

  5. 
  6. If it isn't there, click Add rate and select it from the carrier options. If it isn't available to select, reach out to Intelligems Support for help troubleshooting.

• Does your Shopify plan include carrier-calculated shipping? Without it the experience can't serve rates at checkout, however it's configured.

• Do any of your rates in that zone come from a third-party rate app rather than from Shopify natively? Those can't be replaced, and a third-party carrier registered on the zone can override the Intelligems rate at checkout even when everything on your site looks right.

Open preview

Open your experience and go to the Preview tab. Pick the theme you want to preview, then click Open preview to load your site with the Intelligems widget enabled. Preview on mobile gives you a QR code and a Copy link button per variation.

Check your on-site messaging

This part applies to both types, and it's where most of a shipping experience is visible to shoppers.

• Set up any Content Edits? Visit every place you changed copy about shipping, like an announcement bar, a PDP note, or a shipping FAQ page. Switch groups in the widget and confirm the copy updates. You may need a hard refresh after switching.

• Added a Progress Bar to your cart? Switch groups and confirm the threshold changes with the group, then add and remove items and confirm the math on the bar.

• Does anything else on your site set a shipping threshold, like theme code on the cart? If your theme sets it too, it can overwrite the value Intelligems set and show some visitors the wrong number. The threshold should be coming from one place.

Shipping Rates: check your rates at checkout

A Shipping Rates experience always runs as a test, so every group gets its own rate.

Pick a group and empty your cart. Add a product, or a few, and proceed to checkout. Enter address information, either your own or a test address inside the profile and zone you're testing, so you can reach the shipping step. Then confirm your test rates are there.

Two things to expect:

• Your test rate will include "Intelligems Preview:" at the beginning of the rate name. This is expected, and it only shows up in preview mode so you know where the rate is coming from in case it has the same name as one of your normal rates.

• You will still see the rate(s) you've chosen to test. Intelligems removes those from Shopify when you start the experience, not when you save it, so preview shows you both. On a group whose test rate sits above your current free shipping threshold, your existing free rate will still appear alongside it.

• Every group's rates appear, with the prefix, at the checkout shipping step.

• The rate names and amounts match what you configured for that group.

• Rates you didn't select to replace are still working as they do today.

Complete these steps in each group, emptying your cart before you switch and starting a fresh incognito session.

Free Shipping: check your offer

A Free Shipping experience runs as a Test or a Personalization, so check whichever groups or audiences you built.

• Build the cart up to just below your minimum, then just above it, and confirm shipping goes to $0 at checkout only above the line.

• The discount name you set is the one shown to the shopper.

• Rates you excluded under Choose eligible rates are still charging. If you matched on rate name, check a rate that should match and one that shouldn't.

• Limited your ship-to countries? Check a country that qualifies and one that doesn't. Shoppers see your cart messaging until they pick a country at checkout, at which point the discount may drop off.

• In a test, each group's minimum applies to that group.

A free shipping offer discounts a rate to $0. It doesn't remove the free shipping your store already has, so a group set to a higher minimum will still get free shipping at your existing, lower one. If that's what you're seeing, the experience isn't broken and you want a Shipping Rates experience instead.

Scenarios worth running

Not all of these will apply to your store, and this list isn't exhaustive.

• Below your threshold, above your threshold, and immediately either side of it

• Every tier, if you're testing tiered rates, not just the first and last

• With a discount code applied

• Adding to cart from the PDP, the collection page and the quick-add, if you have all three

• Products from more than one shipping profile or zone

• Each currency you sell in

• Desktop Chrome and mobile Safari, since the cart drawer behaves differently on a small screen

Before you start it

• Confirm your targeting is what you want it to be, not what you set for QA.

• Give the experience enough time. A threshold or rate change needs at least two weeks to account for normal weekday and weekend variation. A couple of days of data on a shipping experience isn't a result.

If you notice any issues, or have any questions, please feel free to reach out to Intelligems support!