Tests vs. Personalizations

Every change you make in Intelligems, whether it's a new price, a checkout badge, a shipping threshold, or a homepage banner, starts the same way: you build it once. From there, you decide how it goes live.

Ship it as a Personalization and it runs always-on for the audience you choose. Everyone who matches your targeting sees the change, all the time.

Run it as a Test and your traffic splits between a control group and one or more variants, so you can measure what the change is worth before you give it to everyone.

Nothing about the setup changes between the two.

Which one to pick

You don't need to know in advance whether something "should" be a test or a personalization. A good rule of thumb:

• Not sure it'll work? Test it. Add a control group and run it long enough to get a statistically significant result.

• Already confident, or making a small, low-risk change? Skip straight to a Personalization and ship it live.

On the fence? Test it. You can always roll a winning test group out as a Personalization. You can't go the other way and turn a Personalization into a Test.

How they differ

This works the same way in every module: Content, Merchandising, Pricing, Shipping, Checkout, and Post-purchase.

	Personalization	Test
Audience	Everyone who matches your targeting	Split between a control group and one or more variants
Multiple Groups	None	This can be a control group or alternate change to test against.
Goal	Ship a change you're confident in	Find out what the change is worth in revenue or profit before you commit
Running it	Enable and disable it whenever you want	Start it, pause it, and end it once you have your answer
Reporting	Visitors, orders, and revenue it drove	Full experiment analytics, with significance for every group

Rolling out a winning test group

When a group wins, you don't have to rebuild it. Roll it out and Intelligems ends the test and creates a Personalization carrying the same modifications, ready for you to target and activate. You can roll out more than once from the same test, so if one group won with a particular audience and another won everywhere else, each can go live to the audience it won with.

Three types roll out differently. Price experiences apply the winning prices straight to Shopify, for all visitors, once, at the moment you end them. A theme change rolled out to all visitors should be done directly to Shopify. Shipping experiences can't be rolled out, so you'll need to make the winning rates live in Shopify yourself.

Where to find yours

Your Dashboard lists every experience in your store, and each module in the left nav lists its own. On both, the All Experiences, Tests, and Personalizations tabs filter the list, and every row is tagged with the mode it's running in.

Experiment Analytics

Sitewide Analytics