For the complete documentation index, see llms.txt. This page is also available as Markdown.

Tests vs. Personalizations

Every change you make in Intelligems, whether it's a new price, a checkout badge, a shipping threshold, or a homepage banner, starts the same way: you build it once. From there, you decide how it goes live.

Ship it as a Personalization and it runs always-on for the audience you choose. Everyone who matches your targeting sees the change, all the time.

Run it as a Test and your traffic splits between a control group and one or more variants, so you can measure what the change is worth before you give it to everyone.

Nothing about the setup changes between the two.

Which one to pick

You don't need to know in advance whether something "should" be a test or a personalization. A good rule of thumb:

  • Not sure it'll work? Test it. Add a control group and run it long enough to get a statistically significant result.

  • Already confident, or making a small, low-risk change? Skip straight to a Personalization and ship it live.

On the fence? Test it. You can always roll a winning test group out as a Personalization. You can't go the other way and turn a Personalization into a Test.

How they differ

This works the same way in every module: Content, Merchandising, Pricing, Shipping, Checkout, and Post-purchase.

Personalization
Test

Audience

Everyone who matches your targeting

Split between a control group and one or more variants

Multiple Groups

None

This can be a control group or alternate change to test against.

Goal

Ship a change you're confident in

Find out what the change is worth in revenue or profit before you commit

Running it

Enable and disable it whenever you want

Start it, pause it, and end it once you have your answer

Reporting

Visitors, orders, and revenue it drove

Full experiment analytics, with significance for every group

Rolling out a winning test group

When a group wins, you don't have to rebuild it. Roll it out and Intelligems ends the test and creates a Personalization carrying the same modifications, ready for you to target and activate. You can roll out more than once from the same test, so if one group won with a particular audience and another won everywhere else, each can go live to the audience it won with.

Three types roll out differently. Price experiences apply the winning prices straight to Shopify, for all visitors, once, at the moment you end them. A theme change rolled out to all visitors should be done directly to Shopify. Shipping experiences can't be rolled out, so you'll need to make the winning rates live in Shopify yourself.

Where to find yours

Your Dashboard lists every experience in your store, and each module in the left nav lists its own. On both, the All Experiences, Tests, and Personalizations tabs filter the list, and every row is tagged with the mode it's running in.

Experiment Analytics

Sitewide Analytics

Last updated

Was this helpful?