> For the complete documentation index, see [llms.txt](https://docs.intelligems.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.intelligems.io/content-experiences/content-testing-faqs.md).
# Content Experiences FAQs
Common Questions about Content Testing
## General
Why should I run a template or content replacement test instead of a theme test?
Theme tests are a great way to test large changes to your store, or experiments that would otherwise require a developer to implement. There are a few drawbacks to keep in mind, however, so you may choose to run a template test or content replacement test instead:
1. A theme test means you have multiple live themes. Your customers are accessing your store via multiple themes, so you need to ensure any changes unrelated to the test (e.g., adding a landing page, editing copy or imagery) are made in all themes in the test.
2. Every visitor goes through a redirect. As mentioned above, theme tests are relatively high performing, but every visitor placed into a non-control group will still go through a one-time redirect to bring them to the correct theme for their group. While the UX impact of this one-time redirect is generally low, you may be able to avoid it altogether by using a content replacement test to dynamically edit content or inject HTML/CSS/JS.
3. You won't be able to delete any themes in the test for a while. During the experiment, visitors are sent to draft themes, and their browser will "remember" which theme to open the next time they visit your store via a session cookie. Once the test is over, if a visitor who was in one of the test groups re-visits your store and the cookie is still active, Shopify will first load the draft theme, and Intelligems will then immediately reset them back to the live theme. However, if the draft theme that was in the test has since been deleted, Intelligems won't be loaded and won't have the opportunity to reset the visitor's theme. Instead, the visitor see an error from Shopify, since they're trying to load a theme that does not exist.\
\
So, it's important to leave any themes that were tested in draft mode (rather than deleting them) for a while after the experiment (we recommend at least 1 month), to ensure any returning visitors are reset back to the live theme.
I'm running a content test and noticed the control content (i.e. URL, theme, language, etc.) flashes for a second before I see the test group content. How can I fix this?
Intelligems has two options for installing our JavaScript in your theme:
1. By using our app embed
2. By manually installing it into the theme code
The app embed works really well in most cases, but it can cause some flashing for certain tests and sites. If you notice this happening on your site, we recommend following these steps:
1. Turn the app embed off in all theme included in the test. You can do this by going to Shopify and in the online store section, click "Customize" next to each theme. In the left side menu, click the bottom option, which is "App Embeds". Turn the toggle off next to Intelligems and hit save.
2. Follow [these steps](/developer-resources/theme-and-storefront-setup/intelligems-theme-snippets.md#option-2-add-script-tag-in-theme-head) to manually add the Intelligems script tag to your theme's head in the theme.liquid and all theme.\*.liquid files.
If you still notice any flashing after making that change, please reach out to our support team [here](broken://spaces/q32N3K9wK9X5CCWMhuVe) - there are a few settings we can turn on that allows us to load more information more quickly to avoid any flashing.
Can I test the One Page vs Three Page checkout, or test specific components on the checkout page, such as whether to include upsells?
Unfortunately we cannot A/B test the One Page vs Three Page checkout currently - this option has to be either on or off, so there isn't a way for us to turn it on for some visitors and not others. This is a Shopify limitation.
We can test other components on the checkout page if you are on Shopify Plus. You can test components on that page using our Checkout Experiences feature, which you can read more about [here](/checkout-experiences/getting-started-with-checkout/testing-checkout-experiences.md). This feature is limited to Shopify Plus because it uses Shopify's Checkout Blocks feature, which is only available for brands on Shopify Plus.
## Onsite Edits Tests
Can we test an image vs. a video (MP4)?
Yes! This can be done using an [Visual Experience Builder](/general-features/editors/visual-builder.md).
What is a query selector and why does Intelligems use them?
A query selector is a way to select an element on a webpage using CSS-style syntax in JavaScript. In this case, we created a query selector based on the element you clicked on, which means it automatically targets that specific part of the page. However, this selector is just a starting point—you can edit it to be more general, more specific, or to target a different element altogether, depending on what you need your code to do.
Can we test our Pop-Ups with Intelligems?
Yes!
Yes, whether your pop-up comes from a third-party app or is built into your theme. Both setup routes, the code you'll need, and how to track engagement are covered here:
[Testing Pop-Ups](/content-experiences/content-experiences-overview/content-test-common-use-cases/testing-pop-ups.md)
## Redirect (Split URL)
Why must visitors visit the Origin URL to be included in the test?
Visitors must visit the Origin URL because this is the only page where Intelligems can detect their arrival and assign them to a test group. When a visitor lands on the Origin URL, our system:
1. Identifies that they've entered the experience
2. Assigns them to either the Control Group or a test variation
3. Redirects test group visitors to their designated destination URL
Without visiting the Origin URL first, there's no trigger point for test group assignment, so visitors remain outside the test entirely. This also keeps the test's denominator limited to people who have been exposed to the test, keeping your data as actionable as possible.
Another thing to note is that URL redirect tests work in **one direction only**: from the Origin URL to the test destination URLs. Therefore, if a visitor lands directly on a destination URL (bypassing the Origin URL), they won't be included in the test. This is because:
* They never triggered the test assignment process
* Intelligems doesn't know they should be part of the test
* They won't be tracked in test results or analytics
To ensure accurate test results, visitors must enter through the Origin URL to be properly allocated and tracked.
I got an error that said I am on the wrong destination URL for my test group while previewing my test - what does this mean?
In a split URL test, Intelligems monitors site visitors who visit your Origin URL, then either keeps them there or redirects them based on their assigned test group. Our preview widget detected that you're on a destination URL for a test group that you're not currently previewing. Please note, Intelligems will not redirect you away from a destination URL, as our redirects only function in one direction: from the origin to the destination.
To continue testing your split URL test, please navigate to the Origin URL again.
I got an error that said I have enabled the setting for "Redirect One Time" for this test, and I have already been redirected while previewing my test - what does this mean?
Intelligems has two options for how a visitor can be redirect in a split URL test -
1. Redirect One Time: A visitor will only be redirected the first time they land on the origin URL
2. Redirect Every Time: A visitor will be redirected every time they land on the origin URL
Because you have enabled "Redirect One Time" for your test, and you have already been redirected one time, Intelligems will not redirect you again, unless you clear your redirect history by clicking the button to do so in the preview widget.
If you'd prefer to change this setting, please see our help guide [here](/content-experiences/content-experiences-overview/how-to-set-up-a-redirect-experience.md) on where you can do so.
Can I run Redirect tests on multiple domains?
Yes! If your brand operates multiple domains under one Shopify account, you can register all of them in **Settings → Domain Settings**. This allows you to:
* Set up Split URL tests for any registered domain
* Use subdomain redirects for each domain (e.g., shop.yourbrand.com → landing.yourbrand.com)
**Important limitation:** Cross-domain redirects are not supported. You cannot redirect visitors from one top-level domain to another (e.g., yourbrand.com → yourbrand.co.uk). Redirects must stay within the same domain or its subdomains.
## Template Tests
Why must visitors visit a page assigned to the Control Group's template in Shopify to be included in a Template test?
Visitors must visit a page that uses the Control Group's template because this is the only way Intelligems can detect their arrival and assign them to a test group. When a visitor lands on a page using the control template, our system:
1. Identifies that they've entered the test
2. Assigns them to either the Control Group or a test variation
3. Dynamically serves the appropriate template based on their test group
Without visiting a page that uses the control template, there's no trigger point for test group assignment, so visitors remain outside the test entirely.
You can check which pages are assigned to a template in Shopify by following [these steps](https://www.loom.com/share/5f4cd7c9e008488287d4086d370a50f3?sid=b4028c6b-b07d-4b6b-8690-ff1507766205), or in your Shopify admin.
If your control template isn't assigned to any pages in Shopify, no visitors will be included in your test. To fix this:
1. Assign your control template to relevant pages in Shopify
2. Or choose a different template to use as your Control that's already assigned to pages you want to test
How can I roll out a winning homepage template?
Shopify doesn't allow you to switch homepage templates. The winning template will have to be recreated in the original template.
An easy way to do this is within the theme editor. Find the test template and copy its contents into the default template (usually `index.json`).
.png?alt=media)
Is it safe to delete variant template files from Shopify after a Template Test ends?
Yes — once a template test is fully **Ended** in the Intelligems dashboard, it is safe to delete the variant template files from your Shopify theme. If a returning visitor with a stale assignment cookie visits a page that was previously served by the deleted variant template, Intelligems will redirect them to the control template instead. They will not see an error or experience an infinite redirect loop.
This is different from **theme tests**, where deleting a variant (draft) theme while visitors still have active cookies can cause those visitors to see a Shopify error page, since Shopify tries to load the now-deleted theme before Intelligems can intervene. Template tests do not carry this same risk.
**Note:** This safe fallback behavior applies to **ended** tests. If you delete a variant template while a test is still **live or paused**, Intelligems may not be able to properly serve the variant, which can cause unexpected behavior for visitors in that test group. Always end the test before deleting variant templates.
## Theme Tests
Why do I see the Shopify theme preview bar during my theme test?
During a properly functioning Intelligems theme test, visitors will never see the Shopify theme preview bar. If you're seeing the preview bar, it's likely because the Intelligems JavaScript is not installed in all themes being tested. The Intelligems script needs to be in the live theme, plus all themes in the test.
If the script is in only the live theme, for example, visitors will be put into test groups and will be redirected to a draft theme, but since the Intelligems script is not in the draft theme, the preview bar will not be hidden once they're there, and Intelligems' analytics tracking will not function properly.
To fix this issue, install the Intelligems script in the live theme and all themes that you're testing. Since theme tests require a redirect, we recommend installing the Intelligems script synchronously by adding the script tag to your theme's head in the theme.liquid and all theme.\*.liquid files (learn more [here](/developer-resources/theme-and-storefront-setup/intelligems-theme-snippets.md#option-2-add-script-tag-in-theme-head)).
How does a theme test impact page load speeds?
Theme tests are relatively high performing, however, there may be a performance impact on a visitor's first page view. If Intelligems detects that a visitor is in the wrong theme, it will immediately reload the page in the correct theme. From then on, that visitor will be in the correct theme for their test group, which may be an unpublished theme. Our benchmarking shows that there is no performance impact on these subsequent page views, even for unpublished themes. If you're concerned abotu the potential performance impact of a reload on the first page view, you might consider using an onsite edit test instead.
After my theme test ended, some of my team still sees an old theme from time to time. Are real visitors affected too?
In almost every case, this is an internal-only thing and your real visitors are fine.
When a visitor is placed in a theme test, their group is stored in a first-party session cookie that tells their browser which draft theme to load. Once the test ends, if a returning visitor still has that cookie, Shopify loads the draft theme first and Intelligems immediately resets them back to your live theme. So actual shoppers land on your live theme, not the old one (as long as the draft theme still exists, see [Ending a Theme Experience](/content-experiences/content-experiences-overview/how-to-set-up-a-theme-change-experience/ending-a-theme-test.md) for more details).
What your team is usually running into is different, and it comes from previewing themes in Shopify Admin rather than from the visitor cookie:
1. Previewing a theme that was part of a test can reassign you to that test's group, so you land on the theme for your group.[ More on this in the preview FAQs on the theme experience setup page.](/content-experiences/content-experiences-overview/how-to-set-up-a-theme-change-experience.md)
2. After a test ends, there's a one-time redirect **per theme** so nobody stays stuck on a preview theme. The first time you open each of those themes after the test, you may get bounced once, and then it previews normally.
Because your team is in the admin previewing themes regularly, this can show up "from time to time" for them while never affecting customers. Anyone who wants it to stop immediately can clear their cookies and cache, or open the store in an incognito window.
A couple of things worth knowing:
* **This is not caused by caching on our end.** Intelligems doesn't run an edge or CDN layer that stores or serves theme pages. Theme selection is a client-side redirect driven by the cookie and our script running inside the theme, so there's no cached variant page for us to clear.
* **Don't delete the old draft themes.** While a tested theme still exists as a draft, the reset-to-live works cleanly. If you delete one, a returning visitor who still has its cookie can hit a Shopify error page instead, since there's no theme left for us to load and redirect from. We recommend leaving tested themes in draft for at least a couple of months after the test ends.
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://docs.intelligems.io/content-experiences/content-testing-faqs.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.