Price Testing FAQs

Can I have multiple products in the same test? Is there a limit on how many?

Yes, you can test multiple products and variants!

If you are on the Shopify, Shopify Advanced, or Shopify Basic plan: you are limited to testing 100 products or 500 variants at a time - this is because we must create a duplicate product for each product and test group, and Shopify tends to time out if we attempt to create too many duplicate products at one time. If you are on the Shopify Plus plan: we don't recommend testing more than 500 products or 5,000 variants at a time. It is possible, but it may cause the test to take a bit longer to load, or result in time outs while setting it up and making edits in the app.

In general, the more products you include in a test, the more data you’re collecting because you’re opening up the aperture of the test. However, the less specific that data is to one product. In most cases though, we recommend testing all variants of a product, or entire collections, at once, in order to control for interaction effects: a simple example is you sell shoes in two colors, red and blue. If you only tested the red shoe, it may seem like raising the price is worse because you didn’t test raising the price of the blue shoe and people went and bought that instead. In this example, the results will be more significant by including both colors.

Do I need to duplicate products to run a price test and does it happen automatically?

Duplicating products may be necessary if:

1. Your store is not on Shopify Plus - this is because non-Shopify Plus stores do not have access to Checkout Scripts.

2. You offer subscriptions through 'Old Recharge'.

3. You are testing significant PDP changes as part of your test.

If your store is not on Shopify Plus, duplicate products will automatically be created in your Shopify store once a Price Test is saved within the Intelligems app. If you are on Shopify Plus, but need to use duplicate products for one of the above or another reason, please reach out to [e-mail] for assistance with turning this setting on.

What will duplicate products look like in Shopify and how can I find them?

The duplicate products will be nearly identical to the originals with a few key differences:

1. They will have different product and variant IDs. Note that duplicates will have the same SKUs as their originals.

2. They will be tagged with a ‘price_test’ tag, as well as a tag with the experiment and group ID.

3. They will have added metafields from Intelligems to hide them from certain collections pages - this avoids showing multiple identical products on a collection page.

4. They will be set to ‘Draft’ status when created.

Can I delete duplicate products once a price test is over?

We typically recommend waiting a few days after a test is over - customers will not be able to see the duplicates, unless they have them in their cart still. Keeping the duplicates live for a few days will allow these edge-case customers to still be able to checkout, creating a better customer experience. When ending a test you will be asked if you would like to archive the products now or within 48 hours. Intelligems will automatically archive duplicates after 48 hours.

Additionally, if you have subscriptions, you should make sure any subscriptions that occurred on the duplicate products get mapped to non-duplicate products.

Other than that, there are not issues with deleting them, but archiving may be a safer option in case you need to make any changes at a later time. You will be able to find all duplicate products created by Intelligems by searching for the products with the tag 'price_test' in Shopify.

What additional steps do I need to take if I use another software to manage product prices?

If you are on Shopify Plus, when you start a price test, the Intelligems Wizard will automatically update your product prices in Shopify to the highest price in the test for each product. If you use another software (like NetSuite) to manage/push prices to Shopify, you'll need to also update the product prices in NetSuite to the highest price for each product.

1. You can download a CSV with your product IDs, SKUs, and test group prices in the "Set Prices" step of test setup. Please note that this file will contain prices for each test group, though you'll want to import only the highest price for each product. You can edit this CSV for import into your price management software (like NetSuite) according to the required format.

2. Start your price test in the Intelligems App. Please select 'Yes, update my prices and start my test' in the pop-up.

3. Update prices to the highest test prices in your price management software right after starting the test (using the data from the downloaded CSV if preferred). As long as prices are updated before the software syncs to Shopify, then the test will work as expected.

What should I do with my Google Shopping Feed during a price test?

This depends on whether you are running your price test using Checkout Scripts (typical for Shopify Plus) or Duplicate Products (typical for non-Shopify Plus).

If you are using Checkout Scripts, Intelligems updates the price in Shopify to the highest price in the test and sends that price to your shopping feed. We recommend triggering a feed update once the test is live, especially if you push your updates to the feed manually or if you use an app to manage your Google Feed. This means that for visitors coming into your site through Google ads, they may see a lower price when they get to the site, but never higher prices.

If you are using Duplicate Products, Google will continue to display the control price in the Google Shopping feed, which means users may see a different price when they get to the site. We recommend sending the higher prices to Google so that if users do see a different price when they land onsite, it's a lower price and therefore better for the customer experience. To do this, the original, non-duplicate products must have the highest price points in the test. This corresponds to the column on the left when you are setting up a test in Intelligems. Additionally, you will want to make sure you remove duplicate products from all third-party channels so that the products don't show up on Google multiple time with different prices. You can read more on how to do that here.

I'm in Preview Mode to QA my Price Test but when I add a test product to cart the price is not right. Why?

Shopify Plus members using checkout scripts for their integration may notice a discrepancy in the cart prices during preview mode that will go away once a test is live if they are testing prices that are higher than their control price! The control price and any price groups lower than the control should show up correctly in the cart if your checkout script is live, while the higher price group will continue to add to cart at the control price.

This is because there is one integration step left, which will occur when the test is started. At that time, the Intelligems app will update the Shopify price to be the highest price in the test, and then for users who are in a lower price group, we will calculate a discount behind the scenes.

If you are only testing prices lower than the control, then you should not see any discrepancies when testing in Preview Mode since the control price is the highest price in the test.

Why is the discount showing on ShopPay checkout?

There is currently no way to hide a checkout script discount from appearing on ShopPay checkout. That said, we can change the name of the discount to appear as you'd like - if you would like to change the name of your discount, please reach out to [e-mail].

Does Intelligems integrate with Global-E?

We do not directly integrate with Global-E, but do have steps to follow if you are on Shopify Plus and running a price test with higher prices. If neither of those apply, nothing needs to be done to work with Global-E. If those do apply, when you start your price test, we raise your prices in Shopify to the highest prices in the test. Because Global-E is converting prices in other markets based off of what is in Shopify, the price will be higher in all other markets while the test is running. To avoid this, you will need Global-E to set a fixed price in the foreign currency for each product included in the test. You can do this by providing Global-E with a CSV file. They will also need to revert this at the end of the test so the prices can sync with the current exchange rate.

Does Intelligems work with subscription softwares like Recharge?

Yes! Intelligems works with Recharge and other subscription management services to facilitate tests around subscription pricing. However, given these integration are often complex, we highly recommend having Intelligems perform this integration. Learn more here about integrating with subscription platforms.