🅰️A/A Test
What is A/A testing?
While A/B testing involves driving traffic to two (or more) different variants to see which version converts better, an A/A test pits two identical variants against each other. Instead of learning which version performs the best, the goal of an A/A test is to start getting comfortable with Intelligems, while also confirming that all the data and analytics are tracking smoothly.
How do I set up an A/A test in Intelligems?
There are a lot of different ways that you could set up an A/A test in Intelligems, but here is what we recommend doing to keep things simple:
Click "Create New Test" in the top right corner of Intelligems > Content Test > Create Test > Onsite Edits > Create Test.
Enter your test name, description and goal - this only impacts how you view the results! Hit continue.
Create your test groups - in most cases, it makes sense to leave traffic split 50/50 for an A/A test.
You can skip setting up any targeting, and hit continue.
Click "Save Test."
Hit the play button in the "Start" column of your experiments list to take your A/A test live! 🎉
Data will start to be available for this test within 2 hours of starting your test, and will update that often as well! Because the groups in this test are identical, you should see nearly identical results across the board. You may notice differences in the results for each group early on, but the longer you run your A/A test, the closer the results should get.
We recommend ending an A/A test after:
2 full weeks of testing
300+ orders per test group
If the above has been hit, and Stat Sig shows 95%+ confidence that one A group is better than the other, this may be a sign that something is wrong, since equal groups should perform similarly. Please reach out to Intelligems support for help troubleshooting your test setup.
One note to keep in mind: running other tests simultaneously can cause some noise in these results. You can read more about interference between tests here.
Last updated