Atlas QA
Overview
Atlas QA is Intelligems' quality assurance (QA) agent. It QAs your test before it goes live — reviewing your test configuration, then browsing your live storefront the way a shopper would, switching between test groups, checking prices, and comparing desktop against mobile. You get back a report that tells you what's wrong, why it matters, and how to fix it, with screenshots as evidence.
Atlas QA is available on all Intelligems accounts, on every Shopify plan, for both tests and personalizations.
Running a QA check
Open the test or personalization you want to check
Go to the QA & Preview tab
Click Start QA check
Click View Live to follow along as the agent works, or leave the page — we'll email you when the report is ready
Fix anything the agent flags, then re-run the check to confirm
What Atlas QA checks
Atlas QA runs two passes. The configuration pass reads your saved test settings and validates them against each other — traffic percentages, targeting rules, price math, offer tiers. The storefront pass opens your preview in a real browser and confirms that what you configured is actually what shoppers see.
Atlas QA is test-type aware. It runs the checks that apply to your test type — price, content, offer, shipping, checkout, theme, template, and split URL — and skips the ones that don't.
Findings are always grouped into the same nine categories, in the same order, no matter what you're testing:
Test setup
Your saved settings, variation setup, and control group
Traffic split
Whether traffic is reaching more than one variation
Targeting
Page, audience, cart, and theme targeting rules
Offer conflicts
Offer tiers and discount stacking
Pricing
Price math, compare-at prices, subscriptions, and prices across your store
Installation
Whether Intelligems is installed once and pointing at the right store
Appearance
Whether your changes are visible and consistent across devices
Links, redirects, & hidden products
Preview links, redirect behavior, and duplicate products
Checkout
Cart, checkout, offer, and shipping behavior
Reading your report
Every report opens with an overall status, followed by findings grouped by category. Each finding carries one of the following:
🔴 Error
The test can't produce valid results as configured
Fix it before you launch
🟡 Recommendation
A non-blocking issue or a best-practice gap
Worth addressing, your call
Needs manual review
Atlas QA couldn't fully verify this check on its own
Confirm it yourself
✅ Ready to launch
Every check passed
You're good to go
Click any finding to open the full detail view. Each one gives you the problem, why it matters, and step-by-step instructions to fix it. Screenshots and visual diffs are the primary evidence, so you can see the issue rather than read a description of it. Technical output like console logs sits in a separate developer section.
Needs Manual Review
"Needs manual review" is not a failure. It doesn't mean a check failed — it means Atlas QA couldn't confirm the check on its own and is handing it to you. Three different things land findings here, and they call for different responses.
Atlas QA couldn't reach your storefront. Browsing can fail on stores with bot protection, slow pages, or a preview URL that didn't load.
A description of what the agent couldn't load or verify
Re-run the check. This often clears on a second attempt.
The check was skipped because something it depends on was flagged. Rather than reporting one root cause as several unrelated failures, Atlas QA skips the checks downstream of it.
A Skipped badge and a note naming the issue to fix first
Fix the issue it points to, then re-run. The skipped check will verify itself.
The check genuinely needs a person. Some things can't be confirmed without a real order or a manual look — post-purchase upsells, subscription refill pricing, JavaScript-expression targeting.
Steps under How to fix it telling you what to check yourself
Do the check, then click Mark as reviewed.
Click How to review on any finding to see which of these applies and what to do next.
Sharing a report
Most QA findings get handed to someone else to fix. Click Share link on any report to generate a link straight to it — the recipient lands on the findings, the fix steps, and the screenshots, instead of you pasting them into Slack one at a time.
The link opens the report for anyone who can already sign in to this Intelligems account. To share with an outside developer or agency, add them as a user first.
Email notifications
Atlas QA emails you when a report is ready, so you don't have to sit and watch. You'll get one of two emails depending on the outcome: no issues found, or issues found with a link straight to the report.
Only the user who pressed Start QA check gets the email.
Launching with unresolved findings
Atlas QA won't stop you from starting a test. If you launch with unresolved errors, you'll be asked to confirm that you've QA'd the test yourself and accept the risk of launching with those errors open.
We'd rather you fix errors first — they're the ones that reach shoppers — but the decision is yours. You know your store better than the agent does.
What Atlas QA doesn't cover
Atlas QA is a strong first pass at quality assurance, not a guarantee. A few things to keep in mind:
It checks before launch, not after. Atlas QA doesn't monitor a test once it's live. Keep doing periodic sweeps of your site during a test.
Some setups need a real order. Things like post-purchase upsells can only be fully verified by placing a test order, which an agent can't do.
It reports, it doesn't repair. Atlas QA tells you what to fix and how, but the change is yours to make.
It can be wrong. If Atlas QA flags something that isn't actually a problem, you can proceed. Let us know when that happens — false positives are the findings we most want to hear about.
Tips
Run QA before every launch. It takes a few minutes and catches the errors that make you doubt a result three weeks later.
Fix errors first, then weigh recommendations. Errors block valid results. Recommendations are judgment calls.
Re-run after you fix something. A clean report is the confirmation that your fix actually worked.
Trust the screenshots. They're the fastest way to see what's actually happening on your storefront.
Last updated
Was this helpful?