Data Reconciliation
This feature is in beta, and is not enabled by default. Because it's in beta, results may not be 100% accurate. Please contact the Intelligems support team if you'd like this feature enabled for your account.
It is not uncommon for some orders to be excluded from a test's analytics. These exclusions are based on predefined criteria or conditions not aligned with the experiment's parameters. For example, orders may be excluded if they were placed just before the experiment started or during a pause. Orders may also be filtered out if the visitor who placed them didn't meet targeting conditions.
The data reconciliation tab in the Intelligems analytics dashboard breaks down all orders that were placed between the beginning of the day on which the test started and the end of the day on which the test ended (or the current time if the test is still live). This allows you to reconcile the order numbers by viewing Shopify's analytics and getting the count of orders created between the date of the experiment start and the date of the experiment end.
There will be a reason associated with every order that has occurred, but is not included in Intelligems' data. One of the most common reasons orders are excluded from a split URL test is if the visitor never visited the origin URL in the test's setup. In a template test, orders will be excluded if the visitor never visited a page assigned to the template in the control group.
Use this tab to get a quick overview of why orders have not been included, or click the Export button on this page to generate a spreadsheet of the order exclusion data.
The Data Reconciliation tab in the Intelligems analytics dashboard is there to help you understand why certain orders may have been excluded. In the table below, you will see all of the possible exclusion reasons. You may not see all of these reasons in your data reconciliation dashboard.
Before timeframe start
Non-error
The order was placed on the same day as the filtered start time, but before the time
After timeframe end
Non-error
The order was placed on the same day as the filtered end time, but after the time
During experiment pause
Non-error
The order was placed while the experiment was paused
Subscription refill
Non-error
The order is a subscription refill (i.e., not the first order in the subscription series)
Draft order
Non-error
The order is a Shopify draft order
Non-online store order
Non-error
The order originated from a channel outside of the Shopify online store (e.g., Shopify POS, Instagram Shopping, etc.)
Visitor is a bot or crawler
Non-error
The visitor who placed this order is excluded because they are a bot or web crawler
Visitor has opened Intelligems preview mode
Non-error
The visitor who placed this order is excluded because they have opened Intelligems' preview mode before
Visitor never entered the experiment
Non-error
The visitor who placed this order never entered this experiment (e.g., they never visited a targeted page, or never went through a redirect in a redirect test, etc.)
Visitor was excluded from the experiment by audience targeting
Non-error
The visitor who placed this order is excluded by audience targeting
Visitor never visited the redirect page
Non-error
The visitor who placed this order visitor never went through an experiment redirect
Visitor did not visit during the selected timeframe
Non-error
The visitor who placed this order did not visit the store during the selected timeframe
Associated test group has been deleted
Non-error
The test group this visitor was placed into has been deleted
Net revenue is <= $0
Non-error
The order's net revenue is $0 or less
No Intelligems ID
Error
There is no Intelligems ID noted on the order. The most common causes for this are "Buy Now" buttons that lead directly to checkout, or unintegrated cart permalinks.
No page views
Error
There are no page views recorded for this visitor
Filtered out: no experiment products
Filtered out
The order is filtered out because it did not contain any experiment products
Filtered out: outlier
Filtered out
The order is filtered out because it is an outlier in net revenue
Filtered out: device type
Filtered out
The order is filtered out by the device type filter
Filtered out: channel
Filtered out
The order is filtered out by the channel filter
Filtered out: new vs. returning visitor
Filtered out
The order is filtered out by the new vs. returning visitor filter
Filtered out: custom event
Filtered out
The order is filtered out by the custom event filter
Filtered out: country
Filtered out
The order is filtered out by the country filter
Filtered out: added to cart filter
Filtered out
The order is filtered out by the added to cart filter
Filtered out: started checkout filter
Filtered out
The order is filtered out by the started checkout filter
Not excluded
Not excluded
The order is included in experiment analytics
Last updated