Data Reconciliation

This feature is in beta, and is not enabled by default. Because it's in beta, results may not be 100% accurate. Please contact the Intelligems support team if you'd like this feature enabled for your account.

It is not uncommon for some orders to be excluded from a test's analytics. These exclusions are based on predefined criteria or conditions not aligned with the experiment's parameters. For example, orders may be excluded if they were placed just before the experiment started or during a pause. Orders may also be filtered out if the visitor who placed them didn't meet targeting conditions.

The data reconciliation tab in the Intelligems analytics dashboard breaks down all orders that were placed between the beginning of the day on which the test started and the end of the day on which the test ended (or the current time if the test is still live). This allows you to reconcile the order numbers by viewing Shopify's analytics and getting the count of orders created between the date of the experiment start and the date of the experiment end.

There will be a reason associated with every order that has occurred, but is not included in Intelligems' data. One of the most common reasons orders are excluded from a split URL test is if the visitor never visited the origin URL in the test's setup. In a template test, orders will be excluded if the visitor never visited a page assigned to the template in the control group.

Use this tab to get a quick overview of why orders have not been included, or click the Export button on this page to generate a spreadsheet of the order exclusion data.

The Data Reconciliation tab in the Intelligems analytics dashboard is there to help you understand why certain orders may have been excluded. In the table below, you will see all of the possible exclusion reasons. You may not see all of these reasons in your data reconciliation dashboard.

Reason	Type	Description
Before timeframe start	Non-error	The order was placed on the same day as the filtered start time, but before the time
After timeframe end	Non-error	The order was placed on the same day as the filtered end time, but after the time
During experiment pause	Non-error	The order was placed while the experiment was paused
Subscription refill	Non-error	The order is a subscription refill (i.e., not the first order in the subscription series)
Draft order	Non-error	The order is a Shopify draft order
Non-online store order	Non-error	The order originated from a channel outside of the Shopify online store (e.g., Shopify POS, Instagram Shopping, etc.)
Visitor is a bot or crawler	Non-error	The visitor who placed this order is excluded because they are a bot or web crawler
Visitor has opened Intelligems preview mode	Non-error	The visitor who placed this order is excluded because they have opened Intelligems' preview mode before
Visitor never entered the experiment	Non-error	The visitor who placed this order never entered this experiment (e.g., they never visited a targeted page, or never went through a redirect in a redirect test, etc.)
Visitor was excluded from the experiment by audience targeting	Non-error	The visitor who placed this order is excluded by audience targeting
Visitor never visited the redirect page	Non-error	The visitor who placed this order visitor never went through an experiment redirect
Visitor did not visit during the selected timeframe	Non-error	The visitor who placed this order did not visit the store during the selected timeframe
Associated test group has been deleted	Non-error	The test group this visitor was placed into has been deleted
Net revenue is <= $0	Non-error	The order's net revenue is $0 or less
No Intelligems ID	Error	There is no Intelligems ID noted on the order
No page views	Error	There are no page views recorded for this visitor
Filtered out: no experiment products	Filtered out	The order is filtered out because it did not contain any experiment products
Filtered out: outlier	Filtered out	The order is filtered out because it is an outlier in net revenue
Filtered out: device type	Filtered out	The order is filtered out by the device type filter
Filtered out: channel	Filtered out	The order is filtered out by the channel filter
Filtered out: new vs. returning visitor	Filtered out	The order is filtered out by the new vs. returning visitor filter
Filtered out: custom event	Filtered out	The order is filtered out by the custom event filter
Filtered out: country	Filtered out	The order is filtered out by the country filter
Filtered out: added to cart filter	Filtered out	The order is filtered out by the added to cart filter
Filtered out: started checkout filter	Filtered out	The order is filtered out by the started checkout filter
Not excluded	Not excluded	The order is included in experiment analytics