START HERE · THE CORE QUESTION · 4 MIN READ
Shopify discount incrementality: did the offer create the order?
A discounted order proves that a purchase happened. It does not prove the discount caused it. Here is the measurement approach that separates the two.
The outcome your order report cannot show
Imagine a shopper buys a $180 product with 10% off. The order shows an $18 discount and a $162 payment. It does not show whether that same shopper would have paid $180 if no discount had appeared. That missing alternative is the counterfactual.
If the shopper would have bought anyway, the offer changed the amount paid, not the decision to purchase. If they would not have bought, the offer may have created an additional sale. One receipt cannot distinguish these stories. The $18 is observed discount spend; calling it either waste or a successful investment requires more evidence.
For a closer look at this arithmetic, read discount spend versus incremental revenue.
Incrementality is a comparison between groups
You cannot show both offers to the same shopper in the same moment. Instead, define an eligible population and randomly assign shoppers before their purchase outcome. In the first Invisible Offer experiment, one group keeps the normal eligible 10% discount; the other receives no experiment offer.
Random assignment aims to make the groups comparable at the start. Then ask whether withholding the discount changed purchase probability or revenue per eligible shopper. The difference between groups estimates the offer’s incremental effect for that population and period—not the motive of any named customer.
This is why “some no-offer shoppers purchased” is not a conclusion. Some shoppers in both arms will buy. The question is whether the rates and economics differ enough to matter. See how to design the holdout test.
A credible answer needs a complete chain
Count all eligible experimental units, not only shoppers who later made a purchase. Keep each assignment sticky through the journey and persist it before the order. Verify that the intended offer was actually applied or withheld. Finally, link the Shopify order—and any later refund—back to the assignment using durable, auditable identifiers.
An unattributed order cannot be silently assigned to the arm that makes the result look better. A failed intervention should be visible as a delivery problem. If the groups become imbalanced or outcomes are missing, the report should say so rather than manufacture confidence.
For the outcome side of this chain, read how to measure Shopify discounts after refunds.
Read conversion and economics together
At a minimum, compare conversion, gross revenue per eligible shopper, discount spend per eligible shopper, and refunds across the arms. Do not stop at a lower discount bill. If the no-offer arm loses enough orders, avoided discounts may be outweighed by lost revenue.
Equally, a small observed conversion gap is not proof that the groups are equivalent. Sample size, uncertainty, the planned evaluation window, and delivery quality determine whether a decision is supported. Contribution margin can add useful context when reliable product costs are available, but it should not be fabricated from order totals alone.
Invisible Offer is built around this sequence: randomize, observe actual Shopify outcomes, build evidence, then decide whether the discount should change. AI-derived shopper signals may describe behavior; they do not replace the experiment as evidence of causality.
CONTINUE THE READING PATH
