Incrementality Testing for Mobile Apps
Attribution tells you which channel got the credit. Incrementality tells you which spend actually caused installs. They are not the same thing.
Key takeaways
- Incrementality measures the installs you would not have gotten without the spend, not just the attributed ones.
- Geo-tests and holdout groups are the practical ways to measure it.
Attribution answers who gets credit for an install. Incrementality answers a harder and more important question: would that install have happened anyway? The gap between the two is where a lot of UA budget quietly leaks.
Why attribution overstates paid
Last-touch models hand credit to whichever ad a user saw last, even if they were already going to install. Branded search and retargeting are the worst offenders, often claiming installs that organic would have delivered for free.
How to test it
The clean methods are geo-experiments (turn a channel off in matched regions and compare) and holdout groups (withhold ads from a random slice and measure the difference). Both isolate the true causal lift of the spend.
When it is worth it
Incrementality testing pays off on large budgets, on retargeting and brand, and before big channel decisions. You do not run it weekly, but when a channel looks suspiciously efficient, it is how you find out if the lift is real.
Suspect a channel is taking credit it did not earn?
Get a UA audit