What is incremental conversion lift?

How is incremental lift calculated?

Subtract the control group's conversion rate from the treated group's. Expressed relatively, divide that difference by the control rate: a control at 2.0% and a treated group at 2.3% is 0.3 percentage points absolute, or 15% relative.

Both forms are correct and they are wildly different numbers, which is why the form should always be stated. A relative lift of 15% sounds like a transformation; the same result described as three extra conversions per thousand visitors sounds like what it is. Neither framing is dishonest — reporting only the flattering one is.

The comparison is only meaningful if the two groups were assigned at random beforehand. Everything else in this calculation is arithmetic; that part is the measurement.

Why is incremental lift lower than the uplift most tools report?

Because most reported uplift includes conversions that would have happened anyway. Attributing every conversion that followed an intervention to that intervention counts the people who were already going to buy.

This is the single largest source of overstated results in conversion optimization, and it is rarely deliberate. A visitor who was ready to convert is also the visitor most likely to be shown something and most likely to engage with it. Counting them credits the system for an outcome it inherited.

Incrementality is the discipline of asking what would have happened otherwise, and a control group is the only way to answer it that does not require a model of what would have happened otherwise.

When is a lift number trustworthy?

When you can say what the two groups were and who decided who went in which. If the answer is 'visitors who saw it versus visitors who did not', the number describes the system's selection as much as its effect.

The second question is volume. A difference between two rates has an uncertainty attached, and on small numbers that uncertainty is usually larger than the difference. A lift figure quoted without any sense of its confidence interval is a headline, not a measurement — the honest version says how sure it is.