New checkout in one region: the diff-in-diff estimate

Question

You launch a new checkout flow in Region A but not in Region B, a comparable market you keep as a control. Conversion rates before and after:

Region A (treated): 10% before, 14% after.
Region B (control): 10% before, 11% after.

Compute the difference-in-differences estimate of the new checkout's effect, and explain what it corrects for that a simple before-and-after would miss.

Accepted Answer

The naive number is too generous. A before-and-after look at Region A alone says conversion rose 4 points (from 10% to 14%) and credits all of it to the checkout. But something lifted Region B too, which got no new checkout: it rose 1 point (from 10% to 11%). That 1 point is the background trend, seasonality, a broad marketing tailwind, general growth, that would have raised Region A as well, checkout or not. Difference-in-differences. The idea is to subtract the control group's change from the treated group's change: 	ext{DiD} = (A{	ext{after}} - A{	ext{before}}) - (B{	ext{after}} - B{	…

New checkout in one region: the diff-in-diff estimate

Your answer

Discussion

Learn the concepts

Related questions