Treat one city, use the neighboring city as control?

Question

To test a delivery promotion, a team turns it on for all customers in City A and uses City B, a similar nearby city, as the control. Sales rise more in City A, and the team declares the promotion a winner.

Why is a whole-city treatment versus a whole-city control a shaky comparison, and how would randomizing help?

Accepted Answer

Two cities differ in countless ways. Even similar cities differ in income, age mix, weather, local competition, a different sports season, a factory that just opened or closed, and dozens of other factors. Any of these can move sales on its own, so each is a potential confounder. With only one city on each side, you cannot tell the promotion's effect from a background difference between the two places, and there is no way to adjust for the many differences you never measured. One unit per arm gives no traction. Because there is a single city in treatment and a single city in control, city-lev…

Treat one city, use the neighboring city as control?

Your answer

More Statistics questions