A strong CUPED covariate at correlation 0.8

Question

A subscription business runs CUPED on monthly spend per user. Because spending is very stable month to month, each user's prior-month spend correlates a strong $\rho = 0.8$ with the in-test metric.

By what percentage does CUPED reduce the variance of the metric?

Accepted Answer

Why a strong covariate is so useful. When behavior is stable over time, the pre-experiment value predicts the in-test value very well. Most of the spread in the metric is just users being their usual selves, which is pure noise for measuring a treatment effect. CUPED subtracts that predictable part, leaving a much cleaner metric. The formula. Variance is multiplied by (1 - ho^2): 	ext{new variance} = (1 - ho^2) 	imes 	ext{old variance}. Plug in ho = 0.8. ho^2 = 0.64, \qquad 1 - ho^2 = 0.36. The variance falls to just 36% of its original value, a 64% reduction. What that buys. A 6…

A strong CUPED covariate at correlation 0.8

Your answer

More Statistics questions