A/B test conversion rate with a Beta prior

Question

A product team estimates the true conversion probability $p$ of a new landing page. Past pages make them cautious, so the prior is $\text{Beta}(4, 6)$ : centered around $0.4$ and moderately informative, like having already seen 4 conversions and 6 non-conversions. The new page gets 40 visitors, of whom 15 convert.

Using Bayesian updating, what is the posterior mean estimate of the conversion probability $p$ ?

Accepted Answer

Intuition first. A Beta prior is a bundle of pretend past outcomes. 	ext{Beta}(4,6) is like 4 pretend conversions and 6 pretend non-conversions, a mild opinion centered near 0.4. Updating is bookkeeping: add real conversions to the first number, real non-conversions to the second. The rule. Beta is conjugate to convert/not-convert data, so the posterior stays a Beta: 	ext{Beta}(a,b)\ +\ k 	ext{ conversions and } (n-k) 	ext{ non-conversions}\ ightarrow\ 	ext{Beta}(a+k,\ b+n-k). Plug in. Here a=4, b=6, with 15 conversions and 25 non-conversions: 	ext{posterior} = 	ext{Beta}(4+15,\ 6+25…

A/B test conversion rate with a Beta prior

Your answer

More Statistics questions