A 20% versus 24% test on 1500 users each

Question

An onboarding experiment runs 1500 users per arm. Control completes at $20\%$ (300 users); treatment completes at $24\%$ (360 users).

Compute the two-sided p-value for the difference in completion rates using a two-proportion z-test.

Accepted Answer

Pooled rate under the null. \hat{p} = \frac{300 + 360}{1500 + 1500} = \frac{660}{3000} = 0.22. Standard error. SE = \sqrt{0.22 	imes 0.78 	imes \left(	frac{1}{1500} + 	frac{1}{1500}ight)}. Here 0.22 	imes 0.78 = 0.1716, times 	frac{2}{1500} = 0.0013333, giving 0.00022879, and \sqrt{0.00022879} \approx 0.015127. z and p. The observed difference is 0.24 - 0.20 = 0.04: z = \frac{0.04}{0.015127} \approx 2.64, \qquad p = 2\,[1 - \Phi(2.64)] = 2 	imes 0.00415 \approx 0.008. Interpretation. At p \approx 0.008, the lift is significant at both the 5% and the 1% levels. If the true completion …

A 20% versus 24% test on 1500 users each

Your answer

Discussion

Learn the concepts

Related questions