Power for a longer session-time test

Question

A product team tests whether a redesign raises the mean session time above the old baseline of $40$ seconds: $H_0: \mu = 40$ against $H_1: \mu > 40$ . Session times have a known standard deviation of $\sigma = 15$ s. They collect $n = 225$ sessions and test at one-sided $\alpha = 0.05$ . Suppose the true mean is $\mu = 43$ s.

What is the power of this test?

Accepted Answer

Standard error. \sigma/\sqrt{n} = 15/\sqrt{225} = 15/15 = 1.0 s. Effect in SE units. The true mean is (43 - 40)/1.0 = 3.0 standard errors above the null, a big gap because the large sample made the standard error tiny. Power. 	ext{power} = \Phi(	ext{effect} - z\alpha) = \Phi(3.0 - 1.645) = \Phi(1.355) \approx 0.91. The true lift here (3 seconds) is not large in absolute terms, but 225 sessions shrink the standard error to 1 second, so the effect stands out clearly and the test detects it about 91% of the time. Takeaway. Power depends on the effect measured in standard errors, not raw units.…

Power for a longer session-time test

Your answer

Discussion

Learn the concepts

Related questions