Sample size for a latency test at 90% power

Question

You want to compare the mean response latency of two servers. Each server's per-request latency has a standard deviation of about $\sigma = 10$ ms. You want to detect a true mean difference of $\Delta = 5$ ms, using a two-sided test at $\alpha = 0.05$ , but this time with $90\%$ power.

Roughly how many requests per server do you need?

Accepted Answer

The formula. n \approx \frac{2\,\sigma^2 \left(z{\alpha/2} + z\betaight)^2}{\Delta^2}. Constants. For 90% power, z\beta = 1.28, so (z{\alpha/2} + z\beta)^2 = (1.96 + 1.28)^2 = (3.24)^2 = 10.5. Note this is larger than the 7.84 used for 80% power: demanding more power raises the multiplier. Plug in. \sigma^2 = 100, \Delta^2 = 25: n \approx \frac{2 	imes 100 	imes 10.5}{25} = \frac{2101.6}{25} \approx 85. So about 85 requests per server. The effect-to-noise ratio is the same as a half-sigma difference, but the jump from 80% to 90% power inflated the count by roughly a third. Takeaway. Raisi…

Sample size for a latency test at 90% power

Your answer

Discussion

Learn the concepts

Related questions