Sample size to compare two execution algos

Question

You want to compare the mean execution cost of two routing algos, A and B, by splitting order flow between them. Each algo's per-trade cost has a standard deviation of about $\sigma = 5$ bps. You want to be able to detect a true mean difference of $\Delta = 2$ bps, using a two-sided test at $\alpha = 0.05$ with $80\%$ power.

Roughly how many trades per arm do you need?

Accepted Answer

The formula. For comparing two independent means with equal variance and equal allocation, the sample size per arm is n \approx \frac{2\,\sigma^2 \left(z{\alpha/2} + z\betaight)^2}{\Delta^2}. The factor of 2 appears because the difference of two group means carries the noise of both groups. The constants. z{\alpha/2} = 1.96 (two-sided 5%) and z\beta = 0.84 (80% power), so (1.96 + 0.84)^2 = (2.80)^2 = 7.84. Plug in. \sigma^2 = 25, \Delta^2 = 4: n \approx \frac{2 	imes 25 	imes 7.84}{4} = \frac{392}{4} \approx 98. Rounding up, you need about 99 trades per arm, roughly 200 total. People rout…

Sample size to compare two execution algos

Your answer

More Statistics questions