Same edge, longer sample, much smaller p

Question

A signal with a fixed per-observation signal-to-noise ratio of $0.1$ is now measured over $n = 900$ observations. Its z-statistic is $z = 0.1\sqrt{n}$ .

Compute the two-sided p-value at $n = 900$ , and explain why it is so much smaller than the $p \approx 0.32$ this same edge produced at $n = 100$ .

Accepted Answer

The computation. At n = 900: z = 0.1 	imes \sqrt{900} = 0.1 	imes 30 = 3.0. The two-sided p-value is p = 2\,[1 - \Phi(3.0)] = 2 	imes 0.00135 = 0.0027. Why it shrank. The per-observation edge is identical to the n = 100 case, where z = 0.1 	imes 10 = 1.0 gave p \approx 0.32. Nothing about the strategy improved. What changed is the precision of the estimate: with nine times the data, the noise in the estimated mean falls by a factor of \sqrt{9} = 3, so the same true edge stands out three times more clearly. The z tripled and the p-value collapsed. The caution. A p-value of 0.0027 looks imp…

Same edge, longer sample, much smaller p

Your answer

More Statistics questions