Detecting a Biased Coin From Few Flips

Question. You flip a coin 10 times and get 7 heads. Is the coin biased? Most people's gut reaction is "70% heads, sounds biased", but the honest answer, worked out properly, is that 10 flips give you almost no ability to distinguish a fair coin from a moderately biased one. This question is really testing whether you understand statistical power: how much evidence a given sample size can actually provide, separate from whether the observed number "looks" unusual.

Setting it up: what would a fair coin do?

If the coin is fair, the number of heads in 10 flips follows a Binomial(10, 0.5) distribution. The probability of getting exactly 7 heads is

That alone isn't damning, plenty of individually-unlikely outcomes happen routinely (every specific sequence of 10 flips has probability $1/1024$). What matters for "is this evidence of bias" is the probability of getting a result at least this extreme, in either direction, under the fair-coin assumption.

Worked example: computing the two-sided p-value

Computing: $\binom{10}{7}=120$, $\binom{10}{8}=45$, $\binom{10}{9}=10$, $\binom{10}{10}=1$, summing to 176, so $P(X\geq 7) = 176/1024 \approx 17.2\%$, and the two-sided figure is about $34.4\%$. A result this extreme (or more) happens roughly one time in three just from a perfectly fair coin, far too common to count as meaningful evidence of bias by any conventional standard (typically you'd want this probability below 5% before calling it suspicious).

How many flips would actually settle it?

To reliably detect, say, a coin biased at 60% heads instead of 50%, you need enough flips that the sampling distributions under the two hypotheses barely overlap. Roughly, distinguishing a $p=0.5$ coin from a $p=0.6$ coin with reasonable confidence needs on the order of a few hundred flips, not ten, the gap between the two hypotheses (0.1 in probability) is small relative to the natural flip-to-flip noise, and that noise only shrinks proportional to $1/\sqrt{n}$. Going from 10 flips to 1,000 flips only cuts your uncertainty band by a factor of 10, not 100, noise shrinks slowly.

Drag $n$ up in the explorer above and watch how the binomial distribution's spread narrows relative to its center, at $n=10$ the distribution is wide enough that 7 heads sits well within normal fair-coin variation; only at much larger $n$ does a 70%-heads outcome become genuinely surprising under a fair-coin hypothesis.