Detecting a Biased Coin From Few Flips

Question. You flip a coin 10 times and get 7 heads. Is the coin biased? Most people's gut reaction is "70% heads, sounds biased" — but the honest answer, worked out properly, is that 10 flips give you almost no ability to distinguish a fair coin from a moderately biased one. This question is really testing whether you understand statistical power: how much evidence a given sample size can actually provide, separate from whether the observed number "looks" unusual.

Setting it up: what would a fair coin do?

If the coin is fair, the number of heads in 10 flips follows a Binomial(10, 0.5) distribution. The probability of getting exactly 7 heads is

$$P(X=7) = \binom{10}{7}(0.5)^{10} = 120 \times \frac{1}{1024} \approx 11.7\% .$$

That alone isn't damning — plenty of individually-unlikely outcomes happen routinely (every specific sequence of 10 flips has probability $1/1024$). What matters for "is this evidence of bias" is the probability of getting a result at least this extreme, in either direction, under the fair-coin assumption.

Worked example: computing the two-sided p-value

$$P(X \geq 7 \text{ or } X \leq 3) = 2 \times P(X \geq 7) = 2\left[\binom{10}{7}+\binom{10}{8}+\binom{10}{9}+\binom{10}{10}\right](0.5)^{10} .$$

Computing: $\binom{10}{7}=120$, $\binom{10}{8}=45$, $\binom{10}{9}=10$, $\binom{10}{10}=1$, summing to 176, so $P(X\geq 7) = 176/1024 \approx 17.2\%$, and the two-sided figure is about $34.4\%$. A result this extreme (or more) happens roughly one time in three just from a perfectly fair coin — far too common to count as meaningful evidence of bias by any conventional standard (typically you'd want this probability below 5% before calling it suspicious).

How many flips would actually settle it?

To reliably detect, say, a coin biased at 60% heads instead of 50%, you need enough flips that the sampling distributions under the two hypotheses barely overlap. Roughly, distinguishing a $p=0.5$ coin from a $p=0.6$ coin with reasonable confidence needs on the order of a few hundred flips, not ten — the gap between the two hypotheses (0.1 in probability) is small relative to the natural flip-to-flip noise, and that noise only shrinks proportional to $1/\sqrt{n}$. Going from 10 flips to 1,000 flips only cuts your uncertainty band by a factor of 10, not 100 — noise shrinks slowly.

Drag $n$ up in the explorer above and watch how the binomial distribution's spread narrows relative to its center — at $n=10$ the distribution is wide enough that 7 heads sits well within normal fair-coin variation; only at much larger $n$ does a 70%-heads outcome become genuinely surprising under a fair-coin hypothesis.