Type I and Type II Errors

Any test that answers "is this real, or is it just noise?" can be wrong in exactly two ways. It can raise a false alarm, shouting "real!" when nothing is there. Or it can miss, staying quiet when something real is happening. Statisticians give these two mistakes deliberately dull names, type I and type II, but the ideas are simple, and keeping them straight is the single most tested piece of statistics in a quant interview.

Every test starts from a skeptical default, the null hypothesis $H_0$: "there is no effect" (this coin is fair, this signal has no edge). The rival claim is the alternative $H_1$: "there is an effect." You look at the data and either reject the null or don't. Cross that decision with what's actually true and you get four boxes, two right and two wrong.

	$H_0$ is true (no effect)	$H_1$ is true (real effect)
Reject $H_0$	Type I error — false alarm	correct — you caught it
Don't reject $H_0$	correct — quiet rightly	Type II error — a miss

The type I error rate is called $\alpha$ (alpha): the chance of a false alarm when nothing is really there. The type II error rate is called $\beta$ (beta): the chance of missing a real effect. The flip side of a miss is the power of the test, $1-\beta$, the chance you do catch a real effect. You choose $\alpha$ up front, usually 5%, and then try to make $\beta$ as small as you can.

Why you can't shrink both at once

Look at the dashed cutoff. Move it right to shrink the red $\alpha$ region, and the amber $\beta$ region swells, you're now so cautious you miss real effects. Move it left to catch more real effects and the false-alarm area balloons. With a fixed amount of data, tightening one error loosens the other. The only way to shrink both is to pull the two curves apart or make them skinnier, and the honest way to make them skinnier is to collect more data. Each curve's width is a sampling distribution, and its spread falls as you gather more observations.

Worked example: testing a coin with 100 flips

You suspect a coin is biased toward heads. $H_0$: it's fair, $p = 0.5$. $H_1$: it's loaded, $p = 0.6$. You flip it $n = 100$ times and will call it biased if you see enough heads.

Under $H_0$ the number of heads is centered at $50$ with a standard deviation of $\sqrt{100 \cdot 0.5 \cdot 0.5} = 5$. For a one-sided 5% test you reject when heads exceed $50 + 1.645 \times 5 \approx 58$. That threshold defines $\alpha \approx 0.05$: a fair coin lands above 58 heads about 5% of the time, and each of those is a false alarm.

Now the miss rate. If the coin really is loaded ($p = 0.6$), heads are centered at $60$ with standard deviation $\sqrt{100 \cdot 0.6 \cdot 0.4} \approx 4.9$. You fail to reject when heads come in below 58:

$$\beta = \mathbb{P}(\text{heads} < 58 \mid p = 0.6) \approx \mathbb{P}\!\left(Z < \frac{58 - 60}{4.9}\right) = \mathbb{P}(Z < -0.37) \approx 0.36.$$

So even with a real 10-percentage-point bias, you'd miss it about a third of the time. The power is only $1 - 0.36 = 0.64$. Bump the sample to $n = 400$ and the same math gives $\beta \approx 0.01$, power $\approx 0.99$. Four times the data crushes the miss rate. That $\sqrt{n}$ lever, not cleverness, is what buys you a reliable test.

Common pitfalls

• Swapping the two. Type I is the false alarm ($\alpha$, "crying wolf"); type II is the miss ($\beta$, "wolf slips by"). Mixing them up is an instant tell in an interview. A useful anchor: type I rejects a true null.
• Treating "didn't reject" as "no effect." A miss is common when data are short and effects are small, exactly the case in finance. Failing to reject usually means underpowered, not nothing there. See Hypothesis Testing.
• Forgetting power entirely. People obsess over $\alpha$ and never ask about $\beta$. A test with 5% false alarms but only 30% power will overlook most real signals; report power, not just significance.
• Running many tests. Each test at $\alpha = 5\%$ is a fresh chance at a false alarm. Screen 100 dead signals and you'll "discover" about 5 by luck alone. See p-values and Multiple Testing.

The one-sentence version: $\alpha$ is how often you fool yourself into seeing something that isn't there, $\beta$ is how often you overlook something that is, and the only free lunch that shrinks both is more data.