Setting the threshold when the two errors cost differently

Question

A signal either fires (reject H0) or stays quiet. A false alarm costs \1; a missed true signal costs \20. Your team reflexively uses \alpha = 0.05. Explain why 5% is a convention rather than an optimum here, how the Neyman–Pearson framework and the relative costs should set the threshold, and what the Neyman–Pearson lemma guarantees.

Accepted Answer

Why 5% is arbitrary. The 0.05 cutoff is a historical convention, not a law. It fixes the type I error rate at 5% regardless of what a type I or type II error actually costs you. When the two errors have very different consequences, blindly controlling only \alpha leaves money on the table. The decision-theoretic view. Suppose the null and alternative are each somewhat plausible, and you want to minimize expected cost. Firing when you shouldn't costs \1; staying quiet when you should fire costs \20, a 20:1 penalty for misses. That asymmetry argues for a trigger-happy rule: accept a higher fals…

Setting the threshold when the two errors cost differently

Your answer

More Statistics questions