A p-value of 0.03: what does it decide, and what does it mean?

Question

A study reports a p-value of 0.03. A colleague says, "So there is a 3% chance the null hypothesis is true." At the 5% level, what is the decision, and is your colleague's interpretation correct?

Accepted Answer

The decision rule: reject H0 if the p-value is below \alpha. Here 0.03 < 0.05, so reject H0. The colleague is wrong. The p-value is not the probability that the null is true. It is the probability of observing data at least as extreme as what was seen, computed assuming the null is true. Those are different statements: the p-value conditions on the null, it does not measure the null's probability. (For reference, at a stricter \alpha = 0.01 the same p-value of 0.03 would not be significant, since 0.03 0.01.) Takeaway. A p-value answers "how surprising is this data if the null holds?", not "ho…

A p-value of 0.03: what does it decide, and what does it mean?

Your answer

More Statistics questions