The Duality of Tests and Confidence Intervals

Textbooks teach hypothesis testing and confidence intervals as two separate techniques, with two separate sets of formulas to memorize. In fact they are the same underlying computation, read in two different directions. If you understand one, you already understand the other — you've just been looking at the same object from a different angle. Seeing this saves you from memorizing twice as many formulas as necessary, and it also explains why a "95% confidence interval that excludes zero" and "rejecting $H_0: \theta=0$ at the 5% level" are always, without exception, the same statement.

An analogy: a bouncer's guest list

Imagine a bouncer who tests, one name at a time, "would this specific name be allowed in tonight?" If you ask the bouncer about every possible name in the world, the ones that pass form a guest list — and that guest list is the confidence interval. Running a single hypothesis test is asking the bouncer about one particular name. Building the confidence interval is asking about every name and writing down who gets in. They are the same bouncer, doing the same check, just asked once versus asked for everyone.

The rule, one symbol at a time

Suppose for every candidate value $\theta_0$ you have a level-$\alpha$ test of $H_0: \theta = \theta_0$ — a rule that rejects or fails to reject, controlling the false-rejection rate at $\alpha$. Define the set

$$C(X) = \{\, \theta_0 : \text{the test of } H_0: \theta=\theta_0 \text{ does NOT reject, given data } X \,\} .$$

In plain English: collect every candidate value that the test would let through, and call that collection your confidence set. The duality theorem says this $C(X)$ is automatically a valid $(1-\alpha)$ confidence interval — meaning it contains the true $\theta$ with probability $1-\alpha$ across repeated sampling — precisely because each individual test controls its false-rejection rate at $\alpha$. Running the theorem backward: given a $(1-\alpha)$ confidence interval, you can test any $H_0: \theta=\theta_0$ by simply checking whether $\theta_0$ falls inside the interval — reject if it doesn't, fail to reject if it does. Both directions are exactly equivalent; neither adds information the other lacks.

Slide a hypothesized $\theta_0$ across the horizontal axis of a curve like the one above, re-centered at your estimate. Every value that would sit inside the ordinary bulk of the curve fails to be rejected; sweeping across all of them traces out exactly the confidence interval.

Worked example 1: a t-test and a confidence interval agree

A sample of $n=30$ trade returns has mean $\bar{X} = 1.2\%$ and standard error $\text{SE} = 0.5\%$. Test $H_0: \mu = 0$ at $\alpha=0.05$ (two-sided, using the normal approximation for simplicity). The test statistic is $z = (1.2-0)/0.5 = 2.4$, and since $|2.4| > 1.96$, reject $H_0$.

Now build the 95% confidence interval directly: $\bar{X} \pm 1.96 \times \text{SE} = 1.2 \pm 1.96(0.5) = 1.2 \pm 0.98$, so the interval runs from $0.22\%$ to $2.18\%$. Does it contain $0$? No — $0$ is well below $0.22$. The interval excludes the null value exactly when the test rejects it. Same conclusion, same underlying arithmetic, two presentations.

Worked example 2: sweeping through candidate values by hand

Using the same $\bar{X}=1.2$, $\text{SE}=0.5$, test three different candidate null values directly, instead of building the interval formula: for $\theta_0 = 0.3$, $z = (1.2-0.3)/0.5 = 1.8$, and since $|1.8| < 1.96$, do NOT reject — so $0.3$ belongs in the confidence set. For $\theta_0 = 0.1$, $z = (1.2-0.1)/0.5 = 2.2$, and $|2.2| > 1.96$, so reject — $0.1$ is excluded. For $\theta_0 = 2.0$, $z = (1.2-2.0)/0.5 = -1.6$, and $|-1.6| < 1.96$, so do NOT reject — $2.0$ belongs in the set. Notice these three verdicts ($0.3$ in, $0.1$ out, $2.0$ in) are exactly consistent with the interval $[0.22, 2.18]$ computed in Example 1, point by point, confirming the duality directly rather than by formula.

What this means in practice

• You only need to build one machine. Implement a valid hypothesis test once, and inverting it for every candidate value gives you a confidence interval for free — this is literally how software computes many nonstandard confidence intervals.
• Reading a report. If a paper reports "the 95% CI for alpha is [0.02, 0.09]," you already know that a test of $H_0: \alpha=0$ at the 5% level rejects, without being told the test statistic at all.
• Asymmetric or irregular intervals. Because the duality holds test-by-test, it works even when the interval isn't a simple symmetric "estimate plus or minus," which is exactly how confidence intervals are built for skewed statistics like variances or odds ratios.