Range Restriction in Selected Samples

A hedge fund wants to know whether its hiring interview score predicts on-the-job trading performance. It computes the correlation between interview scores and one-year P&L, but only among people it actually hired — the correlation comes out to a disappointing 0.15, barely above noise, and the firm concludes the interview process isn't very predictive. But this measurement is quietly rigged against finding a strong relationship: the firm only hired people who scored well on the interview in the first place, so the sample being correlated has almost no low-scoring people in it at all. Correlation needs variation to reveal itself, and hiring has already chopped off most of the variation in the one variable (interview score) that the correlation depends on.

An analogy: judging a shoe size vs. height relationship using only NBA players

If you tried to measure the correlation between height and shoe size using only NBA players, you'd find it weak — nearly everyone in the sample is between 6'3" and 7'0", too narrow a slice of the full height range for much variation to correlate against. Measure the same relationship across the general population, where heights range from 5'0" to 6'8", and the correlation is much stronger, because there's far more genuine variation for shoe size to track. The underlying physical relationship hasn't changed — only the range of height you're allowed to see has been artificially narrowed by pre-selecting a group already filtered on a variable correlated with height.

The mechanics, one piece at a time

If the true correlation in the full population is $\rho$, and a selection process restricts the standard deviation of one variable (say $X$) from $\sigma_X$ to a smaller $\sigma_{X}'$ in the observed subgroup, the classic range-restriction correction (Pearson-Thorndike formula) estimates the population correlation from the restricted-sample correlation $r'$ as

$$\hat{\rho} = \frac{r' \left(\frac{\sigma_X}{\sigma_X'}\right)}{\sqrt{1 - r'^2 + r'^2 \left(\frac{\sigma_X}{\sigma_X'}\right)^2}}.$$

In plain English: the formula inflates the observed, artificially-shrunk correlation back up in proportion to how much the selection process narrowed $X$'s range — the more severely $X$'s variation was restricted (the smaller $\sigma_X'/\sigma_X$), the larger the correction needed to recover an honest estimate of the true, unrestricted relationship.

Worked example 1: correcting the interview-score correlation

Suppose the full applicant pool's interview scores have a standard deviation of $\sigma_X = 15$ points, but among people actually hired (who all scored above a cutoff), the standard deviation shrinks to $\sigma_X' = 6$ points — hiring has restricted the range to 40% of its original spread. The observed correlation among hires is $r' = 0.15$. Applying the correction with $\sigma_X/\sigma_X' = 15/6 = 2.5$: numerator $= 0.15 \times 2.5 = 0.375$; denominator $= \sqrt{1 - 0.15^2 + 0.15^2 \times 2.5^2} = \sqrt{1-0.0225+0.1406} = \sqrt{1.118} \approx 1.057$. So $\hat{\rho} \approx 0.375/1.057 \approx 0.355$ — more than double the naively observed 0.15. The interview process may actually be a substantially better predictor than the hired-only sample suggested; the weak observed correlation was largely a mechanical consequence of only ever measuring it among people who'd already cleared a high bar.

Worked example 2: a stock screen and factor correlation

A quant screens for stocks with a quality score above the 70th percentile before backtesting a value signal within that screened universe, finding the value-signal-to-return correlation is only 0.08. Quality and value are themselves correlated ($\rho_{QV} = 0.3$ in the full universe), so screening on quality has indirectly restricted the range of the value signal too — high-quality stocks cluster in a narrower value-score band than the full universe does. An analogous correction can reveal a true unrestricted value-return correlation closer to 0.18–0.20 — the screen didn't make value stop working, it made the measurement of value's effectiveness look weaker than it truly is.

Set the correlation near the corrected estimate below (about 0.35, the interview-score example's true population value) and imagine only the rightmost sliver of the x-axis being visible — that sliver is what a hired-only sample sees.

What this means in practice

Range restriction quietly understates correlations anywhere a sample has already been filtered on a variable related to what you're measuring: performance evaluation of hired employees, backtests run only on stocks that survived a prior screen, or factor efficacy measured only within an already-curated investable universe. Recognizing it prevents the mistaken conclusion "this predictor doesn't work" when the real issue is "this predictor's variation has been artificially narrowed by the selection process itself."