Partial and Semipartial Correlation

A raw signal has a 0.35 correlation with next-quarter returns, but it's also correlated with market cap, and small-cap stocks happen to have both higher average returns and higher average signal values. Is the signal actually predictive, or is it just a disguised size effect? You want to know the correlation between the signal and returns after removing what market cap explains — but there are two different, both-reasonable ways to do that removal, and they answer subtly different questions. Partial correlation removes the confound's influence from both variables; semipartial correlation removes it from only one.

An analogy: adjusting for a noisy microphone

Imagine two singers recorded in a room with a hum from a nearby generator that bleeds into both microphones. You want to know how well the two singers are actually harmonizing, independent of the shared hum. One approach: filter the hum out of both recordings before comparing them — this is like partial correlation, cleaning both sides. A different approach: filter the hum out of only one recording (say, keep the hum in singer A's raw track but remove it from singer B's) and then compare — this is like semipartial correlation, cleaning only one side. Both give you a "hum-adjusted" sense of the harmony, but they're not the same number, because the first comparison is entirely on cleaned ground while the second still has the raw hum affecting one side of the comparison.

The mechanics, one piece at a time

The partial correlation between $X$ and $Y$, controlling for $Z$, is the correlation between the residuals of $X$ after regressing out $Z$, and the residuals of $Y$ after regressing out $Z$:

$$r_{XY \cdot Z} = \frac{r_{XY} - r_{XZ}r_{YZ}}{\sqrt{(1-r_{XZ}^2)(1-r_{YZ}^2)}}.$$

In plain English: it's the leftover relationship between $X$ and $Y$ once you've stripped $Z$'s influence out of both of them — the answer to "how related are $X$ and $Y$ among observations that are otherwise similar on $Z$?"

The semipartial (or "part") correlation instead strips $Z$ out of only $X$ (or only $Y$), leaving the other variable in its raw form:

$$r_{Y(X \cdot Z)} = \frac{r_{XY} - r_{XZ}r_{YZ}}{\sqrt{1-r_{XZ}^2}}.$$

In plain English: it asks "how much of $Y$'s total, unadjusted variance can the $Z$-purified part of $X$ still explain?" — a slightly different, usually smaller-magnitude question, because the denominator only removes $Z$'s effect from one side, leaving more total variance in play than the fully-adjusted partial correlation's denominator does.

Worked example 1: signal vs. returns, controlling for size

Suppose $r_{XY} = 0.35$ (signal vs. return), $r_{XZ} = 0.50$ (signal vs. market cap), $r_{YZ} = 0.40$ (return vs. market cap). Partial correlation: $r_{XY\cdot Z} = \frac{0.35 - (0.50)(0.40)}{\sqrt{(1-0.25)(1-0.16)}} = \frac{0.35-0.20}{\sqrt{0.75 \times 0.84}} = \frac{0.15}{\sqrt{0.63}} = \frac{0.15}{0.794} \approx 0.189$. So roughly half of the raw 0.35 correlation was attributable to the shared size relationship — a meaningful, but far smaller, genuine relationship remains once size is controlled for on both sides.

Worked example 2: the semipartial version of the same numbers

Using the same $r_{XY}=0.35$, $r_{XZ}=0.50$, $r_{YZ}=0.40$: semipartial correlation of $Y$ with the $Z$-purified $X$ is $r_{Y(X\cdot Z)} = \frac{0.35 - (0.50)(0.40)}{\sqrt{1-0.25}} = \frac{0.15}{\sqrt{0.75}} = \frac{0.15}{0.866} \approx 0.173$. Close to the partial correlation (0.189) in this example, but not identical — the semipartial's denominator is larger (it doesn't shrink for $Z$'s effect on $Y$), so semipartial correlations are always equal to or smaller in magnitude than the corresponding partial correlation. This semipartial number has a direct, practical interpretation: its square, $0.173^2 \approx 0.030$, is exactly the additional variance in raw, unadjusted returns explained by the signal once market cap is already in a regression — the number a stepwise regression's incremental $R^2$ reports.

Drag the correlation strength below to build intuition for what "0.19" versus "0.35" actually looks like scattered on a page — this is the strength of the leftover relationship after Z has been stripped out.

What this means in practice

Partial correlation is the right tool when you want a symmetric, apples-to-apples measure of relationship strength after controlling for a confound on both sides — useful for factor research, comparing signal-return relationships across a controlled universe. Semipartial correlation is the right tool when you specifically care about how much unexplained variance in the raw outcome a new variable adds on top of what's already known, which is exactly the quantity reported as incremental $R^2$ when adding a variable to a regression one step at a time.