The IC t-Statistic and Its Pitfalls

A researcher backtests a signal and computes its information coefficient (IC) — the correlation each period between the signal and subsequent returns — averaged across many periods. The average IC looks decent, say 0.04. Is that real skill, or could it plausibly be noise? The natural instinct is to run a standard t-test: divide the average IC by its standard error. That instinct is right in spirit, but the naive version of this test — treating every period's IC as an independent draw — is one of the most common ways researchers fool themselves in alpha research, because monthly or daily ICs are rarely as independent as the formula assumes.

An analogy: polling the same undecided voters every week

Imagine trying to measure whether a candidate's support is above 50% by polling the same 1,000 people every single week and treating each week's poll as a fresh, independent data point. It isn't — the same people's opinions carry over from week to week, so ten weekly polls give you far less genuine new information than ten polls of ten different groups of 1,000 people would. If you compute a standard error assuming ten independent samples, you'll understate your uncertainty and overstate your confidence in the estimate. A signal's period-by-period IC has exactly this problem: successive periods' returns and characteristics are serially correlated (momentum, mean reversion, macro regimes persisting), so the naive count of "number of periods" overstates the true amount of independent information.

The idea, one symbol at a time

Let $IC_1, \ldots, IC_T$ be the information coefficient measured in each of $T$ periods, with sample mean $\overline{IC}$ and sample standard deviation $s_{IC}$. The naive t-statistic, assuming independence across periods, is:

$$t_{\text{naive}} = \frac{\overline{IC}}{s_{IC}/\sqrt{T}} .$$

In plain English: divide the average IC by its estimated standard error, where the standard error shrinks with the square root of the number of periods — exactly the standard-error formula that assumes each period is an independent piece of evidence. If periods are actually autocorrelated, the true number of independent observations is smaller than $T$, so a corrected version uses an effective sample size $T_{\text{eff}} < T$:

$$T_{\text{eff}} \approx T \times \frac{1 - \phi}{1 + \phi} ,$$

where $\phi$ is the lag-1 autocorrelation of the IC series (a rough approximation valid for simple AR(1)-type persistence). In plain English: the stronger the period-to-period persistence $\phi$, the fewer genuinely independent observations you actually have, even though you measured $T$ periods — and the corrected t-statistic $t_{\text{corrected}} = \overline{IC}/(s_{IC}/\sqrt{T_{\text{eff}}})$ is smaller (less significant) than the naive one whenever $\phi > 0$.

Worked example 1: naive versus corrected, monthly IC

A signal is tested over $T = 60$ monthly periods, with $\overline{IC} = 0.04$ and $s_{IC} = 0.15$. Naive t-statistic:

$$t_{\text{naive}} = \frac{0.04}{0.15/\sqrt{60}} = \frac{0.04}{0.0194} \approx 2.07 .$$

This clears the usual "$|t| > 2$" bar for significance. But suppose the IC series has lag-1 autocorrelation $\phi = 0.5$ (moderate persistence, plausible for a slow-moving factor):

$$T_{\text{eff}} \approx 60 \times \frac{1 - 0.5}{1 + 0.5} = 60 \times \frac{0.5}{1.5} = 20 .$$

Corrected t-statistic: $t_{\text{corrected}} = 0.04 / (0.15/\sqrt{20}) = 0.04/0.0335 \approx 1.19$ — well below the significance threshold. The naive test's "significant" finding evaporates once persistence is properly accounted for.

Worked example 2: a fast, low-persistence signal for contrast

Now a different signal, tested over the same $T=60$ months with the same $\overline{IC}=0.04$, $s_{IC}=0.15$, but this time it's a fast-turnover signal with negligible persistence, $\phi = 0.05$:

$$T_{\text{eff}} \approx 60 \times \frac{1 - 0.05}{1 + 0.05} = 60 \times 0.905 \approx 54.3 .$$

Corrected t-statistic: $0.04/(0.15/\sqrt{54.3}) = 0.04/0.0203 \approx 1.97$ — barely changed from the naive 2.07, and still near the significance threshold. Two signals with identical raw ICs and identical $T$ can have wildly different true statistical confidence, purely because of how persistent their period-to-period IC values are.

What this means in practice

Before trusting a backtested IC t-statistic, a researcher should always check the autocorrelation of the period-by-period IC series, not just its mean and standard deviation — this is especially important for slower, longer-horizon signals (value, quality) which tend to have highly persistent ICs, versus fast signals (short-term reversal) which tend not to. Standard fixes include using Newey-West or block-bootstrap standard errors instead of the naive formula, or explicitly computing an effective sample size as shown here, before declaring a signal statistically significant.