Signal Health Monitoring

A pilot doesn't wait for the engine to stop mid-flight to check its health — there's a panel of gauges: oil pressure, fuel flow, temperature, all watched continuously so a small problem shows up as a needle drifting off-normal long before it becomes an emergency. A live trading signal needs the same panel. Its "engine" is a statistical relationship between the signal and future returns, and that relationship can weaken slowly — crowding, a changed market regime, a data vendor quietly altering their feed — long before the strategy's P&L makes it obvious. Signal health monitoring is building the gauges and deciding, in advance, what a red needle looks like.

The core gauge: rolling information coefficient

The information coefficient (IC) is the correlation between a signal's cross-sectional ranking and the return that follows it. Tracked over time as a rolling window, it becomes the primary gauge:

$IC_t = \text{corr}(\text{signal}_{i,t}, \, r_{i,t+1})$

In words: each period, rank stocks by the signal, rank them by what they actually returned next, and measure how well those two rankings agree — a number between $-1$ and $1$ where higher (positive) means the signal is doing its job. A healthy signal's rolling IC bounces around a stable, positive mean; a decaying one shows a visible downward trend in that mean, even while any single month's IC is too noisy to draw conclusions from alone.

Because a single period's IC is dominated by noise, monitoring uses its standard error to judge whether a run of weak months is a real trend or normal variation:

$SE(\overline{IC}) = \frac{\sigma_{IC}}{\sqrt{n}}$

In words: the uncertainty in an average IC over $n$ periods shrinks with the square root of how many periods you've averaged — one bad month is nearly meaningless, but twelve bad months in a row, with a standard error small enough that the average is statistically indistinguishable from zero, is a real signal of decay, not noise (see IC Standard Errors and Significance).

Worked example 1. A signal has run with average monthly $IC = 0.04$ and a period-to-period standard deviation $\sigma_{IC} = 0.12$ over its live history. Over the last 6 months, the average IC has dropped to $0.005$. Is that a real deterioration or noise? The standard error of a 6-month average is $SE = 0.12/\sqrt{6} \approx 0.049$. The observed $0.005$ is well within one standard error of the historical mean of $0.04$ ($0.04 - 0.005 = 0.035 < 0.049$) — not yet statistically distinguishable from normal variation. A monitoring system that fires an alarm here is likely crying wolf; one that waits for 24 months of the same pattern, where $SE = 0.12/\sqrt{24} \approx 0.024$ and the gap of $0.035$ now clears that bar, has real evidence.

Beyond IC: hit rate and turnover drift

Worked example 2. IC alone can hide a specific failure mode: a signal that still ranks stocks correctly on average but has stopped making money because its turnover has crept up, so trading costs now eat what the IC still delivers. Suppose a signal's IC has held steady at $0.04$, but its monthly portfolio turnover has risen from 15% (at launch) to 45% (a year later) as the underlying data has gotten noisier and the ranking flips positions more often. At 15% turnover and 10bps round-trip cost, monthly cost drag is $0.15 \times 0.10\% \times 2 = 0.03\%$. At 45% turnover, it's $0.45 \times 0.10\% \times 2 = 0.09\%$ — a threefold increase in cost drag with an unchanged IC. A monitoring dashboard that only tracks IC would show a green light while net returns quietly turn negative.

What this means in practice

Real monitoring systems set pre-committed thresholds before a signal goes live — for instance, "if the 12-month rolling IC average falls below one standard error of zero for two consecutive quarters, cut allocation by half and escalate for review" — precisely because a researcher watching their own signal decay in real time is prone to explaining away every bad month as noise until the losses are undeniable. The threshold has to be set by someone without an emotional stake in the signal staying alive, or written down far enough in advance that hindsight can't move it.