From Raw Data Field to Tradeable Signal

Buy a dataset of, say, quarterly analyst revision counts, and what you own is a table of numbers with a company ID and a date attached. That table is not a signal. A signal is a number, one per name per day, that you are willing to rank a trade book on — and getting from the raw field to that number is a fixed sequence of decisions, each of which can silently ruin the result if taken out of order or skipped.

The chain, in order

1. Timestamp it correctly. The date a data provider stamps a row is often not the date the information was knowable. Revision counts might be stamped to the quarter-end even though the last revision in that count arrived three weeks later. Using the field on the stamped date rather than the date it was actually available is look-ahead bias, and it is the single most common way a promising signal turns out to be fake. Every field needs its own answered question: when could I have known this?

2. Choose the transform. Raw fields are rarely usable as-is. A revision count of "+3" means something different for a stock with 30 analysts than one with 4 — so you might convert to a fraction of analysts revising up. A price-based field might need a log transform because raw price ratios are skewed. This step encodes your hypothesis about what shape of relationship connects the field to future returns.

3. Scale it comparably across names. A signal has to be put on the same footing across a universe of stocks that differ enormously in size, volatility and liquidity, or the ranking will just reflect those differences instead of the thing you meant to measure. Two common choices: rank the universe and convert to a percentile, or z-score (subtract the mean, divide by the standard deviation) within some group. Ranking is robust to outliers; z-scoring keeps more information about how extreme a value is, at the cost of being sensitive to fat tails.

4. Neutralise what you don't want to bet on. If the raw signal is secretly a size bet — small caps happen to have noisier analyst counts — that exposure will show up in your Sharpe ratio and then vanish the day size rotates. Demeaning within sector, or against a set of known factors, strips out exposures you did not intend to take.

5. Lag it to a tradeable time. The signal as of the close on day t can only be traded from the open on day t+1 at the earliest, and in practice later once execution latency is accounted for. Skipping this step, even by one day, inflates a back-tested Sharpe in a way that is invisible until the strategy goes live.

A worked example

Take a raw field: the change in a company's short interest as a percentage of float, reported twice a month with a settlement lag.

• Timestamp: the exchange reports short interest as of the 15th and month-end, but publishes it roughly eight business days later. Using the reporting date, not the as-of date, would be look-ahead — you would be trading on a number you could not have seen yet.
• Transform: use the change in short interest, not the level, since the level mostly reflects a stock's persistent characteristics (float size, index membership) rather than new information.
• Scale: rank the change within the investable universe each period, producing a 0–1 percentile per name.
• Neutralise: demean within sector, since short interest tends to cluster by industry during sector-wide narratives (all semiconductor names shorted together, say), and you want the idiosyncratic outlier, not a sector bet.
• Lag: trade from the first available date after the eight-day publication lag, never the as-of date.

The resulting signal — sector-neutral percentile rank of the change in short interest, lagged to actual availability — is what gets back-tested. Skipping the neutralisation step on this exact field, in practice, has produced back-tests that look like a clean short-interest effect and are really a sector-momentum bet wearing a disguise.

Drag the correlation slider down toward zero: that is what neutralisation is doing to the raw field's relationship with the sector it was riding on, while (ideally) leaving its relationship with the return you actually want intact. A signal survives neutralisation if the scatter stays meaningfully non-random after the confound is removed — that is the difference between an idiosyncratic edge and a repackaged factor bet.