Framing Alpha Research as a Prediction Problem

"Predict which stocks will outperform" sounds like a single, obvious task, but turning it into something a model can actually be trained on forces a long chain of decisions: predict the return itself, or just its sign, or its rank against other stocks that day? Over what horizon — one day, twenty days? Scored by what metric — mean squared error, an information coefficient, or the Sharpe ratio of a portfolio built from the predictions? Each of these choices is a legitimate way to frame "find alpha" as a supervised learning problem, and they are not interchangeable: a model that's excellent by one framing's metric can be a poor trading strategy by another's, because the framing decides what the model is actually optimizing for.

The gap between a clean prediction target and a tradeable strategy

A model trained to minimize mean squared error on next-day returns is optimizing to be close to the true value on average — but a strategy's profit depends on getting the sign right on the trades it actually takes, weighted by position size, not on minimizing squared distance across every prediction including the ones near zero that will never be traded. A model can have excellent MSE by being very accurate on the easy, low-volatility, low-signal stocks that don't matter for the portfolio, while being mediocre exactly on the volatile, high-conviction names the strategy actually needs to get right. This is the same gap covered generally by objective functions versus evaluation metrics, but alpha research adds a further twist: the metric that ultimately matters — realized, cost-adjusted portfolio return — is not something any single-stock prediction loss function directly optimizes for at all.

Worked example: same predictions, different rankings by two framings

A model produces predicted next-day returns for five stocks: A: +0.8%, B: +0.3%, C: -0.1%, D: +1.5%, E: -0.6%. Realized returns turn out to be: A: +0.2%, B: +0.4%, C: -0.3%, D: +0.9%, E: -1.0%.

Scored by mean squared error on the raw values, the model does reasonably well across the board — no huge misses. Scored by rank correlation (Spearman), which is what matters if the strategy simply longs the top-ranked names and shorts the bottom-ranked ones: predicted ranking (best to worst) is D, A, B, C, E; realized ranking is D, B, A, C, E. The model got D right at the top and E right at the bottom — the two names that would actually be traded with the most conviction in a long-short strategy — but swapped A and B in the middle. A cross-sectional long-short strategy trading only the top and bottom name would have done well here; an MSE-based evaluation, which weighs every stock's error equally regardless of where it sits in the ranking, would not have highlighted this as the reason the model is actually useful.

What this means in practice

The practical fix is to design the prediction problem backward from the strategy, not forward from whatever is easiest to model. If the eventual strategy is a cross-sectional long-short portfolio, the target should be something rank-based (predicted return relative to the day's cross-section, not the absolute value) and the evaluation should be an information coefficient or a simulated portfolio Sharpe ratio, not raw MSE. If the strategy trades based on a directional threshold rather than sizing continuously, framing the problem as classification (up/down, or up/flat/down) with a cost-aware evaluation (see utility-based model evaluation) is often closer to what actually matters than a regression loss that treats a near-miss on magnitude the same as a sign error. This backward design also has to account for realistic costs and constraints from the start — a model evaluated purely on unconstrained IC can look strong while implying a portfolio that would be prohibitively expensive to actually trade once turnover and market impact are included.

Practice

1. A model predicts absolute next-week returns and is evaluated by MSE. The eventual strategy only trades the top and bottom decile of predictions each week. Propose a better evaluation metric and explain what it would capture that MSE misses.
2. Why might a classification framing ("will this stock beat the sector median?") be more aligned with a long-short strategy's actual objective than a regression framing ("predict the exact return"), even though regression uses strictly more information?