Learning to Rank
Some problems only care about the order of predictions, not their exact values — picking the top 20 stocks to overweight this month needs a correct ranking, and training a model to predict raw returns is not the same as training it to rank well.
Prerequisites: Ranking vs Regression Objectives for Alpha, Gradient Boosting as Functional Gradient Descent
A portfolio manager building a monthly long book from 2,000 stocks doesn't need to know that stock A will return exactly 4.3% and stock B exactly 4.1% — she needs to know that A belongs above B. Training a regression model to predict returns exactly and then sorting the outputs is one way to get a ranking, but it's optimizing the wrong thing: a model can nail the ranking of the top 50 stocks while being wildly wrong about their absolute return level, and squared-error loss doesn't know or care that getting the order right is all that mattered here.
Learning-to-rank trains a model directly against ordering, not magnitude — its loss function penalizes pairs (or whole lists) that come out in the wrong relative order, so the model is optimized for exactly the thing that will be used downstream: who's above whom.
Three levels of ranking loss
Pointwise approaches are what regression already does: predict a single score per item and sort by it. Cheap and simple, but as above, it optimizes absolute accuracy rather than relative order.
Pairwise approaches, like RankNet and LambdaMART, instead take every pair of items where the true order is known — stock A outperformed stock B — and penalize the model whenever it scores B above A. The loss is a function of the difference in predicted scores for the pair, not either score's absolute value, which directly targets ranking correctness.
Listwise approaches go further and optimize a metric computed over an entire ranked list at once — commonly a smoothed version of NDCG (normalized discounted cumulative gain), which rewards getting the top of the list right more than the middle, matching how a portfolio manager who only trades the top 20 names actually cares about the list.
Worked example
A cross-sectional model ranks 500 stocks each month, and only the top 20 get bought. Trained with plain squared-error regression, the model achieves a respectable overall , but among its top-20 picks, the actual rank correlation with realized forward returns is only 0.31 — mediocre, because most of the model's fitting effort went into getting the bulk of mid-ranked, unimportant stocks approximately right. Retrained with a pairwise ranking loss (LambdaMART) using the same features, the overall -style fit metric barely changes, but the rank correlation among the top 20 rises to 0.52 — because every training pair involving a top-tier stock now directly penalizes the model whenever it's out of order, whereas before that signal was diluted across 500 stocks' worth of squared-error gradient.
What this means in practice
Learning-to-rank is the right tool whenever the deployment decision is itself a ranking — top-decile stock selection, order-routing venue preference, candidate-search relevance — and the wrong tool whenever the actual number matters, such as forecasting a portfolio's expected return or sizing a position by expected magnitude.
A model can have excellent rank correlation across the whole universe and still rank the specific top 20 you'll actually trade poorly — always evaluate ranking loss on the slice of the list you'll act on, not the aggregate rank correlation across everything, since listwise metrics like NDCG were designed precisely to catch this gap.
Related concepts
Practice in interviews
Further reading
- Burges et al., 'Learning to Rank Using Gradient Descent' (RankNet, 2005)
- Liu, Learning to Rank for Information Retrieval (2009)