Precision-at-K and Top-Decile Evaluation

A long-short equity strategy doesn't trade every stock in the universe — it ranks a few thousand names by predicted return and only acts on the top and bottom deciles, maybe 200 names on each side. If the model's overall accuracy across all 3,000 stocks is excellent but its top 200 picks are mediocre, the strategy loses money even though the model "works" on average. Precision-at-K asks a narrower, more relevant question: of the K candidates the model ranked highest, how many actually turned out to be winners? That's the number a decision that only touches the top of a ranked list actually needs.

The metric

Rank all $n$ candidates by the model's predicted score, take the top $K$, and compute:

$$\text{Precision@K} = \frac{\text{number of true positives in the top } K}{K}$$

In plain English: out of the K items you actually acted on, what fraction were genuinely good? This ignores everything about how the model performs on the thousands of candidates you never touched, which is exactly the point — those never affected any decision, so their contribution to overall accuracy is a distraction from what matters. Top-decile evaluation is the same idea applied to a fixed fraction (the top 10%) rather than a fixed count, which is more natural when the universe size changes over time (200 stocks out of 2,000 one month, 220 out of 2,200 the next).

Worked example: comparing two signals by their top decile

Two return-prediction signals are each scored on 1,000 stocks with known next-month returns. Both have a similar overall rank correlation with actual returns. Looking only at the top decile (the 100 highest-ranked names by each signal): Signal A's top 100 names had 68 that outperformed the market that month; Signal B's top 100 had 54. Precision-at-100 for A is $68/100 = 0.68$; for B it's $54/100 = 0.54$. A strategy that only trades the top decile would clearly prefer Signal A, even if a broader correlation measure across all 1,000 names showed the two signals as roughly tied — the broad measure is diluted by the 900 names nobody was going to trade anyway.

The explorer above illustrates the general shape: as $K$ shrinks toward the very top of a ranking, precision tends to rise for a genuinely informative signal (the model is most confident, and most often right, about its extremes) — plotting Precision@K against K for a few candidate signals is a fast way to see which one degrades least as you tighten the cutoff.

What this means in practice

Whenever a decision only touches a fixed slice of a ranked list — the top N loan applicants approved, the top decile of stocks longed, the top 50 alerts an analyst actually reviews — evaluate the model on precision-at-K rather than overall accuracy or AUC, because those aggregate metrics can be dominated by performance on the vast middle of the ranking that never gets acted on. It's also a natural metric to track over time: a signal whose top-decile precision decays even while its overall correlation looks stable is a warning that the part of the model actually being traded is degrading first.