What Papers Leave Out

A published anomaly comes with a clean abstract, a headline t-statistic, and a handful of robustness tables — and it's tempting to treat that as the full picture. It isn't, and not necessarily because anyone was dishonest. Academic publishing has structural incentives and page constraints that systematically leave out exactly the details a quant would most want before risking real capital on a result.

The idea

Journals reward novel, statistically significant findings and rarely publish null results, which means the published literature as a whole is a filtered, survivorship-biased sample of everything researchers actually tried — a version of the multiple-testing problem playing out at the level of an entire academic field rather than one researcher's project. A paper that reports a strong anomaly says nothing about how many similar hypotheses other researchers quietly tested and abandoned because they didn't work, information that would matter enormously for judging how likely the published result is to be a fluke.

Beyond that field-level bias, individual papers routinely omit the operational details a trading strategy actually depends on. Academic backtests typically ignore transaction costs, market impact, and borrowing costs for short positions — reasonable simplifications for testing a statistical hypothesis about return predictability, but exactly the costs that determine whether a real strategy is profitable. Papers also rarely disclose the full list of alternative specifications, universe definitions, or sample periods the authors tried before settling on the one reported, which makes it impossible for a reader to know how sensitive the result is to those choices without redoing the work independently.

A concrete example

A published paper reports a factor with a strong, statistically significant average return over a twenty-year sample, tested on a broad universe including micro-cap stocks that are cheap to include in an academic backtest but expensive or impossible to actually trade at scale. The paper doesn't report returns separately for the largest, most liquid names, doesn't model bid-ask spreads or market impact, and doesn't mention whether the authors tested and discarded a 12-month or 6-month version of the signal before landing on the reported 9-month lookback. A quant reading the paper adversarially reconstructs the result on large-cap names only, applies realistic transaction cost assumptions, and finds the anomaly's magnitude shrinks by more than half — not evidence the original finding was fabricated, just evidence that the parts of the analysis that mattered most for tradeability were never the parts the paper was optimized to report.

What this means in practice

Reading a paper for trading purposes means actively asking what isn't there: what would the result look like restricted to liquid, tradeable names; what would it look like net of realistic costs; how many nearby specifications were probably tried and not reported. None of this makes the paper worthless — a real, published anomaly is still a far better starting point than a hypothesis invented from nothing — but treating the reported number as directly tradeable, rather than as a first estimate to be independently re-derived under realistic constraints, is a common and costly mistake.