Definition Mismatches That Break a Replication
Why trying to reproduce a published trading result often fails not because the strategy is wrong, but because a variable's exact definition — timing, universe, or adjustment convention — differs quietly from the paper's.
A researcher tries to reproduce a published anomaly's returns and gets a Sharpe ratio half of what the paper reports — the strategy logic is coded correctly, yet the numbers don't match. Far more often than an actual coding bug, the culprit is a quiet mismatch in how some ordinary-sounding variable is defined: "market cap" measured at month-end versus at the rebalance date, "book equity" using the fiscal-year-end figure versus the most recently available figure (which lags fiscal year-end by months due to reporting delay), or a stock universe that includes versus excludes financial firms and ADRs.
These mismatches are dangerous precisely because they're invisible in the paper's prose — "we sort stocks on book-to-market" sounds unambiguous until you realize there are several defensible conventions for exactly which book value and which market value go into that ratio, and the paper's footnotes or appendix (if read carefully) are usually the only place the actual choice is disclosed.
The single highest-value habit for avoiding this is reproducing a well-known baseline result first — the market factor's own historical Sharpe ratio, say — using the exact same data pipeline before trying to replicate the paper's actual claim, since a mismatch on a well-known baseline is far easier to spot than a mismatch buried inside a novel result.
Failed replications are usually definition mismatches, not logic bugs — timing conventions, universe filters, and reporting lags rarely appear explicitly in a paper's main text, so checking the exact footnoted definitions and reproducing a known baseline first is the fastest way to find them.
Related concepts
Further reading
- Hou, Xue, Zhang, 'Replicating Anomalies', Review of Financial Studies (2020)