The Research Standards Document
A research standards document is the written checklist a fund holds every backtest to before capital follows it — what counts as a valid test, what must be disclosed, and who has to sign off. Without one, every researcher quietly invents their own rules.
Prerequisites: Backtest Overfitting, Look-Ahead Bias
A mid-sized fund runs a post-mortem after a statistical arbitrage strategy loses back eighteen months of paper profits in six weeks live. The backtest had shown a Sharpe of 2.1 over twelve years. Nobody had done anything fraudulent. What the post-mortem actually found was that four different researchers, over four different years, had each tested the idea their own way: one used adjusted prices that silently baked in look-ahead from stock splits, one had rebalanced daily in the backtest but the desk could only trade the strategy on a two-day settlement cycle, one had quietly dropped three delisted names because the data vendor's feed didn't carry them, and the version that finally got funded stitched together the best-looking segments of all four without anyone noticing the composite was internally inconsistent. Each individual decision was defensible in isolation. There was no document anyone could have checked their work against, and no single reviewer had seen all four versions side by side.
That gap is what a research standards document closes. It is a short, written, firm-wide reference that answers, in advance, the questions every researcher would otherwise answer for themselves under time pressure and the pull of a good-looking result.
What it actually contains
Not a style guide and not a compliance manual — a methodology contract. A working document typically fixes:
| Area | What it pins down |
|---|---|
| Universe and data | Which point-in-time data source is authoritative, how delistings are handled, which corporate actions are applied and when |
| Costs and constraints | The cost model every backtest must use unless explicitly justified otherwise, and the trading calendar and settlement lag the desk actually faces |
| Validation | The minimum out-of-sample or walk-forward test required before a strategy can be proposed for capital, and how many prior configurations were tried |
| Disclosure | That every submission states how many variants were tested, not only the one that is being presented |
| Sign-off | Who reviews a submission before it reaches a portfolio manager, and what they are required to check |
None of this is exotic. Every item on that list is something an experienced researcher already does out of habit. The document's value is not novelty — it is that the habit becomes mandatory for the researcher who is new, in a hurry, or quietly attached to a result they want to be true.
The standards document does not make research more rigorous by adding new statistical machinery. It makes rigor uniform, so that a strategy's Sharpe ratio reflects the idea and not which researcher happened to build the test, in which year, with which shortcuts.
Why "just use good judgment" fails
The instinct at a small fund is to skip this: everyone is smart, everyone knows about look-ahead bias, why write it down? The failure mode above is the answer. Good judgment is not the scarce resource — consistency under deadline pressure is. A researcher racing to present at Friday's investment committee will, without malice, take the version of the backtest that survived the fewest revisions rather than the one that is most correct, because nobody told them explicitly which one the firm requires. A written standard removes that choice from the researcher in the moment it matters, by making it beforehand, when nobody has a result to defend.
It also creates the one artifact a reviewer actually needs. Without a shared checklist, an investment-committee review — see Investment Committee Sign-Off — degenerates into "does this look right to me," which is a test of the reviewer's pattern-matching, not the strategy's validity. With a checklist, the review becomes "did this follow the process," which is answerable and auditable, and which catches the sat-in-a-drawer version of the bank-pairs failure above before it reaches a live book.
A standards document that nobody actually enforces is worse than none at all — it gives every subsequently-blown-up strategy a paper trail suggesting it was checked, when in practice the checklist was filled in after the fact to match a result the team already believed. The document only has value if a strategy can be, and sometimes is, rejected at the door for failing it.
Keeping it alive
A standards document is not written once. It grows a line item each time a specific failure gets past it — that is precisely how the pairs-arbitrage post-mortem above should end: not with a firing, but with one new required field in the submission template ("state the settlement lag assumed and confirm it matches the desk's actual cycle"). Treat every blow-up as a missing checklist item rather than a one-off mistake, and the document becomes a running institutional memory that survives any single researcher leaving.
If your fund or personal research process has no such document, write one page for yourself: five bullet points fixing your data source, cost assumptions, minimum validation bar, and a rule that every result you show yourself states how many variants you tried to get it. It costs an afternoon and it is the single cheapest defense against fooling yourself that exists.
Related concepts
Practice in interviews
Further reading
- López de Prado, Advances in Financial Machine Learning (Ch. 1, on the research factory)
- Harvey, Liu & Zhu, ...and the Cross-Section of Expected Returns