Running a Research Review Meeting
The structure that keeps a research review focused on whether a strategy's evidence is solid, instead of turning into an unstructured debate.
A research review meeting is where a researcher presents a candidate strategy or signal to peers or a risk committee before it's allocated capital. Without structure, these meetings drift into whoever argues loudest, or into re-litigating the researcher's methodology choices one at a time rather than assessing the overall case — both of which waste time and let weak strategies through on charisma.
A working structure fixes the order of questions everyone must answer, in this sequence: what is the economic or behavioral rationale for the effect existing at all (not just a backtest that found it); what is the sample size and out-of-sample performance, not just the in-sample fit; what are the known ways this specific backtest could be overstating performance (look-ahead bias, survivorship, overfitting to a fixed data window); and what is the capacity and cost estimate once realistic slippage is included. Presenting the report in this fixed order — evidence before implementation detail — prevents the meeting from getting stuck debating a parameter choice before anyone has agreed the core idea is sound.
A useful discipline is assigning one attendee as a designated skeptic whose job is specifically to find the strongest reason the result is spurious, rather than leaving skepticism to whoever happens to speak up — this surfaces objections the presenter didn't think to preempt, without making the meeting adversarial by default.
Fix the order of questions in advance — rationale, then out-of-sample evidence, then bias checks, then capacity/costs — and assign a designated skeptic, so the review tests the strategy's evidence rather than becoming a debate about whoever argues best.
Further reading
- Grinold and Kahn, Active Portfolio Management, ch. 1