Triaging a Research Backlog

A research desk almost always has more candidate ideas — new signals, dataset ideas, model tweaks — than time to test them properly, and testing each one thoroughly is expensive in researcher-hours and computing time. Triage is the discipline of quickly ranking a backlog of ideas before committing to a full backtest, so the limited research budget goes to the ideas most likely to be worth it.

A simple triage framework scores each candidate along three axes: expected payoff (how much better would this make the portfolio if it works, based on similar published or in-house results), cost to test properly (does it need new data, new infrastructure, or just a quick script against data already on hand), and overlap (is this really a new source of edge, or a small variant of a signal the desk already runs, which would mostly just add correlated risk rather than new return). An idea that scores well on payoff and low on cost, with little overlap to existing signals, jumps the queue; an idea requiring months of new data pipeline work to test a hunch with weak theoretical grounding gets deprioritized or shelved.

For instance, given three candidate ideas — one reusing an existing dataset with a half-day script, one requiring a new alternative-data vendor contract and months of history backfill, and one that's essentially a re-parameterization of a signal already live — a triage pass would likely test the first immediately, park the second pending a cheaper proxy test, and deprioritize the third unless the re-parameterization has a specific, well-argued reason to outperform the original.

Doing this triage step explicitly, rather than testing ideas in whatever order they arrive, is what keeps a research pipeline from spending its scarcest resource — researcher attention — on ideas that were never likely to clear the bar, however interesting they sounded at first.