Scoring Research Ideas by Expected Value

A research team always has more candidate ideas than time to test them — a backlog of "what if" questions from meetings, papers, and half-finished side projects. Picking which one to work on next by gut feel tends to favor whichever idea was mentioned most recently or sounds most exciting, which isn't the same as whichever idea is actually worth the team's limited time. Scoring ideas by expected value means weighing three things together for each candidate: how big the payoff would be if it works, how likely it is to actually work, and how much time it would take to find out.

Why all three factors matter together

An idea with a huge potential payoff but a tiny chance of working, and that takes months to test, can be a worse use of time than a modest idea that's likely to pan out and testable in a week — even though the first idea sounds more exciting in a meeting. This isn't a precise formula most desks compute numerically; it's a habit of asking all three questions explicitly rather than defaulting to whichever idea feels most compelling. An idea that would take a long time to even find out whether it works deserves extra scrutiny, because time spent testing it is time not spent on several faster ideas that might have paid off just as well in aggregate.

Worked example

A team is choosing between two candidate projects for the next month. Idea A is a well-trodden factor variant: modest expected payoff if it works, a decent chance of a positive result based on similar past work, testable in about a week. Idea B is a novel alternative-data signal: a much bigger payoff if it works, but a low chance of panning out and a full month to properly test given data-cleaning work needed first. Scoring by rough expected value rather than raw excitement, the team runs Idea A first — not because B is a bad idea, but because A returns a likely answer in a fraction of the time, and the team can decide whether to invest the fuller month in B once A is out of the way, informed by whatever A turns up.

What this means in practice

Expected-value scoring is a way of turning "which idea should I work on" from a mood-driven decision into a structured comparison, without needing precise numbers to do it usefully — the discipline of estimating payoff, likelihood, and time cost even roughly is what improves prioritization, not the arithmetic itself. It also protects against a common failure mode: pouring months into one high-glamour idea while a queue of faster, similarly valuable ideas sits untouched.