Prior Elicitation and Prior Choice

Before you can run Bayes' theorem on anything, you need a prior — a probability distribution over the unknown quantity that describes what you believed before looking at the data. Where does that number actually come from? "Just pick a reasonable prior" sounds simple until you're staring at a blank distribution and a new strategy with three months of live data, wondering whether its true Sharpe ratio sits near 0.3 or 1.5. Prior elicitation is the discipline of answering that honestly and turning it into a distribution you can compute with.

An analogy: pricing a used car before the inspection

Before a mechanic inspects a used car, you already have some idea of what it's worth — from the make, model, year, mileage, and asking price of similar cars sold nearby. That rough range is your prior: not a guess pulled from nowhere, but a summary of everything you knew before this specific inspection. A buyer with no market knowledge might have a wide, vague prior ("somewhere between $2,000 and $20,000"); a buyer who's flipped a hundred similar cars has a narrow, confident prior ("almost certainly $8,000–$9,500"). Both update the same way once the inspection report (the data) comes in, but the narrow prior moves less — it takes more evidence to shift a confident belief than a vague one.

The maths, one symbol at a time

Bayes' theorem combines a prior $p(\theta)$ with a likelihood $p(D \mid \theta)$ to get a posterior $p(\theta \mid D)$. Prior elicitation is choosing $p(\theta)$ itself, and the standard toolkit has three routes:

1. Subjective elicitation: interview a domain expert and translate their stated beliefs ("I'd be surprised if the true win rate were below 45% or above 65%") into a distribution's parameters, often by matching stated percentiles to a chosen family (e.g. a Beta distribution).
2. Empirical priors: set $p(\theta)$ from a related, larger dataset — for instance, the historical distribution of Sharpe ratios across hundreds of past strategies at a fund, used as a prior for a brand-new strategy (see empirical Bayes).
3. Weakly informative or noninformative priors: when you genuinely have little to say, choose a deliberately wide or standardized prior that lets the data dominate (see Jeffreys priors).

The key quantity to reason about is how much the prior can be overridden by data. For a Beta$(\alpha, \beta)$ prior on a probability $\theta$ (like a win rate), the prior behaves like $\alpha + \beta - 2$ "pseudo-observations" already baked in before real data arrives — a useful rule of thumb for judging whether a proposed prior is strong or weak relative to the sample size you expect.

Worked example 1: eliciting a Beta prior for a win rate

A quant is told by a senior trader: "I'd guess this new signal's win rate is around 55%, and I'd be surprised if it were below 45% or above 65%." Matching "center near 0.55, most mass within ±0.10" to a Beta$(\alpha,\beta)$ distribution gives roughly $\alpha = 55$, $\beta = 45$ (mean $55/100 = 0.55$). This prior behaves like having already observed 100 trades with 55 wins — a fairly strong starting point. If only 20 real trades come in next, they'll nudge the estimate only modestly, since the prior's implicit 100 pseudo-trades outweigh 20 real ones.

Worked example 2: how the choice changes the answer with little data

Suppose 12 live trades come in: 9 wins, 3 losses (a raw rate of 75%). Combining with the strong prior from example 1 (Beta$(55,45)$), the posterior is Beta$(64, 48)$, mean $64/112 \approx 0.571$ — barely moved from 0.55. Combining the same 12 trades with a weak prior, Beta$(2,2)$, gives a posterior of Beta$(11,5)$, mean $11/16 \approx 0.688$ — much closer to the raw 75%. Same data, two defensible priors, two meaningfully different conclusions — exactly why the prior must be stated and justified, not buried.

Drag the prior strength in the explorer above with a small number of observations, and watch how a strong prior barely moves while a weak one snaps toward the data — this is the same effect as the two posteriors above.

What this means in practice

Prior elicitation matters most exactly when it's tempting to skip it: small samples, new strategies, thin live-trading history. A fund reusing its whole strategy book's historical Sharpe distribution as a prior will size a new strategy far more conservatively in its first months than one using a flat prior — a real risk-management decision, not a technicality. Documenting the prior and its source is what lets someone else audit that decision later.