Jeffreys and Noninformative Priors

If you want a Bayesian analysis but genuinely have no strong opinion about an unknown parameter, the obvious move seems to be: use a "flat" prior that treats every value as equally likely. That instinct is quietly broken — a prior that's flat for one way of writing the parameter can be sharply peaked for an equivalent, relabeled version of the same problem. Jeffreys priors fix this: a rule for constructing an "as uninformative as reasonably possible" prior that gives the same answer no matter how you parametrize the unknown.

An analogy: a map that doesn't care which units you use

Imagine describing "no strong opinion about a distance" by saying every value in miles from 0 to 100 is equally plausible. Converted to kilometers, it's still flat — no contradiction yet. But suppose your unknown is a rate, like miles per gallon, and you declare every value from 0 to 100 mpg equally plausible. Converted to the reciprocal — gallons per mile — that "flat" belief is no longer flat at all; it's badly skewed toward small values. A belief that felt neutral under one labeling secretly favors certain outcomes under an equivalent relabeling. Jeffreys' idea builds a notion of "neutral" that survives this — the prior transforms exactly the way it should when you change units, so "no strong opinion" means the same thing regardless of how you write the parameter down.

The maths, one symbol at a time

Let $\theta$ be the unknown parameter and $p(D \mid \theta)$ the likelihood of data $D$ given $\theta$. The Jeffreys prior is defined as

$$p(\theta) \propto \sqrt{I(\theta)}$$

where $I(\theta)$ is the Fisher information — a measure of how sharply the likelihood changes as $\theta$ moves, i.e. how much information a single data point carries about $\theta$ at that value. In words: the prior is deliberately made larger wherever the data is more informative about $\theta$, and smaller where the data says little — this specific weighting is exactly what makes the prior invariant to reparametrization (relabeling $\theta$ as some function of itself leaves the resulting posterior conclusions unchanged, unlike a plain flat prior). It is "noninformative" in the sense of encoding no preference beyond how the likelihood itself is shaped — not in the sense of encoding literally nothing.

For a binomial proportion $\theta$ (like a win rate), the Jeffreys prior works out to Beta$(1/2, 1/2)$ — a distribution slightly favoring values near 0 and 1 over 0.5, because that's where a coin flip's outcome is most informative about $\theta$ per observation.

Worked example 1: Jeffreys prior for a win rate

A strategy has produced 3 wins out of 4 trades. Using the Jeffreys prior Beta$(0.5, 0.5)$, the posterior is Beta$(3.5, 1.5)$, with posterior mean $3.5/5 = 0.70$. Using a flat Beta$(1,1)$ prior instead gives posterior Beta$(4,2)$, mean $4/6 \approx 0.667$. Both are close to the raw rate of $0.75$, but not identical — the Jeffreys prior's slight pull toward the extremes shows up as a marginally higher posterior mean. With only 4 trades, the choice of "uninformative" prior still visibly matters.

Worked example 2: why flat isn't invariant, concretely

Suppose an analyst places a flat prior on $\theta \in [0,1]$, a win rate. Reparametrize using the log-odds $\phi = \ln(\theta / (1-\theta))$, an equally valid way to describe the same unknown. Transforming the flat-in-$\theta$ prior into the $\phi$ scale (via the standard change-of-variables Jacobian) produces a prior that is not flat — it's shaped like a logistic density, putting more mass near $\phi=0$ ($\theta=0.5$) than near the extremes. So "I have no opinion, all win rates are equally likely" silently became "I lean toward believing the win rate is near 50-50" the moment you switched to the log-odds scale. The Jeffreys prior transforms consistently under this exact change of variables — the same belief comes out the same way regardless of scale.

The logistic curve above is the same shape that appears when a flat prior in one parametrization gets pushed through to the log-odds scale — a reminder that "flat" is not a parametrization-free concept.

What this means in practice

Jeffreys priors show up as software defaults when an analyst wants a "let the data speak" analysis without secretly baking in an opinion through an arbitrary parametrization — useful for a first pass on a new signal where no domain prior exists yet. They're rarely the final word in a serious quant workflow, since a genuine domain prior (see prior elicitation) usually beats a mechanically "neutral" one once real information is available.