Profile Likelihood

Suppose you've fit a model with two parameters — say a volatility model with a mean-reversion speed $\theta$ you actually care about, and a noise-scale parameter $\sigma$ you don't. You want a confidence interval for $\theta$ alone. The naive approach — fix $\sigma$ at its fitted value and vary only $\theta$ — understates the true uncertainty, because it pretends $\sigma$ is known exactly when it was also estimated from the same noisy data. Profile likelihood fixes this: for each candidate value of $\theta$, it re-maximizes the likelihood over $\sigma$ before recording the likelihood value, so the uncertainty in $\sigma$ is folded into the interval for $\theta$ rather than ignored.

Formally, the profile likelihood is

$$L_p(\theta) = \max_{\sigma} L(\theta, \sigma)$$

— at each $\theta$, find the best-fitting $\sigma$ given that $\theta$, and use that maximized value. Plotting $L_p(\theta)$ (or its log) against $\theta$ gives a curve; the confidence interval is the set of $\theta$ where this curve stays within a cutoff of its peak, exactly the same cutoff used in a likelihood-ratio test.

Worked illustration. Fitting a two-parameter model, the ordinary log-likelihood peaks at $\hat\theta = 1.5$. Naively fixing $\sigma$ at its fitted value and varying only $\theta$ might show the log-likelihood dropping by the critical amount (1.92, for a 95% interval with one free parameter) at $\theta = 1.3$ and $\theta = 1.7$. Profiling out $\sigma$ properly — re-optimizing it at every trial $\theta$ — typically widens this to something like $\theta \in [1.15, 1.9]$, because part of what looked like precision in $\theta$ was actually the model quietly borrowing certainty from an unrealistically fixed $\sigma$.