Optimal shrinkage straight from the signal-to-noise ratio

Question

You shrink a noisy estimate x toward an anchor b using \hat\mu = w\,x + (1-w)\,b. Instead of the raw numbers, you are told the signal-to-noise ratio, defined as the squared distance from the anchor over the noise variance: 	ext{SNR} = d^2/\sigma^2 = 3. What weight w minimizes the mean squared error?

Accepted Answer

The idea. The optimal shrinkage weight depends only on how the signal (distance from the anchor) compares to the noise. Once you know their ratio, you do not need the individual values. Rewrite the optimal weight. The MSE-minimizing weight is w^\star = \frac{d^2}{d^2 + \sigma^2}. Divide top and bottom by \sigma^2 to express it purely through the signal-to-noise ratio: w^\star = \frac{d^2/\sigma^2}{d^2/\sigma^2 + 1} = \frac{	ext{SNR}}{	ext{SNR} + 1} = \frac{3}{3 + 1} = 0.75. Keep three-quarters of the raw estimate. Sanity check the extremes. If the SNR is huge (clean signal), w^\star 	o 1: …

Optimal shrinkage straight from the signal-to-noise ratio

Your answer

Discussion

Learn the concepts

Related questions