Variance of a ten-model ensemble with mild correlation

Question

You average B = 10 model predictions. Each has variance \sigma^2 = 1, and any two are correlated with \rho = 0.2. What is the variance of the averaged prediction?

Accepted Answer

The idea. The ensemble variance is a shared part that correlation keeps alive plus a private part that averaging shrinks. Averaging more models attacks only the private part. The formula. 	ext{Var}(	ext{avg}) = ho\sigma^2 + \frac{1 - ho}{B}\sigma^2. Plug in \sigma^2 = 1, ho = 0.2, B = 10: 	ext{Var}(	ext{avg}) = 0.2(1) + \frac{1 - 0.2}{10}(1) = 0.2 + 0.08 = 0.28. The variance fell from 1 to 0.28. Because the correlation is low, most of the drop is real reduction, and the surviving floor (ho\sigma^2 = 0.2) is small. This is why bagging methods deliberately decorrelate their members …

Variance of a ten-model ensemble with mild correlation

Your answer

Discussion

Learn the concepts

Related questions