Averaging independent predictors

Question

You average B = 5 model predictions that are mutually independent, each with variance \sigma^2 = 200. What is the variance of the averaged prediction?

Accepted Answer

The idea. Independence is the best case for averaging: with no shared component, every model adds fresh information and the variance falls in full proportion to how many you average. The formula with correlation ho = 0: 	ext{Var}(	ext{avg}) = ho\sigma^2 + \frac{1 - ho}{B}\sigma^2 = 0 + \frac{\sigma^2}{B} = \frac{\sigma^2}{B}. Plug in \sigma^2 = 200, B = 5: 	ext{Var}(	ext{avg}) = \frac{200}{5} = 40. The variance dropped by the full factor of B = 5, from 200 to 40. This is the same \sigma^2/n rule that governs the variance of a sample mean, and it is the ideal that ensembles chase but…

Averaging independent predictors

Your answer

More Statistics questions