Chance a data point is left out of a bootstrap sample

Question

You have n = 1000 observations and draw a bootstrap sample by picking n of them with replacement (each draw is uniform over all 1000 points, independent of the others). What is the probability that one particular observation is never selected, i.e. it is left out of the bootstrap sample entirely?

Accepted Answer

On a single draw, the chance of not picking a specific point is 1 - 	frac{1}{n}. The n draws are independent, and the point is omitted only if it misses on every one of them. Probability of omission. \left(1 - \frac{1}{n}ight)^{n} = \left(1 - \frac{1}{1000}ight)^{1000} \approx 0.368. This is a famous limit: as n grows, \left(1 - 	frac{1}{n}ight)^{n} 	o e^{-1} \approx 0.368. So in any bootstrap sample, roughly 36.8\% of the original observations are missing, and about 63.2\% appear at least once. The left-out points are the "out-of-bag" sample used for out-of-bag error in bagging and …

Chance a data point is left out of a bootstrap sample

Your answer

Discussion

Learn the concepts

Related questions