Sequential Bootstrap Sampling

An ordinary bootstrap resamples observations with replacement, assuming each draw is independent of the others — a fine assumption for coin flips, but a bad one for financial labels that overlap in time, such as a label spanning "this trade's outcome over the next 5 days," where many labels share days with their neighbors. Sampling such overlapping labels as if independent overstates how much genuinely new information the resample contains, since drawing two heavily overlapping labels barely adds any new information versus drawing one of them twice.

Sequential bootstrap sampling fixes this by drawing observations one at a time and, after each draw, temporarily lowering the probability of drawing another observation that overlaps heavily with what's already been picked. Concretely, each candidate observation's draw probability is scaled by its average uniqueness — how much of its time span is not already covered by previously drawn observations — so a label that barely overlaps anything already chosen keeps close to its original weight, while one that mostly duplicates already-picked labels becomes much less likely to be drawn again.

The resulting resampled set behaves closer to a genuinely independent sample, which matters directly for training an ensemble model (like a random forest) on overlapping-label financial data — a naive bootstrap would repeatedly duplicate near-identical, correlated labels, making the ensemble's trees more similar to each other than to independent estimators, and understating the model's true out-of-sample uncertainty.