Topic · Machine Learning & AI
← All topicsStatistical Learning Theory
30 articles · 5 checkpoints · 15 deeper reads · 10 reference notes
A standalone topic: it is on no roadmap, so read it on its own terms.
Every article, in reading order
plant a flag as you finish eachRead these first
The generalization gap is the distance between how well a model scores on the data it was fitted to and how well it does on data it has never seen. Understanding where that distance comes from explains why backtests flatter and why more data helps.
Every standard learning bound assumes your observations are independent draws. Market data is not, so the row count in the formula overstates how much you actually know. This page shows what the honest bound looks like and how much narrower your evidence really is.
Rademacher complexity measures how well a family of models can fit pure coin-flip noise. That single number turns out to control how far a fitted model's training score can drift from its true score, and unlike VC dimension you can estimate it on your own data.
Fitting harder always lowers training error, so training error cannot tell you when to stop. Structural risk minimization stacks your model families from simple to complex, charges each one a complexity fee decided in advance, and picks the level where fit plus fee is smallest.
The law of large numbers promises that one model's training score converges to its true score. Learning needs something stronger, that every model in your class is measured accurately at once, and that stronger promise is what makes fitting from data safe.
Then the rest