Supervised Learning

Read these first

1. 01The Supervised Learning Framework

   The shared skeleton behind every predictive model, labelled examples, a family of candidate rules, a loss function and a search. Once you see the four pieces, every algorithm looks like a different choice of the same four things.

2. 02Decision Trees and CART

   A model that is literally a flowchart of yes/no questions, grown greedily by always asking whichever question separates the outcomes best. Easy to read, easy to overfit, and the building block of every forest and boosted ensemble.

3. 03Empirical Risk Minimization

   The rule behind almost every fitted model, since you cannot measure error on data you have not seen, minimise average error on the data you have. It works, right up until the model starts memorising the sample instead of learning the world.

4. 04Label Noise and Learning with Noisy Targets

   When the labels you're training on are themselves partly wrong, a common situation in finance, where the 'true' outcome is a noisy realisation of an underlying signal, a model that fits them too closely is fitting mistakes, not patterns.

5. 05Support Vector Machines

   A classifier that does not just separate two groups but separates them by the widest possible corridor. The width is set by a handful of borderline points, and everything else in the dataset is ignored.

Then the rest

1. 06K-Nearest Neighbors
2. 07AdaBoost and Exponential Loss
3. 08Bagging and Variance Reduction
4. 09Class Imbalance and Resampling
5. 10Double Descent
6. 11Feature Selection: Filter, Wrapper, Embedded
7. 12Generalized Additive Models and Splines
8. 13Huber and Robust Loss Functions
9. 14Learning to Rank
10. 15Linear Discriminant Analysis
11. 16Loss Functions for Classification
12. 17Loss Functions for Regression
13. 18Missing Data Imputation
14. 19Naive Bayes Classifiers
15. 20Online Learning and Regret Bounds
16. 21Probability Calibration
17. 22Random Forests and Out-of-Bag Error
18. 23Semi-Supervised Learning
19. 24The Curse of Dimensionality
20. 25XGBoost, LightGBM and CatBoost Compared
21. 26Gaussian Process Regression
22. 27The Kernel Trick and Kernel Methods