Classical Algorithms

Read these first

1. 01Choosing a Classical Model for Tabular Data

   For the spreadsheet-shaped data most quant problems actually present, a well-tuned gradient-boosted tree usually beats a neural network, knowing why, and when that's not true, saves a lot of wasted modelling effort.

2. 02Soft-Margin SVMs and the C Parameter

   Real data overlaps, so an SVM has to let some points sit inside its corridor. C is the fine it charges them. Turning C up buys a boundary that contorts to accommodate every last point; turning it down buys a smooth one that shrugs at outliers.

3. 03Splitting Criteria: Gini, Entropy and Variance Reduction

   A decision tree has to score every candidate question and keep the best one. The score is always the same shape, how mixed was the parent, minus how mixed the children are, and the three standard ways of measuring "mixed" agree far more often than the arguments about them suggest.

4. 04The Perceptron Algorithm

   The oldest learning algorithm still in use. It looks at one example at a time, does nothing when it is right, and nudges its weights toward the answer when it is wrong. If the data can be separated by a straight line, it is guaranteed to find one.

5. 05The RBF Kernel and Bandwidth Selection

   The RBF kernel says two rows are similar if they are close and strangers if they are not, with a single knob controlling how fast "close" turns into "far". That knob decides whether your model is a constant, a sensible curve, or a lookup table of the training set.

Then the rest

1. 06Categorical Splits in Decision Trees
2. 07Choosing k and Distance Weighting in k-NN
3. 08Cost-Complexity Pruning
4. 09Distance Metrics and Metric Learning
5. 10Gamma and Tweedie Regression
6. 11Linear SVM vs Logistic Regression
7. 12Link Functions and the Exponential Family
8. 13MARS: Multivariate Adaptive Regression Splines
9. 14Multi-Label Classification
10. 15Multinomial and Ordinal Logistic Regression
11. 16One-vs-Rest and One-vs-One Multiclass Strategies
12. 17Orthogonal Matching Pursuit
13. 18Poisson and Count Regression
14. 19Polynomial Regression and Basis Expansions
15. 20Quadratic Discriminant Analysis
16. 21Support Vector Regression
17. 22Surrogate Splits and Missing Values in Trees
18. 23The Adaptive Lasso
19. 24Factorization Machines
20. 25Kernel Ridge Regression
21. 26Mercer's Condition and Valid Kernels
22. 27Nystrom Approximation and Random Fourier Features
23. 28The Representer Theorem
24. 29The SVM Dual and Support Vectors