XGBoost, LightGBM and CatBoost Compared

By the mid-2010s, gradient boosting had become the default choice for tabular prediction — return forecasting, credit models, click-through rates — and three open-source libraries came to dominate it: XGBoost, LightGBM, and CatBoost. All three fit an additive sequence of trees to the residual errors of the trees before them. What differs is how each tree is grown and how categorical features are handled, and those differences translate directly into training speed and which library tends to win on which kind of data.

Three ways to grow a tree

XGBoost splits level-wise: it considers every leaf at the current depth before moving deeper, and its objective adds an explicit L1/L2 penalty on leaf weights plus a penalty per leaf added, so the optimizer is directly discouraged from growing trees that don't earn their complexity back.

LightGBM splits leaf-wise: at each step it finds the single leaf, anywhere in the tree, whose split would reduce the loss the most, and grows only that one. This tends to reach a lower training loss with fewer splits, but because depth isn't controlled implicitly the way level-wise growth controls it, LightGBM trees can grow unexpectedly deep and overfit unless max_depth or num_leaves is capped explicitly.

CatBoost also grows trees (symmetric ones, where every node at a given depth uses the same split condition), but its headline difference is how it handles categorical features. Naively encoding a category by its average target value uses each row's own label to help encode that row — a subtle leak. CatBoost's ordered boosting computes each row's target statistic using only rows that came before it in a random permutation, so a row never sees its own label baked into its own feature.

Worked example

A quant fits a next-day-return classifier on 2 million rows with 40 numeric features and 8 categorical ones (sector, exchange, index membership). LightGBM trains in roughly a third of the time XGBoost takes on the same hardware, because leaf-wise growth needs fewer total splits to reach a given loss — but the validation AUC on LightGBM's default settings is slightly worse until num_leaves is tuned down, because the leaf-wise trees over-grew on the noisier features. Switching sector and exchange to CatBoost's ordered categorical encoding, instead of one-hot encoding them, improves out-of-sample AUC by a small but consistent margin, because the one-hot version was letting a rarely-traded sector's few training rows overfit through 40-plus extra sparse columns.

What this means in practice

None of the three is unconditionally "best" — LightGBM is usually the fastest to iterate with on large tabular data, XGBoost's explicit regularization terms make it a safe conservative default, and CatBoost's categorical handling is worth the switch specifically when high-cardinality categorical features (tickers, sectors, counterparties) are important predictors.