Choosing a Classical Model for Tabular Data

Deep learning dominates images, text, and audio, but a large share of quant modelling problems are neither — they're a spreadsheet: a table of rows (dates, or trades, or securities) and columns (features like momentum, spread, volume), with no spatial or sequential structure a convolution or attention mechanism is built to exploit. On exactly this kind of data, deep networks routinely lose to much older, cheaper methods, and knowing why prevents a lot of wasted effort defaulting to whatever's newest.

The analogy: a Swiss Army knife versus the tool built for the job

A neural network is a Swiss Army knife: extraordinarily capable in the hands of an expert, and its flexibility is exactly the point when the task's structure is unknown or irregular, like recognising an object in a photo from any angle. But tabular data usually already comes with useful structure built in — each column has its own meaning and scale, relationships are often threshold-like ("above this leverage ratio, risk changes") rather than smooth, and there's no spatial pattern to exploit. A tool built specifically for that shape of problem — a decision tree, which naturally splits on thresholds column by column — tends to beat the general-purpose tool, the same way a screwdriver beats a Swiss Army knife's screwdriver attachment for actually driving screws all day.

The decision, roughly

For a genuinely tabular problem with a moderate number of rows (thousands to low millions) and mixed numeric/categorical features, the practical hierarchy quant teams tend to reach for, in order of how often they win:

Gradient-boosted trees (XGBoost, LightGBM, CatBoost) are usually the strong default. They handle mixed feature types and missing values natively, need little feature scaling, and their tree-based splits match tabular data's often threshold-like, non-smooth relationships well. See Gradient Boosting as Functional Gradient Descent for the mechanics.

Linear or logistic regression with regularization (see Regularization as a Prior) is worth trying first regardless, both as a fast baseline and because it can genuinely win when the true relationship really is close to linear and the dataset is small — a regime where a flexible tree ensemble's extra capacity just adds variance without adding signal.

Random forests sit between the two: less tuning-sensitive than boosted trees, often slightly less accurate, useful when you want a robust result fast without much hyperparameter search.

Deep networks earn their place mainly when the "tabular" framing is misleading — when a column actually encodes something with internal structure, like an embedded time series, text, or a large number of high-cardinality categorical features (thousands of distinct tickers, say) that benefit from a learned embedding rather than one-hot encoding.

Worked example: a concrete comparison on a small research task

A team builds a model to predict whether a stock will outperform its sector next month, using 40 engineered features (valuation ratios, momentum measures, size, liquidity) across 5 years of monthly data for 2,000 stocks — roughly 120,000 rows, a squarely tabular, moderate-size problem.

Logistic regression with L2 regularization, minimal tuning: 54.2% out-of-sample accuracy (against a 50% base rate for a roughly balanced label), AUC 0.56.

A tuned LightGBM model with a modest grid search over tree depth and learning rate: 57.8% accuracy, AUC 0.61 — a meaningful jump, consistent with genuine nonlinear and interaction effects among the 40 features that the linear model can't capture.

A feedforward neural network with two hidden layers, tuned with a comparable amount of effort: 56.9% accuracy, AUC 0.60 — close to the boosted tree, sometimes even matching it with enough tuning and regularization, but taking roughly ten times longer to train and search over on this dataset size, for no accuracy improvement. On this shape of problem, the extra engineering cost of the neural network bought nothing the tree ensemble hadn't already captured.

The explorer above shows how a flexible classifier's decision boundary can bend to fit training points closely at the cost of overfitting; tree ensembles and neural networks both live on the flexible end of that spectrum for tabular data, and the practical choice between them usually comes down to which one reaches a good boundary with less tuning effort and lower variance, not which one is theoretically more expressive.

What this means in practice

None of this means neural networks are never worth trying on tabular data — for very large datasets, or where the raw features genuinely benefit from learned representations rather than the engineered ones a tree-based model relies on, the gap narrows or reverses. But as a starting default, spending the first modelling cycle on a well-tuned gradient-boosted tree and a regularized linear baseline, rather than reaching straight for a deep architecture, tends to be the faster path to a strong result on the kind of tabular problems that dominate quant research — factor models, credit scoring, trade classification — and it leaves a clear, well-understood baseline to beat before investing in anything more complex.