Deep Learning for Tabular Data

Deep learning's biggest wins — images, language, audio — all share something a spreadsheet of financial features doesn't: nearby inputs mean nearly the same thing. Shift a photo of a cat one pixel to the right and it's still obviously a cat; a convolution can exploit that local smoothness directly. Change one word in a sentence to a close synonym and the meaning barely shifts; attention can exploit that too. A row of financial features has no such structure — momentum, market cap, and a sector dummy variable are not "nearby" to each other in any meaningful geometric sense, and there's no reason to expect the relationship between them to be smooth just because their column order happens to be adjacent.

Think of it like the difference between reading a photograph and reading a jumbled list of unrelated facts about a company. A photograph has spatial relationships a specialized tool can exploit — this pixel's neighbors tell you something about this pixel. A jumbled fact list has no such geometry; "revenue: $40m" sitting next to "founded: 1998" in a table carries no implication that those two facts are related just because they're adjacent columns. Tools built to exploit smooth local structure lose much of their advantage on data that has none.

The mechanics, one comparison at a time

A gradient-boosted tree model builds decision rules directly on each feature's actual values: "if momentum > 0.3 and volatility < 0.2, predict up." It handles each feature's raw scale and its interactions with other features through explicit, greedy splits, one at a time, and the resulting boundary can be a sharp, irregular step function — exactly the kind of shape tabular relationships often actually have (a discontinuous rule like a specific credit-rating threshold, not a smooth curve).

$$\hat{y} = \sum_{t=1}^{T} f_t(x), \qquad f_t \text{ a decision tree}$$

In words: the prediction is a sum of many small trees, each one correcting the errors of the ones before it — this is gradient boosting's core mechanism, and every tree can carve out sharp, irregular decision boundaries with no need for the target function to be smooth.

A neural network, by contrast, is built from smooth building blocks — weighted sums and continuous activation functions — which naturally favor smoothly varying predictions. It can represent a sharp step function, but only awkwardly, needing many units and careful weight tuning to approximate what a tree produces with one clean split. On the messy, irregular, small-to-medium-sized tables typical in finance, that mismatch between the model's natural inductive bias and the data's actual shape shows up directly as worse accuracy, not just slower training.

Worked example 1: a sharp threshold, tree versus network

The true relationship: a stock's expected return jumps sharply once market cap crosses $2 billion (a well-documented small-cap liquidity effect), and is roughly flat on either side. A tree with one split, "market cap > $2bn," represents this exactly: two leaf values, one for each side, a perfect fit with a single rule.

An MLP has to approximate a step using smooth sigmoids or ReLUs. A single hidden unit with a very steep, precisely-tuned weight can get close, but "precisely tuned" is the catch: gradient descent has to find that exact steepness and threshold from noisy data, competing against thousands of other parameters being fit simultaneously, and it often settles for a smoothed-out transition — a gradual ramp over a $300–$500 million range instead of a clean jump at $2 billion — because a slightly-blurred boundary produces almost as low a training loss and is easier for gradient descent to find.

Worked example 2: sample efficiency on a mid-sized dataset

A dataset has 20,000 rows and 40 features, typical for a monthly cross-sectional equity study. A gradient-boosted tree model with default settings often reaches good performance with modest tuning, because each tree split is a simple, low-variance decision (compare one feature to one threshold) and boosting combines many such simple decisions.

A neural network with, say, 3 hidden layers of 128 units each has on the order of tens of thousands of parameters — comparable to or larger than the number of training rows. Getting it to generalize (rather than overfit) on 20,000 rows typically needs heavier regularization, more careful learning-rate tuning, and often still lands behind the tree model's out-of-sample accuracy in published comparisons on tabular benchmarks — not because neural networks are worse in principle, but because trees need less data to find the right shape when the true relationship is irregular and sample size is limited.

Use the explorer above to see the general trade-off directly: a flexible model needs more data to avoid landing in the overfit region. Trees reach useful flexibility with less data on tabular problems specifically because their split-based flexibility matches the data's typically irregular shape; a network needs to spend some of its flexibility budget just learning to approximate the tree-like boundaries the data actually has.

What this means in practice

The practical default for tabular financial prediction — factor models, credit scoring, fraud flags on structured data — remains gradient-boosted trees (XGBoost, LightGBM, CatBoost), which are also faster to train, easier to tune, and more interpretable via feature importance and split structure. Deep learning earns its place on tabular data specifically when the problem has structure a network can exploit that a tree can't: high-cardinality categorical features benefiting from learned embeddings, a need to combine tabular features with images or text in one end-to-end model, or genuinely enormous datasets (millions of rows) where a network's capacity to keep improving with more data starts to outpace a tree ensemble's. Absent one of those specific reasons, reaching for a neural network on a plain spreadsheet-shaped dataset is usually solving a harder optimization problem for no accuracy gain.