Neural Oblivious Decision Ensembles

Gradient-boosted trees usually beat plain neural networks on tabular data — spreadsheet-style rows of numeric and categorical features, the format most trading data comes in — because trees split on hard thresholds that suit that kind of data well, while ordinary neural layers don't have an equivalent inductive bias. Neural Oblivious Decision Ensembles (NODE) try to get the best of both: a layer built from "oblivious" decision trees, where every node at the same depth uses the same feature and same threshold test (unlike ordinary trees, where each node can split on a different feature), which makes the whole tree representable as a smooth, differentiable function trainable by ordinary backpropagation.

Because the layer is differentiable, many oblivious trees can be stacked and trained end-to-end just like layers in a deep network — combining tree-like splitting behavior, which works well on tabular features, with the ability to learn representations jointly across layers, which trees alone can't do.

A worked example

On a tabular dataset predicting loan default from income, credit score, and debt ratio, a single oblivious tree of depth 4 uses the same threshold (say, "credit score > 680") at every node at a given depth across all its branches, letting the whole layer's output be computed with soft, differentiable comparisons instead of hard if-then branches — enabling gradient descent to tune those thresholds jointly with several stacked layers, something a standard gradient-boosted tree, trained greedily one tree at a time, cannot do.