The Multilayer Perceptron

A linear model gives every input a fixed importance. Momentum is worth $0.3$, value is worth $0.2$, and that stays true on quiet days, panicked days, and every day in between. That rigidity is often fine — and it is exactly why linear models are hard to beat. But it makes some perfectly ordinary statements impossible to write down. "Momentum works, unless volatility is high" is not a straight line. Neither is "this signal helps small caps and hurts large caps". Every relationship where the effect of one input depends on another is out of reach.

You could hand-build the interaction terms, but you have to know in advance which ones matter. A multilayer perceptron finds them for you.

The analogy: a desk of junior analysts

Picture a research desk. Every junior analyst sees the same raw inputs, but each one cares about a different combination of them. Analyst A watches "momentum minus volatility". Analyst B watches "value plus liquidity". Crucially, each one only speaks up when their own number crosses a threshold — below it they stay silent and contribute nothing.

The portfolio manager does not look at the raw data at all. She takes a weighted vote of whichever analysts happened to speak.

Nothing in that room is sophisticated. Every analyst computes a weighted sum and applies a threshold; the PM computes a weighted sum of what she hears. But because analysts go quiet in some market conditions and loud in others, the PM's final view is not a fixed formula of the inputs. Its shape changes with the situation. That is a multilayer perceptron, and the analysts are its hidden layer.

One unit

A single artificial neuron takes inputs $x_1, \dots, x_n$, multiplies each by a weight $w_i$ (how much this unit cares about that input), adds a bias $b$ (its baseline, equivalently the negative of its threshold), and passes the result through an activation function $\varphi$:

$$z = \sum_{i=1}^{n} w_i x_i + b, \qquad a = \varphi(z)$$

In words: score the inputs, then bend the score. The first equation alone is just linear regression. The second one, $\varphi$, is what makes everything else possible.

Two activations cover most of what you will see. ReLU is $\varphi(z) = \max(0, z)$: pass positive scores through untouched, output zero for anything negative. That is literally the analyst who stays silent. The logistic (sigmoid) is $\varphi(z) = 1/(1 + e^{-z})$, an S-shape that squashes any score into the range 0 to 1, useful when you want a probability at the end.

Drag the steepness below to see the sigmoid go from a gentle slope to a near-vertical switch. A steep activation is close to a hard yes/no gate; a shallow one blends smoothly.

Stacking units into layers

Put several units side by side, all reading the same inputs, and you have a layer. Feed that layer's outputs into another layer and you have a multilayer perceptron. Written compactly for a two-layer network:

$$h = \varphi(W^{(1)}x + b^{(1)}), \qquad \hat{y} = W^{(2)}h + b^{(2)}$$

In words: the first line says every hidden unit scores the raw inputs and bends the result; the second line says the output is an ordinary weighted sum of those bent scores. $W^{(1)}$ is a table of weights with one row per hidden unit, $h$ is the vector of what the hidden units said, and $\hat{y}$ is the prediction.

Worked example 1: one forward pass by hand

Two standardised inputs: momentum $x_1 = 1.2$ and volatility $x_2 = -0.5$ (so vol is below average). Two ReLU hidden units, one sigmoid output.

Hidden unit 1 has weights $(0.8,\ -0.6)$ and bias $0.1$:

$$z_1 = 0.8(1.2) + (-0.6)(-0.5) + 0.1 = 0.96 + 0.30 + 0.10 = 1.36$$

That is positive, so ReLU passes it: $h_1 = 1.36$. This unit likes high momentum and dislikes high volatility, and today both conditions favour it, so it speaks loudly.

Hidden unit 2 has weights $(-0.4,\ 0.9)$ and bias $0.2$:

$$z_2 = -0.4(1.2) + 0.9(-0.5) + 0.2 = -0.48 - 0.45 + 0.20 = -0.73$$

Negative, so ReLU clamps it: $h_2 = 0$. This unit stays silent today, and therefore contributes exactly nothing to the answer.

Output with weights $(1.1,\ -0.7)$ and bias $-0.3$:

$$z_{\text{out}} = 1.1(1.36) - 0.7(0) - 0.3 = 1.496 - 0.3 = 1.196$$ $$\hat{y} = \frac{1}{1 + e^{-1.196}} = \frac{1}{1 + 0.302} = 0.77$$

A 77% chance of an up move. Note what happened: the second unit was switched off, so today the network behaved like a different, simpler model than it would on a high-volatility day. Same weights, different active pathway.

Worked example 2: the thing a line cannot do

Two flags: $a$ is 1 when momentum is positive, $b$ is 1 when volatility is low. The target pays 1 when exactly one flag is on — the classic XOR pattern.

$a$	$b$	target
0	0	0
1	0	1
0	1	1
1	1	0

No straight line can do this. A linear rule fires when $w_1 a + w_2 b + c > 0$. It must be positive at $(1,0)$ and $(0,1)$, so adding those two conditions gives $w_1 + w_2 + 2c > 0$. It must be negative at $(0,0)$ and $(1,1)$, and adding those gives $w_1 + w_2 + 2c < 0$. The same quantity cannot be both. No weights exist, ever.

Now two ReLU hidden units. Let $h_1 = \max(0,\ a + b)$ and $h_2 = \max(0,\ a + b - 1)$, with output $\hat{y} = h_1 - 2h_2$:

• $(0,0)$: $h_1 = 0$, $h_2 = 0$, output $0$. ✓
• $(1,0)$: $h_1 = 1$, $h_2 = \max(0, 0) = 0$, output $1$. ✓
• $(0,1)$: $h_1 = 1$, $h_2 = 0$, output $1$. ✓
• $(1,1)$: $h_1 = 2$, $h_2 = \max(0, 1) = 1$, output $2 - 2 = 0$. ✓

Exact, with two hidden units and six numbers. The second unit only wakes up when both flags are on, and its job is to cancel the first unit's enthusiasm. That cancellation is the interaction term you never had to specify.

What this means in practice

Add more hidden units and the boundary the network can draw gets more intricate. The The Universal Approximation Theorem says one wide enough hidden layer can approximate essentially any continuous function — but "can represent" is not "will find from noisy data". Financial signals have low signal-to-noise, so an MLP flexible enough to fit the training set will happily fit the noise too. The explorer below shows the trade-off directly: turn up flexibility and watch the boundary contort around individual points.

In practice the inputs must be standardised (a unit weighing raw price against raw volume is hopeless), the weights are found by Stochastic Gradient Descent using gradients from Backpropagation, and depth is usually kept modest on tabular financial data — two or three hidden layers, heavily regularised, is a typical honest setup. Where MLPs genuinely earn their keep is when interactions matter and you cannot name them in advance.

Three choices are yours to make. Width is how many units per layer, which controls how many distinct "analysts" the model can hire. Depth is how many layers, which lets later units build on combinations that earlier units already found rather than starting from the raw inputs each time. And the output layer is not bent at all for a regression target — you want a plain weighted sum that can take any value — but gets a sigmoid for a two-way classification, so the answer arrives as a probability. Get the last one wrong and the model cannot express the answer you asked for, no matter how well the rest trains.