Convolutional Neural Networks

Hand a The Multilayer Perceptron a 256-day window of prices and it faces a problem it cannot see its way out of: to the network, the 40th input and the 41st input are two unrelated quantities. If a sharp reversal on day 40 predicts something, the model learns that fact with one set of weights. When the same reversal shows up on day 41 — as it inevitably will, since patterns do not politely align to fixed offsets — none of that learning transfers. It has to learn the shape again, from scratch, at every possible position.

The bill arrives twice. First in data, because each position needs its own examples. Second in parameters: a dense layer from 256 inputs to 64 units already carries 16,448 weights, and none of them share anything.

The analogy: one stencil, slid across the page

Cut a shape out of cardboard — say a V, for a dip and recovery. Now slide that stencil along a printed price chart, and at every position write down how well the chart matches the cutout underneath. You get a new series: a running score of "how V-ish is the chart around here". One stencil, used at every position, and it works just as well on a chart of 8 bars as on one of 8,000.

That is a convolution, and quants have been running them by hand forever. A 20-day moving average is a convolution with a stencil of twenty equal weights. A momentum signal that subtracts the old price from the recent one is a convolution with a stencil that is negative at the back and positive at the front. The only thing a convolutional network adds is that it learns the shape of the stencil from the data instead of you choosing it, and it learns several stencils at once.

The maths

For a one-dimensional series $x$ and a kernel (the stencil) of $k$ weights $w_0, \dots, w_{k-1}$, the output at position $t$ is

$$s[t] = \sum_{j=0}^{k-1} w_j \, x[t+j] + b$$

In words: line the kernel up starting at position $t$, multiply each weight by the value beneath it, add them up, add a bias. Then move one position right and do it again. The resulting series $s$ is called a feature map — one number per position saying how strongly this particular pattern fired there.

Every symbol matters and there are only four. $w$ is the pattern being looked for. $k$ is how many bars the pattern spans. $b$ is a threshold offset. $t$ is where you currently are. The kernel weights are the same at every $t$ — that is weight sharing, and it is the whole source of the savings.

Worked example 1: a convolution by hand

Take eight closing prices: $100,\ 101,\ 103,\ 102,\ 106,\ 108,\ 107,\ 109$. Use the kernel $(-1,\ 0,\ +1)$ with $b = 0$, which measures "how much higher is the price two bars on".

Slide it along, one position at a time:

• Position 1 covers $(100, 101, 103)$: $-100 + 0 + 103 = 3$
• Position 2 covers $(101, 103, 102)$: $-101 + 102 = 1$
• Position 3 covers $(103, 102, 106)$: $-103 + 106 = 3$
• Position 4 covers $(102, 106, 108)$: $-102 + 108 = 6$
• Position 5 covers $(106, 108, 107)$: $-106 + 107 = 1$
• Position 6 covers $(108, 107, 109)$: $-108 + 109 = 1$

The feature map is $(3, 1, 3, 6, 1, 1)$ and its peak at position 4 correctly flags the steepest advance. Three numbers described the whole detector. Pass this through a ReLU and downward moves become zero, leaving a clean "rising" channel; a second kernel $(+1, 0, -1)$ gives you the matching "falling" channel.

Note the length: eight inputs became six outputs. In general the output length is

$$\left\lfloor \frac{n - k + 2p}{s} \right\rfloor + 1$$

where $n$ is the input length, $k$ the kernel width, $p$ the padding added at the edges and $s$ the stride (how far the kernel jumps each time). In words: count how many places the kernel fits. With $n = 256$, $k = 5$, no padding and stride 1 you get $252$ outputs; add $p = 2$ and you get $256$, preserving length; use stride $2$ instead and you get $128$, halving the series.

Worked example 2: what weight sharing actually saves

Compare two layers reading the same 256-bar window.

Dense layer, 64 units. Every unit has one weight per input plus a bias:

$$256 \times 64 + 64 = 16{,}448 \text{ parameters}$$

Convolutional layer, 8 kernels of width 5. Every kernel has five weights plus a bias, and that is all — the same five are reused at all 252 positions:

$$8 \times 5 + 8 = 48 \text{ parameters}$$

That is 343 times fewer parameters, and it produces more numbers: 8 channels by 252 positions, or 2,016 values, versus 64. Fewer things to learn, more structure extracted. On noisy financial data, where the binding constraint is almost always the amount of independent signal in the sample, that ratio is the difference between a model that generalises and one that memorises.

Depth buys width of view

A width-5 kernel only ever sees five bars. Stack a second convolutional layer on top and each of its outputs summarises five outputs of the first, which between them covered nine input bars. This span is the receptive field, and for $L$ stacked layers of width $k$ it is $1 + L(k-1)$ — so four layers of width 5 see 17 bars.

That hierarchy is the second half of the idea. On images, early kernels find edges, later ones find corners and textures, later still whole objects. On a price series, early kernels find single-bar jumps and gaps, later ones find multi-day patterns built from those pieces.

What this means in practice

Convolutions apply to any input where neighbouring positions genuinely belong together: a price or volume series, an order book snapshot across price levels, a volatility surface across strike and tenor. Multiple related series become channels — feed open, high, low, close and volume as five channels and the kernels learn patterns across them jointly, exactly as the three colour channels work in an image.

They are the wrong tool when position is arbitrary. A row of cross-sectional factor exposures — value, momentum, quality, size — has no meaningful ordering, so "neighbouring columns belong together" is simply false and a plain MLP or gradient-boosted tree is the honest choice.

Practically, kernels are learned by ordinary Backpropagation with one twist: because a weight is used at every position, its gradient is the sum of the gradients from all those positions. That is also why convolutional layers train stably even when they are deep.