Topic · Machine Learning & AI
← All topicsDeep Learning
77 articles · 13 checkpoints · 46 deeper reads · 18 reference notes
Every article, in reading order
plant a flag as you finish eachRead these first
Walking every number through a small two-layer network's forward and backward pass, with no shortcuts, is the fastest way to trust that backpropagation is just arithmetic, not magic.
Backpropagation is how a network works out, for every one of its weights at once, whether nudging that weight up or down would reduce the error. It is the chain rule from calculus, applied backwards through the network so that a single sweep answers millions of questions.
A neural network's output layer produces numbers that look like probabilities, but a network trained only to get the label right has no incentive to get the confidence right too. Calibration is the separate step that fixes this.
A convolutional network learns a small set of pattern detectors and slides each one across the whole input, so a shape learned in one place is recognised everywhere. That single idea cuts the parameter count by orders of magnitude and is what makes networks work on images and price series.
Stochastic gradient descent estimates the downhill direction from a small random sample of the data instead of all of it, then steps immediately. Each step is noisier, but you take hundreds of them for the price of one exact step, and that trade is why large models can be trained at all.
The multilayer perceptron is the plain vanilla neural network, a stack of weighted sums with each one bent by a simple nonlinear function before being passed on. That bending is the whole trick, and it is what lets the model learn the "it depends" relationships a straight line cannot express.
A computer can compute the exact derivative of any function you can write as code, without you ever doing calculus by hand, by recording every elementary operation as a graph and applying the chain rule mechanically along it.
Instead of deriving a hedge ratio from a model and then apologising for the assumptions, train a neural network to choose the hedge directly by minimising a risk measure of the final profit and loss, with trading costs and discrete rebalancing built in from the start.
Gated recurrent networks that keep a protected memory line running through time, with small learned valves deciding what to erase, what to add, and what to reveal. They are the standard fix for a plain RNN forgetting anything that happened more than a handful of steps ago.
Two trained models can have identical training loss and still behave completely differently under small perturbations, because one sits in a wide, gentle valley of the loss surface and the other sits in a narrow, steep-walled canyon.
The Sharpe ratio is built entirely from a mean and a standard deviation, both of which have well-defined gradients, which means a model can be trained to directly maximize risk-adjusted return instead of a proxy like mean squared error.
A sequence model that throws away recurrence entirely and lets every position look at every other position in one hop, using attention weights it computes on the fly. The dot-product-scale-softmax-average recipe is the whole engine.
Off-the-shelf losses like mean squared error assume every mistake costs the same. Real trading mistakes don't, and a loss function can be built from scratch to charge the model exactly what each kind of error actually costs.
Then the rest