The Supervised Learning Framework

Every supervised learning problem starts from the same slightly awkward position. You have a pile of past cases where you know both the question and the answer, and you want a rule that answers a question you have never seen. A bank has ten years of loan applications and knows which ones defaulted; it wants to score the applicant walking in today. A desk has five years of daily bars and knows which days rose; it wants a view on tomorrow. Supervised learning is the machinery that turns "here are answered examples" into "here is a rule."

The word supervised just means the answers came attached — somebody labelled them, or time revealed them. That is the only thing separating this from unsupervised learning, where you get inputs and nothing else.

The four pieces

Every supervised setup, from a straight line to a hundred-layer network, is built from exactly four parts. Learning the parts is worth more than learning any one algorithm, because every algorithm is just a different choice for one of them.

The data. Pairs $(x_i, y_i)$ for $i = 1, \dots, n$. Each $x_i$ is a vector of features — the things you know at decision time, like yesterday's return or an option's moneyness. Each $y_i$ is the label, the thing you want. If $y$ is a number you have a regression problem; if it is a category you have a classification problem. Nothing else about the machinery changes.

The hypothesis class. The set $\mathcal{H}$ of rules you are willing to consider: all straight lines, all trees of depth at most five, all networks with a fixed architecture. You never search "every possible function" — you search a family you chose in advance. This is the single biggest modelling decision you make, and it is usually made in thirty seconds by importing a library.

The loss function. A number $\ell(\hat{y}, y)$ saying how bad it is to predict $\hat{y}$ when the truth was $y$ — squared error $(\hat{y} - y)^2$ for regression, a cost per mistake for classification. Your real preferences live here. If missing a blow-up costs ten times a false alarm, that belongs in the loss, not in an apology afterwards.

The learning algorithm. The search procedure that walks $\mathcal{H}$ looking for the rule with the smallest average loss on your data. That search has a name: Empirical Risk Minimization.

The goal is the population, not the sample

Here is the part beginners skip. What you actually care about is the risk: the average loss over all future cases the world could hand you.

$$R(f) = \mathbb{E}_{(X,Y)\sim P}\big[\ell(f(X),\, Y)\big]$$

In plain words: if the universe kept dealing you fresh examples forever, this is how wrong your rule would be on average. You cannot compute it, because you do not have the universe — you have $n$ rows in a file. So you compute the average loss on the rows you do have and hope it tracks the real number. Every difficult thing in machine learning is a consequence of that gap.

A worked example: six days in, six days out

Suppose the feature is "did the market rise yesterday?" and the label is "did it rise today?". Six training days:

Day	Yesterday	Today
1	up	up
2	up	down
3	up	up
4	down	down
5	down	down
6	down	up

The hypothesis class here is tiny — with one binary feature there are only four possible rules. Take the candidate "predict today equals yesterday". It is right on days 1, 3, 4, 5 and wrong on 2 and 6, so the training error under 0-1 loss is

$$\hat{R} = \frac{2}{6} = 0.333,$$

an accuracy of $66.7\%$. Encouraging. Now run the frozen rule on six held-out days and suppose it is right on three. The test error is $3/6 = 0.5$ — coin-flip. The rule did not get worse; it was never that good. The $66.7\%$ was the score of the best of four candidates on six noisy points, and best of anything is biased upward.

Drag the complexity slider below and watch the two curves separate. Training error falls forever; test error bottoms out and then climbs. The distance between them is the price of having searched.

What this looks like on a desk

Finance strains the framework in three specific ways. The pairs are not independent — overlapping windows mean day 100 and day 101 share most of their information, so your effective sample is far smaller than $n$. The world is not stationary, so the $P$ you trained on may not be the $P$ you trade in (Regime Detection). And the label is itself a modelling choice: "did it go up?" is a much worse target than "did it hit the profit barrier before the stop?" (Triple-Barrier Labeling).