Inner Products and Orthogonality

Two trading signals might overlap heavily in what they capture, or might be measuring completely independent things — and correlation, the number analysts use to check this, is secretly a rescaled inner product. Inner products give vectors a notion of angle and length using nothing but multiplication and addition, and the special case of a zero inner product — orthogonality — is the mathematical meaning of "these two directions share nothing in common."

An analogy: how much two flashlights overlap

Point two flashlights at a wall from the same spot. If they point in exactly the same direction, their beams overlap completely. If they point at right angles to each other, their beams don't overlap at all — knowing where one beam hits tells you nothing about where the other hits. Anywhere in between, the beams partially overlap, and the amount of overlap can be measured precisely. The inner product of two vectors captures exactly this idea of overlap: large and positive when vectors point the same way, zero when they point in genuinely unrelated ("perpendicular") directions, negative when they point opposite ways.

The mathematics, one symbol at a time

For vectors $u = (u_1, \ldots, u_n)$ and $v = (v_1, \ldots, v_n)$, the standard inner product (dot product) is

$$\langle u, v \rangle = \sum_{i=1}^{n} u_i v_i = \|u\|\,\|v\|\cos\theta,$$

where $\|u\|$ and $\|v\|$ are the vectors' lengths and $\theta$ is the angle between them. In words: multiply corresponding entries and sum them, and the result equals the product of the two lengths scaled by how aligned they are (via the cosine of the angle between them) — fully aligned vectors ($\theta=0$) give the largest possible positive value, and perpendicular vectors give zero regardless of their lengths. Two vectors are orthogonal when $\langle u,v\rangle = 0$, meaning $\theta = 90°$: they point in genuinely independent directions, and neither has any component lying along the other. Correlation between two return series is exactly this cosine, computed on the vectors after subtracting their means — correlation is the cosine of the angle between two (mean-centered) data vectors.

Worked example 1: computing an inner product and checking orthogonality

Let $u = (3, 4)$ and $v = (4, -3)$. Inner product: $\langle u,v\rangle = 3(4) + 4(-3) = 12 - 12 = 0$. These two vectors are exactly orthogonal — perpendicular in the plane — which you can also see by noting $u$'s direction is "3 right, 4 up" and $v$'s is "4 right, 3 down," a 90° rotation of the first. Now check $w = (1,1)$ against $u$: $\langle u,w\rangle = 3(1)+4(1)=7 \neq 0$, so $u$ and $w$ are not orthogonal — they share some common direction.

Worked example 2: correlation as a cosine

Two mean-centered daily return series (deviations from their own averages) are $u=(2,-1,-1)$ and $v=(1,0,-1)$. Inner product: $\langle u,v\rangle = 2(1)+(-1)(0)+(-1)(-1)=2+0+1=3$. Lengths: $\|u\|=\sqrt{4+1+1}=\sqrt6\approx2.449$, $\|v\|=\sqrt{1+0+1}=\sqrt2\approx1.414$. Correlation $=\cos\theta = \langle u,v\rangle/(\|u\|\|v\|) = 3/(2.449\times1.414)\approx 3/3.464\approx 0.866$. A correlation of 0.866 corresponds to an angle of about $30°$ between the two return vectors — strong alignment, far from the 90° that would signal orthogonal (uncorrelated) signals.

Drag the correlation slider above: as $\rho$ moves toward 1 or −1, the scatter tightens into a line — the two variables' underlying vectors are nearly aligned or nearly opposite — and as $\rho$ approaches 0, the cloud rounds out, matching two vectors sitting near 90° apart.

What this means in practice

Orthogonality is the goal behind constructing "clean," non-overlapping factors or signals — a signal orthogonalized against known market factors carries genuinely new information rather than repackaging an existing exposure (see Orthogonalising a Signal Against Known Factors). Inner products also underlie projection, least-squares fitting, and the Gram-Schmidt process for building an orthogonal basis from any starting set of vectors.