SIMD Vectorization

A conventional ("scalar") CPU instruction adds one number to one number. SIMD — Single Instruction, Multiple Data — packs several numbers into one wide register and adds all of them in a single instruction. A 256-bit AVX register holds four 64-bit doubles side by side; one vaddpd instruction adds four pairs of doubles at once, in roughly the same time a scalar instruction adds one pair. Applied across a loop with millions of iterations, that's a potential 4x (or 8x with 32-bit floats) reduction in instruction count for the exact same arithmetic.

This is the mechanism underneath NumPy's speed advantage over plain Python loops, and underneath why a well-written C++ loop can outrun a naively-written one doing identical math: the compiler, or an explicit intrinsic, packs multiple elements into one SIMD register and processes them together instead of one at a time.

Scalar versus vectorized addition

Worked example

Element-wise adding two arrays of 100 million doubles, compiled with auto-vectorization disabled (-fno-tree-vectorize), takes roughly 210 milliseconds — one scalar add per pair. The identical loop, compiled with auto-vectorization enabled and targeting AVX2 (which processes 4 doubles per instruction), takes roughly 55 milliseconds — close to a 4x speedup, with not a single line of source code changed. The compiler recognized the loop's simple, sequential, branch-free structure and packed it into SIMD instructions on its own.

Now change the loop body to include a data-dependent branch — say, adding only when a third array's flag is true — and auto-vectorization often fails silently: the compiler can't safely pack a conditional operation into an unconditional wide instruction without extra work (masked SIMD instructions can do this, but naive auto-vectorization frequently gives up), and the loop falls back to scalar speed with no warning unless compiler diagnostics are explicitly requested.

What this means in practice

SIMD is why "vectorize your NumPy code" and "keep hot C++ loops simple and branch-free" are really the same piece of advice at different levels of abstraction: both are about giving the underlying hardware operations it can pack together instead of doing one at a time. Pricing engines evaluating the same formula across thousands of strikes, or risk calculations applied uniformly across a portfolio, are prime candidates — the same arithmetic, repeated over dense, contiguous data, with no data-dependent branching in between.