Float32 vs Float64 in Model Inference

Every number a computer stores as a decimal is actually an approximation, and the question is only how many bits of approximation you're willing to pay for. Float64 ("double precision") uses 64 bits per number and can represent roughly 15–17 significant decimal digits; float32 ("single precision") uses 32 bits and gives roughly 7 digits. It's tempting to assume a trading model should always use the more precise option, but in practice the opposite is usually true: float32, and increasingly even lower-precision formats, dominate production inference.

The reason is that float32 uses half the memory, moves through memory bandwidth twice as fast, and runs roughly 2x faster on most CPUs and dramatically faster on GPUs, which are built with far more float32 (and float16) compute units than float64 ones. A model's actual prediction accuracy is almost never limited by the 8th decimal digit of a weight — it's limited by noisy data and imperfect features — so spending double the compute for precision the signal can't use is close to pure waste.

Worked number. A neural network with 100 million parameters stored as float64 needs 800MB just for weights; stored as float32 it needs 400MB, doubling how many models fit in GPU memory and roughly doubling inference throughput on typical hardware — with no measurable change in prediction quality for most trading models.