Memoising Expensive Signal Computations

A typical research workflow recomputes the same expensive signal — a rolling volatility estimate, a factor exposure, a cross-sectional rank — over and over across many backtest runs that only tweak an unrelated parameter, like a position-sizing rule or a stop-loss threshold. If the signal computation takes minutes and doesn't actually depend on what changed, recomputing it every run wastes enormous amounts of researcher time for no benefit. Memoisation fixes this by storing ("caching") the output of a computation the first time it runs, keyed by its inputs, and returning the stored result instantly on any later call with the same inputs.

The idea is the same as a student who solves a hard homework problem once, writes the answer in the margin, and just copies it down if the same problem appears again on a later worksheet — no need to redo the work, because nothing about the problem has changed.

In practice, memoisation for signal computations usually means hashing the function's inputs (the raw data, parameters, and code version) into a cache key, and storing results either in memory for a single research session or on disk for reuse across sessions and across a whole research team. The main pitfall is a stale cache: if the underlying data or the signal's code changes but the cache key doesn't reflect that change, the cached (now-wrong) result gets silently reused, producing a backtest that quietly runs on outdated numbers.