Testing Stochastic Code

A normal unit test says: call the function, check the output equals exactly this. That breaks the moment the function involves randomness — a Monte Carlo pricer, a bootstrap resampler, a stochastic simulation of a trading strategy. Run it twice and you get two different numbers, both correct. Testing stochastic code means testing it anyway, without either disabling the randomness you're trying to verify or writing a flaky test that fails one run in twenty for no real reason.

The idea: pin the seed, test properties not exact values

Two separate techniques do the job, and they are not the same thing.

Fix the seed for reproducibility tests. Every random number generator in practice is a pseudorandom generator: given the same seed, it produces the exact same sequence every time. Passing an explicit seed turns "random" into "deterministic but complicated," which means you can assert an exact expected output — as long as you accept that the test is now pinned to one specific generator's specific algorithm.

Test statistical properties for everything else. For code meant to work across seeds — meant to be genuinely random in production — assert things that are true of the distribution, not of any one draw: the sample mean lands within a tolerance of the true mean, the sample falls within a known confidence interval, the count of a rare event is roughly what theory predicts. This is where the math from Numerical Tolerance in Tests becomes unavoidable: a stochastic test needs a tolerance wide enough that it essentially never fails on a correct implementation, and it should be justified by an actual confidence interval, not a guessed number.

Worked example 1: pinning a seed for exact reproducibility

import numpy as np

def simulate_gbm_terminal(s0, mu, sigma, t, seed):
    rng = np.random.default_rng(seed)      # explicit, isolated generator
    z = rng.standard_normal()
    return s0 * np.exp((mu - 0.5 * sigma**2) * t + sigma * np.sqrt(t) * z)

def test_gbm_terminal_is_reproducible():
    result = simulate_gbm_terminal(100, 0.05, 0.2, 1.0, seed=42)
    assert abs(result - 104.86) < 0.01   # exact value for this seed, computed once and pinned

Run this test a thousand times and it produces the exact same 104.86 every time, because seed=42 fixes the entire random sequence. This test does not verify the model is statistically correct — it only verifies the code hasn't silently changed. That distinction matters: it will catch a refactor that accidentally flips a sign, but it will not catch a mathematically wrong formula that was wrong on day one and has stayed consistently wrong ever since.

Worked example 2: testing a statistical property across many seeds

def test_gbm_terminal_mean_matches_theory():
    s0, mu, sigma, t = 100, 0.05, 0.2, 1.0
    n = 20_000
    results = [simulate_gbm_terminal(s0, mu, sigma, t, seed=i) for i in range(n)]
    sample_mean = np.mean(results)

    theoretical_mean = s0 * np.exp(mu * t)          # 105.127...
    std_err = np.std(results) / np.sqrt(n)          # standard error of the sample mean

    assert abs(sample_mean - theoretical_mean) < 4 * std_err   # ~4-sigma tolerance

Here the tolerance isn't guessed — it's built from the standard error of the mean, which shrinks as $1/\sqrt{n}$. With $n=20{,}000$ draws, the sample mean should land within a few standard errors of the known theoretical mean under geometric Brownian motion, $s_0 e^{\mu t}$. Using $4\sigma$ as the bound makes a false failure on correct code astronomically rare (about 1 in 16,000 by the normal tail), while still catching a genuinely broken drift term, which would miss by a mile.

What this means in practice

Complexity is dominated by how many draws the property test needs: the standard error shrinks as $1/\sqrt{n}$, so cutting the tolerance in half requires roughly four times as many samples. Budget test runtime accordingly — a property test with $n=1{,}000{,}000$ draws that runs on every commit will slow the whole CI suite down; most teams cap statistical tests to a slower, less frequent test tier.

Where it shows up

Quant-dev interviews sometimes ask directly: "how would you unit-test a Monte Carlo pricer?" — the expected answer is exactly this split between seeded regression tests and statistical property tests, plus mentioning Random Seed Management so parallel test runs don't accidentally share state. In production, every Monte Carlo risk engine, bootstrap-based backtest, and randomized execution simulator needs this treatment before it can be trusted in CI — an unseeded, no-tolerance test on stochastic code is worse than no test, because it trains the team to ignore its failures.