Property-Based Testing

A hand-written unit test checks a function against specific numbers you thought of — usually the typical case and one or two edge cases you happened to remember. That leaves a huge space of inputs untested, and bugs love to hide in exactly the combinations nobody thought to try: a negative size, a price of zero, a portfolio with one asset instead of several. Property-based testing flips the approach — instead of picking specific inputs, you state a property that should hold for every valid input, and a library generates hundreds of random inputs (including deliberately weird ones) to try to break it.

The idea: describe a rule, let the machine hunt for counterexamples

A property is something that should always be true regardless of the specific numbers — "sorting a list twice gives the same result as sorting it once," "a portfolio's weights should always sum to 1 after normalization," "encoding then decoding a message returns the original message." The testing library generates random inputs matching a type or shape you specify, runs the function, and checks the property on every one; if it finds a case where the property fails, it automatically shrinks that case down to the smallest input that still fails, so you get a minimal, readable counterexample instead of a huge random one.

from hypothesis import given, strategies as st

def normalize_weights(weights):
    total = sum(weights)
    return [w / total for w in weights]

@given(st.lists(st.floats(min_value=0.01, max_value=1e6), min_size=1, max_size=20))
def test_weights_sum_to_one(weights):
    result = normalize_weights(weights)
    assert abs(sum(result) - 1.0) < 1e-9

Worked example: finding a bug that examples missed

normalize_weights looks correct and passes every hand-written example: [1, 1, 1] -> [0.333, 0.333, 0.333] sums to 1, [10, 20, 30] -> [0.166, 0.333, 0.5] sums to 1. But Hypothesis, generating random lists, eventually tries something like [1e6, 0.01] — a huge value next to a tiny one — and finds that floating-point division introduces enough rounding error that the sum drifts to 0.9999999991, failing the 1e-9 tolerance. It then shrinks the failing example, discarding elements and simplifying values while the failure persists, until it reports something minimal like [1000000.0, 0.01] instead of the original random list of twenty numbers — pointing straight at the numerically unstable case: extreme ratios between weights.

Falsifying example: test_weights_sum_to_one(
    weights=[1000000.0, 0.01],
)
AssertionError: assert abs(0.9999999990999999 - 1.0) < 1e-9

Where it shows up

Interviewers occasionally ask you to state a property for a function you just wrote as a sanity check — "what's true about this output no matter the input?" is a quick way to test whether you understand your own code's contract, independent of whether you use a property-testing library at all.

In production quant systems, property-based testing earns its keep on exactly the kind of numerically finicky code hand examples tend to under-test: portfolio weight normalization, position sizing under leverage constraints, order book invariants (bid always below ask), and serialization round-trips for market data messages. It pairs naturally with Numerical Tolerance in Tests, since floating-point properties almost always need an explicit tolerance rather than exact equality.