Single-Producer Single-Consumer Ring Buffers

A market-data thread receives packets from the network as fast as they arrive; a strategy thread wants to consume them and react. Put a mutex between the two and every single message pays a lock-acquire cost, plus the risk of the strategy thread blocking the data thread (or vice versa) at exactly the wrong moment. When there's exactly one writer and one reader, there's a much cheaper option: a single-producer single-consumer (SPSC) ring buffer — a fixed-size circular array where the producer writes into the next free slot and the consumer reads from the next filled slot, with no lock at all.

The trick that makes this safe without a lock is restricting the situation to exactly one writer and one reader. The buffer keeps two index counters, a write_index (owned and only ever modified by the producer) and a read_index (owned and only ever modified by the consumer). Because each index has exactly one writer, updates to it don't need to be atomic in the "multiple writers" sense — they just need the other thread to see the update promptly and in the right order relative to the data it protects, which is exactly what a memory barrier provides. The producer writes the data into a slot, then — after a release barrier — advances write_index; the consumer reads write_index — with an acquire barrier — before reading the slot's data, guaranteeing it never reads a slot the producer hasn't fully finished writing. The buffer is circular: once the indices reach the end of the fixed-size array, they wrap back to the start, using modular arithmetic (index % capacity), and it's "full" when the producer would catch up to the consumer, "empty" when the consumer has caught up to the producer.

Worked example: sizing the buffer against a burst

A feed handler produces market-data events at up to 2 million per second during a burst, and the strategy thread consuming them takes 800 nanoseconds per event on average, i.e. it can keep up with roughly $1/800\text{ns} \approx 1.25$ million events per second. During a one-millisecond burst at the 2 million/sec rate, the producer generates $2{,}000{,}000 \times 0.001 = 2{,}000$ events, while the consumer only drains $1{,}250{,}000 \times 0.001 \approx 1{,}250$ events in that same millisecond — a shortfall of about 750 events the buffer must absorb without the producer blocking or the consumer falling permanently behind. A ring buffer sized for only a few hundred slots would fill up mid-burst; sizing it to comfortably exceed the worst observed burst (say, 4,096 slots, a power of two for cheap modular wraparound) lets the producer keep writing through the burst while the consumer catches up afterward during the quieter period that follows.

What this means in practice

SPSC ring buffers are the standard way to hand data between a network-facing thread and a processing thread in latency-sensitive systems, because they avoid both the lock overhead of a mutex-protected queue and the cache-coherence traffic of a lock-free structure designed for many producers. Sizing matters: a buffer that's too small drops or stalls under bursts, one that's needlessly huge wastes cache and adds no benefit, and slot size should ideally be padded to avoid false sharing between the producer's and consumer's cache lines. The one-writer-one-reader restriction is absolute — adding a second producer or consumer breaks the correctness argument entirely and requires a different, more complex structure (MPSC, SPMC, or MPMC).