Mutexes, Spinlocks and Contention

Two threads both need to update the same order book: one processes incoming market data, the other reads the book to decide whether to quote. If both touch the book's internal data at the same instant, the reader can see it half-updated — a corrupted, meaningless state. The threads need to take turns, and the software mechanism that enforces "only one thread at a time" is a lock. The two everyday flavors are a mutex and a spinlock, and they differ in what a thread does while it's waiting.

A mutex ("mutual exclusion") puts a waiting thread to sleep: it asks the operating system to park the thread and wake it back up once the lock is free. This costs almost no CPU while waiting, but the sleep-and-wake round trip goes through the OS scheduler and can take microseconds — an eternity on a latency-critical path. A spinlock instead makes the waiting thread busy-wait: it sits in a tight loop repeatedly checking "is the lock free yet?", burning CPU cycles the entire time but able to grab the lock the instant it's released, with no OS round trip at all. The right choice depends entirely on how long the lock is expected to be held: for a lock held for a few nanoseconds (a quick pointer swap), spinning wins easily because the OS round trip of a mutex would dwarf the actual wait; for a lock that might be held for milliseconds (writing to disk), spinning would waste enormous CPU time for nothing, and sleeping is clearly better.

Contention is what happens when many threads want the same lock at once. Under low contention, the choice of lock barely matters — one thread usually just walks straight through. Under high contention, a spinlock can degrade badly: every waiting thread is burning a full CPU core doing nothing but polling, competing for the same cache line that holds the lock's state and repeatedly invalidating each other's cached copy of it, which slows down even the thread that's supposed to be making progress.

Worked example: choosing for a shared order-book update

A book-update lock is held for about 50 nanoseconds per critical section (a single pointer swap), and an OS context switch to sleep and wake a mutex-waiting thread costs roughly 2,000 nanoseconds round trip. If a thread contends for this lock and uses a mutex, it pays a 2,000 ns penalty to protect a 50 ns operation — a 40x overhead. Using a spinlock instead, the waiting thread polls in a loop and typically picks up the lock within a few tens of nanoseconds of it freeing, since 50 ns of hold time means the wait, if any, is brief. Here the spinlock is the clear win. If instead the same lock protected a slow disk write held for 5 milliseconds, a spinning thread would burn 5 milliseconds of a full CPU core doing nothing productive — the mutex's 2,000 ns sleep/wake overhead is negligible by comparison, and sleeping is clearly better.

What this means in practice

Low-latency trading systems typically avoid locks on their hottest paths altogether, reaching for lock-free data structures instead, but where a lock genuinely is needed, the hold-time-vs-wake-cost tradeoff above is the deciding factor. Spinlocks also interact badly with CPU oversubscription: if there are more runnable threads than cores, a spinning thread can burn CPU time that the actual lock-holder needed to finish and release the lock, a pathology called lock holder preemption — one more reason latency-sensitive systems pin threads to dedicated, isolated cores.