NUMA Architecture
Non-Uniform Memory Access: in multi-socket servers, each CPU has its own directly attached memory, so accessing memory attached to a different CPU is slower than accessing local memory — a fact that matters for low-latency trading systems.
A modern trading server often has more than one physical CPU socket, and in a NUMA (Non-Uniform Memory Access) design, each socket has its own bank of RAM wired directly to it. A CPU can read its own "local" memory quickly, but reading memory attached to the other socket has to travel across an interconnect between the two chips, adding extra latency — often 30-100% slower than a local access, depending on the hardware.
This matters in low-latency trading because a thread pinned to CPU 0 that allocates a large buffer, only to have that memory physically placed on the CPU 1 node (because the operating system's default allocator didn't know better), pays a remote-access penalty on every single read and write to that buffer — a cost that's invisible in ordinary correctness testing but shows up as extra microseconds on the hot path of an order-processing loop.
The practical fix is NUMA-aware placement: pinning a process's threads to cores on one socket and ensuring its memory is allocated on that same socket's local RAM (using OS calls like numactl or libnuma to control this explicitly), so that every memory access on the critical path stays local. Systems that skip this can see tail latencies that vary depending on which socket happened to serve a given request — an inconsistency that's hard to diagnose without knowing NUMA topology is in play.
On multi-socket servers, memory local to a CPU's own socket is meaningfully faster to access than memory on another socket's bank, so low-latency systems pin threads and allocate memory together on the same NUMA node to keep every access on the hot path local.
Practice in interviews
Further reading
- Drepper, What Every Programmer Should Know About Memory