The TLB and Huge Pages

Every time a program reads or writes memory using a virtual address, the CPU has to translate that address into a real physical memory location — a lookup performed via a multi-level page table stored in main memory. Doing this full lookup on every single memory access would be crushingly slow, so CPUs cache recent translations in a small, extremely fast on-chip cache called the Translation Lookaside Buffer (TLB). A TLB hit resolves an address in essentially zero extra cycles; a TLB miss forces the slow page-table walk, which can cost tens to hundreds of cycles — directly on the hot path of a latency-sensitive trading system.

The standard memory page size on most systems is 4 kilobytes, so a large in-memory order book or market-data cache spanning gigabytes needs a huge number of distinct pages, far more than the TLB's limited number of entries (often only a few dozen to a few hundred) can track at once — causing frequent TLB misses as the working set churns through more pages than the cache can hold. Huge pages (commonly 2MB or 1GB instead of 4KB) fix this by covering vastly more memory per TLB entry, so the same working set needs far fewer entries and fits comfortably in the TLB, cutting miss rates and shaving microseconds off memory-bound hot paths.

Because 4KB is the default almost everywhere, using huge pages usually requires deliberate OS configuration or explicit allocation calls — a common, high-payoff tuning step in exchange gateways and matching-adjacent systems where every microsecond of jitter matters.