NIC Timestamping and Clock Synchronization
Why software timestamps aren't accurate enough to measure microsecond latency, and how hardware timestamping on the network card plus protocols like PTP let different machines agree on 'when' to sub-microsecond precision.
Prerequisites: Latency Percentiles and Tail Latency
To measure how long a packet took to travel from an exchange to your server, you need to know two things precisely: the instant it left the exchange, and the instant it arrived at you. If either timestamp is recorded by ordinary application software — calling a clock function after the operating system has already handled the packet, moved it through several buffers, and scheduled your process to run — the timestamp is contaminated by all of that handling delay, which is exactly the thing you're trying to measure separately from the network itself. Worse, if the exchange's clock and your server's clock aren't precisely synchronized, the two timestamps might disagree about "now" by more than the actual delay you're trying to measure, making the whole comparison meaningless.
NIC timestamping solves the first problem by moving the timestamp as early — and as late — as physically possible: modern network interface cards (NICs) can stamp a packet with a hardware timestamp at the instant it physically leaves or arrives on the wire, before any operating system software gets involved at all. This removes the OS scheduling, interrupt handling, and buffering delay from the measurement entirely, leaving only the genuine network transit time plus the NIC's own tiny, consistent processing delay.
That only helps if both ends' clocks agree on what time it is, which is the second problem: ordinary machine clocks drift and can be milliseconds apart even after periodic NTP synchronization, a difference far too large to matter for millisecond-scale software but enormous for microsecond-scale trading latency. PTP (Precision Time Protocol) is designed for exactly this: by exchanging timestamped messages back and forth and accounting for the network delay between them, PTP-synchronized clocks across a network of machines can typically agree to within tens to hundreds of nanoseconds, rather than NTP's typical millisecond-scale agreement — a difference of roughly 10,000x, which is the gap between "close enough for a web server" and "close enough to measure trading latency at all."
Worked example: why the wrong clock reference invalidates a latency number
An engineer measures "exchange-to-server" latency by subtracting an application-level timestamp recorded after the packet is fully processed by the OS from a timestamp the exchange reports in its own logs. The application timestamp is recorded perhaps 4 microseconds after the packet actually hit the server's NIC (OS and scheduling delay), and the two clocks, synchronized only via NTP, can disagree by up to 1 millisecond. The reported "latency" could easily read anywhere from a nonsensical negative number to several milliseconds, when the true wire transit time is perhaps 50 microseconds — the measurement error, driven by clock disagreement and software-timestamp delay, dwarfs the quantity being measured by 20-1000x. Switching both ends to NIC-hardware timestamps synchronized via PTP shrinks the clock disagreement to under a microsecond and removes the OS delay from the timestamp entirely, making the 50 microsecond figure trustworthy for the first time.
What this means in practice
Any serious latency measurement or attribution — deciding whether a slowdown is in the network, the OS, or the application — depends on having timestamps accurate and synchronized well below the latency being measured. This is why colocated trading infrastructure invests in PTP-synchronized clocks and NIC hardware timestamping as basic plumbing, not an optional extra: without it, every latency number downstream — dashboards, alerts, backtested execution-cost models — is built on a foundation with more noise than signal.
Hardware timestamps taken directly on the NIC, combined with PTP clock synchronization across machines, remove OS scheduling delay and clock-drift error from latency measurements — both of which, left unaddressed, are typically far larger than the microsecond-scale latency being measured.
A latency number is only as trustworthy as the clock synchronization behind it. Comparing timestamps from two machines synchronized only by NTP (millisecond-scale agreement) to draw microsecond-scale conclusions is a common and easy-to-miss error — the "measurement" can be dominated by clock disagreement rather than anything real.
Related concepts
Practice in interviews
Further reading
- Gregg, Systems Performance, ch. 10 (Network)