Timeliness: When Does the Data Actually Arrive?
The date stamped on a data record and the date you could actually have used it are usually two different things, and the gap between them is delivery latency.
Prerequisites: Profiling a Raw Feed: Rows, Keys and Nulls
A vendor's file for July 24th says "as of July 24th." That does not mean it was on your servers on July 24th. It might have been compiled overnight and delivered at 6am on the 25th, or — worse — computed with information that only existed by the 26th and then backdated to the period it describes. Timeliness is the study of how long it actually takes a piece of data to travel from the real-world event it describes to a form your research code can use.
Three different timestamps matter for the same record: when the underlying event happened, when the vendor's file claims it happened, and when the file actually landed on your infrastructure. Only the last one is safe to use in a backtest.
The three clocks
- Event time — when the thing really occurred (an earnings call, a trade print).
- Vendor timestamp — the date the vendor's file labels the record with, which may be the event date even if delivery came later.
- Receipt time — when the file actually arrived and became queryable in your systems.
Worked example
A desk logs the exact wall-clock time each new file from an alternative-data vendor lands on its ingestion server, alongside the "as of" date printed inside the file, for 60 trading days. The vendor's documentation claims same-day delivery. The logs show the file consistently lands between 4:40am and 5:15am the next calendar day — usable before the following day's open, but not on the day it is labeled for. A backtest that joins this data to same-day prices, trusting the vendor's own timestamp, is trading on information a full day before it was really available; shifting every observation forward by one day removes roughly a third of the strategy's apparent edge.
What this means in practice
Always log your own receipt timestamps independently of what a vendor's file claims, and build backtests off receipt time, not vendor-label time. When historical receipt logs do not exist (the common case for backfilled history), add a conservative, documented latency assumption rather than assuming same-day availability.
The most dangerous latency problems are invisible in the data itself — the vendor's timestamp looks perfectly clean and same-day. The only way to catch delayed delivery is to independently measure when files actually arrive, going forward, and apply a matching assumption to the backfilled history.
Related concepts
Practice in interviews
Further reading
- Ang, Asset Management, ch. 12 on data timing