Decision Time Versus Data Timestamp
Every row of market data carries a time, but it is usually the time the event happened, not the time you could have known about it. Indexing a backtest by the wrong one is the single most productive way to invent an edge that does not exist.
Prerequisites: Look-Ahead Bias, Point-in-Time Data
A dataframe row has one time column, so it feels like there is one time. There are at least three, and they can be months apart:
- Event time — when the thing happened. The quarter ended on 31 March.
- Arrival time — when the information reached you. The earnings were released on 12 May at 16:05, and the vendor's row appeared in your database on 13 May at 02:00.
- Decision time — when you are allowed to act on it. Some time strictly after arrival, plus however long your pipeline takes.
The rule the whole page reduces to: a backtest must be indexed by arrival time, and may only trade at a decision time after it. Vendors overwhelmingly index by event time, because that is what an analyst wants to look at. If you join their file to a price series on its native date column, you have built a time machine and you will not notice, because nothing errors.
Worked example: a fundamental signal that was never tradeable
A cross-sectional value strategy on 1,500 US stocks. Rank on earnings yield, long the cheapest quintile, short the most expensive, rebalance quarterly. The fundamentals file has a period_end column, so the join uses it: on 1 April, use the quarter ending 31 March.
Backtest result: 19.4% a year, volatility 11%, Sharpe 1.76, positive in 17 of 20 years. It looks like a career.
Now re-run with the vendor's report_date field instead, and add the two-day settling lag a real pipeline needs. The median US filer reports about 42 days after quarter end; the slow tail runs past 70. So the 31 March number becomes usable around 14 May, not 1 April.
Same rule, same universe, same 20 years: 3.1% a year, volatility 11%, Sharpe 0.28. Net of 30 bps round-trip costs at 160% turnover it is roughly 2.6%, indistinguishable from cash.
Where did 16 points of annual return go? Nowhere — it was never there. Prices move on the earnings release. Knowing the number six weeks before the market did means you were front-running the announcement in every one of the 80 quarters in the sample. The "value edge" was the announcement drift, harvested with a time machine.
Index every dataset by when you could have known it, never by when it happened. If a column is called date, period, as_of, or report_period, assume it is event time until you have proven otherwise — and a fundamental data point is not knowable until roughly 45 days after the quarter it describes.
Worked example: one minute of the future, at high frequency
The same bug in miniature, where it is even easier to miss. You have 1-minute bars. Your feed labels each bar with the time it started: the bar tagged 09:31:00 covers 09:31:00 to 09:32:00. Your research code assumes the label is the end of the bar, which is the other common convention.
The strategy trades a short-horizon reversal, entering at what the code believes is the bar close and exiting one bar later. Backtest: average +3.9 bps per trade, roughly 26,000 trades a year, hit rate 58%.
Correct the convention — the label is the bar's open, so a decision made "at" 09:31:00 is really made 60 seconds before that data exists — and re-run: +0.6 bps per trade, hit rate 50.4%. The spread on these names is about 1.4 bps, so half a bp of gross edge is a loss before you have paid anyone.
The arithmetic is worth sitting with. A US trading year holds about 5.9 million seconds, so a stock with 25% annualised volatility moves roughly — about 8 bps — in a typical minute. Perfectly knowing the sign of that move would be worth around 6 bps a trade. A "discovered" edge of 3.9 bps is therefore not a small effect that got lucky; it is most of one minute of foresight, which is exactly what the off-by-one label handed you. Whenever a one-bar shift swings the result by several multiples, the timing is the result.
Three secondary traps hide behind the same column. Restatements: the value in the file today may be a corrected figure published years later, so even the right timestamp can carry the wrong number. Time zones: an exchange feed in local time joined to a news feed in UTC can hand you a five-and-a-half hour head start. Snapshot versus tick: a "daily close" from a vendor is often a consolidated print finalised well after 16:00, not something you could have traded on at 16:00:00.
Two checks that cost minutes. First, plot the distribution of arrival_time − event_time for every dataset you use; if it is a spike at zero, the vendor has not given you arrival times at all. Second, add one extra period of lag everywhere and re-run: a real edge decays gently, a timestamp bug falls off a cliff. See Signal-to-Execution Lag.
In practice
Store both times and index on arrival time, keeping event time as an ordinary column for analysis. Build the decision timestamp explicitly — arrival plus feed latency plus compute time plus the wait for the next tradeable moment — and make the simulator refuse any order stamped earlier. An Event-Driven Backtest Architecture enforces this for free, which is most of the argument for using one. When an interviewer asks how you would validate a fundamentals-driven backtest, your first sentence should be about report dates, not Sharpe.
Related concepts
Practice in interviews
Further reading
- López de Prado, Advances in Financial Machine Learning (Ch. 2, 4)
- Harvey & Liu, Backtesting
- Compustat / I/B/E/S point-in-time documentation