Revision Analysis: How Much Does History Change?

GDP for last quarter gets published as an estimate, then revised a month later, then revised again a year later as more complete source data comes in. Company revenue gets restated after an accounting review. A "final" number in today's database file may be the third or fourth version of a figure that looked quite different on the day it first appeared. Revision analysis measures how much history typically changes after its first release, so you know whether the version of the data you are backtesting on ever actually existed in real time.

Measuring the size of a revision

For any given data point, compare its first-released value against its value as of today (or as of some later, "settled" date):

$$\Delta = x_{\text{final}} - x_{\text{first release}}$$

In words: the revision is simply how far the number moved between the earliest version anyone could have traded on and the version sitting in the database now. Tracking the distribution of $\Delta$ across many data points — not just its average, which can hide large individual swings that cancel out — tells you how much a strategy relying on the "final" number is overstating what was knowable in real time.

Worked example

A researcher pulls quarterly revenue for 400 companies, comparing the figure as originally reported in each quarter's press release against the figure sitting in a standard fundamentals database today. For 92% of company-quarters the difference is under 0.5% — rounding and minor reclassification. For 3% of company-quarters the difference exceeds 5%, driven by restatements after accounting reviews or M&A-related reclassifications. A signal built on "surprise versus consensus" using the database's current, revised revenue figure would, for that 3% of cases, be comparing analysts' real-time forecasts against a number that did not exist until long after the trade would have been made.

What this means in practice

Where the vendor supports it, always request point-in-time or "as originally reported" fields rather than the current, fully-revised ones. Where it does not, run this revision analysis on a sample to estimate how big the problem is before deciding whether the dataset is usable for backtesting at all.