Missing Data and Forward Fill

Real market data has holes. A stock gets halted for an hour, an exchange holiday skips a day entirely, a vendor's feed drops a field, or two assets that should line up on the same calendar simply don't — a US stock trades Monday, its European ADR doesn't. Every one of these shows up in a dataframe as NaN, and what you do with that NaN is a modeling decision, not a formatting one.

The most common fix is forward fill: carry the last known value forward until a new one arrives. It is popular because it is the closest thing to "reasonable" — if you don't know today's price, yesterday's is your best guess. But forward fill is also the easiest way to accidentally tell your backtest something it couldn't have known.

Filling strategies compared

Method	What it does	When it's right
Forward fill (ffill)	Carries the last known value forward	Prices, when nothing traded — the last trade is the current price until proven otherwise
Backward fill (bfill)	Carries the next known value backward	Rarely correct for live use — it looks into the future
Linear interpolation	Draws a straight line between the two nearest known points	Smooth physical quantities (e.g., temperature), not prices, which don't interpolate meaningfully
Drop	Removes the row entirely	When a feature is required and there's no honest way to guess it
Fill with a constant (e.g., 0)	Replaces gaps with a fixed value	Only when 0 is a genuinely meaningful default, like missing volume meaning no trades

Worked example

A stock's daily closing prices: Mon 50.00, Tue NaN (halted), Wed NaN (halted), Thu 52.00, Fri 51.50.

Forward filling gives: Mon 50.00, Tue 50.00, Wed 50.00, Thu 52.00, Fri 51.50. Tuesday and Wednesday both show the same 50.00 — reasonable, since the stock genuinely did not trade and 50.00 was the last real price. Compute a daily return series on the filled data and Tuesday's and Wednesday's returns come out as exactly 0%, which is correct: nothing happened, so nothing should be reported as having happened.

Now contrast with linear interpolation, which would give Tuesday 50.67 and Wednesday 51.33 — smoothly walking the price from 50.00 to 52.00 across the halt. Those numbers were never real prices anyone could have traded at; a backtest that marks a position to that interpolated value on Tuesday is pricing off data that didn't exist yet, since the 52.00 endpoint wasn't known until Thursday.

What this means in practice

The test for any fill method is: could a system running live, at that exact timestamp, have produced this value using only data available up to that moment? Forward fill passes that test for prices. Interpolation fails it, because computing the value at Tuesday requires already knowing Thursday's close. The same logic applies to fundamentals data — filling a missing quarterly earnings figure with the next reported quarter's number is backward fill wearing a disguise, and it is one of the most common sources of look-ahead bias in factor research.