Handling Irregular and Missing Timestamps

An ordinary recurrent network processes a sequence one step at a time and implicitly assumes every step represents the same amount of elapsed time — step $t$ to step $t{+}1$ is always "one tick," whatever that tick means. Real data rarely cooperates: a stock might trade every few seconds during the open and not at all overnight, a sensor might drop readings when a connection fails, an economic release might land on an unpredictable schedule. If you feed such data into a standard model as if the gaps between observations were all equal, the model has no way to tell a 2-second gap from a 16-hour one — it treats both as "one step," silently discarding exactly the information (how much time actually passed) that often matters most.

The analogy: reading a diary with no dates

Imagine reading someone's diary where every entry is written down but none of the dates survived — you only know entry 5 came after entry 4. If the entries were actually written daily, you'd be fine treating "entry to entry" as "day to day." But if the writer sometimes wrote three entries in one afternoon and then went silent for two months, reading the diary as if each entry were one uniform day apart would badly mislead you about how the writer's mood or circumstances evolved — you'd see a jump between two entries and have no way to know it spanned an afternoon or a season. The fix is not to guess — it's to keep the actual elapsed time as information the reader (or model) explicitly uses, not to discard it by treating "next entry" as "next day."

The mechanics: two families of fixes

Time-aware inputs. The simplest fix is to compute the elapsed time $\Delta t_i = t_i - t_{i-1}$ between each pair of real observations and feed it into the model as an extra input alongside the observed value, so the model can learn to treat a short gap differently from a long one:

$$h_i = f(h_{i-1}, x_i, \Delta t_i)$$

In words: the hidden state update now explicitly depends on how much time passed since the last observation, not just on the previous state and the new value — the model can learn, for example, to "decay" its memory of stale information faster across a long gap than a short one.

Continuous-time models. A more thorough fix treats the hidden state as evolving continuously between observations rather than only updating at observation instants — for example, a hidden state that follows an exponential decay toward some baseline between observations, then gets a fresh update when a new observation arrives:

$$h(t) = h_{i-1} \, e^{-\lambda (t - t_{i-1})}, \qquad t_{i-1} \le t < t_i$$

In words: between two real observations, the model's belief doesn't just sit frozen — it decays continuously toward a baseline at rate $\lambda$, so a query at any point in time, not just at observation instants, gets a well-defined, time-consistent answer, and a long gap naturally produces more decay than a short one.

Worked example 1: a $\Delta t$-aware update by hand

Suppose a hidden state has value $h=10$ (say, "confidence a signal is still active") right after an observation, decay rate $\lambda = 0.1$ per hour, and the next observation arrives 3 hours later. Naive equal-spacing treatment would carry $h=10$ forward unchanged into the next step. The decay-aware version computes $h(3) = 10 \, e^{-0.1 \times 3} = 10 \times 0.741 = 7.41$ — the model correctly recognizes that after 3 quiet hours, the earlier signal should carry less weight than it would after, say, 10 minutes ($h(0.167) = 10\, e^{-0.0167} = 9.83$, barely decayed). Equal-spacing treatment cannot distinguish these two very different situations.

Worked example 2: quantifying the distortion from ignoring gaps

Suppose a trading signal series has 100 observations, 90 of them 1 minute apart and 10 of them following a 6-hour overnight gap. If you feed all 100 into a standard RNN as uniform steps, the 10 overnight transitions — each representing 360x more elapsed time than a typical step — get exactly the same "one tick" treatment as every 1-minute transition. Those 10 transitions are only 10% of the steps but represent the vast majority of the actual elapsed time in the dataset (10 × 360 = 3,600 minutes versus 90 × 1 = 90 minutes, i.e. the overnight gaps are over 97% of total elapsed time yet get 10% of the model's "step budget"). A model trained this way is effectively blind to almost all the real time the series spans.

The decay curve above is the shape a $\Delta t$-aware hidden state follows between observations — steep early, flattening out — which is exactly why treating a long gap the same as a short one throws away a real, learnable pattern in how influence should fade.

What this means in practice

This matters anywhere observations are event-driven rather than clock-driven: trade prints, order book updates, credit events, sensor telemetry with dropped packets. Ignoring irregular timing is a common, quiet source of poor model performance that doesn't show up as an obvious bug — the model trains, the loss goes down, but it has learned a subtly wrong notion of time. It's closely related to Neural Hawkes Processes for Event Sequences, which model the timing of events itself as part of what's being predicted, not just a side input.

Practice

1. Given $h=20$, $\lambda=0.2$/hour, compute $h(t)$ after a 2-hour gap and after a 30-minute gap, and compare.
2. Explain why feeding raw timestamps directly (rather than $\Delta t$ between observations) is usually a worse choice of input.
3. In the 100-observation example, what fraction of total elapsed time do the 90 one-minute-apart observations represent?