Group Sequential Designs and Alpha Spending

A quant team backtests a new signal and plans to run it live for six months, checking performance at the end of each month before deciding whether to kill it early or let it run. If they use the ordinary 5% significance threshold at every one of those six checks, the chance that at least one of the six checks falsely flags the signal as working, purely by luck, is much higher than 5% — closer to 20% once you account for six independent-ish looks. A group sequential design fixes this by planning the checks in advance and spending a shrinking piece of the total 5% "false alarm budget" at each one, so that no matter how many times you look, the overall chance of a false alarm across the whole plan stays at 5%.

An analogy: a household budget for suspicion

Think of the 5% significance level as a fixed budget of "how much doubt you're allowed to spend being wrong," for the entire monitoring plan, not per look. If you had one look, you'd spend the whole 5% on it. With six planned looks, you instead allocate a little sliver of that budget to each one — say, hardly any at the first look (it's early, unlikely to be conclusive, don't risk much doubt on it) and progressively more at later looks, as more evidence has accumulated and a false alarm becomes less likely to occur by chance alone. As long as the total spent across every planned look sums to no more than 5%, the overall false-alarm rate for the whole multi-look plan is controlled, just like a household budget stays balanced as long as sub-budgets across the month sum to the total.

The mechanics, one symbol at a time

Let $\alpha$ be the overall false-positive budget for the whole study (typically 0.05), and suppose there are $K$ planned interim looks. An alpha spending function $\alpha(t)$, where $t$ ranges from 0 to 1 and represents the fraction of total planned information (roughly, fraction of data) collected so far, specifies how much of the budget has been spent by each point: $\alpha(0) = 0$, $\alpha(1) = \alpha$, and $\alpha(t)$ increases along the way. A common choice is the O'Brien-Fleming-style function, which spends almost nothing early and most of the budget near the end:

$$\alpha(t) = 2 \left[ 1 - \Phi\!\left( \frac{z_{\alpha/2}}{\sqrt{t}} \right) \right],$$

where $\Phi$ is the standard normal cumulative distribution function and $z_{\alpha/2}$ is the usual two-sided critical value. In plain English: this formula converts "how far through the planned data collection you are" into "how much of your 5% doubt-budget you're allowed to have spent by now," and it's built so early interim looks get a very strict, hard-to-cross bar (protecting against a lucky early fluke) while the final look ends up close to the ordinary single-look threshold.

Worked example 1: a two-look plan, spending the budget

Suppose a strategy is monitored at the halfway point ($t = 0.5$) and at the end ($t = 1$) of a planned six-month test, with total budget $\alpha = 0.05$. Using an O'Brien-Fleming-style schedule, a typical allocation spends about $0.0031 of the budget at the halfway look and the remaining $0.0469 at the final look — the boundary at the halfway point corresponds to needing a z-score above roughly 2.96 (much stricter than the usual 1.96), while the final look's boundary sits close to 2.0. If the interim monthly Sharpe ratio z-score comes in at 2.4 at month three, that's not enough to stop early and declare success, because 2.4 falls short of the 2.96 bar reserved for that early look — even though 2.4 would easily clear the ordinary single-test bar of 1.96.

Worked example 2: what happens without the correction

Now compare an uncorrected version of the same plan: checking at month 3 and month 6 with the ordinary $z > 1.96$ threshold applied both times, with no budget adjustment. Each look independently has roughly a 2.5% chance of a false positive on one side, and across two mostly-independent looks the combined chance of at least one false alarm is approximately

$$1 - (1 - 0.025)^2 = 1 - 0.9506 = 0.0494 \approx 4.9\% \text{ one-sided, or about } 9\text{–}10\% \text{ two-sided across the pair.}$$

That's roughly double the nominal 5%, and it only gets worse with more looks — six uncorrected monthly checks push the true false-alarm rate toward 20%. Alpha spending exists precisely to claw that inflation back down to the advertised 5%, by making early boundaries strict enough to offset the extra chances taken.

What this means in practice

Group sequential designs with alpha spending are the standard tool for pre-registered, staged decisions: a scheduled quarterly review of a live strategy's performance with a built-in "kill it early" option, or a research team's plan to check an experiment at fixed calendar milestones rather than continuously. Unlike always-valid e-value monitoring (see Always-Valid Inference and e-Values), which allows looking at literally any moment, group sequential designs suit settings where the checks happen at a small, known number of scheduled reviews — a board meeting, a monthly risk committee — and the spending function is chosen once, before any data is seen.