Context Windows and Long Documents

A language model's context window is the maximum number of tokens — roughly, word pieces — it can read and reason over in a single pass. Everything the model is asked to consider, whether that's a prompt, a document, or prior conversation, has to fit inside that window at once, and anything beyond it simply isn't seen.

This matters directly for financial text analysis. A single 10-K annual report can run 50,000 to 100,000+ words, and a portfolio of filings across many companies is far larger still. Even a model with a context window of 128,000 tokens can struggle to hold an entire filing plus a prior year's filing plus the analyst's question all at once, especially once formatting overhead and instructions are included.

Worked example. An analyst wants a model to compare the risk-factors section of this year's 10-K against last year's. If both sections plus the surrounding prompt exceed the context window, a common fix is retrieval: only the paragraphs most relevant to "risk factors" are pulled from each filing and fed in, rather than the whole document. This works, but it depends entirely on the retrieval step finding the right paragraphs — a new risk buried in an unexpected section can be missed even though the model itself never had a chance to see it.

Larger context windows reduce how often this chunking is necessary, but they don't eliminate the tradeoff: longer inputs are slower and costlier to process, and models still tend to pay less attention to information buried in the middle of a very long context than to text near the start or end.