Text Classification for Filings

A large-cap company files a 10-K every year, and most of its risk-factor section is boilerplate copied nearly verbatim from the prior year. Buried in there might be one new paragraph describing a genuinely new risk — a supply concentration, a fresh lawsuit, a covenant near breach. Reading every filing end to end to find that one paragraph doesn't scale across thousands of companies. Text classification automates the sorting: train a model on labeled examples of "new/material risk language" versus "routine boilerplate," and let it triage new filings.

The analogy: an inbox filter for filings

An email spam filter doesn't read every email with equal attention — it learns from thousands of examples labeled "spam" or "not spam" what distinguishes the two, then scores new emails automatically. Text classification for filings works the same way: instead of "spam" versus "not spam," the categories might be "material change" versus "boilerplate," or "positive tone" versus "negative tone," and the model learns the distinguishing features from a labeled training set of past filings rather than a human reading every single one going forward.

The pipeline, piece by piece

1. Represent the text as features. Turn each document or paragraph into a numeric vector — TF-IDF weights over a vocabulary, or a Loughran-McDonald tone score, are common starting points.
2. Label a training set. A human (or a proxy like "did the stock move more than 3% around this filing") labels a sample of documents with the target category.
3. Fit a classifier. Logistic regression or a gradient-boosted tree learns a decision rule mapping the feature vector to a predicted category, typically outputting a probability like $\Pr(\text{material change} \mid \text{text})$.
4. Evaluate and threshold. Held-out filings check whether the classifier generalizes, and a probability threshold is chosen based on how much false-positive review time is acceptable versus how many true risks can be missed.

In plain English: the model doesn't understand English the way a person does — it has learned that certain word patterns co-occurred with the target label in the past, and it's betting that the same pattern will hold on new text.

Worked example: classifying a risk-factor paragraph

Suppose a logistic regression trained on TF-IDF features assigns weight $+2.1$ to the feature "single supplier," $+1.8$ to "covenant breach," and $-0.4$ to "no material changes." A new paragraph contains "single supplier" once and no other flagged features, giving a linear score of roughly $2.1 - 3.0$ (intercept) $= -0.9$. Passed through the logistic function, $\Pr(\text{material}) = \frac{1}{1+e^{0.9}} \approx 0.29$ — below a typical 0.5 threshold, so this paragraph is classified as routine, even though it does mention a real risk factor, because the single weak signal wasn't strong enough on its own.

Worked example: choosing a threshold under an analyst's time budget

An analyst can only review 50 flagged paragraphs a day out of 2,000 filed. If the classifier's default 0.5 threshold flags 300 paragraphs, raising the threshold to 0.85 might cut that to 45 paragraphs a day — fitting the review budget — but at the cost of missing some true risks that scored between 0.5 and 0.85. This tradeoff, between how many paragraphs get reviewed and how many genuine risks are missed, is a direct consequence of where the threshold is set, not something a single "best" model output resolves automatically.

What this means in practice

Text classification lets a small analyst team triage a large volume of filings by directing attention to the fraction a model flags as most likely to matter, rather than reading everything or reading nothing. Its usefulness depends entirely on the quality and relevance of the labels used to train it — a classifier trained to predict "did the stock move" learns something different from one trained on human-annotated "is this genuinely new language," and conflating the two is a common source of disappointing live performance.