Document Embeddings as Alpha Features

Bag-of-words counts and keyword lists work fine until two articles say the exact same thing in different words: "the company slashed its outlook" and "management cut full-year guidance" share almost no words in common, yet describe an identical event. A feature built from raw word counts sees these as unrelated documents. What a strategy actually wants is a representation of meaning that treats paraphrases as similar and unrelated stories as different, regardless of vocabulary.

A document embedding is a fixed-length vector — typically a few hundred numbers — produced by a language model such that documents with similar meaning end up close together in that vector space, and documents about unrelated topics end up far apart. Instead of hand-picking keywords, a quant treats the embedding itself, or distances computed from it, as a feature.

From text to a usable feature

Two ways embeddings turn into alpha features in practice:

• Direct feature vectors. The embedding's coordinates (often reduced with PCA to a manageable handful of dimensions) go straight into a downstream model alongside price and fundamental features, letting a tree-based or linear model learn which directions in "meaning space" correlate with returns.
• Similarity-based features. More commonly, a desk computes the embedding's distance to a reference set — e.g., cosine similarity to a labeled cluster of historically bearish guidance-cut articles — and uses that similarity score as an interpretable feature: "how much does today's story resemble the pattern of stories that historically preceded a drawdown."

The second approach is popular precisely because a raw 384-dimensional vector isn't interpretable on its own, while "this story is 0.89 similar to our labeled bad-news cluster" is.

Worked example

A desk embeds all news stories about a stock using a pretrained sentence encoder, and maintains a small reference set of 40 historically-labeled "guidance cut" articles, averaged into one centroid vector. For a new incoming story, the pipeline computes the cosine similarity between the story's embedding and that centroid. A story reading "XYZ Corp lowers full-year revenue outlook citing weak demand" produces a similarity of $0.86$ to the centroid — very close, despite sharing few exact words with the labeled examples, which mostly used phrases like "cuts guidance" and "reduces forecast." A story about a routine product launch produces a similarity of $0.11$ — far from the centroid, correctly flagged as unrelated to the guidance-cut pattern. Backtesting shows that a similarity above roughly $0.6$ historically preceded an average next-day abnormal return of $-1.8\%$, giving the desk a continuous, tunable signal built entirely from meaning rather than exact keyword matches.

What this means in practice

Embedding features let a desk build signals from concepts too diffuse to define with a keyword list — "management sounds defensive," "this reads like past supply-chain disruption stories" — without hand-engineering rules for every phrasing. The tradeoff is that embeddings drift as language models are updated or fine-tuned (see Embedding Drift and Model Refresh), so a similarity threshold calibrated on one model version can silently stop meaning the same thing after an upgrade, and needs to be re-validated rather than assumed stable.