Embedding-Based Topic Modelling

Classical topic models like LDA (see Topic Modeling with LDA) discover themes by looking at which words tend to co-occur across documents — but that approach struggles badly on the kind of text a finance desk actually deals with. A tweet, a headline, or a two-sentence analyst note is too short to give a co-occurrence-based model enough signal, and two documents using entirely different vocabulary to describe the same event ("chip shortage" vs. "semiconductor supply constraints") get treated as unrelated, splitting what should be one coherent topic into two weak, noisy ones.

Embedding-based topic modelling sidesteps the word-counting problem entirely: it first converts every document into a dense embedding vector (see Document Embeddings as Alpha Features) that captures meaning rather than exact wording, then discovers topics by clustering those vectors — documents that land near each other in embedding space, regardless of their surface vocabulary, are grouped into the same topic.

The pipeline

The typical recipe (popularized by methods like BERTopic) has three stages:

1. Embed every document with a sentence-level encoder.
2. Cluster the embeddings — commonly after reducing dimensionality first — using a density-based algorithm that can leave genuinely unclustered documents as noise rather than forcing every document into some topic.
3. Label each resulting cluster by extracting its most distinctive words relative to the rest of the corpus, giving a human-readable name to what would otherwise just be a group of nearby vectors.

Because clustering happens on meaning rather than word overlap, a topic can absorb documents phrased in completely different ways, and short, informal text (a headline, a tweet) works about as well as a full article — a major practical advantage over classical bag-of-words topic models, which need long, vocabulary-rich documents to find reliable co-occurrence patterns.

Worked example

A desk applies embedding-based topic modelling to six months of short-form financial social-media posts, where classical LDA had previously produced unusable results (the posts were too short for meaningful word co-occurrence counts). The embedding-based pipeline successfully groups "chip shortage crushing margins," "semiconductor supply still constrained," and "fab capacity remains the bottleneck" into a single coherent cluster, despite the three posts sharing almost no words in common beyond "the" and "and." Labelling the cluster by its most distinctive terms across all its members surfaces "semiconductor," "capacity," "shortage," and "fab" — a clean, interpretable topic name. A parallel run of classical LDA on the same short-post dataset, by contrast, split superficially similar posts into three separate weak topics purely because of vocabulary mismatch, none of which reached a coherence score high enough to be usable as a feature.

What this means in practice

Embedding-based topic modelling is the practical choice whenever a desk's text sources are short, informal, or highly varied in phrasing — social media, chat transcripts, headline feeds — where classical co-occurrence topic models simply don't have enough signal per document to work well. The tradeoff is compute and dependency on the quality of the underlying embedding model: a poorly-suited or outdated encoder (see Embedding Drift and Model Refresh) will produce a clustering that looks coherent on the surface but doesn't track the distinctions that actually matter for markets.