Vector Databases and Embedding Search

A keyword search for "supply chain disruption" over a year of earnings-call transcripts will miss a call that only says "our vendors are struggling to keep pace with orders" — same meaning, no shared words. Retrieving by meaning rather than exact text match requires representing documents as points in a numeric space where nearby points mean similar things, and a way to search that space fast even across millions of documents. That's what embeddings plus a vector database provide.

The analogy: a library organized by meaning, not alphabet

A normal library shelves books alphabetically by title — finding "similar" books requires already knowing what you're looking for. Imagine instead a library where books sit near other books on closely related topics regardless of title, so walking up to any book on recessions naturally puts you near other books on the same subject. Embeddings do this for text: documents are placed in a high-dimensional space where distance corresponds to similarity of meaning, learned by a model rather than assigned by hand.

The pipeline, piece by piece

1. Embed each document. A neural embedding model converts each chunk of text into a fixed-length vector, typically hundreds of dimensions, trained so that texts with similar meaning map to nearby vectors.
2. Index the vectors. A vector database stores millions of these vectors in a structure — commonly an approximate nearest-neighbor index like HNSW — built specifically so that finding the closest vectors to a query doesn't require comparing against every single stored vector one by one.
3. Embed the query the same way. A user's question is passed through the same embedding model to get a query vector in the same space as the stored documents.
4. Retrieve by similarity. The database returns the $k$ stored vectors closest to the query vector, typically measured by cosine similarity, ranked from most to least similar.

In plain English: the search never looks at the words in the query and the words in the documents directly — it only compares where their respective vectors landed in the learned meaning-space, which is exactly why it can match a query and a document that share no vocabulary at all.

Worked example: retrieving a semantically related transcript passage

Query: "companies worried about component shortages." Suppose after embedding, cosine similarity to three candidate transcript passages comes out as: Passage A ("we're seeing vendor delays on key parts") $= 0.86$; Passage B ("Q3 revenue grew 4% year over year") $= 0.11$; Passage C ("our supplier lead times have stretched considerably") $= 0.91$. Ranked by similarity, Passage C and Passage A are retrieved as the top matches even though neither shares a single content word with the query — "component shortages" versus "supplier lead times" — because the embedding model placed them near each other in meaning-space. Passage B, on an unrelated topic, correctly scores far lower.

Worked example: sizing an approximate search

A firm has 50 million embedded filing paragraphs, each a 768-dimensional vector. A brute-force search comparing a query against all 50 million vectors for every question would take seconds to minutes — too slow for an interactive tool. An approximate nearest-neighbor index restructures the vectors into a graph or tree so that a search only examines a small fraction of the 50 million vectors — often under 1% — while still returning the true nearest neighbors with high probability, trading a small amount of exactness for a search that completes in milliseconds.

What this means in practice

Vector databases are the retrieval half of retrieval-augmented generation (RAG) pipelines, letting an LLM-based tool pull the most relevant passages out of a much larger document set before generating an answer, rather than fitting an entire archive into the model's context window. Retrieval quality depends entirely on the embedding model: one trained on general web text may not place financial jargon near its true synonyms as reliably as one fine-tuned on financial documents, and a poor embedding model quietly returns mediocre matches with no error message to flag it.