Embedding Drift and Model Refresh

Embedding vectors from one version of a model are not compatible with embedding vectors from a different version or a different model entirely, even if both are meant to capture "meaning," the two vector spaces have unrelated geometries, so a distance that meant "similar" under the old model is meaningless under the new one. This is embedding drift: whenever a team upgrades its embedding model to a newer, better one, every document previously embedded with the old model becomes stale and has to be re-embedded from scratch, not merely refreshed.

This is easy to overlook because the upgrade itself often looks harmless, same input text, same interface, just a newer model version, but skipping the re-embedding step silently corrupts anything downstream that compares vectors, like a search index or a clustering pipeline, because it ends up comparing old-model vectors against new-model vectors as if they lived in the same space.

Worked example

A firm's news-search index has 5 million articles embedded with model v1. It upgrades to model v2, which scores better on internal benchmarks, and starts embedding new incoming articles with v2 while leaving the 5 million old vectors untouched to save recomputation cost. Searches now silently degrade: a v2 query vector compared against a v1 document vector returns a similarity score that reflects nothing meaningful, so old articles stop surfacing in search results even when they're the most relevant match, and nothing in the system raises an error to say why.