Mapping a Vendor Dataset to Your Universe

A new alternative dataset arrives with company names, or a vendor-specific ID, or a ticker column that looks close enough to what a trading system uses — and it's tempting to just join it onto the existing universe and start researching. This step, done carelessly, is one of the most common sources of silent bugs in quant research, because a bad join doesn't crash, it just quietly loses or misassigns data.

The idea

Every dataset identifies companies somehow, and vendors rarely use the same identifier a firm's own systems use internally. Tickers change when companies rename or get delisted and get recycled for entirely different companies later. Company names have inconsistent formatting across vendors — "Alphabet Inc." versus "Alphabet Inc" versus "GOOGL" versus "Google". CUSIPs and other standard identifiers change when a company reorganizes its share structure, even though the underlying business hasn't changed at all. A naive join on any single one of these fields will silently drop rows that don't match exactly, and will occasionally match the wrong two rows together — a dropped row shows up as a coverage gap you might notice, but a wrong match shows up as data that looks completely normal and is simply attached to the wrong company.

The fix is to use a stable, permanent identifier wherever one exists — a security master maintained internally or through a vendor built for this purpose — and to treat the mapping itself as a first-class research artifact, not a throwaway join. That means checking match rates explicitly (what fraction of the vendor's names or IDs successfully mapped to something in the universe?), spot-checking a sample of matches by hand, and specifically looking for cases where a match rate looks suspiciously perfect, since a 100% match rate on messy real-world identifiers is itself often a sign that the join is too permissive and silently pairing near-misses.

A concrete example

A researcher maps a vendor's alternative-data feed, identified by company name, onto an equity universe identified by ticker. A naive fuzzy-string match reports 94% coverage, which sounds reassuring. On closer inspection, several rows for "Meta Platforms" mapped to an old delisted ticker still carrying the name "Facebook" in the trading system's static reference table, silently splitting what should have been one company's signal across two dead identifiers — invisible in aggregate statistics, but enough to understate the real signal by roughly half for that name. The fix required building a proper crosswalk using a permanent identifier and a maintained corporate-actions history, rather than matching on names or tickers directly.

What this means in practice

Treat the identifier mapping as a deliverable with its own QA step, separate from the signal research that depends on it — report the match rate, investigate unmatched names rather than silently dropping them, and be specifically suspicious of matches that look too clean. A signal built on top of a bad mapping can look statistically fine in a backtest while actually measuring something subtly different from what the researcher believes it's measuring.