MultiIndex and Panel Data

Quant data is naturally two-dimensional in a way a single row-index doesn't capture: you have both a date and a ticker for every observation, and you regularly want to slice by either one — "give me all tickers on this date" or "give me this ticker across all dates." A MultiIndex is pandas's way of stacking multiple key columns into a single hierarchical row (or column) index, so both kinds of slicing stay fast and expressive instead of requiring a manual filter-and-loop every time.

A typical panel is built by setting two columns as the index: df.set_index(["date", "ticker"]) produces rows labeled by the pair (date, ticker). From there, df.loc["2024-01-05"] pulls every ticker on that date, df.loc[(slice(None), "AAPL"), :] pulls every date for one ticker, and df.unstack("ticker") pivots the ticker level out into columns, turning a long panel into a wide date-by-ticker matrix — exactly the shape needed for a covariance matrix or a cross-sectional regression. df.stack() reverses that, folding the wide matrix back into the long panel form.

The main trap is that MultiIndex operations are markedly slower than they look if the index isn't sorted; df.sort_index() after building a MultiIndex, before doing repeated .loc lookups, is close to mandatory in practice, since an unsorted MultiIndex forces pandas to fall back to a full scan instead of a binary search on each level.