Quant Memo
Core

Chained Assignment and Copy-on-Write

The classic pandas trap of writing df[df.x > 0]["y"] = 1 and having it silently fail to update the original DataFrame, plus how pandas's newer copy-on-write mode changes and mostly fixes this behavior.

Prerequisites: Pandas DataFrame Internals, Index Alignment in Pandas

A line like df[df.pnl < 0]["flag"] = "loss" looks like it should set a column on every losing row, and it often runs without any error — yet the original df is left completely unchanged. This is chained assignment: the first bracket, df[df.pnl < 0], may return either a view into the original data or a brand-new copy, depending on internal details pandas doesn't guarantee to you, and the second bracket assignment then modifies whichever one it happened to get. When it gets a copy, your update vanishes into a temporary object that's immediately discarded, with no error raised — historically only a SettingWithCopyWarning, easy to miss in a noisy log.

The fix has always been to do the row and column selection in a single .loc call instead of two chained bracket operations: df.loc[df.pnl < 0, "flag"] = "loss" unambiguously operates on the original DataFrame in one step, with no intermediate object whose view-or-copy status is ambiguous.

Recent pandas versions introduce copy-on-write (default from pandas 3.0 onward), which changes the underlying mechanics: every selection now behaves as if it returned an independent copy, so chained assignment fails loudly and consistently — the assignment simply has no effect on the original, with a clear warning — rather than sometimes silently working and sometimes silently failing depending on internal memory layout. Copy-on-write removes the ambiguity, but .loc in one step is still the correct and unambiguous way to write the update.

Chained assignment (df[mask]["col"] = value) may silently fail to modify the original DataFrame because the first bracket can return a copy rather than a view; use a single .loc[mask, "col"] = value call instead, which is unambiguous under both classic and copy-on-write pandas.

Related concepts

Practice in interviews

Further reading

  • pandas documentation, 'Copy-on-Write'
ShareTwitterLinkedIn