Qm
Core

Boolean Masking and Fancy Indexing

Selecting array elements with a condition instead of positions, how NumPy lets you write "all returns above 2%" as a single expression instead of a loop.

Prerequisites: NumPy Broadcasting

You have a year of daily returns for a stock and want every day the stock moved more than 2%. The loop version, walk every element, check the condition, append to a list, works but is slow and verbose for large arrays. NumPy's boolean masking lets you write returns[returns > 0.02] and get exactly that subset back in one vectorized pass, with no explicit Python loop.

A boolean mask is an array of True/False the same shape as the data, usually built from a comparison like returns > 0.02. Indexing an array with that mask keeps only the positions where it's True. Fancy indexing does the same job with a list or array of explicit positions instead of a condition.

Two related tools

Boolean masking: returns > 0.02 doesn't compute a single true/false, applied to an array, comparisons broadcast elementwise, producing a same-shaped array of booleans. Indexing with that array, returns[mask], pulls out only the True positions, always as a flat 1D result regardless of the original shape.

Fancy indexing: instead of a condition, you supply an explicit list or array of positions: returns0, 5, 9 returns the 1st, 6th, and 10th elements. np.where(mask) converts a boolean mask into the integer positions where it's True, bridging the two approaches, useful when you need the locations of a condition, not just the values.

Both return copies, not views, because the selected elements generally aren't a regular stride pattern NumPy can describe as a reinterpretation of the original memory.

returns: 1%3%-1%4%0% mask >2%: FTFTF result: 3% 4% only the True positions survive, flattened into a new array
The mask marks positions; indexing with it collapses the array down to just the matching values.

Worked example

returns = np.array([0.010, 0.030, -0.010, 0.040, 0.000])
mask = returns > 0.02                       # [False, True, False, True, False]
big_moves = returns[mask]                    # [0.03, 0.04]
positions = np.where(mask)[0]                # [1, 3]
returns[mask] = 0.02                         # cap: set every big move to exactly 0.02
print(returns)                               # [0.01, 0.02, -0.01, 0.02, 0.00]

The last line shows the other direction: a mask isn't just for reading, returns[mask] = 0.02 writes back to every True position at once, capping outliers in a single vectorized statement, no explicit loop over indices needed either way.

What this means in practice

Masking and fancy indexing are the standard way to filter, cap, and flag data in a quant pipeline: pulling out days a strategy triggered, capping extreme returns before computing volatility, or selecting the subset of a universe passing a liquidity filter. Combining conditions uses &, |, ~ (not Python's and/or/not, which don't broadcast) with parentheses around each condition: returns[(returns > 0.02) & (volume > 1e6)].

Forgetting parentheses around each condition when combining masks, returns > 0.02 & volume > 1e6 instead of (returns > 0.02) & (volume > 1e6), is a classic bug, because & binds tighter than > in Python, so the expression parses in an unintended order and either raises an error or silently computes the wrong mask.

Discussion

Sign in to join the discussion · reading is open to everyone

💡 Discussion rules

  1. Ask and answer about this concept. Off-topic gets removed.
  2. No homework dumps. Show what you tried first.
  3. Corrections are welcome. Cite a source when you claim an error.

Loading discussion…

Related concepts

Practice in interviews

Further reading

  • NumPy documentation, 'Indexing on ndarrays'
  • VanderPlas, Python Data Science Handbook (ch. 2)
ShareTwitterLinkedIn