Boolean Masking and Fancy Indexing
Selecting array elements with a condition instead of positions — how NumPy lets you write "all returns above 2%" as a single expression instead of a loop.
Prerequisites: NumPy Broadcasting
You have a year of daily returns for a stock and want every day the stock moved more than 2%. The loop version — walk every element, check the condition, append to a list — works but is slow and verbose for large arrays. NumPy's boolean masking lets you write returns[returns > 0.02] and get exactly that subset back in one vectorized pass, with no explicit Python loop.
A boolean mask is an array of True/False the same shape as the data, usually built from a comparison like returns > 0.02. Indexing an array with that mask keeps only the positions where it's True. Fancy indexing does the same job with a list or array of explicit positions instead of a condition.
Two related tools
Boolean masking: returns > 0.02 doesn't compute a single true/false — applied to an array, comparisons broadcast elementwise, producing a same-shaped array of booleans. Indexing with that array, returns[mask], pulls out only the True positions, always as a flat 1D result regardless of the original shape.
Fancy indexing: instead of a condition, you supply an explicit list or array of positions: returns0, 5, 9 returns the 1st, 6th, and 10th elements. np.where(mask) converts a boolean mask into the integer positions where it's True, bridging the two approaches — useful when you need the locations of a condition, not just the values.
Both return copies, not views, because the selected elements generally aren't a regular stride pattern NumPy can describe as a reinterpretation of the original memory.
Worked example
returns = np.array([0.010, 0.030, -0.010, 0.040, 0.000])
mask = returns > 0.02 # [False, True, False, True, False]
big_moves = returns[mask] # [0.03, 0.04]
positions = np.where(mask)[0] # [1, 3]
returns[mask] = 0.02 # cap: set every big move to exactly 0.02
print(returns) # [0.01, 0.02, -0.01, 0.02, 0.00]
The last line shows the other direction: a mask isn't just for reading, returns[mask] = 0.02 writes back to every True position at once, capping outliers in a single vectorized statement — no explicit loop over indices needed either way.
What this means in practice
Masking and fancy indexing are the standard way to filter, cap, and flag data in a quant pipeline: pulling out days a strategy triggered, capping extreme returns before computing volatility, or selecting the subset of a universe passing a liquidity filter. Combining conditions uses &, |, ~ (not Python's and/or/not, which don't broadcast) with parentheses around each condition: returns[(returns > 0.02) & (volume > 1e6)].
Forgetting parentheses around each condition when combining masks — returns > 0.02 & volume > 1e6 instead of (returns > 0.02) & (volume > 1e6) — is a classic bug, because & binds tighter than > in Python, so the expression parses in an unintended order and either raises an error or silently computes the wrong mask.
Related concepts
Practice in interviews
Further reading
- NumPy documentation, 'Indexing on ndarrays'
- VanderPlas, Python Data Science Handbook (ch. 2)