Notebooks vs Scripts in Research

Notebooks (Jupyter and similar) let a researcher run one cell at a time, look at a plot, tweak a parameter, and re-run — which is exactly the loop of early-stage idea exploration. Plain Python scripts run top to bottom every time, with no persistent state between runs. Both have a real place in a research workflow, but using the wrong one for the wrong stage is a common source of quiet bugs.

The specific failure mode is hidden state: a notebook lets you delete or reorder cells, but variables from cells you've since deleted or changed can still linger in memory, so a notebook that "runs fine" interactively can fail — or worse, silently produce a different result — when re-run fresh from top to bottom. A backtest built this way can look profitable in the notebook session that created it and be unreproducible (or wrong) the next day.

The standard discipline is to treat notebooks as disposable scratch space for exploration only, and to promote any logic that's going to be reused — a signal calculation, a backtest engine, a data loader — into a script or module as soon as it's stable, then call that module from the notebook rather than duplicating the logic inline. Before trusting a notebook's results, restart the kernel and run all cells top to bottom; if the output changes, the notebook had hidden state.