Explaining a Trading Model's Decisions

A model flags a trade a portfolio manager doesn't understand, or a risk committee asks why exposure jumped, or a regulator asks why an order was routed the way it was. "The model said so" satisfies no one. Explaining a trading model's decisions means picking the right explanation for the right audience and question — because "why did the model do this?" splits into several genuinely different questions that call for different tools.

Matching the question to the method

Three questions come up constantly, each wanting a different kind of answer. "Why this trade, right now?" is a local, single-prediction question, best answered by a per-instance attribution method like SHAP or TreeSHAP, which decomposes one prediction into each feature's contribution for that specific input. "What does the model generally do?" is a global question, best answered by a surrogate model or partial-dependence-style summaries that describe average behavior across many inputs. "Would the model have done something different if X had been slightly different?" is a counterfactual question, answered by perturbing one input and re-running the model, or by a dedicated counterfactual-explanation method.

Picking the wrong one produces a technically correct but useless answer — showing a portfolio manager a global surrogate when they asked about one specific trade tells them what the model does on average, not what happened in the case in front of them.

Worked example

A long-short equity model shorts a stock, and the PM asks why. The desk runs TreeSHAP on that single prediction and gets a ranked list: earnings-quality score contributed −0.6 to the short signal, 20-day momentum contributed −0.3, sector-relative valuation contributed −0.1, and four other features contributed less than 0.05 combined. This local explanation says: this specific short was driven mostly by a deteriorating earnings-quality score, with momentum adding a smaller push in the same direction. Separately, compliance asks a broader question — does the model systematically short small-cap names more than large-cap ones? — which the local SHAP values for one trade can't answer; the desk instead runs a global surrogate tree over six months of predictions and finds market-cap is not among its top five splits, evidence against systematic small-cap bias, though not a complete guarantee of one.

What this means in practice

Build the explanation pipeline before you need it under pressure: log the exact feature values feeding every live prediction so a SHAP or TreeSHAP explanation can be regenerated for any past trade on demand, and keep a periodically refreshed global surrogate on hand for the "what does it do generally" conversations that come up in risk reviews. An explanation produced after the fact from re-created, approximate feature values is far less convincing — and less trustworthy — than one generated from the exact inputs the model actually saw.