SmoothGrad and Attribution Noise

A basic saliency map explains a model's prediction by taking the gradient of the output with respect to each input feature — how much a tiny nudge to that feature would move the prediction. For deep, highly nonlinear models this gradient is often jagged and visually noisy, lighting up seemingly irrelevant features alongside the genuinely important ones, because the gradient at any single point can be dominated by local wiggles in the model's function that have nothing to do with the broader pattern it learned.

SmoothGrad fixes this with a deliberately counterintuitive trick: add small random noise to the input, compute the gradient-based attribution again, repeat this many times with different random noise each time, and average the results. Averaging over many noisy perturbations cancels out the jagged, point-specific artifacts while the genuinely important features — whose gradients stay large and consistent across nearby points — survive the averaging and stand out clearly.

Worked example

A model classifies news headlines by sentiment. A raw gradient-based attribution on one headline highlights a handful of scattered, seemingly random words alongside the genuinely sentiment-bearing ones. Running SmoothGrad — adding small random perturbations fifty times, recomputing the gradient attribution each time, and averaging — causes the scattered noise to wash out across the fifty runs while the words consistently flagged as important across nearly all of them (the actual sentiment-carrying words) remain highlighted.