DeepLIFT and Layer-Wise Relevance Propagation

Plain gradient-based attribution measures how much a tiny nudge to an input would change a neural network's output right now, but it can be misleading when the network has flat or saturated regions where the gradient is near zero even though the feature clearly mattered to get the prediction where it is. DeepLIFT and layer-wise relevance propagation (LRP) instead compare the actual input to a chosen reference input (like an all-zero or average baseline) and trace the difference in activation, layer by layer, back to the original input features.

Rather than asking "what is the local slope here," these methods ask "how much of the difference between this prediction and the reference prediction is attributable to each neuron, and in turn to each input feature," propagating that credit backward through the network using rules designed so the attributions sum up consistently layer to layer. This sidesteps the zero-gradient problem that plain gradients run into, at the cost of needing a sensible reference input to compare against.

Worked example

A network uses a ReLU activation that has fully saturated (output pinned at zero) for a particular input, making the local gradient exactly zero and hiding the feature's true influence from a plain gradient-based method. DeepLIFT instead compares this input against a reference input (say, all-zero features) and asks how much of the difference in the final output between the actual input and the reference traces back to each feature through the network's actual computation path — correctly assigning credit to a feature even though its local gradient reads as zero.