Interpretability & XAI

Read these first

1. 01Counterfactual Explanations

   Instead of scoring how much each feature mattered, a counterfactual explanation just answers "what's the smallest change to this case that would have flipped the model's decision?", often the more actionable question.

2. 02Intrinsically Interpretable vs Post-Hoc Explained Models

   There are two ways to get an understandable model, build one simple enough to read directly, or build a complex one and attach a separate explanation afterward, and the second approach explains a story about the model, not necessarily the model itself.

3. 03Attention Is Not Explanation

   Attention weights show which inputs a model looked at hardest, which is tempting to read as "why" it made a decision, but looking hard at something and that thing driving the output turn out to be different claims.

4. 04The Shapley Axioms for Feature Attribution

   SHAP values split a model's prediction fairly among its input features by borrowing a rule from cooperative game theory, the only way to divide credit that satisfies four common-sense fairness properties at once.

Then the rest

1. 05Global vs Local Explanations
2. 06Interpretability vs Explainability
3. 07Choosing the Baseline in Feature Attribution
4. 08Explaining a Trading Model's Decisions
5. 09Explanation Stability and Robustness
6. 10Feature-Effect Plots When Features Are Correlated
7. 11Global Surrogate Models
8. 12Measuring Explanation Faithfulness
9. 13Sanity Checks for Saliency Methods
10. 14TreeSHAP
11. 15Concept Activation Vectors and TCAV
12. 16Integrated Gradients
13. 17KernelSHAP
14. 18Mechanistic Interpretability and Circuits
15. 19Saliency Maps and Raw Gradient Attribution