Delta-Audit: Explaining What Changes When Models Change

Afsaneh Fatemi; Arshia Hemmat

REVIEW 4 major objections 3 minor 22 references

Delta-Audit: Explaining What Changes When Models Change

T0 review · 4 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Delta-Attribution explains model updates by subtracting old attributions from new ones.

desk verdict A reasonable packaged audit tool whose headline evidence doesn't support the claim that the deltas explain feature-level behavior; BAC and DCE are permutation-invariant and several 'benign' controls are identical models. read the letter →

arxiv 2508.19589 v1 pith:C33CA7WT submitted 2025-08-27 cs.LG

Arshia Hemmat , Afsaneh Fatemi This is my paper

classification cs.LG

keywords explainableAIfeatureattributiondeltamodelupdatesocclusionauditbehaviour-attributioncouplingdistributionshift

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Model updates often change accuracy but not the reason decisions changed. This paper proposes Delta-Attribution, a model-agnostic framework that explains a version-to-version update by subtracting per-feature explanations: Δφ(x)=φ_B(x)−φ_A(x). To judge whether that difference means anything, it introduces a quality suite covering magnitude, sparsity, rank overlap, Jensen–Shannon divergence, a Delta Conservation Error, a Behaviour–Attribution Coupling score, and robustness checks, then runs it over 45 settings spanning five classical model families and three datasets. The finding is that structural changes such as kernel, depth, and feature-rule choices produce large, behaviour-aligned deltas, while cosmetic tweaks such as solver, search, or gamma settings produce near-zero deltas with perfect rank overlap. If this holds, update audits can flag benign, meaningful, or risky reliance shifts cheaply, complementing accuracy dashboards.

What carries the argument

The central object is the delta-attribution vector Δφ(x)=φ_B(x)−φ_A(x), the per-feature difference of two local explanation vectors. The behaviour-linked metrics carry the argument: the Delta Conservation Error DCE=E|Σ_j Δφ_j−Δf| checks how close the attribution difference comes to explaining the output difference additively, while the Behaviour–Attribution Coupling BAC is the Pearson correlation between ∥Δφ∥₁ and |Δf| across samples. Jensen–Shannon divergence and rank overlap separate redistribution from reweighting; grouped occlusion probes interactions. Together they form an audit: high BAC with large Δ means a coherent reliance shift, near-zero Δ with high rank overlap means a cosmetic u

What would settle it

A direct falsifier: take a model update that changes many weights but leaves all test predictions (and hence Δf) unchanged; if Δφ still has large L1 norm and BAC remains high, the coupling is measuring change size, not explanatory power. A sharper version: permute the feature labels of φ_B before differencing; if BAC and DCE stay close to their unpermuted values, the delta metrics do not depend on which features are attributed.

Watch

Extended reading notes

Core claim

Delta-Attribution claims that the difference of local attributions between two versions of a model is itself a faithful explanation of what changed. Given any per-feature explainer, define Δφ(x)=φ_B(x)−φ_A(x), and anchor behaviour to the class predicted by the updated model, Δf(x)=f_B(x)−f_A(x). The paper instantiates this with fast occlusion/clamping in standardized space, using shared baselines and a class-anchored margin. Across 45 A/B pairs, the strongest results show near-perfect behaviour–attribution coupling for inductive-bias changes—SVC poly→rbf on Breast Cancer with BAC≈0.998 and DCE≈6.6, Random Forest feature-rule swap on Digits with BAC≈0.997—and exact conservation (DCE=0) with p

Load-bearing premise

The framework assumes that a high Pearson coupling between ∥Δφ∥₁ and |Δf| means the attribution difference explains the output difference, rather than both simply growing together because the model changed.

Editorial extensions

If this is right

Practitioners can run a version-to-version audit in one pass: if BAC is high and Δ is large, the update reallocated reliance coherently, and if accuracy also improves it can be deployed with a light spot-check.
Cosmetic updates—solver, search, or gamma changes that leave top-10 features intact and DCE≈0—can be treated as safe without deep review.
A low-BAC, high-JSD update signals a reliance redistribution that accuracy alone misses, and should trigger inspection of the largest-Δ features.
Because the occlusion audit is O(nd) forward passes and the paper caps at 256 stratified samples, the suite is cheap enough for continuous-integration gates on tabular models.
The DCE diagnostic tells the user when occlusion is misleading; high DCE is an instruction to switch to a path-additive explainer before trusting the delta view.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper's central evidence is correlational: BAC couples Δφ size to Δf size. A natural stress test it does not report is to compare BAC against a null model in which feature attributions are permuted, or Δφ is drawn from the same model twice; if the correlation survives, BAC would be tracking the magnitude of model change rather than genuine explanation.
The reference-class choice (anchor to the updated model's predicted class) makes the audit optimistic toward B; anchoring to the true label could expose deltas that look benign but shift reliance off correct evidence, a testable variant for error analysis.
The method is defined for any explainer, but the paper only instantiates occlusion; plugging in path-additive explainers would change DCE's meaning and could turn DCE from a diagnostic into a hard conservation check.
The thresholds suggested for benign, aligned, and risky updates are tuned to tabular data with d≤64; extending to text or vision would likely need sparse or grouped occlusion, a direction the paper names as future work.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper packages a simple idea—difference two attribution vectors and watch the norm and rank change—into an "audit suite" with code, and that packaging is the actual contribution. The empirical findings do not support the headline claim that Delta-Attribution explains what changed at the feature level. The two behaviour-linked metrics, BAC and DCE, are symmetric functions of the coordinates of Δφ: permute the feature indices (even per-sample) and both numbers stay exactly the same. So BAC≈0.998 for a kernel change only says that the total magnitude of the attribution shift tracks the total magnitude of the output shift; it says nothing about whether the features carrying the shift are the ones responsible for the output change. That is a load-bearing gap, because the paper's "behaviour-aligned delta" evidence is built on BAC and DCE.

What is genuinely useful: the suite includes feature-sensitive checks (rank overlap, JSD, TopK, COΔF), and the robustness/noise checks are sensible. The 45-setting audit is cleanly organized, and the code release with figure hashes is a good reproducibility practice. The paper is also honest that occlusion is non-additive and DCE is only a diagnostic.

The soft spots are not minor. Several "perfect" results come from A/B pairs that are mathematically identical after standardization: SVC gamma=scale vs auto (1/d), kNN algorithm auto vs ball_tree, RF sqrt vs log2 on 13 features. Those give DCE=0 and RankOverlap=1 by construction, so the benign-vs-meaningful separation compares identical models against large structural changes. The actionable thresholds in Section VI.B are fit to the same data they describe, and no error bars or significance tests appear anywhere. For an empirical paper, that is a real gap.

Who gets value from this: someone building a quick CI check could reasonably use the suite as a first-pass heuristic, but should not read BAC as feature-level alignment. The conclusion overreaches the evidence. I would ask the authors to add a sanity check that actually manipulates feature alignment (e.g., permute Δφ coordinates and show BAC/DCE do not move) or to replace BAC with a metric that is feature-sensitive. That would fix the central problem.

I would send this to peer review rather than desk reject, because the question is practical and the flaw is addressable. But as it stands, the central interpretative claim is unsupported.

Referee Report

4 major / 3 minor

Summary. The paper proposes Delta-Attribution (Δ-Attribution), a model-agnostic framework for auditing model updates by differencing per-feature attributions between two versions, Δφ(x)=φ_B(x)−φ_A(x). The authors instantiate this with an occlusion/clamping explainer in standardized space and define a Δ-Attribution Quality Suite of metrics covering magnitude/sparsity (ℓ1, TopK, entropy), agreement/shift (rank overlap, JSD), behavioural alignment (DCE, BAC, COΔF), and robustness (noise, baselines, grouped occlusion). They report an audit of 45 settings (5 classical model families × 3 datasets × 3 A/B pairs), claiming that inductive-bias changes yield large, behaviour-aligned deltas (BAC up to ≈0.998), while cosmetic tweaks yield DCE≈0 and rank overlap 1.0, and that the method can separate benign updates from behaviourally meaningful or risky reliance shifts. The paper also provides a public code repository and reproducibility artifacts.

Significance. If the claims are sound, Δ-Attribution fills a practical gap: a lightweight, model-agnostic update audit that complements accuracy and single-model explanations. The paper’s strengths include a clear problem framing, a reproducible pipeline, and an explicit acknowledgment of occlusion non-additivity as a diagnostic limitation. However, the empirical evidence as presented does not yet support the central claim that Δφ explains what changed in a feature-specific way: the headline behaviour-linked metrics are invariant to feature permutation, and several control pairs are mathematically identical models. The contribution is promising but the evaluation needs substantial reworking to be load-bearing.

major comments (4)

[Section III.C / Section V.A] BAC and DCE are coordinate-symmetric functions of Δφ: BAC = corr(||Δφ||_1, |Δf|) and DCE = E|Σ_j Δφ_j − Δf|. Any permutation of feature indices—even a different permutation per sample—leaves these metrics unchanged. Consequently, the headline BAC≈0.998 values (Section V.A) do not establish that Δφ identifies the features responsible for the output change; a random relabeling of the same Δφ magnitudes would give identical numbers. The only feature-sensitive metric, COΔF, measures how much Δ mass falls on globally important features of B, not whether the per-sample pattern of Δφ explains Δf. To support the “explains what changed” claim, the paper needs a feature-level evaluation (e.g., comparing Δφ against known ground-truth feature switches or against a permutation baseline).
[Table I / Section V.A] Several “cosmetic” controls are exact identity pairs after standardization. With StandardScaler, SVC gamma='scale' equals 1/(n_features × variance) = 1/d, so gamma='auto' is the same value; thus svcP2 (rbf gamma=scale vs auto) compares identical models. kNN algorithm='auto' vs 'ball_tree' also yields identical predictions because auto selects ball_tree for these data. Therefore the reported DCE=0 and RankOverlap=1 for these pairs (Section V.A) are tautological and cannot serve as validations that the suite recognizes benign changes. The authors should replace these with genuinely non-identical but benign updates (e.g., different solvers converging to the same optimum or small regularization differences that alter predictions only within noise).
[Section IV / Tables III–IV and Section VI.B] All metrics are point estimates computed on a single stratified subset of up to 256 test samples, with no error bars, bootstrap confidence intervals, or repeated-seed experiments. The “±” values in Table III report standard deviations across A/B pairs, not uncertainty of the estimates. Consequently, qualitative distinctions such as “Logistic Regression changes are small and stable” and the actionable thresholds in Section VI.B (BAC<0.2, JSD>0.15, etc.) are not statistically supported. Without uncertainty quantification, the paper cannot justify that the observed separation between benign and meaningful updates is reliable rather than noise.
[Section VI.B] The “actionable thresholds” are fitted to the same 45 settings that they are used to describe, with no held-out validation or sensitivity analysis. This is a circular procedure: the thresholds summarize the authors’ own results rather than providing tested engineering guidance. The paper should either present these as descriptive observations (clearly labeled) or validate the thresholds on held-out A/B pairs, synthetic perturbations, or additional datasets.

minor comments (3)

[Abstract / Section I] The phrase “behaviour-aligned deltas” is used before the metric BAC is defined; a short parenthetical definition (e.g., “high correlation between Δφ magnitude and |Δf|”) would improve readability.
[Section III.A] The three-way definition of f(x) is awkwardly formatted; the third case (log-odds with epsilon) also has ambiguous parentheses. Please clarify the formula and ensure consistent notation across Sections III and IV.
[Section III.C / Table II] The metric name COΔF is not expanded consistently; in the text it is introduced as “class-outcome focus” but the underlying quantity is defined via permutation importance. Consider renaming or defining explicitly to avoid confusion.

Circularity Check

2 steps flagged · score 5.0 of 10

The 'benign/cosmetic' evidence is partly circular: the DCE=0 and RankOverlap=1 results come from A/B pairs that are functionally identical models, so those values are forced by construction; the Section VI.B thresholds are also fit to the same data they are used to describe. The structural-change findings (high BAC/JSD for genuinely different models) remain independent.

self definitional [Section V.A, 'Exact conservation under occlusion in small-change controls'; Table I configurations]
"Exact conservation under occlusion in small-change controls. Five A/B pairs exhibit DCE = 0.0: digits–knn–pair3, breast_cancer–svc–pair2, wine–knn–pair3, wine–rf–pair3, and breast_cancer–knn–pair3. These serve as sanity checks that our explainer/baseline choices can yield perfect delta conservation when updates are cosmetic."

For each listed pair the two 'versions' are functionally identical under the paper's own setup: with a StandardScaler, SVC gamma='scale' and gamma='auto' both give 1/d; kNN algorithm='auto' vs 'ball_tree' does not alter the decision function; and for Wine (13 features) RandomForest max_features='sqrt' and 'log2' both select 3 features. Hence phi_A=phi_B and f_A=f_B, so Delta_phi=0 and Delta_f=0 by definition, forcing DCE=0 and RankOverlap@10=1. Presenting these forced identities as evidence that the suite 'can yield perfect delta conservation' or separates 'benign changes' makes the benign-side result equivalent to the input identity rather than an empirical property of the explainer.
fitted input called prediction [Section VI.B, 'Actionable thresholds (engineering defaults)']
"We found the following decision heuristics useful in CI: • Benign update: BAC<0.2, ΔMagℓ1 in the bottom quartile, RankOverlap@10>0.9, DCE≈0. Treat as safe to deploy."

The thresholds are derived from the same 45 experimental settings they are then used to describe (the benign/risky categories in Section V). No independent or held-out validation is given; the 'decision heuristics' are a re-encoding of observed BAC/Mag/RankOverlap/DCE distributions. Thus the conclusion that Delta-Attribution 'flags benign updates, behaviour-aligned improvements, and risky reliance redistributions in a single pass' is a restatement of the fitting data rather than a validated prediction.

full rationale

The paper does not have a formal derivation chain, so circularity must be assessed on how empirical claims are supported. The main structural-change findings (e.g., SVC poly→rbf BAC≈0.998, RF depth-change JSD≈0.357) are not circular: BAC and DCE are transparently defined statistics and the high values for genuinely different models are empirical. However, the 'benign' side of the headline separation is supported by five A/B pairs that are actually identical models after standardization (SVC gamma, kNN algorithm, RF max_features on Wine), making DCE=0 and RankOverlap=1 exact identities. Calling these 'small-change controls' and using them as evidence of the audit's ability to detect cosmetic updates is a self-definitional result. The actionable thresholds in Section VI.B are also fit to the same data they are then applied to, a mild fitted-input-as-guidance circularity. The BAC/DCE permutation-invariance issue is a validity limitation, not a circularity, because the metrics are explicitly defined as symmetric functions of the coordinates; it does not make the reported numbers equivalent to the paper's inputs. No load-bearing self-citations or imported uniqueness theorems are present. Overall, circularity is partial and concentrated in the benign/control evidence; the core descriptive findings retain independent content.

Assumptions & free parameters 6 free parameters · 5 assumptions · 1 invented entities

The framework introduces no physical entities. The main hand-set quantities are the metric hyperparameters and the actionability thresholds, which are calibrated on the same data they interpret. The interpretative bridge from correlation to explanation is the least supported component.

free parameters (6)

benign BAC threshold = 0.2
Hand-set threshold in Section VI.B to classify benign updates; derived from the observed spread of BAC values, not from an independent validation set.
risky JSD threshold = 0.15
Hand-set threshold in Section VI.B for 'risky shift'; same post-hoc derivation issue.
top-k for TopK and rank overlap = 10
Fixed k=10 for DeltaTopK@10 and RankOverlap@10 in Section III.C; choice not justified by feature dimension.
top-m for CODF = 10
m=10 globally relevant features via permutation importance in Section III.C.
grouped occlusion top-k = 2
Joint clamp top-2 features in Section III.B; arbitrary choice.
noise sigma = 0.01, 0.05
Gaussian noise levels in the robustness section; no sensitivity study of these values.

assumptions (5)

domain assumption Occlusion attribution scores are meaningful proxies for feature reliance in the standardized input space.
The whole Delta-phi construction assumes clamping a feature to its mean and measuring the margin drop reflects how much the model relies on that feature.
domain assumption The class-anchored margin Delta-f(x) = f_B(x) - f_A(x) for the class predicted by fB is a suitable scalar proxy for behaviour change.
Section III.A anchors to fB's predicted class; the choice is justified only by an ablation in Section VI.F, not by theory.
domain assumption Pearson correlation (BAC) across test samples between ||Delta-phi||_1 and |Delta-f| indicates that attribution shifts explain output shifts.
This interpretative leap is asserted in Section III.C and never validated against ground truth or compared with baselines.
domain assumption DCE measures additive inconsistency; small DCE implies the occlusion delta explanation is well-behaved.
Section III.C defines DCE as a diagnostic; the paper itself notes occlusion is non-additive, so the semantics of DCE are unclear.
domain assumption Mean/median baseline averaging removes baseline artifacts.
Section IV.C states this without formal justification; extreme distributions may need domain baselines, as acknowledged in Section VI.E.

invented entities (1)

CODF (Class-Outcome Focus) metric
purpose: Measure whether delta attribution mass concentrates on globally relevant features for fixes and regressions.
New metric defined in Section III.C; no external validation of what a high or low CODF means in practice.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Delta-Audit: Explaining What Changes When Models Change." pith.science (2026). https://pith.science/paper/C33CA7WT

@misc{pith2026250819589,
  author       = {Pith},
  title        = {Pith review of: Delta-Audit: Explaining What Changes When Models Change},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/C33CA7WT}},
  note         = {Machine review of arXiv:2508.19589}
}

abstract

Model updates (new hyperparameters, kernels, depths, solvers, or data) change performance, but the \emph{reason} often remains opaque. We introduce \textbf{Delta-Attribution} (\mbox{$\Delta$-Attribution}), a model-agnostic framework that explains \emph{what changed} between versions $A$ and $B$ by differencing per-feature attributions: $\Delta\phi(x)=\phi_B(x)-\phi_A(x)$. We evaluate $\Delta\phi$ with a \emph{$\Delta$-Attribution Quality Suite} covering magnitude/sparsity (L1, Top-$k$, entropy), agreement/shift (rank-overlap@10, Jensen--Shannon divergence), behavioural alignment (Delta Conservation Error, DCE; Behaviour--Attribution Coupling, BAC; CO$\Delta$F), and robustness (noise, baseline sensitivity, grouped occlusion). Instantiated via fast occlusion/clamping in standardized space with a class-anchored margin and baseline averaging, we audit 45 settings: five classical families (Logistic Regression, SVC, Random Forests, Gradient Boosting, $k$NN), three datasets (Breast Cancer, Wine, Digits), and three A/B pairs per family. \textbf{Findings.} Inductive-bias changes yield large, behaviour-aligned deltas (e.g., SVC poly$\!\rightarrow$rbf on Breast Cancer: BAC$\approx$0.998, DCE$\approx$6.6; Random Forest feature-rule swap on Digits: BAC$\approx$0.997, DCE$\approx$7.5), while ``cosmetic'' tweaks (SVC \texttt{gamma=scale} vs.\ \texttt{auto}, $k$NN search) show rank-overlap@10$=1.0$ and DCE$\approx$0. The largest redistribution appears for deeper GB on Breast Cancer (JSD$\approx$0.357). $\Delta$-Attribution offers a lightweight update audit that complements accuracy by distinguishing benign changes from behaviourally meaningful or risky reliance shifts.

Figures

Figures reproduced from arXiv: 2508.19589 by the authors.

**Figure 1.** Overview of ∆-Attribution results across all (dataset, algorithm, pair). (a) BAC vs. DCE with marker size ∝ ∥∆ϕ∥1; (b) mean BAC and DCE by algorithm; (c) mean ∆-magnitude by algorithm. toggle regularization, inductive bias (kernel/depth), or search strategy (Table I). All models are wrapped in a Pipeline with a shared StandardScaler. C. Score Function and Explainer For each test sample we anchor to the class predict… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

22 extracted references · 18 canonical work pages

[1]

A unified approach to interpreting model predictions,

S. M. Lundberg and S.-I. Lee, “A unified approach to interpreting model predictions,” in NeurIPS, 2017. [Online]. Available: https://proceedings.neurips.cc/paper/7062-a-unified- approach-to-interpreting-model-predictions.pdf

work page 2017
[2]

From local explanations to global understanding with explainable AI for trees,

S. M. Lundberg et al. , “From local explanations to global understanding with explainable AI for trees,”Nature Machine Intelligence, vol. 2, no. 1, pp. 56–67, 2020

work page 2020
[3]

Anchors: High- precision model-agnostic explanations,

M. T. Ribeiro, S. Singh, and C. Guestrin, “Anchors: High- precision model-agnostic explanations,” inAAAI, 2018. [Online]. Available: https://homes.cs.washington.edu/~marcotcr/aaai18. pdf

work page 2018
[4]

Visualizing and understanding convolutional networks,

M. D. Zeiler and R. Fergus, “Visualizing and understanding convolutional networks,” inECCV, 2014, pp. 818–833

work page 2014
[5]

Sanity checks for saliency maps,

J. Adebayo, J. Gilmer, M. Muelly, I. Goodfellow, M. Hardt, and B. Kim, “Sanity checks for saliency maps,” inNeurIPS, 2018. [Online]. Available: https://papers.nips.cc/paper/8160-sanity- checks-for-saliency-maps.pdf

work page 2018
[6]

Fooling LIME and SHAP: Adversarial attacks on post hoc explanation methods,

D. Slack, S. Hilgard, E. Jia, S. Singh, and H. Lakkaraju, “Fooling LIME and SHAP: Adversarial attacks on post hoc explanation methods,” in AIES, 2020, pp. 180–186. [Online]. Available: https: //www.aies-conference.com/2020/wp-content/papers/174.pdf

work page 2020
[8]

Explanation shift: How did the distribution shift impact the model?

——, “Explanation shift: How did the distribution shift impact the model?” Transactions on Machine Learning Research, 2025. [Online]. Available: https://openreview.net/pdf?id=MO1slfU9xy

work page 2025
[9]

Training for stable explanation for free,

C. Chen, C. Guo, R. Chen, G. Ma, M. Zeng, X. Liao, X. Zhang, and S. Xie, “Training for stable explanation for free,” in NeurIPS, 2024. [Online]. Avail- able: https://proceedings.neurips.cc/paper_files/paper/2024/ file/0626822954674a06ccd9c234e3f0d572-Paper-Conference.pdf

work page arXiv 2024

Show all 22 references

[10]

An information-theoretic approach to distribution shifts,

M. Federici, R. Tomioka, and P. Forré, “An information-theoretic approach to distribution shifts,” in NeurIPS, 2021. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/ 2021/file/93661c10ed346f9692f4d512319799b3-Paper.pdf

2021
[11]

“why did the model fail?

H. Zhang, H. Singh, M. Ghassemi, and S. Joshi, ““why did the model fail?”: Attributing model performance changes to distribu- tion shifts,” inICML, 2023, pp. 41824–41846. [Online]. Available: https://proceedings.mlr.press/v202/zhang23ai/zhang23ai.pdf

2023
[12]

Locating and editing factual associations in GPT,

K. Meng, A. Andonian, D. Bau, and Y. Belinkov, “Locating and editing factual associations in GPT,” in NeurIPS, 2022. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2022/file/ 6f1d43d5a82a37e89b0665b33bf3a182-Paper-Conference.pdf

2022
[13]

Fast model editing at scale,

E. Mitchell, C. Lin, A. Bosselut, C. Finn, and C. D. Manning, “Fast model editing at scale,”arXiv:2110.11309, 2022. [Online]. Available: https://arxiv.org/abs/2110.11309

2022 arXiv
[14]

Mass-editing memory in a transformer,

K. Meng, A. Sen Sharma, A. Andonian, Y. Belinkov, and D. Bau, “Mass-editing memory in a transformer,” inICLR, 2023. [Online]. Available: https://openreview.net/forum?id=MkbcAHIYgyS

2023
[15]

Rebuilding ROME: Resolving model collapse during sequential model editing,

A. Gupta, S. Baskaran, and G. Anumanchipalli, “Rebuilding ROME: Resolving model collapse during sequential model editing,” in EMNLP, 2024. [Online]. Available: https:// aclanthology.org/2024.emnlp-main.1210.pdf

2024
[16]

Axiomatic attribution for deep networks,

M. Sundararajan, A. Taly, and Q. Yan, “Axiomatic attribution for deep networks,” in ICML, 2017. [Online]. Available: https://arxiv.org/abs/1703.01365

2017 arXiv
[18]

Grad-CAM: Visual explanations from deep networks via gradient-based localization,

R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-CAM: Visual explanations from deep networks via gradient-based localization,”International Journal of Computer Vision , vol. 128, no. 2, pp. 336–359, 2020

2020
[19]

A benchmark for interpretability methods in deep neural networks,

S. Hooker, D. Erhan, P.-J. Kindermans, and B. Kim, “A benchmark for interpretability methods in deep neural networks,” in NeurIPS, 2019. [Online]. Available: https://papers.nips.cc/paper_files/paper/2019/file/ fe4b8556000d0f0cae99daa5c5c5a410-Paper.pdf

2019
[20]

Reliable evaluation of attribution maps in cnns,

L. Nieradzik, F. Müller, and D. Ward, “Reliable evaluation of attribution maps in cnns,”International Journal of Computer Vision, 2024. [Online]. Available: https://link.springer.com/ article/10.1007/s11263-024-02282-6

2024 doi
[21]

Smoothed ge- ometry for robust attribution,

Z. J. Wang and J. Jamieson, “Smoothed ge- ometry for robust attribution,” in NeurIPS, 2020. [Online]. Available: https://papers.nips.cc/paper/2020/file/ 9d94c8981a48d12adfeecfe1ae6e0ec1-Paper.pdf

2020
[22]

Divergence measures based on the shannon entropy,

J. Lin, “Divergence measures based on the shannon entropy,” IEEE Transactions on Information Theory , vol. 37, no. 1, pp. 145–151, 1991

1991
[2017]

Available: https://arxiv.org/abs/1706.03825

[Online]. Available: https://arxiv.org/abs/1706.03825

arXiv
[2022]

Available: https://arxiv.org/abs/2210.12369

[Online]. Available: https://arxiv.org/abs/2210.12369

arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

A unified approach to interpreting model predictions,

S. M. Lundberg and S.-I. Lee, “A unified approach to interpreting model predictions,” in NeurIPS, 2017. [Online]. Available: https://proceedings.neurips.cc/paper/7062-a-unified- approach-to-interpreting-model-predictions.pdf

work page 2017

[2] [2]

From local explanations to global understanding with explainable AI for trees,

S. M. Lundberg et al. , “From local explanations to global understanding with explainable AI for trees,”Nature Machine Intelligence, vol. 2, no. 1, pp. 56–67, 2020

work page 2020

[3] [3]

Anchors: High- precision model-agnostic explanations,

M. T. Ribeiro, S. Singh, and C. Guestrin, “Anchors: High- precision model-agnostic explanations,” inAAAI, 2018. [Online]. Available: https://homes.cs.washington.edu/~marcotcr/aaai18. pdf

work page 2018

[4] [4]

Visualizing and understanding convolutional networks,

M. D. Zeiler and R. Fergus, “Visualizing and understanding convolutional networks,” inECCV, 2014, pp. 818–833

work page 2014

[5] [5]

Sanity checks for saliency maps,

J. Adebayo, J. Gilmer, M. Muelly, I. Goodfellow, M. Hardt, and B. Kim, “Sanity checks for saliency maps,” inNeurIPS, 2018. [Online]. Available: https://papers.nips.cc/paper/8160-sanity- checks-for-saliency-maps.pdf

work page 2018

[6] [6]

Fooling LIME and SHAP: Adversarial attacks on post hoc explanation methods,

D. Slack, S. Hilgard, E. Jia, S. Singh, and H. Lakkaraju, “Fooling LIME and SHAP: Adversarial attacks on post hoc explanation methods,” in AIES, 2020, pp. 180–186. [Online]. Available: https: //www.aies-conference.com/2020/wp-content/papers/174.pdf

work page 2020

[7] [8]

Explanation shift: How did the distribution shift impact the model?

——, “Explanation shift: How did the distribution shift impact the model?” Transactions on Machine Learning Research, 2025. [Online]. Available: https://openreview.net/pdf?id=MO1slfU9xy

work page 2025

[8] [9]

Training for stable explanation for free,

C. Chen, C. Guo, R. Chen, G. Ma, M. Zeng, X. Liao, X. Zhang, and S. Xie, “Training for stable explanation for free,” in NeurIPS, 2024. [Online]. Avail- able: https://proceedings.neurips.cc/paper_files/paper/2024/ file/0626822954674a06ccd9c234e3f0d572-Paper-Conference.pdf

work page arXiv 2024