REVIEW 4 major objections 4 minor 33 references
CAMs as Shapley Value-based Explainers
T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read The paper claims that standard Class Activation Mapping heatmaps are Shapley-value attributions of a Taylor-expanded network score, and that adding a Hessian correction yields ShapleyCAM, a closed-form second-order explainer computable in…
desk verdict A clean game-theoretic reframing of CAMs whose load-bearing Taylor-surrogate assumption is unvalidated and whose empirical wins are small; worth a serious referee but needs revision. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the Content Reserved Game-theoretic (CRG) Explainer, which defines a cooperative game at each pixel position: the $d$ spatial positions of the downsampled activation maps are players, and the utility $U$ is the network output. The explanation is $E = \sum_i g(W^i) \odot A^i$; Type-I keeps pixel-wise weights ($g(W^i)=W^i$, game-theoretic) and Type-II keeps only the channel mean ($g(W^i)=\bar{W}^i \mathbf{1}_d$, content-reserving). The proof machinery is the Taylor expansion of $U$ at $X_D$: first order gives $U'(X_D)X_j$, and the second-order contribution collapses through the subset-sum identity to $-\frac{1}{2} X_D^\top H_D X_j$. Only a Hessian-vector product is needed, so the computation costs one extra backward pass relative to GradCAM. The ReST utility $U_{\mathrm{ReST}} = y_c + \ln(\mathrm{softmax}(y)_c)$ ties together the pre- and post-softmax analyses.
What would settle it
Enumerate all subsets of a small activation tensor (say a $3\times3$ spatial grid over a few channels), compute the exact Shapley value of the second-order Taylor-expanded utility using the definition, and compare each value with Equation (14); exact agreement is required if the derivation is correct. A second check is to evaluate the Taylor remainder on random subsets for a Swin Transformer's self-attention layer, where the paper itself reports that the approximation stops being predictive.
Extended reading notes
Core claim
Under the second-order Taylor expansion of the utility at the full activation tensor $X_D$, the Shapley value of the $j$-th spatial position is $\phi(j;U) = U'(X_D)X_j - \frac{1}{2} X_D^\top H_D X_j$, where $X_j$ keeps only the $j$-th column across all activation maps and $H_D$ is the Hessian of the utility at $X_D$. Because this expression is linear in the entries of $X_j$, it decomposes as $\sum_i W^i_j A^i_j$ with $W^i = [U'(X_D) - \frac{1}{2} X_D^\top H_D]^i$, exactly the weighting ShapleyCAM uses. The paper proves that ShapleyCAM-H is a Type-I CRG Explainer (per-pixel game-theoretic) and ShapleyCAM is a Type-II CRG Explainer (content-reserving); at first order the same theorem identifies HiResCAM and GradCAM as two valid truncations of one Shapley formula. It further proves that a post-softmax GradCAM heatmap is a probability-weighted sum of differences of pre-softmax heatmaps, which motivates adding the pre-softmax term back to form the ReST utility.
Load-bearing premise
The derivation assumes the second-order Taylor expansion of the utility at the full activation tensor is a good approximation for every subset of pixel positions, which the paper's own Swin Transformer experiments show does not always hold.
Editorial extensions
If this is right
- ShapleyCAM, ShapleyCAM-H, and ShapleyCAM-E compute exact Shapley values for the second-order Taylor game, with only one additional backward pass beyond GradCAM.
- GradCAM and HiResCAM inherit a Shapley-value interpretation as first-order approximations, and the placement of ReLU is settled: apply it after summing the weighted maps, not before.
- The ReST utility removes both failure modes of the two standard score choices: off-target highlighting from pre-softmax and vanishing gradients from post-softmax.
- Across the twelve tested networks, the second-order correction typically improves the ADCC, IC, and ADD explanation metrics over the corresponding first-order methods, especially for CNN architectures.
- For Swin Transformers at deep self-attention target layers, the Taylor approximation is not reliable: most CAM methods, including ShapleyCAM, do not beat random-weight heatmaps, identifying a boundary of the theory.
Reading between the lines
- If the identity holds generally, any linear CAM weighting scheme can be read as a Taylor order of a Shapley expansion; an order-$k$ expansion would use $k$-th order derivative information and yield a hierarchy of CAM explainers trading precision against cost.
- The proof technique transfers naturally to other cooperative-game solution concepts that differ from Shapley only in how subset marginals are averaged, so the same second-order closed form may produce new attribution methods beyond ShapleyCAM.
- The ReST construction suggests a one-parameter family $U_\lambda = y_c + \lambda \ln p_c$; tuning $\lambda$ could interpolate between localization behavior and gradient stability for different confidence regimes.
- The paper's explainability-versus-localization argument implies that current metrics such as ADCC reward foreground localization, so a target-class-specific evaluation, measuring whether the heatmap changes when the target class changes, would be a sharper test for CAM explainers.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces the Content Reserved Game-theoretic (CRG) Explainer, a cooperative-game framework in which each pixel of the downsampled explanation map is a player whose utility is the network output as a function of the target-layer activations. Within this framework, the paper proves that HiResCAM and GradCAM are first-order Taylor approximations of Shapley values, and it proposes ShapleyCAM, ShapleyCAM-H, and ShapleyCAM-E, which use a second-order Taylor expansion to obtain closed-form weights involving the gradient and Hessian of a proposed ReST utility function. The paper also gives a theoretical relation between pre-softmax and post-softmax GradCAM heatmaps and evaluates the methods on 12 ImageNet backbones with 6 metrics. The central closed-form Shapley derivation in Theorem 2 is mathematically correct for the quadratic surrogate, but the paper does not validate the fidelity of that surrogate across the coalition space, and the empirical margins over existing CAM methods are generally very small.
Significance. If the Taylor-surrogate assumption held uniformly over all coalitions, the paper would provide a valuable bridge between heuristic CAM methods and Shapley-based attribution: a one-extra-backward-pass method that produces exact Shapley values for an explicit quadratic approximation. The paper's main strengths are its identification of GradCAM and HiResCAM as first-order Shapley approximations, the correct closed-form second-order correction in Eq. (14), the ReST utility analysis, the full-ImageNet evaluation, and the public code release. However, the significance is currently limited by three concerns: the exact-Shapley claim is conditional on an unvalidated Taylor expansion, the empirical advantages over GradCAM are often fractions of a point in ADCC, and the proof of Theorem 3 contains an internal sign inconsistency. These issues are fixable but require additional analysis and experiments.
major comments (4)
- [Section 3.4, proof of Theorem 3 and Eq. (15)] The chain-rule identity in the proof of Theorem 3 is stated with the wrong sign. The correct identity is ∂p_c/∂A_i = p_c Σ_{k≠c} p_k (∂y_c/∂A_i − ∂y_k/∂A_i), not p_c Σ_{k≠c} p_k (∂y_k/∂A_i − ∂y_c/∂A_i). The subsequent displayed equation g(∂p_c/∂A_i) = p_c Σ_{k≠c} p_k (g(∂y_k/∂A_i) − g(∂y_c/∂A_i)) inherits this sign error, yet Eq. (15) uses Epre_c − Epre_k, which would follow from the correct sign. As written, the derivation is internally inconsistent: the final formula is correct only after a second, unstated sign change. Because ReST is motivated by this relationship, the proof must be corrected before publication.
- [Tables 1 and 3, Section 4.4] The claim that ShapleyCAM and ShapleyCAM-H 'consistently outperform' other CAM methods in ADCC, IC, and ADD is based on point estimates whose differences are very small. For example, in Table 1 ResNet-18, ShapleyCAM versus GradCAM gives ADCC 77.22 versus 77.10, IC 33.92 versus 33.77, and ADD 51.90 versus 51.89; in Table 3 ResNet-18 last-convolutional-layer, ShapleyCAM-H versus HiResCAM gives ADCC 77.21 versus 77.12, IC 30.50 versus 30.48, and ADD 49.37 versus 49.39. No confidence intervals, repeated runs, or significance tests are reported anywhere in the paper. Given the magnitude of these differences, the empirical support for the central claim of improved explainability is not yet established and needs a more rigorous reporting of variability or statistical testing.
- [Sections 3.2–3.3 and 4.4, Eq. (14), Table 4] The exact-Shapley interpretation of ShapleyCAM is conditional on a second-order Taylor surrogate for U(X_S), but the paper provides no diagnostic that this surrogate is accurate across the coalition distribution. For typical small coalitions, X_S is far from the expansion point X_D, and the paper gives no bound on the Taylor remainder and no comparison against exact or Monte Carlo Shapley values on any configuration. The load-bearing nature of this assumption is visible in the paper's own Table 4: on the first normalization layer of Swin Transformer blocks, ShapleyCAM-H achieves ADCC 39.48 versus RandomCAM 49.21 for Swin-S, and most CAM methods fail to beat RandomCAM; Section 4.4 attributes this to the polynomial approximation of self-attention but leaves the hypothesis untested. A concrete test, such as computing Monte Carlo Shapley values for a reduced target layer and reporting correlation or error against ShapleyCAM weights, is needed to support the central claim.
- [Section 4.6, Table 5] The ReST ablation does not support the statement that ReST 'outperforms pre-softmax and post-softmax scores, except in Complexity.' Under the paper's primary composite metric ADCC, pre-softmax achieves 77.80 while ReST achieves 77.22; ReST is better on AD, Coh, IC, and ADD, but worse on Complexity and, overall, worse on ADCC. The text as written is misleading because the 'except in Complexity' exception is precisely what makes ReST lose the primary metric. The claim should be qualified, or the choice of primary metric should be defended in light of this result.
minor comments (4)
- [Section 2.2] The text contains a typo: 'CIF AR-10' should be 'CIFAR-10'.
- [Section 5] The word 'proccess' should be 'process'.
- [Equations (6) and (10)–(14)] The notation for row and column vectors is inconsistent: U'(X_D) is written both as a row vector (e.g., U'(X_D)X_j) and as a quantity involving transposes (e.g., X_D^T H_D). Defining all vectors as column vectors and using explicit transposes throughout would remove ambiguity.
- [Section 3.5] The justification for placing ReLU outside the summation relies on 'Shapley values greater than 0 indicating positive contribution,' but this does not by itself imply that negative Shapley values should be discarded; the paper should acknowledge that clipping is a visualization choice that removes potentially informative negative attributions.
Circularity Check
No circularity found: the central result is a closed-form Shapley value for an explicitly stated second-order Taylor surrogate, derived from the Shapley definition with no fitted parameters and no self-citation chain.
full rationale
The paper's load-bearing derivation (Theorem 2, Eq. 14) is self-contained: starting from Definition 1 (Shapley 1953), it substitutes the second-order Taylor expansion of U(X_S) around X_D, applies the Linearity axiom to split the first- and second-order parts, and evaluates the combinatorial sums to obtain phi(j;U) = U'(X_D)X_j - (1/2) X_D^T H_D X_j. The gradient and Hessian are computed from the fixed trained network, not fit to any explanation data, and the ReST utility is an explicit modeling choice whose value is then used in the same expansion. GradCAM and HiResCAM are related to the first-order part by the same algebra, and the claim that they are 'Optimal CRG Explainers' at the pre-GAP layer follows from the constancy of gradients through GAP, as cited to [5] and checked by the definition. The only citation used as a technique is [17] for the idea of Taylor-expanding the utility; the expansion itself is written out and proved in the paper, so the result does not reduce to that citation. The empirical comparison to RandomCAM and other CAMs is external evidence. The acknowledged failure on Swin self-attention layers (Section 4.4) is a limitation of the Taylor-surrogate fidelity, not a sign that the Shapley claim is fitted or definitionally forced. Hence no circular step can be quoted and exhibited.
Assumptions & free parameters
assumptions (4)
- standard math The Shapley value axioms (dummy, symmetry, efficiency, linearity) and the uniqueness theorem for the Shapley value.
- domain assumption The utility function U, representing the network from target layer to output, is differentiable and its second-order Taylor expansion around X_D is an accurate approximation for all subsets X_S.
- domain assumption The cooperative game is defined on the activation maps: each pixel position j is a player, and the utility U can be evaluated at X_S, a vector where only pixels in S contribute.
- standard math For Theorem 3, the averaging map g(A)=mean(A) 1_d is linear.
Cite this review
Pith. "Pith review of CAMs as Shapley Value-based Explainers." pith.science (2026). https://pith.science/paper/RG4OTYRE
@misc{pith2026250106261,
author = {Pith},
title = {Pith review of: CAMs as Shapley Value-based Explainers},
year = {2026},
howpublished = {\url{https://pith.science/paper/RG4OTYRE}},
note = {Machine review of arXiv:2501.06261}
}
read the original abstract
Class Activation Mapping (CAM) methods are widely used to visualize neural network decisions, yet their underlying mechanisms remain incompletely understood. To enhance the understanding of CAM methods and improve their explainability, we introduce the Content Reserved Game-theoretic (CRG) Explainer. This theoretical framework clarifies the theoretical foundations of GradCAM and HiResCAM by modeling the neural network prediction process as a cooperative game. Within this framework, we develop ShapleyCAM, a new method that leverages gradients and the Hessian matrix to provide more precise and theoretically grounded visual explanations. Due to the computational infeasibility of exact Shapley value calculation, ShapleyCAM employs a second-order Taylor expansion of the cooperative game's utility function to derive a closed-form expression. Additionally, we propose the Residual Softmax Target-Class (ReST) utility function to address the limitations of pre-softmax and post-softmax scores. Extensive experiments across 12 popular networks on the ImageNet validation set demonstrate the effectiveness of ShapleyCAM and its variants. Our findings not only advance CAM explainability but also bridge the gap between heuristic-driven CAM methods and compute-intensive Shapley value-based methods. The code is available at \url{https://github.com/caihuaiguang/pytorch-shapley-cam}.
Reference graph
Works this paper leans on
-
[1]
BMC Medical Informat- ics and Decision Making 20, 1–9 (2020)
Amann, J., Blasimme, A., Vayena, E., Frey, D., Madai, V.I.: Explainability for Artifi- cial Intelligence in Healthcare: A Multidisci- plinary Perspective. BMC Medical Informat- ics and Decision Making 20, 1–9 (2020)
work page 2020
-
[2]
IEEE Transactions on Intelligent Trans- portation Systems 23, 10142–10162 (2022)
Omeiza, D., Webb, H., Jirotka, M., Kunze, L.: Explanations in Autonomous Driving: A Sur- vey. IEEE Transactions on Intelligent Trans- portation Systems 23, 10142–10162 (2022)
work page 2022
-
[3]
In: Conference on Computer Vision and Pattern Recogni- tion, pp
Zhou, B., Khosla, A., Lapedriza, `A., Oliva, A., Torralba, A.: Learning Deep Features for Discriminative Localization. In: Conference on Computer Vision and Pattern Recogni- tion, pp. 2921–2929 (2016)
work page 2016
-
[4]
arXiv preprint arXiv:2011.08891 (2020)
Draelos, R.L., Carin, L.: Use HiResCAM instead of Grad-CAM for Faithful Expla- nations of Convolutional Neural Networks. arXiv preprint arXiv:2011.08891 (2020)
arXiv 2020
-
[5]
In: International Joint Conference on Artificial Intelligence, pp
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad- CAM: Visual Explanations from Deep Net- works via Gradient-Based Localization. In: International Joint Conference on Artificial Intelligence, pp. 618–626 (2017) 18
work page 2017
-
[6]
In: Conference on Computer Vision and Pattern Recognition Workshops, pp
Wang, H., Wang, Z., Du, M., Yang, F., Zhang, Z., Ding, S., Mardziel, P., Hu, X.: Score-CAM: Score-Weighted Visual Explana- tions for Convolutional Neural Networks. In: Conference on Computer Vision and Pattern Recognition Workshops, pp. 111–119 (2020)
work page 2020
-
[7]
In: Winter Conference on Applications of Com- puter Vision, pp
Desai, S., Ramaswamy, H.G.: Ablation-CAM: Visual Explanations for Deep Convolutional Network via Gradient-free Localization. In: Winter Conference on Applications of Com- puter Vision, pp. 972–980 (2020)
work page 2020
-
[8]
In: Winter Conference on Applications of Computer Vision, pp
Chattopadhay, A., Sarkar, A., Howlader, P., Balasubramanian, V.N.: Grad-CAM++: Improved Visual Explanations for Deep Con- volutional Networks. In: Winter Conference on Applications of Computer Vision, pp. 839– 847 (2018)
work page 2018
Show all 33 references
-
[9]
arXiv preprint arXiv:2008.02312 (2020)
Fu, R., Hu, Q., Dong, X., Guo, Y., Gao, Y., Li, B.: Axiom-based Grad-CAM: Towards Accurate Visualization and Explanation of CNNs. arXiv preprint arXiv:2008.02312 (2020)
2020 arXiv
-
[10]
IEEE Transactions on Image Processing 30, 5875–5888 (2021)
Jiang, P., Zhang, C., Hou, Q., Cheng, M., Wei, Y.: LayerCAM: Exploring Hierarchi- cal Class Activation Maps For Localization. IEEE Transactions on Image Processing 30, 5875–5888 (2021)
2021
-
[11]
In: Inter- national Conference on Computer Vision, pp
Jung, H., Oh, Y.: Towards Better Explana- tions of Class Activation Mapping. In: Inter- national Conference on Computer Vision, pp. 1316–1324 (2021)
2021
-
[12]
Gildenblat, J., contributors: PyTorch library for CAM methods. GitHub. https: //github.com/jacobgil/pytorch-grad-cam [Accessed: (Nov. 11, 2024)] (2021)
2021
-
[13]
Contributions to the Theory of Games 2, 307–317 (1953)
Shapley, L.S.: A Value for N-person Games. Contributions to the Theory of Games 2, 307–317 (1953)
1953
-
[14]
In: International Joint Conference on Artificial Intelligence, pp
Rozemberczki, B., Watson, L., Bayer, P., Yang, H., Kiss, O., Nilsson, S., Sarkar, R.: The Shapley Value in Machine Learning. In: International Joint Conference on Artificial Intelligence, pp. 5572–5579 (2022)
2022
-
[15]
In: Advances in Neural Information Processing Systems, pp
Lundberg, S.M., Lee, S.: A Unified Approach to Interpreting Model Predictions. In: Advances in Neural Information Processing Systems, pp. 4765–4774 (2017)
2017
-
[16]
In: International Conference on Machine Learning, pp
Ghorbani, A., Zou, J.Y.: Data Shapley: Equi- table Valuation of Data for Machine Learn- ing. In: International Conference on Machine Learning, pp. 2242–2251 (2019)
2019
-
[17]
arXiv preprint arXiv:2406.11011 (2024)
Wang, J.T., Mittal, P., Song, D., Jia, R.: Data Shapley in One Training Run. arXiv preprint arXiv:2406.11011 (2024)
2024 arXiv
-
[18]
arXiv preprint arXiv:2309.14304 (2023)
Minh, A.P.T.: Overview of Class Activation Maps for Visualization Explainability. arXiv preprint arXiv:2309.14304 (2023)
2023 arXiv
-
[19]
Fernandez, F.-G.: TorchCAM: class activa- tion explorer. GitHub. https://github.com/ frgfm/torch-cam [Accessed: (Nov. 11, 2024)] (2020)
2020
-
[20]
Algaba, E., Fragnelli, V., S´ anchez-Soriano, J.: Handbook of the Shapley Value, (2019)
2019
-
[21]
In: Advances in Neural Information Processing Systems, pp
Mazumder, M., Banbury, C.R., Yao, X., Kar- las, B., Rojas, W.G., Diamos, S.F., Diamos, G., He, L., Parrish, A., Kirk, H.R., Quaye, J., Rastogi, C., Kiela, D., Jurado, D., Kanter, D., Mosquera, R., Cukierski, W., Ciro, J., Aroyo, L., Acun, B., Chen, L., Raje, M., Bartolo, M., E...
2023
-
[22]
https://iclr-blogposts.github.io/2024/blog/ bench-hvp/ [Accessed: (Nov
Dagr´ eou, M., Ablin, P., Vaiter, S., Moreau, T.: How to compute Hessian-vector prod- ucts? In: ICLR Blogposts 2024 (2024). https://iclr-blogposts.github.io/2024/blog/ bench-hvp/ [Accessed: (Nov. 11, 2024)]
2024
-
[23]
1–4 (2023)
Lerma, M., Lucas, M.: Pre or Post-Softmax 19 Scores in Gradient-based Attribution Meth- ods, What is Best? In: International Confer- ence on Pattern Recognition Systems, pp. 1–4 (2023)
2023
-
[24]
In: Conference on Computer Vision and Pattern Recognition, pp
He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition. In: Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
2016
-
[25]
In: International Conference on Artificial Intelligence and Statistics, pp
Wang, J.T., Jia, R.: Data Banzhaf: A Robust Data Valuation Framework for Machine Learning. In: International Conference on Artificial Intelligence and Statistics, pp. 6388–6421 (2023)
2023
-
[26]
International Journal of Computer Vision 115, 211–252 (2015)
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M.S., Berg, A.C., Fei-Fei, L.: ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision 115, 211–252 (2015)
2015
-
[27]
In: Conference on Computer Vision and Pattern Recognition, pp
Xie, S., Girshick, R.B., Doll´ ar, P., Tu, Z., He, K.: Aggregated Residual Transformations for Deep Neural Networks. In: Conference on Computer Vision and Pattern Recognition, pp. 5987–5995 (2017)
2017
-
[28]
In: International Conference on Learning Representations (2015)
Simonyan, K., Zisserman, A.: Very Deep Con- volutional Networks for Large-Scale Image Recognition. In: International Conference on Learning Representations (2015)
2015
-
[29]
In: International Conference on Machine Learning, pp
Tan, M., Le, Q.V.: EfficientNet: Rethink- ing Model Scaling for Convolutional Neural Networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019)
2019
-
[30]
In: Con- ference on Computer Vision and Pattern Recognition, pp
Sandler, M., Howard, A., Zhu, M., Zhmogi- nov, A., Chen, L.-C.: MobileNetV2: Inverted Residuals and Linear Bottlenecks. In: Con- ference on Computer Vision and Pattern Recognition, pp. 4510–4520 (2018)
2018
-
[31]
In: International Conference on Computer Vision, pp
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B.: Swin Trans- former: Hierarchical Vision Transformer using Shifted Windows. In: International Conference on Computer Vision, pp. 9992– 10002 (2021)
2021
-
[32]
Wightman, R.: PyTorch Image Models. GitHub. https://github.com/rwightman/ pytorch-image-models [Accessed: (Nov. 11, 2024)] (2019)
2019
-
[33]
In: Conference on Computer Vision and Pat- tern Recognition Workshops, pp
Poppi, S., Cornia, M., Baraldi, L., Cuc- chiara, R.: Revisiting the Evaluation of Class Activation Mapping for Explainability: A Novel Metric and Experimental Analysis. In: Conference on Computer Vision and Pat- tern Recognition Workshops, pp. 2299–2304 (2021) 20
2021
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.