REVIEW 3 major objections 6 minor 30 references
Counterfactual Explanations in Medical Imaging: Exploring SPN-Guided Latent Space Manipulation
T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read A sum-product network in the VAE latent space keeps chest X-ray counterfactuals valid when regularization is weak.
desk verdict New SPN-in-VAE combination for chest X-ray counterfactuals, but the robustness claim rests on a self-referential validity metric that tracks classifier entropy more than counterfactual quality. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is a class-conditional Gaussian sum-product network: a rooted directed acyclic graph whose leaves are univariate Gaussian densities and whose internal nodes are weighted sums and products, learned from latent codes by a recursive structure-learning algorithm. Because the network is complete and decomposable, it gives exact, tractable values of $p_{\mathrm{SPN}}(z)$ and $p_{\mathrm{SPN}}(y|z)$, so one model supplies the classifier term, the density term, and the class-conditioned likelihood used in the latent-space objective. The learned network is converted into a differentiable neural-network form so that the counterfactual code $z_{\mathrm{cf}}$ can be optimized by gradient descent and then decoded into an image.
What would settle it
Recompute validity with a single fixed referee classifier, such as the original MLP or a third independently trained model, labeling both SPN-generated and MLP-generated counterfactuals; if the SPN's validity advantage disappears or reverses, the robustness claim fails. A supporting check is to temperature-scale both classifiers to matching entropy and see whether the validity gap vanishes.
Extended reading notes
Core claim
The authors claim that a structure-learned sum-product network placed over the latent space of a semi-supervised VAE (a VAE trained so its latent codes also predict the class) can serve as both the latent-space density descriptor and the classifier for counterfactual generation. Starting from a sampled latent code $z$ for an image, they search for $z_{\mathrm{cf}}$ by gradient descent to maximize $\log p_{\mathrm{SPN}}(y_{\mathrm{cf}}|z')$, with optional penalties $\beta\|z'-z\|^2$ and $\gamma|\log p_{\mathrm{SPN}}(z')-\log p_{\mathrm{SPN}}(z)|$, then decode $z_{\mathrm{cf}}$ into a counterfactual image. Their empirical claim is that this SPN-guided manipulation maintains validity, the fraction of counterfactuals whose predicted class changes, under weak Kullback-Leibler divergence (KLD) regularization ($\beta_1$ from 0.01 down to 0.0001), while the MLP baseline's validity collapses to near zero. The auxiliary $\beta$ and $\gamma$ weights have little effect, and the visualized counterfactual changes concentrate around the heart region.
Load-bearing premise
The load-bearing premise is that validity measured by each method's own classifier is a fair comparison; if the SPN merely has less confidence in its predictions, its higher flip rate would not mean better counterfactuals.
Editorial extensions
If this is right
- If the SPN robustness holds, weakly regularized VAEs, which reconstruct finer image detail, can still support counterfactual explanation without losing the ability to flip predictions.
- The small effect of $\beta$ and $\gamma$ implies that the target-class probability term, not the proximity or likelihood penalties, is the main driver of the optimized latent shift.
- Absolute $\ell^2$ and FID scores track reconstruction quality more than counterfactual quality, so comparisons across VAE regularizations need relative or class-conditioned versions of these metrics.
- SPN-guided explanations localize changes around the heart across regularization settings, while MLP-guided explanations do so only under strong regularization, which matters if explanations are meant to point clinicians to disease-relevant regions.
Reading between the lines
- Editorial: the validity comparison is not controlled for classifier confidence; the SPN's higher predictive entropy at low $\beta_1$ means its flip rate may reflect softer decisions, so a fixed referee classifier should be used before concluding the SPN produces better counterfactuals.
- Editorial: because the $\beta$ and $\gamma$ penalties barely changed outcomes, a natural next step is to use the SPN log-likelihood itself as the distance penalty, replacing the Euclidean term with a distribution-aware measure.
- Editorial: the result suggests a cheap diagnostic for any latent-space counterfactual method: report predictive entropy alongside validity, since confident classifiers can make otherwise identical edits look invalid.
- Editorial: transferring the approach to other pathologies or multiclass labels would require class-specific FID and $\ell^2$ metrics, because absolute reconstruction metrics are dominated by VAE quality.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a counterfactual explanation pipeline for medical images in which a sum-product network (SPN) is trained on the latent space of a semi-supervised VAE and then used both as a class-posterior model and as a likelihood descriptor for latent-space counterfactual search. After VAE training, the MLP classifier is replaced by a structure-learned Gaussian SPN, and counterfactual latent codes are obtained by maximizing the SPN's target-class log-posterior while penalizing squared distance and log-likelihood deviation (Eq. 2). Experiments on a balanced binary CheXpert subset (cardiomegaly vs. no finding) compare the SPN with the MLP baseline under four KLD strengths beta_1 and with different combinations of beta and gamma. The headline result is that under weak regularization the MLP validity drops to 0.03-0.09 while the SPN remains at 0.43-0.89, which the paper interprets as greater robustness of SPN-guided manipulation; the paper also reports L2 proximity, FID plausibility, and alteration visualizations.
Significance. The proposed integration of tractable SPN inference with VAE latent space manipulation is a sensible and potentially useful direction for probabilistic counterfactual explanations, and the released code and detailed experimental setup are assets. If the robustness claim were established, the paper would provide a practical demonstration that probabilistic classifiers can support counterfactual search under weak latent regularization. However, the current evidence does not establish the claimed cross-method advantage, because the validity metric is evaluated with the same classifier that generates the counterfactual and the comparison is not controlled for classifier confidence; the absence of uncertainty estimates further weakens the empirical claims.
major comments (3)
- [Section 4.4, Eq. (3); Tables 2-3] The central robustness claim is confounded by the self-referential validity measure. In Eq. (3), f is the SPN for SPN counterfactuals and the MLP for MLP counterfactuals, and Eq. (2) maximizes log p(y_cf|z') under the same f. Validity therefore measures how often each method's own optimization pushes latent codes across its own decision boundary, not an independent property of the generated counterfactuals. Table 2 shows that in the weak-regularization regimes where the validity gap appears, the SPN has much higher predictive entropy than the MLP (0.756-2.233 vs. 0.359-0.365), and Section 4.4 explicitly attributes the MLP's difficulty to 'the MLP's low entropy... more challenging to modify'. Hence the higher SPN validity may reflect a softer, less confident classifier rather than better counterfactuals. To support the claim that SPN-guided manipulation is more robust, the authors should evaluate both sets of counterfactuals with a fixed independent classifier, and/or report validity under each classifier in both directions, and/or control for predictive entropy or calibration. This is the main load-bearing issue for the paper's headline conclusion.
- [Section 4.2, Eq. (4)] The FID formula is not the standard Frechet Inception Distance. The correct expression is ||mu_org - mu_cf||^2 + Tr(Sigma_org + Sigma_cf - 2(Sigma_org Sigma_cf)^{1/2}); the manuscript writes the cross-term as -2Tr(Sigma_org Sigma_cf), which is the trace of a matrix product and not a valid distance statistic. The reported FID values should be recomputed with the correct formula before they are used to support any claims about plausibility.
- [Section 4.1, Table 3] All counterfactual statistics are reported as means over the three cross-validation splits without standard deviations, confidence intervals, or significance tests. For the paper's central comparison, the reader cannot tell whether gaps such as 0.55 vs. 0.09 are stable across splits or driven by a single fold; please report per-split results and appropriate uncertainty measures, and test the validity differences when the confound in the first comment is addressed.
minor comments (6)
- [Section 4.3] The text says 'Only for beta_1 = 0.1, the SPN achieves an improvement... However, for smaller beta_1 = 0.1, the MLP outperforms the SPN.' The phrase 'smaller beta_1 = 0.1' should be corrected to 'beta_1 < 0.1' or 'smaller values of beta_1.'
- [Section 4.1, Table 3] The text states that the MLP baseline is run with gamma in {0,1}, but Table 3 lists MLP only for gamma = 0.0; please specify which configurations were actually run and why gamma = 1.0 is omitted for the MLP.
- [Section 4.3] The text mentions 'for both datasets,' but the experiments use only the CheXpert dataset; please remove or correct this phrase.
- [Section 4.2, Eq. (3)] The validity indicator 1[f(x) != f(x_cf)] counts any prediction change, not necessarily a flip to the targeted counterfactual class y_cf; please use 1[f(x_cf) = y_cf] or report both quantities.
- [Section 4.2, Table 3] The 'switch epoch' metric is reported in Table 3 but is not defined in the metrics section; please define how it is computed.
- [Table 2] The reported entropy values exceed 1.0 for binary classification, which suggests a particular logarithm base or averaging convention; please specify the base and clarify whether this is per-class averaged entropy.
Circularity Check
SPN validity is self-referential: the same fitted SPN is both the optimization target (Eq. 2) and the evaluation function (Eq. 3), so the headline robustness gap largely reflects classifier softness; independent content remains in FID/L2 and the raw MLP comparison.
-
self definitional
[Section 3.3 Eq. (2), Section 4.2 Eq. (3), Table 3, Section 4.4]
"z_{cf} = arg max_{z'} log(p(y_{cf}|z')) − β||z'−z||^2 − γ|log(p(z'))−log(p(z))| (2) ... Validity = 1/|X| ∑_{x∈X} 1[f(x)≠f(x_{cf})] (3) ... under weaker regularization, validity significantly declines for the MLP (0.09-0.03), while the SPN retains moderate validity (0.43-0.89)."
Eq. (2)'s first term maximizes the fitted SPN posterior p_spn(y_cf|z'), and Eq. (3)'s validity for SPN rows is the flip rate of that same SPN (f=SPN). Thus SPN validity measures only whether the optimizer pushed the SPN's own posterior over threshold, not objective counterfactual quality. The MLP rows use f=MLP, so the Table 3 gap (0.43–0.89 vs 0.03–0.09) compares the ease of flipping two different decision surfaces; Table 2 shows SPN entropy (0.756–2.233) far above MLP (0.359–0.365) in those regimes. The paper concedes the mechanism: 'the MLP's low entropy ... is more challenging to modify in the latent space.' Hence the headline robustness conclusion is self-referential: the SPN is both generator and judge.
full rationale
The paper's central empirical claim—that SPN-guided counterfactuals retain moderate validity under weak VAE regularization while the MLP baseline collapses—is partially circular because the validity metric (Eq. 3) uses the very classifier that the counterfactual objective (Eq. 2) optimizes. For SPN counterfactuals, f in Eq. 3 is the SPN, and Eq. 2 maximizes the SPN's own class posterior; the reported validity therefore tracks how easily the SPN's soft decision boundary is crossed, not an independent measure of counterfactual quality. This is confirmed by Table 2, where the SPN has substantially higher entropy than the MLP in the weak-regularization regimes where the validity gap appears, and by the paper's own admission in Section 4.4 that the MLP's low entropy is more challenging to modify. However, this is not a full circularity: the comparison with the MLP is an independent empirical observation, the FID and L2 metrics use the external Inception V3 feature extractor and input-space distances, and no self-citation or imported uniqueness theorem is load-bearing. The core methodology (SPN as latent descriptor and classifier) has content beyond the metric definition, but the headline robustness conclusion is weakened by the self-referential validity evaluation.
Assumptions & free parameters
free parameters (5)
- beta_1 (KLD regularization weight) =
{0.1, 0.01, 0.001, 0.0001}
- beta and gamma (counterfactual objective weights) =
{0, 1}
- beta_0 and beta_2 (reconstruction and classification weights) =
not reported
- Latent dimension =
62
- R (number of replicate samples) =
not reported
assumptions (6)
- standard math The VAE ELBO with reparameterization provides a valid training objective for the latent space (Section 3.1).
- standard math SPN completeness and decomposability ensure exact and tractable probabilistic inference (Section 3.2).
- domain assumption LearnSPN learns a structure that accurately models the VAE latent distribution from finite samples (Section 3.2).
- domain assumption The latent space of the VAE is smooth enough that gradient ascent on p(y_cf|z) yields images that are plausible and recognizable (Section 3.3).
- domain assumption FID computed with Inception V3 embeddings is a valid measure of counterfactual plausibility (Section 4.2).
- domain assumption CheXpert labels for cardiomegaly and no finding are treated as ground truth (Section 4.1).
Cite this review
Pith. "Pith review of Counterfactual Explanations in Medical Imaging: Exploring SPN-Guided Latent Space Manipulation." pith.science (2026). https://pith.science/paper/VRIXD5AP
@misc{pith2026250719368,
author = {Pith},
title = {Pith review of: Counterfactual Explanations in Medical Imaging: Exploring SPN-Guided Latent Space Manipulation},
year = {2026},
howpublished = {\url{https://pith.science/paper/VRIXD5AP}},
note = {Machine review of arXiv:2507.19368}
}
read the original abstract
Artificial intelligence is increasingly leveraged across various domains to automate decision-making processes that significantly impact human lives. In medical image analysis, deep learning models have demonstrated remarkable performance. However, their inherent complexity makes them black box systems, raising concerns about reliability and interpretability. Counterfactual explanations provide comprehensible insights into decision processes by presenting hypothetical "what-if" scenarios that alter model classifications. By examining input alterations, counterfactual explanations provide patterns that influence the decision-making process. Despite their potential, generating plausible counterfactuals that adhere to similarity constraints providing human-interpretable explanations remains a challenge. In this paper, we investigate this challenge by a model-specific optimization approach. While deep generative models such as variational autoencoders (VAEs) exhibit significant generative power, probabilistic models like sum-product networks (SPNs) efficiently represent complex joint probability distributions. By modeling the likelihood of a semi-supervised VAE's latent space with an SPN, we leverage its dual role as both a latent space descriptor and a classifier for a given discrimination task. This formulation enables the optimization of latent space counterfactuals that are both close to the original data distribution and aligned with the target class distribution. We conduct experimental evaluation on the cheXpert dataset. To evaluate the effectiveness of the integration of SPNs, our SPN-guided latent space manipulation is compared against a neural network baseline. Additionally, the trade-off between latent variable regularization and counterfactual quality is analyzed.
Figures
Reference graph
Works this paper leans on
-
[1]
Matan Atad, Vitalii Dmytrenko, Yitong Li, Xinyue Zhang, Matthias Keicher, Jan Kirschke, Bene Wiestler, Ashkan Khakzar, and Nassir Navab. 2022. CheXplaining in Style: Counterfactual Explanations for Chest X-rays using StyleGAN. arXiv:2207.07553 [eess.IV] https: //arxiv.org/abs/2207.07553
arXiv 2022
-
[2]
Cory J. Butz, Jhonatan S. Oliveira, André E. dos Santos, and André L. Teixeira. 2019. Deep Convolutional Sum-Product Networks. Proceed- ings of the AAAI Conference on Artificial Intelligence 33, 01 (Jul. 2019), Counterfactual Explanations in Medical Imaging BIOKDD 2025, August 03, 2025, Toronto, Canada 3248–3255. doi:10.1609/aaai.v33i01.33013248
-
[3]
Joseph Paul Cohen, Rupert Brooks, Sovann En, Evan Zucker, Anuj Pa- reek, Matthew P. Lungren, and Akshay Chaudhari. 2021. Gifsplanation via Latent Shift: A Simple Autoencoder Approach to Counterfactual Generation for Chest X-rays. InProceedings of the Fourth Conference on Medical Imaging with Deep Learning (Proceedings of Machine Learning Research, Vol. 14...
work page 2021
-
[4]
Adnan Darwiche. 2002. A Logical Approach to Factoring Belief Net- works. In International Conference on Principles of Knowledge Repre- sentation and Reasoning. https://api.semanticscholar.org/CorpusID: 14032494
work page 2002
-
[5]
Adnan Darwiche. 2003. A differential approach to inference in Bayesian networks. J. ACM 50, 3 (May 2003), 280–305. doi:10.1145/ 765568.765570
arXiv 2003
-
[6]
Fabio De Sousa Ribeiro, Tian Xia, Miguel Monteiro, Nick Pawlowski, and Ben Glocker. 2023. High Fidelity Image Counterfactuals with Probabilistic Causal Models. In Proceedings of the 40th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 202). PMLR, 7390–7425. https://proceedings.mlr.press/ v202/de-sousa-ribeiro23a.html
work page 2023
-
[7]
Robert Gens and Domingos Pedro. 2013. Learning the Structure of Sum- Product Networks. In Proceedings of the 30th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 28). PMLR, Atlanta, Georgia, USA, 873–880. https://proceedings.mlr.press/ v28/gens13.html
work page 2013
-
[8]
Riccardo Guidotti. 2024. Counterfactual explanations and how to find them: literature review and benchmarking. Data Mining and Knowledge Discovery 38, 5 (2024), 2770–2824
work page 2024
Show all 30 references
-
[9]
Victor Guyomard, Françoise Fessant, Tassadit Bouadi, and Thomas Guyet. 2021. Post-hoc Counterfactual Generation with Supervised Autoencoder. In Machine Learning and Principles and Practice of Knowl- edge Discovery in Databases . Springer International Publishing, Cham, 105–114
2021
-
[10]
Anselm Haselhoff, Kevin Trelenberg, Fabian Küppers, and Jonas Schneider. 2024. The Gaussian Discriminant Variational Autoencoder (GdVAE): A Self-explainable Model with Counterfactual Explanations. In Computer Vision – ECCV 2024: 18th European Conference, Milan, Italy, Septembe...
2024 doi
-
[11]
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems 30 (2017)
2017
-
[12]
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner
-
[13]
Mong, Safwan S
Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea- Ilcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, Jayne Seekins, David A. Mong, Safwan S. Hal- abi, Jesse K. Sandberg, Ricky Jones, David B. Larson, Curtis P. Lan- glotz, Bh...
2019
-
[14]
Guillaume Jeanneret, Loïc Simon, and Frédéric Jurie. 2023. Adversarial counterfactual visual explanations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 16425–16435
2023
-
[15]
Bhagirath Athresh Karanam, Saurabh Mathur, Predrag Radivojac, David M Haas, Kristian Kersting, and Sriraam Natarajan. 2022. Explain- ing Deep Tractable Probabilistic Models: The sum-product network case. In Proceedings of The 11th International Conference on Probabilistic Grap...
2022
-
[16]
Mohamadsadegh Khosravani and Sandra Zilles. 2024. Using Sum- Product Networks to Assess Uncertainty in Deep Active Learning. Transactions on Machine Learning Research (2024). https://openreview. net/forum?id=Ai9XpjGxjl
2024
-
[17]
Durk P Kingma, Shakir Mohamed, Danilo Jimenez Rezende, and Max Welling. 2014. Semi-supervised learning with deep generative models. Advances in neural information processing systems 27 (2014)
2014
-
[18]
Diederik P Kingma and Max Welling. 2022. Auto-Encoding Variational Bayes. arXiv:1312.6114 [stat.ML] https://arxiv.org/abs/1312.6114
2022 arXiv
-
[19]
Alejandro Molina, Antonio Vergari, Karl Stelzner, Robert Peharz, Pranav Subramani, Nicola Di Mauro, Pascal Poupart, and Kristian Kersting. 2019. Spflow: An easy and extensible library for deep probabilistic learning using sum-product networks. arXiv preprint arXiv:1901.03704 (2019)
2019 arXiv
-
[20]
Jiří Němeček, Tomáš Pevn `y, and Jakub Marecek. 2025. Generating Likely Counterfactuals Using Sum-Product Networks. In The Thir- teenth International Conference on Learning Representations
2025
-
[21]
Robert Peharz, Sebastian Tschiatschek, Franz Pernkopf, and Pedro Domingos. 2015. On Theoretical Properties of Sum-Product Net- works. In Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learn- ing Research...
2015
-
[22]
Robert Peharz, Antonio Vergari, Karl Stelzner, Alejandro Molina, Xi- aoting Shao, Martin Trapp, Kristian Kersting, and Zoubin Ghahra- mani. 2020. Random Sum-Product Networks: A Simple and Effective Approach to Probabilistic Deep Learning. In Proceedings of The 35th Uncertainty...
2020
-
[23]
Hoifung Poon and Pedro Domingos. 2011. Sum-product networks: A new deep architecture. In 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops). 689–690. doi:10.1109/ ICCVW.2011.6130310
2011
-
[24]
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. 2014. Stochastic backpropagation and approximate inference in deep gener- ative models. In International conference on machine learning . PMLR, 1278–1286
2014
-
[25]
Sumedha Singla, Motahhare Eslami, Brian Pollack, Stephen Wal- lace, and Kayhan Batmanghelich. 2023. Explaining the black-box smoothly—A counterfactual approach. Medical Image Analysis 84 (2023), 102721. doi:10.1016/j.media.2022.102721
2023
-
[26]
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition . 2818–2826
2016
-
[27]
Jos van de Wolfshaar and Andrzej Pronobis. 2020. Deep Gener- alized Convolutional Sum-Product Networks. In Proceedings of the 10th International Conference on Probabilistic Graphical Models (Pro- ceedings of Machine Learning Research, Vol. 138) . PMLR, 533–544. https://proceed...
2020
-
[28]
Payal Varshney, Adriano Lucieri, Christoph Balada, Andreas Dengel, and Sheraz Ahmed. 2025. Generating counterfactual trajectories with BIOKDD 2025, August 03, 2025, Toronto, Canada Julia Siekiera and Stefan Kramer latent diffusion models for concept discovery. In International...
2025
-
[29]
Antonio Vergari, Nicola Di Mauro, and Floriana Esposito. 2019. Visu- alizing and Understanding Sum-Product Networks. Machine Learning 108 (04 2019). doi:10.1007/s10994-018-5760-y
2019 doi
-
[2017]
In International conference on learning repre- sentations
beta-vae: Learning basic visual concepts with a constrained variational framework. In International conference on learning repre- sentations
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.