{"id":"62886aa4-d1ec-4c46-be73-6975a3634e3f","arxiv_id":"2507.19368","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A Sum-Product Network fitted to a VAE latent space acts as both density model and classifier, producing counterfactual chest X-ray images that preserve the original anatomy while altering the predicted diagnosis.","lead":"This paper proposes a counterfactual explanation method for chest X-ray classifiers: it trains a VAE whose latent space is modeled by a Sum-Product Network, then optimizes latent vectors to flip the model's prediction. The authors report that the SPN-based approach flips predictions more reliably than an MLP baseline when latent regularization is weak.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 3 validity gap is confounded: each method is evaluated with its own classifier, and the gap tracks classifier entropy rather than counterfactual quality.","rationale":"The reader's weakest assumption is precisely the load-bearing concern. The empirical case for SPN robustness is built on Table 3's validity column, and that column is computed with method-specific classifiers. Because Eq. 2 optimizes the same classifier's log-probability, a 'valid' counterfactual is almost tautologically one that crosses that classifier's boundary; the metric therefore conflates decision-boundary sharpness with counterfactual quality. Table 2 documents that the SPN is much less confident exactly in the weak-regularization regimes where the validity gap is largest, making the entropy confound the most plausible explanation for the reported difference. A fixed-classifier re-evaluation is the minimal experiment that would either rescue or refute the claim; if the advantage disappears, the conclusion should be reframed as 'SPN counterfactuals flip the SPN classifier more often' rather than 'SPN generates better counterfactuals.' The paper's other contributions—the SPN-in-VAE architecture, the public code, and the honest discussion of metric limitations—are not in question, and the authors' proposed improvements to L2 and FID are reasonable. These strengths justify keeping the reader's conditional verdict; the concern is addressable and does not require rejection, but it must be addressed before the robustness claim can be accepted.","tokens_in":12310,"tokens_out":5033,"duration_ms":49240,"concrete_test":"Recompute Table 3 validity with a single fixed classifier for both methods: use the trained MLP classifier as f in Eq. 3 for both SPN-generated and MLP-generated counterfactuals at beta_1 = 0.01, 0.001, and 0.0001. If SPN validity drops to MLP levels when the target classifier is held fixed, the reported robustness advantage is an artifact of evaluating each method with its own more-mutable classifier. Repeat with the SPN as the fixed classifier as a robustness check, and report per-fold standard deviations for both.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central robustness claim rests on Table 3, but validity in Eq. 3 uses f equal to the classifier that generated the counterfactual: the SPN for SPN counterfactuals and the MLP for MLP counterfactuals. Since Eq. 2 maximizes log p(y_cf|z') with the same f, validity measures how often each method's optimization crosses its own classifier's decision boundary, not an objective quality of the counterfactual. Table 2 shows the SPN has much higher predictive entropy than the MLP in the weak-regularization regimes where the validity gap appears (SPN entropy 0.756-2.233 vs MLP 0.359-0.365 for beta_1 = 0.01, 0.001, 0.0001), so the SPN boundary is softer and easier to flip. The paper even acknowledges this mechanism in Section 4.4 ('the MLP's low entropy... is more challenging to modify') but does not control for it. Consequently, the headline conclusion that the SPN 'consistently maintains at least moderate validity... demonstrating larger robustness' is not currently supported as a statement about counterfactual quality; it may simply reflect classifier uncertainty. The absence of error bars is secondary; the self-referential validity definition is the load-bearing flaw.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a counterfactual explanation pipeline for medical images in which a sum-product network (SPN) is trained on the latent space of a semi-supervised VAE and then used both as a class-posterior model and as a likelihood descriptor for latent-space counterfactual search. After VAE training, the MLP classifier is replaced by a structure-learned Gaussian SPN, and counterfactual latent codes are obtained by maximizing the SPN's target-class log-posterior while penalizing squared distance and log-likelihood deviation (Eq. 2). Experiments on a balanced binary CheXpert subset (cardiomegaly vs. no finding) compare the SPN with the MLP baseline under four KLD strengths beta_1 and with different combinations of beta and gamma. The headline result is that under weak regularization the MLP validity drops to 0.03-0.09 while the SPN remains at 0.43-0.89, which the paper interprets as greater robustness of SPN-guided manipulation; the paper also reports L2 proximity, FID plausibility, and alteration visualizations.","tokens_in":12583,"tokens_out":7588,"duration_ms":68364,"significance":"The proposed integration of tractable SPN inference with VAE latent space manipulation is a sensible and potentially useful direction for probabilistic counterfactual explanations, and the released code and detailed experimental setup are assets. If the robustness claim were established, the paper would provide a practical demonstration that probabilistic classifiers can support counterfactual search under weak latent regularization. However, the current evidence does not establish the claimed cross-method advantage, because the validity metric is evaluated with the same classifier that generates the counterfactual and the comparison is not controlled for classifier confidence; the absence of uncertainty estimates further weakens the empirical claims.","major_comments":[{"comment":"The central robustness claim is confounded by the self-referential validity measure. In Eq. (3), f is the SPN for SPN counterfactuals and the MLP for MLP counterfactuals, and Eq. (2) maximizes log p(y_cf|z') under the same f. Validity therefore measures how often each method's own optimization pushes latent codes across its own decision boundary, not an independent property of the generated counterfactuals. Table 2 shows that in the weak-regularization regimes where the validity gap appears, the SPN has much higher predictive entropy than the MLP (0.756-2.233 vs. 0.359-0.365), and Section 4.4 explicitly attributes the MLP's difficulty to 'the MLP's low entropy... more challenging to modify'. Hence the higher SPN validity may reflect a softer, less confident classifier rather than better counterfactuals. To support the claim that SPN-guided manipulation is more robust, the authors should evaluate both sets of counterfactuals with a fixed independent classifier, and/or report validity under each classifier in both directions, and/or control for predictive entropy or calibration. This is the main load-bearing issue for the paper's headline conclusion.","section":"Section 4.4, Eq. (3); Tables 2-3"},{"comment":"The FID formula is not the standard Frechet Inception Distance. The correct expression is ||mu_org - mu_cf||^2 + Tr(Sigma_org + Sigma_cf - 2(Sigma_org Sigma_cf)^{1/2}); the manuscript writes the cross-term as -2Tr(Sigma_org Sigma_cf), which is the trace of a matrix product and not a valid distance statistic. The reported FID values should be recomputed with the correct formula before they are used to support any claims about plausibility.","section":"Section 4.2, Eq. (4)"},{"comment":"All counterfactual statistics are reported as means over the three cross-validation splits without standard deviations, confidence intervals, or significance tests. For the paper's central comparison, the reader cannot tell whether gaps such as 0.55 vs. 0.09 are stable across splits or driven by a single fold; please report per-split results and appropriate uncertainty measures, and test the validity differences when the confound in the first comment is addressed.","section":"Section 4.1, Table 3"}],"minor_comments":[{"comment":"The text says 'Only for beta_1 = 0.1, the SPN achieves an improvement... However, for smaller beta_1 = 0.1, the MLP outperforms the SPN.' The phrase 'smaller beta_1 = 0.1' should be corrected to 'beta_1 < 0.1' or 'smaller values of beta_1.'","section":"Section 4.3"},{"comment":"The text states that the MLP baseline is run with gamma in {0,1}, but Table 3 lists MLP only for gamma = 0.0; please specify which configurations were actually run and why gamma = 1.0 is omitted for the MLP.","section":"Section 4.1, Table 3"},{"comment":"The text mentions 'for both datasets,' but the experiments use only the CheXpert dataset; please remove or correct this phrase.","section":"Section 4.3"},{"comment":"The validity indicator 1[f(x) != f(x_cf)] counts any prediction change, not necessarily a flip to the targeted counterfactual class y_cf; please use 1[f(x_cf) = y_cf] or report both quantities.","section":"Section 4.2, Eq. (3)"},{"comment":"The 'switch epoch' metric is reported in Table 3 but is not defined in the metrics section; please define how it is computed.","section":"Section 4.2, Table 3"},{"comment":"The reported entropy values exceed 1.0 for binary classification, which suggests a particular logarithm base or averaging convention; please specify the base and clarify whether this is per-class averaged entropy.","section":"Table 2"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look if you work at the SPN/XAI intersection. The paper's new bit is a genuine combination: train a semi-supervised VAE on chest X-rays, replace the latent classifier with a structure-learned SPN, then use the SPN as both density model and classifier to guide latent-space counterfactuals. The method is clearly described, the code is linked, and the authors are upfront about limitations.\n\nThe headline robustness claim, however, is not yet supported. Table 3 shows SPN validity holding at 0.43–0.89 under weak VAE regularization while the MLP falls to 0.03–0.09. But validity (Eq. 3) uses the same classifier that generated the counterfactual: for SPN counterfactuals, f is the SPN; for MLP counterfactuals, f is the MLP. Since the objective (Eq. 2) maximizes log p(y_cf|z') with that same f, the metric mostly tells you how often each method crosses its own decision boundary. Table 2 shows the SPN has much higher predictive entropy in exactly the weak-regularization regimes where the gap appears (0.756–2.233 vs 0.359–0.365 for beta_1 = 0.01, 0.001, 0.0001). A softer boundary flips more easily. The authors even note this mechanism in Section 4.4 ('the MLP's low entropy... is more challenging to modify') but do not control for it. So the claim that the SPN demonstrates 'larger robustness' is not a stable statement about counterfactual quality yet.\n\nThis is fixable and not fatal. Use a fixed external classifier to define validity, or match entropy/confidence between the two models. Error bars would help; their absence is secondary. A comparison with an existing counterfactual method (latent shift, GDVAE) would strengthen the empirical case, though this is a workshop paper and the authors already do a careful ablation of beta and gamma.\n\nThe paper deserves a serious referee because the combination is new and the evaluation can be repaired. As it stands, I would not accept it without a reworked validity analysis. Who gets value: the SPN community and anyone building probabilistic latent-space XAI for medical images.\n\nSend it to review, but expect heavy revision.","headline":"New SPN-in-VAE combination for chest X-ray counterfactuals, but the robustness claim rests on a self-referential validity metric that tracks classifier entropy more than counterfactual quality.","tokens_in":13129,"tokens_out":3631,"would_cite":false,"duration_ms":30829,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A sum-product network in the VAE latent space keeps chest X-ray counterfactuals valid when regularization is weak.","keywords":["counterfactual explanations","sum-product networks","variational autoencoder","latent space manipulation","chest X-ray classification","explainable AI","KLD regularization"],"falsifier":"Recompute validity with a single fixed referee classifier, such as the original MLP or a third independently trained model, labeling both SPN-generated and MLP-generated counterfactuals; if the SPN's validity advantage disappears or reverses, the robustness claim fails. A supporting check is to temperature-scale both classifiers to matching entropy and see whether the validity gap vanishes.","tokens_in":12086,"feed_emoji":"🩻","tokens_out":11016,"duration_ms":90686,"temperature":0.7,"pith_summary":"This paper tries to make counterfactual explanations for a chest X-ray classifier more reliable by describing the VAE's latent space with a sum-product network (SPN). The SPN doubles as the density model of the latent codes and as the classifier, so the search for a counterfactual code can maximize the target-class probability while discouraging large or unlikely moves. On a balanced cardiomegaly-versus-no-finding task, the authors report that when the latent-space regularization is weak, SPN-guided edits flip the prediction in 43-89% of test cases, whereas an MLP baseline flips only 3-9%. The practical interest is that weakly regularized VAEs reconstruct images more faithfully, and the SPN appears to preserve counterfactual validity exactly in the regime where the standard neural classifier fails.","feed_headline":"SPN-guided latent edits keep chest X-ray counterfactuals valid","feed_subtitle":"Weakly regularized VAEs reconstruct better; the SPN still flips cardiomegaly cases 43-89% of the time, an MLP only 3-9%.","key_machinery":"The central mechanism is a class-conditional Gaussian sum-product network: a rooted directed acyclic graph whose leaves are univariate Gaussian densities and whose internal nodes are weighted sums and products, learned from latent codes by a recursive structure-learning algorithm. Because the network is complete and decomposable, it gives exact, tractable values of $p_{\\mathrm{SPN}}(z)$ and $p_{\\mathrm{SPN}}(y|z)$, so one model supplies the classifier term, the density term, and the class-conditioned likelihood used in the latent-space objective. The learned network is converted into a differentiable neural-network form so that the counterfactual code $z_{\\mathrm{cf}}$ can be optimized by gradient descent and then decoded into an image.","core_discovery":"The authors claim that a structure-learned sum-product network placed over the latent space of a semi-supervised VAE (a VAE trained so its latent codes also predict the class) can serve as both the latent-space density descriptor and the classifier for counterfactual generation. Starting from a sampled latent code $z$ for an image, they search for $z_{\\mathrm{cf}}$ by gradient descent to maximize $\\log p_{\\mathrm{SPN}}(y_{\\mathrm{cf}}|z')$, with optional penalties $\\beta\\|z'-z\\|^2$ and $\\gamma|\\log p_{\\mathrm{SPN}}(z')-\\log p_{\\mathrm{SPN}}(z)|$, then decode $z_{\\mathrm{cf}}$ into a counterfactual image. Their empirical claim is that this SPN-guided manipulation maintains validity, the fraction of counterfactuals whose predicted class changes, under weak Kullback-Leibler divergence (KLD) regularization ($\\beta_1$ from 0.01 down to 0.0001), while the MLP baseline's validity collapses to near zero. The auxiliary $\\beta$ and $\\gamma$ weights have little effect, and the visualized counterfactual changes concentrate around the heart region.","pith_inferences":["Editorial: the validity comparison is not controlled for classifier confidence; the SPN's higher predictive entropy at low $\\beta_1$ means its flip rate may reflect softer decisions, so a fixed referee classifier should be used before concluding the SPN produces better counterfactuals.","Editorial: because the $\\beta$ and $\\gamma$ penalties barely changed outcomes, a natural next step is to use the SPN log-likelihood itself as the distance penalty, replacing the Euclidean term with a distribution-aware measure.","Editorial: the result suggests a cheap diagnostic for any latent-space counterfactual method: report predictive entropy alongside validity, since confident classifiers can make otherwise identical edits look invalid.","Editorial: transferring the approach to other pathologies or multiclass labels would require class-specific FID and $\\ell^2$ metrics, because absolute reconstruction metrics are dominated by VAE quality."],"forward_implications":["If the SPN robustness holds, weakly regularized VAEs, which reconstruct finer image detail, can still support counterfactual explanation without losing the ability to flip predictions.","The small effect of $\\beta$ and $\\gamma$ implies that the target-class probability term, not the proximity or likelihood penalties, is the main driver of the optimized latent shift.","Absolute $\\ell^2$ and FID scores track reconstruction quality more than counterfactual quality, so comparisons across VAE regularizations need relative or class-conditioned versions of these metrics.","SPN-guided explanations localize changes around the heart across regularization settings, while MLP-guided explanations do so only under strong regularization, which matters if explanations are meant to point clinicians to disease-relevant regions."],"supporting_citations":[{"why":"introduces sum-product networks and the completeness/decomposability conditions that enable exact inference.","marker":"[23]"},{"why":"supplies the recursive structure-learning algorithm that builds the SPN from latent codes.","marker":"[7]"},{"why":"provides the implementation for learning the SPN structure and converting it into a differentiable network.","marker":"[19]"},{"why":"introduces the variational autoencoder and its evidence lower bound objective.","marker":"[18]"},{"why":"extends VAEs to semi-supervised learning by adding a classifier on the latent representation.","marker":"[17]"},{"why":"supplies the beta-VAE weighting scheme used to tune the KLD regularization term.","marker":"[12]"},{"why":"provides the chest X-ray dataset and the cardiomegaly/no-finding labels used in the evaluation.","marker":"[13]"},{"why":"frames the validity, proximity, and plausibility metrics used to evaluate counterfactuals.","marker":"[8]"},{"why":"shows how SPNs can be trained as constrained neural networks, supporting the differentiable conversion.","marker":"[22]"}],"fun_headline_variants":["Weak VAE regularization? SPN still flips 43-89% of X-ray cases","SPN doubles as latent classifier for robust X-ray counterfactuals","SPN beats MLP for valid counterfactuals in weakly regularized VAEs","SPN latent guide keeps X-ray counterfactuals valid under weak VAE"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that validity measured by each method's own classifier is a fair comparison; if the SPN merely has less confidence in its predictions, its higher flip rate would not mean better counterfactuals.","fun_headline_variants_meta":{"raw":{"variants":["Weak VAE regularization? SPN still flips 43-89% of X-ray cases","SPN doubles as latent classifier for robust X-ray counterfactuals","SPN beats MLP for valid counterfactuals in weakly regularized VAEs","SPN latent guide keeps X-ray counterfactuals valid under weak VAE"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001375,"raw_usage":{"total_tokens":5638,"prompt_tokens":1075,"completion_tokens":4563,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":691,"completion_tokens_details":{"reasoning_tokens":4477}},"tokens_in":691,"tokens_out":4563,"duration_ms":27982,"temperature":1.0,"reasoning_tokens":4477,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:53:46.115852+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute validity with a single fixed referee classifier, such as the original MLP or a third independently trained model, labeling both SPN-generated and MLP-generated counterfactuals; if the SPN's validity advantage disappears or reverses, the robustness claim fails. A supporting check is to temperature-scale both classifiers to matching entropy and see whether the validity gap vanishes.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"extends VAEs to semi-supervised learning by adding a classifier on the latent representation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"frames the validity, proximity, and plausibility metrics used to evaluate counterfactuals."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"introduces sum-product networks and the completeness/decomposability conditions that enable exact inference."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the recursive structure-learning algorithm that builds the SPN from latent codes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"shows how SPNs can be trained as constrained neural networks, supporting the differentiable conversion."}],"review_version":2}