REVIEW 4 major objections 5 minor 26 references
BetaExplainer: A Probabilistic Method to Explain Graph Neural Networks
T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read BetaExplainer frames GNN edge explanation as posterior inference over a Beta-distributed edge mask, returning ranked importance scores with uncertainty and improving explanation fidelity on heterophilic and sparse-feature graphs.
desk verdict A promising idea with a shaky variational derivation and a too-circular evaluation metric; worth a revision, not acceptance as is. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the probabilistic edge mask $M_{ij}$ with a Beta prior $\mathrm{Beta}(\alpha,\beta)$ per edge, learned through mean-field variational inference. Each mask entry controls whether edge $(i,j)$ is retained in the graph $\mathbf{G}_s$ fed to the trained GNN, and the loss is the KL divergence between the GNN output on the masked graph and the original output; optimizing the ELBO is claimed to minimize this divergence. The Beta distribution is chosen because it is the conjugate prior for the Bernoulli likelihood in Eq. 1, and its two shape parameters let the user encode prior beliefs, which the paper argues is what lets the method adapt to heterophilic and sparse-feature graphs.
What would settle it
Run BetaExplainer on a small graph with a known ground-truth edge, set the constant in Eq. 1 to 0.5, and check whether maximizing the stated ELBO recovers that edge and whether the learned posterior separates true from false edges at the 0.5 boundary; then compute both sides of Eq. 5 to verify that the claimed lower-bound identity actually holds numerically.
Extended reading notes
Core claim
The central claim is that a GNN explanation can be obtained by posterior inference over a binary edge mask, rather than by direct optimization of a mask or by subgraph search. BetaExplainer places an independent Beta prior on each edge's mask value, defines a Bernoulli-like likelihood comparing the trained GNN's output on the masked graph with its output on the original graph, and fits the posterior with black-box variational inference. The learned posterior mean provides a ranked importance score, and the spread of the distribution provides an uncertainty estimate that neither GNNExplainer nor SubgraphX offers. On the paper's simulated benchmarks this probabilistic mask yields significantly better Jaccard and F1 scores on some datasets, comparable accuracy on sparse-feature graphs with significantly better F1, and lower unfaithfulness on most datasets, indicating that the subgraph selected by probability matches the ground-truth motif better than the baselines.
Load-bearing premise
The method assumes that the simple Bernoulli-style likelihood with a fixed 0.5 threshold and an unspecified constant is a valid description of how well a masked graph reproduces the GNN's output, and that the variational objective used for training is a genuine lower bound that can be optimized in this form.
Editorial extensions
If this is right
- If BetaExplainer's edge scores are reliable, users can threshold the posterior probabilities to trade precision against recall, choosing a smaller set of high-confidence edges for costly downstream validation.
- Because the method returns a distribution rather than a point mask, repeated runs can be aggregated into per-edge confidence intervals, giving an explanation that is stable across random seeds.
- The improved unfaithfulness on heterophilic graphs suggests the same probabilistic masking strategy can be applied where explainers typically fail, such as protein-structure graphs with mixed node types.
- On sparse gene-expression graphs, better F1 at fixed accuracy means BetaExplainer can highlight a larger fraction of true regulatory interactions without flooding the user with false positives.
- Batched training should make BetaExplainer practical on whole-graph datasets, since the paper shows batching cuts runtime dramatically without changing the reported metric patterns.
Reading between the lines
- The paper does not test calibration of the uncertainty scores; an editor would hypothesize that the posterior probability separates true from false edges by rank, and that a user could set a threshold to target a desired false-positive rate - a testable extension.
- Because the objective is KL divergence between GNN outputs, the same Beta-mask framework could be extended to explain node-level predictions or to produce counterfactual edge deletions, though the paper only evaluates graph-level explanations.
- The fixed 0.5 threshold in the likelihood is a discretization choice; a continuous relaxation might improve separation on datasets where the importance distribution is not symmetric around the threshold.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes BetaExplainer, a method intended to explain graph neural network predictions by learning a Beta-distributed probabilistic edge mask. The stated goal is to provide edge importance scores with uncertainty quantification and to improve explanation accuracy over GNNExplainer and SubgraphX, particularly on simulated datasets with challenging properties such as heterophily, fairness-related feature flips, low information content, and sparse node features. The authors present a variational-inference formulation (Eqs. 1–5), an algorithm in Section 2.1, and experiments on ShapeGGen and SERGIO datasets comparing Jaccard index, F1 score, accuracy, and unfaithfulness. The paper reports that BetaExplainer outperforms baselines on several metrics, especially unfaithfulness, and provides uncertainty scores for edges.
Significance. If the proposed method were correctly formulated, the contribution would be practically useful: a probabilistic edge mask with uncertainty quantification could aid hypothesis generation in domains such as gene-regulatory-network analysis, and the inclusion of sparse-feature datasets addresses a realistic gap in GNN explainability benchmarks. The code is publicly available under an open-source license, which is a strength. However, the significance cannot be assessed from the manuscript as written because the variational derivation is mathematically invalid, the training objective is not well defined, and the main unfaithfulness metric is circular with respect to the training loss. The independent metrics (F1, Jaccard) show only partial advantages, and these are weakened by per-dataset hyperparameter selection. The central claims therefore remain unsubstantiated.
major comments (4)
- [§2.1, Eq. (1)] The likelihood in Eq. (1) is not a valid likelihood for the observed GNN output. It defines P(f(X,E)|M) as p when M ≥ 0.5 and 1−p otherwise, but p is never specified and the expression has no dependence on the actual value of f(X,E). A likelihood that does not depend on the observation cannot update the posterior toward edges that drive the prediction; the posterior would be a fixed reweighting of the Beta prior. This is a load-bearing error because the paper's claimed uncertainty quantification is derived from this likelihood.
- [§2.1, Eqs. (3)–(5)] Equations (3)–(5) do not constitute an evidence lower bound or a variational inference objective. Eq. (3) omits the variational distribution q, contains a mismatched parenthesis, and is written as an expectation of a difference of log-probabilities without the standard ELBO structure. Eq. (5) equates a difference between a log-probability and the ELBO with a KL divergence term that follows from no standard variational identity, and the stated nonnegativity is unsupported. As written, the training objective in Algorithm 1 cannot be derived from variational inference. The code may implement a valid objective, but the manuscript does not provide it.
- [§2.3.3, Eq. (9) and §3.1] The unfaithfulness metric in Eq. (9) is 1−exp(−KL(f(X,G)||f(X,G_s))), while BetaExplainer's stated training objective is to minimize the KL divergence between the GNN output on the masked graph and the original output. The paper itself acknowledges this in Section 3.1: 'BetaExplainer optimizes the KL divergence between the GNN output on the masked graph and the original output, which is also measured by the unfaithfulness metric. This likely explains the decreased unfaithfulness score.' Consequently, the reported improvements in unfaithfulness are circular and do not independently validate the quality of the explanations.
- [§2.3.2, Table 2] The alpha and beta prior parameters are selected per dataset 'based on the most balanced results across metrics' (Section 2.3.2). Because these parameters directly control the sparsity and shape of the learned mask, tuning them on the evaluation metrics and then reporting improvements on those same metrics raises selection-overfitting concerns. The paper does not provide a validation protocol, sensitivity analysis, or held-out hyperparameter selection, which weakens the generality of the empirical claims.
minor comments (5)
- [§1, Abstract] The abstract says 'we proposed BetaExplainer' while the introduction says 'We propose'; the tense should be consistent. The last paragraph of the Introduction also contains 'have evaluated' without a subject, likely 'we have evaluated'.
- [§2.1, Eq. (2)] Equation (2) is a probability density, not a probability mass; the surrounding text refers to it as the 'probability that the mask importance for edge e_ij is value M_ij', which is imprecise and should be reworded.
- [Table 2] In the SERGIO 25% Sparsity row, the Type column contains the word 'Phenomenon', which appears to be a placeholder or typo; this should be corrected or explained.
- [References] Reference [21] lists page numbers '1241–12252', which appear to be a typo; the correct page range should be verified.
- [§3.2, Fig. 4] The discussion of SubgraphX returning no edges would benefit from explicitly stating that an empty explanation graph makes accuracy and F1 comparisons degenerate; the current text mentions this only indirectly.
Circularity Check
Circular evaluation: the unfaithfulness metric is the KL divergence BetaExplainer minimizes, so those gains are by construction; the printed likelihood also makes the posterior reduce to the prior.
-
self definitional
[Section 3.1 Results and Eq. 9 in Section 2.3.3]
"The final metric calculated is unfaithfulness [1], or 1−exp(−KL(f(X,G)||f(X,Gs))) (9) ... BetaExplainer optimizes the KL divergence between the GNN output on the masked graph and the original output, which is also measured by the unfaithfulness metric. This likely explains the decreased unfaithfulness score for BetaExplainer for most datasets and justifies the choice of our formulation."
Eq. 9 defines unfaithfulness as a monotone transform of the KL divergence between the GNN output on the original graph and on the masked graph. Algorithm 1 trains BetaExplainer by minimizing exactly this KL divergence, as the paper states when describing the loss between model output on the masked graph and original graph. Therefore, lower unfaithfulness is not an independent test of explanation quality; it is the training objective renamed as an evaluation metric. The paper's own sentence concedes that the objective explains the decreased score. The unfaithfulness improvements in Fig. 3c are thus forced by construction, although F1 and Jaccard comparisons against simulator ground truth remain non-circular.
-
other
[Section 2.1, Eq. 1 and Eqs. 3–5]
"the likelihood over the GNN or p(f(X,E)) is a Bernoulli distribution described by P(f(X,E)|M)= ( p if M≥0.5, 1−p if M<0.5. (1) ... Thus, the ELBO may be calculated as follows with respect to the original output: E[log(P(Gs,f(X,G)|M)− log(P(f(X,G)))] (3)"
In Eq. 1, the right-hand side depends only on the constant p and on whether M is above 0.5; the observed GNN output f(X,E) does not appear. Consequently, Bayes' rule gives P(M|f(X,E)) proportional to P(f(X,E)|M)P(M), which is just the Beta prior P(M) reweighted by the constants p and 1−p on the two halves of [0,1]. As written, the 'learned' posterior mask is therefore the input prior rescaled, not an explanation derived from the GNN's predictions. Eq. 3 omits the variational distribution q, and Eq. 5 asserts a KL decomposition that does not follow from any standard variational identity. The printed derivation chain thus reduces the claimed uncertainty quantification to the prior rather than grounding it in the GNN output.
full rationale
BetaExplainer's evaluation is partially circular: the unfaithfulness metric (Eq. 9) is a monotone transform of the KL divergence that Algorithm 1 explicitly minimizes, so the reported unfaithfulness improvements are forced by construction, and the paper itself acknowledges this. The F1 and Jaccard results against simulator ground truth are independent, non-circular comparisons, which keeps the circularity partial rather than total. A second, more fundamental issue in the printed derivation is that Eq. 1's likelihood has no dependence on the observed GNN output, so the posterior mask reduces to a reweighted Beta prior; Eq. 3 is not a valid ELBO and Eq. 5 does not establish the claimed KL identity. Per-dataset choices of alpha and beta (Table 2) are selected using the same metrics later reported; this is a tuning-selection concern rather than a derivation-level circularity. There is no load-bearing self-citation or imported uniqueness theorem: reference [3] shares authors but only supports biological motivation. Overall, one central evaluation claim reduces to the training objective, giving a score of 6.
Assumptions & free parameters
free parameters (3)
- p (likelihood constant, Eq. 1) =
unspecified
- alpha (Beta prior, per dataset) =
0.8, 0.7, 0.8, 0.6, 0.8, 0.55, 0.5 (7 datasets)
- beta (Beta prior, per dataset) =
0.6, 0.6, 0.6, 0.6, 0.8, 0.65, 0.95 (7 datasets)
assumptions (3)
- standard math Beta-Bernoulli conjugacy and mean-field variational inference apply to the edge mask posterior.
- domain assumption The ground-truth edges in ShapeGGen are exactly the house motifs that determine node class.
- domain assumption The correlation-threshold graph (0.35) from SERGIO is a valid input for explanation evaluation.
Cite this review
Pith. "Pith review of BetaExplainer: A Probabilistic Method to Explain Graph Neural Networks." pith.science (2026). https://pith.science/paper/FUM6PJN4
@misc{pith2026241211964,
author = {Pith},
title = {Pith review of: BetaExplainer: A Probabilistic Method to Explain Graph Neural Networks},
year = {2026},
howpublished = {\url{https://pith.science/paper/FUM6PJN4}},
note = {Machine review of arXiv:2412.11964}
}
read the original abstract
Graph neural networks (GNNs) are powerful tools for conducting inference on graph data but are often seen as "black boxes" due to difficulty in extracting meaningful subnetworks driving predictive performance. Many interpretable GNN methods exist, but they cannot quantify uncertainty in edge weights and suffer in predictive accuracy when applied to challenging graph structures. In this work, we proposed BetaExplainer which addresses these issues by using a sparsity-inducing prior to mask unimportant edges during model training. To evaluate our approach, we examine various simulated data sets with diverse real-world characteristics. Not only does this implementation provide a notion of edge importance uncertainty, it also improves upon evaluation metrics for challenging datasets compared to state-of-the art explainer methods.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
C. Agarwal, O. Queen, H. Lakkaraju, and M. Zitnik. Evaluating explainability for graph neural networks. Scientific Data, 10(144), 2023
work page 2023
-
[2]
F . Baldassarre and H. Azizpour . Explainability techniques for graph convolutional networks.International Conference on Machine Learning (ICML) W orkshops, 2019
work page 2019
-
[3]
J. Bigness, X. Loinaz, S. Patel, E. Larschan, and R. Singh. Integrating long-range regulatory interactions to predict gene expression using graph convolutional networks.Journal of Computational Biology, 29(5):409–422, 2022
work page 2022
-
[4]
G. A. Bouland, A. Mahfouz, and M. J. T. Reinders. Consequences and opportunities arising due to sparser single-cell rna-seq datasets.Genome Biol, 24(86), 2023. 12
work page 2023
-
[5]
P . Dibaeinia and S. Sinha. Sergio: A single-cell expression simulator guided by gene regulatory networks. Scientific Data, 11(3), 2020
work page 2020
-
[6]
W. Ju, S. Yi, Y. Wang, Z. Xiao, Z. Mao, H. Li, Y. Gu, Y. Qin, N. Yin, S. Wang, X. Liu, X. Luo, P . Yu, and M. Zhang. A survey of graph neural networks in real world: Imbalance, noise, privacy and ood challenges. arxiv, 2024
work page 2024
-
[7]
D. Kim, A. Tran, H. Kim, Y . Lin, J. Y ang, and P . Y ang. Gene regulatory network reconstruction: harnessing the power of single-cell multi-omic data.NPJ Systems Biology and Applications, 9(51), 2023
work page 2023
-
[8]
Z. Liu, G. W an, B. Prakash, M. Lau, and W . Jin. A review of graph neural networks in epidemic modeling. ACM Digital Library, pages 6577 – 6587, 2024
work page 2024
Show all 26 references
-
[9]
D. Luo, W. Cheng, D. Xu, W. Yu, B. Zong, H. Chen, and X. Zhang. Parameterized explainer for graph neural network. Advances in Neural Information Processing Systems, pages 19620–19631, 2020
2020
-
[10]
Perozzi, R
B. Perozzi, R. Al-Rfou, and S. Skiena. Deepwalk: Online learning of social representations.Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 701–710, 2014
2014
-
[11]
P . Pope, S. Kolouri, M. Rostami, C. Martin, and H. Hoffmann. Explainability methods for graph convolutional neural networks.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 10772–10781, 2019
2019
-
[12]
Prˇzulj, D
N. Prˇzulj, D. Corneil, and I. Jurisica. Modeling interactome: scale-free or geometric?Bioinformatics, 20(18):3508–3515, 2004
2004
-
[13]
Simonyan, A
K. Simonyan, A. V edaldi, and A. Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps.ICLR, 2024
2024
-
[14]
Sundararajan, A
M. Sundararajan, A. Taly, and Q. Y an. Axiomatic attribution for deep networks.ICML, page 3319–3328, 2017
2017
-
[15]
Vu and M
M. Vu and M. Thai. Pgm-explainer: Probabilistic graphical model explanations for graph neural networks. Advances in Neural Information Processing Systems, pages 12225–12235, 2020
2020
-
[16]
Waikhom and R
L. Waikhom and R. Patgiri. A survey of graph neural networks in various learning paradigms: methods, applications, and challenges.Artificial Intellegence Review, 56:6295–6364, 2023
2023
-
[17]
Z. Wu, S. Pan, F . Chen, G. Long, C. Zhang, and P . Y u. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 32(1):4–24, 2021
2021
-
[18]
A. Xenos. Simplifying complex machine learning by linearly separable network embedding spaces.arxiv, 2024
2024
-
[19]
R. Ying, D. Bourgeois, J. You, M. Zitnik, and J. Leskovec. Gnnexplainer: Generating explanations for graph neural networks.Advances in Neural Information Processing Systems, 2019
2019
-
[20]
Y uan, H
H. Y uan, H. Y u, S. Gui, and S. Ji. Explainability in graph neural networks: A taxonomic survey.IEEE, 2022
2022
-
[21]
H. Yuan, H. Yu, J. Wang, K. Li, and S. Ji. On explainability of graph neural networks via subgraph explo- rations. Proceedings of The 38th International Conference on Machine Learning, pages 1241–12252, 2021. 13
2021
-
[22]
Zhang, B
H. Zhang, B. Wu, X. Yuan, S. Pan, H. Tong, and J. Pei. Trustworthy graph neural networks: Aspects, methods, and trends.Proceedings of the IEEE, 112(2):97–139, 2024
2024
-
[23]
Zhang, B
H. Zhang, B. Wu, X. Yuan, S. Pan, H. Tong, and J. Pei. Trustworthy graph neural networks: Aspects, methods, and trends.Proceedings of the IEEE, 2024
2024
-
[24]
Zheng, Y
X. Zheng, Y. Wang, Y. Liu, M. Li, M. Zhang, D. Jin, P . Yu, and S. Pan. Graph neural networks for graphs with heterophily: A survey.arvix, 14(8), 2024
2024
-
[25]
J. Zhou, G. Cui, S. Hu, Z. Zhang, C. Yang, Z. Liu, L. Wang, C. Li, and M. Su. Graph neural networks: A review of methods and applications.AI Open, pages 57–81, 2020
2020
-
[26]
Y. Zhou, H. Zheng, S. Hao, D. Li, and J. Zhao. Graph neural networks: Taxonomy, advances, and trends. ACM Transactions on Intelligent Systems and T echnology, 13(15):1–54, 2022. 14 5 Supplementary Figure 5. The architecture for the input GNNs for the explainers are recorded fo...
2022
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.