{"id":"0b9a0048-65cf-4193-b182-e3a3a3ded613","arxiv_id":"2506.04608","paper_version":1,"verdict":"CONDITIONAL","confidence":"LOW","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Preserving edge directionality in GNN preprocessing can substantially improve post-hoc explanation quality compared to symmetrization.","lead":"Symmetrizing a directed graph, the standard preprocessing step for many spectral GNNs, removes edge direction, and this paper argues it also degrades post-hoc explanations. Experiments compare symmetrization with a direction-preserving alternative and report higher explanation fidelity when direction is kept.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The direction-preserving arm swaps the base GNN from GCN to DiGCN, so the empirical gains in Table 1 and Table 2 cannot be attributed to preserving directionality rather than to a different, possibly stronger model.","rationale":"Reader's weakest_assumption identifies the main threat: directionality is not varied independently of architecture. I agree. The paper's Section 4 states it trains 'GCN-based models' yet the L condition uses DiGCN, a directed GNN; this is not a preprocessing-only change. Since no accuracy numbers are given, the explanation AUC gap on DiLink could be a proxy for model-quality gap. My proposed fixed-architecture experiment would settle this. I also note Theorem 3.3's proof contains a sign inconsistency: Eq. (8) and Eq. (11) together would give MI(Y;G_s) <= MI(Y;G_u_s), contradicting the theorem. The theorem statement is nonetheless recoverable via the data processing inequality, so I do not treat this as the primary reason for the conditional verdict. The lack of released code and error bars reinforces the conditionality but is secondary to the confound. Overall, the paper's idea is plausible and worth testing, but the current empirical support is not clean enough for acceptance as-is.","tokens_in":8595,"tokens_out":7335,"duration_ms":86842,"concrete_test":"On DiLink-Motif, fix the base model to DiGCN with identical hyperparameters and training budget, and compare two preprocessing conditions: (i) directed adjacency as in the paper's L arm, (ii) the same adjacency symmetrized by A+A^T (or max(A,A^T)) as in the B arm. Track node-classification test accuracy and GNNExplainer/PGExplainer explanation AUC over at least 10 seeds. If the directed arm's AUC advantage disappears or shrinks to within error bars once both models are matched in accuracy (or after accuracy is included as a covariate), the paper's causal attribution to directionality is not supported. For completeness, also run a GCN baseline on the symmetrized graph with increased capacity to match DiGCN's accuracy.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4 and Table 1 compare 'Bidirectional Symmetrical Relaxation' (standard GCN, ref [18]) against 'Laplacian Normalization' (DiGCN, ref [14]). This changes the convolution, normalization, and trainable capacity at the same time as the edge-direction treatment. The paper reports no node-classification accuracy for either base model, so if DiGCN is more accurate on DiLink, GNNExplainer's higher AUC (0.914 vs 0.793) may simply reflect explaining a better classifier. The abstract's causal claim ('symmetrization compromises explanation fidelity') requires the two pipelines to differ only in how direction is encoded; otherwise the effect is confounded with model quality. The same issue affects Table 2, where Fidelity/Char improvements could be driven by the base model change. A fixed-architecture comparison (DiGCN on directed vs symmetrized adjacency, with matched accuracy) is needed. The theoretical Theorem 3.3 does not resolve this: it assumes a single output variable Y and an optimal explainer, whereas the experiments change Y with the model and use heuristic explainers.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper argues that the standard practice of symmetrizing directed graphs before GNN training discards directional information and thereby reduces the fidelity of post-hoc explanations. It offers a theoretical inequality (Theorem 3.3) about mutual information between a model's output and an optimal explaining subgraph, and reports experiments on synthetic (BA-Shapes, BA-Community, Tree-Cycles, Tree-Grid, and new DiLink variants) and real-world datasets comparing 'Bidirectional Symmetrical Relaxation' with 'Laplacian Normalization' pipelines. The empirical results show higher explanation AUC and Char scores for the direction-preserving pipeline.","tokens_in":8847,"tokens_out":8261,"duration_ms":85379,"significance":"If the causal claim is established, the work would fill a genuine gap in the GNN explainability literature, with practical implications for security-critical applications. The paper is honest about the importance of directionality, and the synthetic DiLink dataset is a reasonable construct for controlled study. However, the current evidence does not yet support the strong causal conclusion because the two compared pipelines differ in both the directionality handling and the base GNN architecture, and the theoretical proof contains an error. The paper does include a formal theorem and reproducible configuration details, which are strengths, but the empirical confound is load-bearing.","major_comments":[{"comment":"The comparison confounds directionality with base GNN architecture. The bidirectional symmetrization pipeline uses GCN [18], while the Laplacian normalization pipeline uses DiGCN [14]. Since no model accuracy, parameter counts, or error bars are reported, the observed explanation improvements (e.g., GNNExplainer AUC 0.793→0.914 on DiLink-Motif) could reflect the higher capacity or accuracy of DiGCN rather than the preservation of direction. A fixed-architecture comparison (e.g., DiGCN trained on directed vs. symmetrized adjacency, or GCN with a direction-aware Laplacian) is required to support the abstract's causal claim.","section":"Section 4, Tables 1 and 2"},{"comment":"The proof contains a sign error. From the premise that G_u_s is a deterministic function of G_s, the correct inequality is H(Y|G_s) ≤ H(Y|G_u_s) (equivalently MI(Y,G_s) ≥ MI(Y,G_u_s)), not \"H(Y|G_s) − H(Y|G_u_s) ≥ 0\" as written. The step from H(G_u_s | Y, G_s) ≤ H(G_u_s | G_s) to the claimed entropy difference is not derived and contradicts the standard data-processing inequality. The theorem statement also needs to specify that G_u_s is the symmetrization of a subgraph of G; if G_u_s ranges over all subgraphs of G_u, the inequality is false.","section":"Section 3.2, Theorem 3.3"},{"comment":"The theorem assumes a fixed output Y and an optimal explainer, but in the experiments Y is the output of two different base models (GCN vs. DiGCN). Thus the empirical results in Tables 1 and 2 are not a direct instantiation of the theorem, and the \"Directional Semantic Gain\" label may overstate what the inequality shows.","section":"Section 3.2 vs. Section 4"},{"comment":"The Char score weights w+ and w− are never specified in the paper. The reported Char values in Table 2 are therefore unreproducible without additional information on how these weights were chosen.","section":"Section 2, Eq. (3)"}],"minor_comments":[{"comment":"The phrase \"Our analysis demonstrates\" is too strong given the empirical confound; suggest softening to \"Our analysis suggests\" or \"Our theoretical and empirical results indicate.\"","section":"Abstract"},{"comment":"\"Our proposed Laplacian normalization\" cites [14] (DiGCN); please clarify whether Lap-Norm is a novel contribution of this paper or a re-use of DiGCN's preprocessing.","section":"Section 4"},{"comment":"\"PGExplaienr\" is a typo for \"PGExplainer.\"","section":"Table 2 caption"},{"comment":"The inequality H_v(G) ≥ H_v(G_u) is asserted without proof or citation; either provide a reference or a brief justification.","section":"Section 3.1"},{"comment":"Reference [11] is cited as GCN, but GCN is reference [18]; please check citation accuracy.","section":"References"},{"comment":"The paper refers to \"Appendix C\" for real-world experiments, but the appendix structure is not clear in the manuscript; ensure appendices are properly included and labeled.","section":"Appendix"}],"recommendation":"major_revision","confidential_remarks":"The paper targets a relevant and under-studied problem, but the main empirical claim is undermined by the architecture confound. The theoretical proof is fixable, but the manuscript would need a substantially extended experimental section (fixed-architecture comparison, accuracy reporting, error bars) before I could recommend acceptance. The self-citation of DiGCN is not a fairness issue per se, but it reinforces the need for a cleaner comparison."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take on arXiv:2506.04608. The paper is right that symmetrization throws away directional information, and it makes a fair point that explainability evaluations have mostly ignored this. The new synthetic DiLink dataset is a small but real contribution, and the empirical direction—larger AUC differences on directed motifs—makes sense. The formal result in Theorem 3.3 is correct but elementary: symmetrization is a deterministic function, so the directed graph carries at least as much mutual information as the symmetrized one. That's data processing, not a deep new bound.\n\nThe soft spot is exactly the one the stress-test flags. The two pipelines don't differ only in edge-direction handling. The symmetrized arm uses GCN; the direction-preserving arm uses DiGCN. That changes convolution, normalization, and capacity at the same time. If DiGCN is simply a better classifier on the directed data, GNNExplainer's higher AUC could reflect explaining a stronger model rather than preserving direction. The paper reports no node-classification accuracy for either base model, no error bars, no significance tests, and no code. As written, the abstract's causal claim—that symmetrization compromises explanation fidelity—is not supported by the experiments.\n\nThe theory doesn't rescue this either. It assumes an optimal explainer and a fixed model output, whereas the experiments change the model and use heuristic explainers like GNNExplainer and PGExplainer.\n\nThe right fix is a controlled comparison: fix the base architecture, use the same GNN for both directed and symmetrized inputs, match test accuracy (or at least report it), and add error bars. Testing with a generic directed GNN other than the authors' own DiGCN would also remove the confound and the self-citation worry.\n\nIf revised cleanly, this could be a solid practical paper for the GNN explainability community, especially in fraud detection and network security. As it stands, the central empirical claim is unverified. I would still send it to peer review if I were an editor—the question matters and the confound is fixable—but I would not cite the empirical result until the controlled comparison is done. Recommendation: major revision, with the architecture-matched experiment as the gate.","headline":"A real but under-controlled claim about directionality in GNN explanations; the empirical comparison is confounded by swapping GCN for DiGCN.","tokens_in":9322,"tokens_out":1759,"would_cite":false,"duration_ms":20493,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Symmetrizing directed graphs strips edge direction and lowers the ceiling on faithful GNN explanations.","keywords":["graph neural networks","post-hoc explanations","graph symmetrization","directed graphs","GNNExplainer","PGExplainer","Laplacian normalization","mutual information"],"falsifier":"Train a single direction-capable base GNN twice on DiLink-Motif—once on the directed adjacency matrix and once on its symmetrized version—with matched node-classification accuracy, then run GNNExplainer on both frozen models. If explanation AUC does not fall under symmetrization once accuracy is matched, the observed gain cannot be attributed to directionality.","tokens_in":8374,"feed_emoji":"🧭","tokens_out":11362,"duration_ms":97007,"temperature":0.7,"pith_summary":"The paper argues that the standard preprocessing step of symmetrizing a directed graph—adding reverse edges so spectral graph neural networks can run—destroys directional information that post-hoc explanations need. It proves a mutual-information inequality stating that the best subgraph explanation available in the original directed graph can never be worse than the best explanation available in the symmetrized version, and it adds a von Neumann entropy argument that symmetrization removes structural complexity. Empirically, on the new DiLink synthetic graphs, GNNExplainer's explanation AUC rises from 0.793 under bidirectional symmetrization to 0.914 under Laplacian normalization, while undirected benchmarks stay roughly unchanged. The authors conclude that direction-aware preprocessing should be part of graph neural network explanation pipelines in applications where edge direction carries causal or temporal meaning, such as financial transaction graphs.","feed_headline":"Graph symmetrization misleads GNN explainers","feed_subtitle":"Keeping edge direction lifts GNNExplainer AUC from 0.79 to 0.91 on the DiLink-Motif benchmark.","key_machinery":"The load-bearing object is Theorem 3.3, the Directional Semantic Gain inequality. It compares the best mutual information a size-$k$ explanation subgraph can carry about the model output in the original directed graph, $\\max_{G_s \\subseteq G} MI(Y, G_s)$, with the same quantity in the symmetrized graph, $\\max_{G_s^u \\subseteq G_u} MI(Y, G_s^u)$, and proves the first is at least the second. The proof chain uses the fact that symmetrization is a deterministic transformation, so $H(G_s^u \\mid G_s) = 0$, and then conditional entropy to conclude $H(Y \\mid G_s) \\le H(Y \\mid G_s^u)$. The paper also uses the von Neumann entropy inequality $H_v(G) \\ge H_v(G_u)$ to argue that symmetrization discards structural complexity. On the empirical side, the direction-preserving pipeline is Laplacian normalization from the DiGCN directed graph convolutional architecture, which replaces the symmetrized adjacency matrix used by the GCN baseline.","core_discovery":"The central claim is that replacing a directed graph with its symmetrized undirected form does not merely lose a modeling nicety: it lowers the best possible quality of a post-hoc explanation. The formal statement is Theorem 3.3, the Directional Semantic Gain inequality, which says the maximum mutual information between a size-$k$ explanation subgraph and the model prediction in the directed graph is at least as large as the corresponding maximum in the symmetrized graph. The experiments show the loss is visible in practice: on DiLink-Motif, GNNExplainer's explanation AUC rises from 0.793 with bidirectional symmetrization to 0.914 with Laplacian normalization, and PGExplainer rises from 0.778 to 0.905, while undirected benchmarks remain essentially flat. The authors read these results as evidence that preserving directional semantics improves explanation fidelity and should be standard when explaining predictions on directed graphs.","pith_inferences":["Beyond the paper's pipeline, the mutual-information inequality suggests that mask-based explainers could be extended to select directed subgraphs—choosing not just which edges matter but which orientation of each edge—rather than inheriting an undirected mask space.","A direct test of the directionality mechanism would randomly shuffle edge directions while keeping the topology fixed; if explanation quality does not fall when directions are randomized, the observed gain is not specifically about directionality.","The entropy inequality implies the explanation gap should widen as graphs become more asymmetric, so datasets dominated by one-way edges should show larger gains from Laplacian normalization than nearly symmetric graphs."],"forward_implications":["On the paper's DiLink synthetic graphs, both tested explainers show higher explanation AUC with Laplacian normalization than with symmetrization, with GNNExplainer rising from 0.793 to 0.914 on DiLink-Motif.","On undirected benchmarks such as BA-Shapes and Tree-Cycles, direction-preserving preprocessing performs comparably to symmetrization, so adopting it does not appear to hurt cases where edge direction is irrelevant.","Real-world citation and product graphs show the same directionality trend in Fidelity and Characterization Score, suggesting the phenomenon is not confined to synthetic data.","If the theorem is correct, no post-hoc explainer working on a symmetrized graph can recover the directional explanations available from the original graph, because the deficit is introduced in the graph representation itself."],"supporting_citations":[{"why":"Supplies GNNExplainer, the mask-based post-hoc explainer whose explanation AUC and fidelity are measured in the empirical study.","marker":"[8]"},{"why":"Supplies PGExplainer, the second explainer baseline across synthetic and real-world datasets.","marker":"[9]"},{"why":"Provides the direction-preserving directed GNN (DiGCN) whose Laplacian normalization is the paper's alternative to symmetrization.","marker":"[14]"},{"why":"Provides GCN, the spectral model that motivates symmetrization and serves as the baseline preprocessing pipeline.","marker":"[18]"},{"why":"Supplies the approximate von Neumann entropy for directed graphs used in the information-loss argument.","marker":"[16]"},{"why":"Supplies the Barabási–Albert graph construction underlying the synthetic graphs, including the new DiLink datasets.","marker":"[17]"},{"why":"Supplies the mutual-information formulation of GNN explanations and frames graph symmetrization as an understudied issue.","marker":"[13]"}],"fun_headline_variants":["Symmetrizing graphs compromises GNN explanation fidelity","Ignoring edge direction hurts GNN explanations","Graph symmetrization lowers explanation quality","Direction-aware GNNs yield more faithful explanations","Direction-aware GNNs raise explanation AUC from 0.79 to 0.91"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The causal conclusion assumes the only meaningful difference between the symmetrized and direction-preserving pipelines is how edge direction is handled; in the experiments the two pipelines also use different base graph neural network architectures, and the paper does not report whether their accuracies were matched.","fun_headline_variants_meta":{"raw":{"variants":["Symmetrizing graphs compromises GNN explanation fidelity","Ignoring edge direction hurts GNN explanations","Graph symmetrization lowers explanation quality","Direction-aware GNNs yield more faithful explanations","Direction-aware GNNs raise explanation AUC from 0.79 to 0.91"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000654,"raw_usage":{"total_tokens":2927,"prompt_tokens":809,"completion_tokens":2118,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":425,"completion_tokens_details":{"reasoning_tokens":2040}},"tokens_in":425,"tokens_out":2118,"duration_ms":16661,"temperature":1.0,"reasoning_tokens":2040,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T10:38:32.740346+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a single direction-capable base GNN twice on DiLink-Motif—once on the directed adjacency matrix and once on its symmetrized version—with matched node-classification accuracy, then run GNNExplainer on both frozen models. If explanation AUC does not fall under symmetrization once accuracy is matched, the observed gain cannot be attributed to directionality.","supporting_citations":[{"cited_title":"Digraph inception convolutional networks.Advances in neural information processing systems, 33:17907–17918, 2020","cited_arxiv_id":null,"evidence_quote":"Provides the direction-preserving directed GNN (DiGCN) whose Laplacian normalization is the paper's alternative to symmetrization."},{"cited_title":"Approximate von neumann entropy for directed graphs.Physical Review E, 89(5):052804, 2014","cited_arxiv_id":null,"evidence_quote":"Supplies the approximate von Neumann entropy for directed graphs used in the information-loss argument."},{"cited_title":"Statistical mechanics of complex networks.Reviews of modern physics, 74(1):47, 2002","cited_arxiv_id":null,"evidence_quote":"Supplies the Barabási–Albert graph construction underlying the synthetic graphs, including the new DiLink datasets."},{"cited_title":"Explainability in graph neural networks: A taxonomic survey.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022","cited_arxiv_id":null,"evidence_quote":"Supplies the mutual-information formulation of GNN explanations and frames graph symmetrization as an understudied issue."}],"review_version":1}