{"id":"61286d96-b006-4487-9bf6-bd5ae79c37f6","arxiv_id":"2412.12880","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"GRBE improves out-of-distribution graph classification and explanation quality by mixing environment subgraphs in the original graph space and refining rationale subgraphs with contrastive learning.","lead":"This paper proposes GRBE, a graph rationalization method that builds more diverse training graphs by mixing environment subgraphs from different graphs in the original graph space, instead of splicing learned representations. It reports large gains on synthetic distribution-shift benchmarks and a sentiment graph dataset, but slight losses on two real-world molecular datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Label preservation of augmented graphs is asserted, not verified; imperfect rationale masks can mislabel augmentations and undermine the claimed OOD gains.","rationale":"The reader's weakest_assumption correctly identifies the most load-bearing step in the argument: the augmented graph's label is taken to be the original graph's label, which is valid only if the extracted rationale subgraph deterministically determines the label and is independent of the environment. The paper's own account of Spmotif, where house and cycle differ by one edge, shows that small mask errors can change the label. The reported rationalization AUC of about 85% on Spmotif-0.9 indicates imperfect masks, yet no experiment quantifies the resulting label noise in augmented samples. Since the augmentation loss La explicitly trains on these labels, mislabeled augmentations could push the predictor away from invariance, undermining both the OOD classification and rationalization claims. This is not an internal inconsistency in the ideal case, but an unverified empirical assumption on which the central contribution rests. A direct oracle-based label-accuracy test on synthetic Spmotif would settle whether the concern lands. I considered whether the block-diagonal construction in Eq. 10 means the 'mixup' is really a disjoint union rather than a structural recombination, but the label-preservation issue is more fundamental because it is required regardless of how the environment subgraph is constructed. Therefore the reader's CONDITIONAL verdict remains appropriate, with the added condition that the authors should measure and report label fidelity of augmented graphs.","tokens_in":18937,"tokens_out":8476,"duration_ms":87169,"concrete_test":"On Spmotif-0.9, after training GRBE, generate an augmented batch and determine the label implied by the learned rationale subgraph Gi,r using oracle motif templates (house/cycle/tree) via subgraph isomorphism or graph-edit distance. Compare this implied label with the assigned label yi from Eq. 12, and report the mismatch rate across training epochs and for several augmentation ratios raug. Then retrain GRBE either excluding mismatched augmented samples or relabeling them, and measure Spmotif-0.9 OOD test accuracy. If the mismatch rate is materially above zero (e.g., >2%) or filtering changes accuracy non-trivially, Eq. 12's label-preservation assumption is violated and the central claim requires qualification.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that environment mixing yields better OOD generalization depends on the label assignment in Eq. 12: Gi,aug = Gi,r + Gmix,e is assigned the label yi. This is justified only by Definition 1's sufficiency and independence conditions, i.e., the rationale subgraph Gi,r fully determines the label and is independent of the environment. But Gi,r is obtained from stochastic Bernoulli masks (Eqs. 2-3) with a soft sparsity penalty (Eq. 4), and the paper itself notes in Section I that in Spmotif 'house' and 'cycle' differ by one edge and that even minor structural errors damage augmented graphs. The reported rationalization AUC is 85.04 on Spmotif-0.9, so the learned masks deviate from ground truth in a non-negligible fraction of cases. Under such errors, Gi,r + Gmix,e may correspond to a different ground-truth label than yi, so the augmentation loss La in Eq. 14/17 trains the classifier on partially mislabeled data. The paper reports no measurement of label accuracy on augmented samples and no filtering or confidence mechanism to remove unreliable augmentations. Without such evidence, the claimed 'superiority' of GRBE is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes GRBE, a graph rationalization method that performs data augmentation in the original graph space rather than in representation space. The method has two modules: PRSE, which refines rationale subgraph extraction through contrastive losses, and EDA, which mixes environment subgraphs from different graphs and attaches the resulting mixed environment to the rationale subgraph, assigning the original graph's label. The authors report experiments on three synthetic Spmotif variants and six real-world datasets, claiming average improvements of 7.65% in rationalization AUC and 6.11% in classification accuracy over state-of-the-art baselines.","tokens_in":19193,"tokens_out":7404,"duration_ms":74930,"significance":"The core idea of boosting environment diversity in the original graph space is a timely and plausible alternative to representation-space augmentation, and the paper provides a concrete implementation with comparisons on nine datasets, ablations, and visualizations. If the label-preservation assumption behind the augmented samples were verified, GRBE would be a useful contribution to graph rationalization. However, the headline claim is weakened by negative results on two datasets and by the lack of direct evidence that augmented graphs are correctly labeled. The significance of the contribution is therefore contingent on additional analysis and validation.","major_comments":[{"comment":"The label assignment of augmented graphs is load-bearing but unverified. Equation (12) sets the label of Gi,aug = Gi,r + Gmix,e to yi, justified only by Definition 1. However, the rationale mask is learned through stochastic Bernoulli sampling (Eqs. 2-3) with a soft sparsity penalty (Eq. 4), and the paper itself notes in Section I that a one-edge difference changes the Spmotif label. Table V reports rationalization AUC of 85.04 on Spmotif-0.9, so a non-negligible fraction of learned masks deviates from ground truth. For those graphs, the augmented sample can correspond to a different ground-truth label than yi, and training on La in Eq. (17) can push the classifier away from invariance. The authors should report label accuracy on augmented samples, add a confidence-based filtering mechanism, or provide a robustness analysis under mask error.","section":"IV-B, Eq. (12)"},{"comment":"The headline average improvements of 7.65% and 6.11% are unweighted means of per-dataset relative improvements and are dominated by the large Spmotif gains. Table IV shows that GRBE is worse than GREA on OGBG-Molbace (-5.56%) and OGBG-Molbbbp (-0.94%), and on MUTAG and Twitter the reported gains are within one standard deviation of the best baseline. The explanation in Section V-C that performance gains track the reported distribution-shift distance is not supported by Table IV: Graph-SST5 has a smaller distance than MUTAG but a larger improvement, while OGBG-Molbbbp has a distance similar to Graph-SST5 yet a negative improvement. The paper should report per-dataset significance tests, use a clearly defined aggregate metric rather than an unweighted mean of relative improvements, and provide a more rigorous account of the negative cases.","section":"Abstract and Table IV"},{"comment":"The contrastive objective in Eq. (9) combines -I(h1,+, h2,+) + I(h, h-), and Eq. (16) is presented as the InfoNCE estimator for I(h', h'') for arbitrary pairs. Using the same estimator for the negative pair (h, h-) and then minimizing it is not a standard negative-pair contrastive objective: minimizing the log-ratio can be achieved by increasing similarity to other negative samples in the denominator, so the intended repulsion between h and h- is not guaranteed. The authors should state the exact loss used in the implementation, including how positive and negative pairs are batched, and provide the derivation or a reference for the negative-pair term.","section":"IV-C, Eq. (16)"}],"minor_comments":[{"comment":"The text states that GRBE 'can produce 34 categories of environment subgraphs... which is significantly more than the environment subgraphs learned by GRBE'; this appears to be a typo and should read 'learned by GREA', since the comparison is with GREA and Table VI lists 34 categories for GRBE.","section":"I, Fig. 2"},{"comment":"The notation rows for Gi,e and Gj,e are duplicated with conflicting descriptions, one saying 'rationale subgraphs' and the other saying 'environment subgraphs'; these rows need to be corrected.","section":"Table I"},{"comment":"The sparsity loss is written without an explicit norm, so it is unclear whether the absolute value or an L1/L2 norm is intended; please define the norm explicitly.","section":"IV-A, Eq. (4)"},{"comment":"The figure plots a 'Distance' between augmented and original distributions, but Table IV also uses 'Distance' to denote training-testing distribution shift. Please clarify whether the same JS divergence is used and whether higher values in Fig. 6 are intended to indicate greater diversity.","section":"V-F, Fig. 6"}],"recommendation":"major_revision","confidential_remarks":"The central empirical claim is not yet established because the label-preservation assumption in Eq. (12) is both load-bearing and acknowledged as fragile in Section I. The revision should add direct validation of augmented-sample labels or a principled mitigation, and should present results in a way that does not rely on an unweighted average of relative improvements. I would be willing to review a revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a plausible methods paper with a genuine new idea, but the authors need to verify their augmentation labels and release code before the stronger claims stand.\n\nThe new thing is straightforward: prior rationalization methods like DIR and GREA splice rationale and environment representations in latent space; GRBE instead mixes environment subgraphs in the original graph space via a Bernoulli-sampled mixup, then reattaches the environment to a learned rationale. That is a real departure, and the motivation (representation-space splicing can't create genuinely novel environments) is clearly argued. The adaptive Bernoulli rationale sampling and the contrastive refinement are useful twists. The empirical study is broad: nine datasets, rationalization AUC plus classification, ablations, hyperparameter sensitivity, and a case study. The gains on Spmotif are large, and the rationalization AUC improvements on the synthetic sets are convincing.\n\nThe soft spots are real but not fatal. The load-bearing assumption is that each augmented graph Gi,r + Gmix,e carries the label yi because the rationale determines the label. That only holds if the mask is perfect. The paper itself notes in Section I that house and cycle differ by one edge, and the learned masks have AUC around 85 on Spmotif-0.9, so a non-negligible fraction of masks deviate from ground truth. The authors never measure label accuracy on augmented samples, nor do they filter or downweight unreliable augmentations. That is a legitimate gap, not a manufactured one. The stress-test note gets this right.\n\nThe evidence also doesn't fully support the abstract's 'superiority' claim. On OGBG-Molbace and OGBG-Molbbbp GRBE is worse than the best baseline, though the authors do acknowledge this and attribute it to small distribution shift. The +7.65% average is heavily weighted by Spmotif. There are no significance tests, and Table III shows per-dataset hyperparameter tuning without a stated validation protocol, so it's hard to know overfitting level. No code or data is released.\n\nWho should read it: anyone working on graph rationalization or OOD augmentation on graphs. It deserves a serious referee, but the referee should ask for code, a clear tuning/selection protocol, significance checks, and—most important—an analysis of whether augmented graphs actually preserve labels. If that analysis holds, the method is worth using; if not, the central mechanism is undermined.","headline":"Original-space environment mixup is a real idea, but the paper's label-preservation assumption for augmented graphs is unverified and the evidence doesn't support the claimed superiority.","tokens_in":19739,"tokens_out":2229,"would_cite":false,"duration_ms":19824,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GRBE augments graphs by mixing environment subgraphs in the original graph space, and reports that this produces more diverse training distributions, better out-of-distribution classification, and more accurate rationale subgraphs.","keywords":["graph rationalization","out-of-distribution generalization","graph neural networks","data augmentation","environment diversity","mixup augmentation","contrastive learning","subgraph extraction"],"falsifier":"Run GRBE on Spurious-Motifs where ground-truth motif masks are known, then inspect the augmented graphs: if a non-trivial fraction of mixed graphs inherit a label that differs from the label determined by their ground-truth motif, the label-inheritance assumption is violated and the reported gains cannot be credited to clean environment diversity.","tokens_in":1612,"feed_emoji":"🧩","tokens_out":5071,"duration_ms":76289,"temperature":0.7,"pith_summary":"Graph rationalization splits each input graph into a label-determining rationale subgraph and a label-independent environment subgraph, then augments the environment so that training becomes robust to distribution shift. This paper argues that prior methods augment in representation space, where the pool of environments is small and unevenly sampled, so the augmented distribution is not diverse enough. GRBE instead mixes environment subgraphs directly in the original graph space: it samples edges from two different environments, splices the resulting subgraph onto a rationale, and labels the new graph with the rationale's label. A contrastive refinement step first sharpens the rationale/environment split so that the new graphs remain valid. The paper reports that this raises classification accuracy and rationale-recovery AUC across benchmark datasets, with the largest gains when the train/test shift is severe.","feed_headline":"Original-space graph mixup lifts OOD accuracy by up to 32%","feed_subtitle":"Splicing environment subgraphs from different graphs creates more diverse training data and cleaner rationales.","key_machinery":"The method couples two modules: PRSE (Precise Rationale Subgraph Extraction) samples rationale edges from a Bernoulli distribution parameterized by a learned mask, uses a Gumbel-Softmax/Concrete relaxation to keep the sampling differentiable, and refines the mask with self-supervised contrastive pairs (perturb the environment while keeping the rationale for positive pairs, perturb the rationale while keeping the environment for negative pairs). EDA (Environment Diversity Augmentation) takes two environment subgraphs, builds a block-diagonal adjacency $A_{\\mathrm{ext}}$ with mask $M_{\\mathrm{mix},e}=\\mathrm{diag}(\\lambda M_{i,e},(1-\\lambda)M_{j,e})$, samples a new environment subgraph from that mixed distribution, and splices it onto a rationale subgraph with $N_{\\mathrm{add}}$ bridge edges. This machinery moves augmentation from representation space into the original graph space, which is what generates the larger and more varied set of augmented environments.","core_discovery":"The paper's central claim is that environment diversity, not merely environment replacement, is the bottleneck for graph rationalization under distribution shift. Existing methods such as GREA recombine rationale and environment representations, so augmentation can only reshuffle existing environment categories; GRBE treats edges as Bernoulli samples and mixes two environment subgraphs in input space, producing far more environment categories (for example, 34 versus 5 on Spmotif-0.9) and correspondingly better out-of-distribution classification and rationale extraction. The authors report average improvements of 7.65% in rationalization performance and 6.11% in classification performance over comparison methods, with Spmotif classification gains ranging from +9.0173% to +31.951%.","pith_inferences":["Editorial: If environment diversity is the active ingredient, sampling $\\lambda$ adaptively (for example, biasing toward rare or dissimilar environment clusters) could push diversity further; the paper keeps $\\lambda$ as a fixed hyperparameter.","Editorial: Original-space environment mixup should transfer to node-level or link-level tasks wherever a rationale subgraph can be identified; the paper validates only graph classification.","Editorial: The label-inheritance rule hides a testable risk: if a mixed environment accidentally contains motif-like structure, the augmented sample may be mislabeled. An oracle experiment using ground-truth rationale masks, with augmentation otherwise unchanged, would isolate how much of the gain comes from environment diversity rather than from label noise.","Editorial: The clustering-based diversity count could be replaced by a quantitative effective-number-of-environments measure and used as a criterion for selecting which augmented samples to keep, a direction the paper leaves as future work."],"forward_implications":["On the Spurious-Motifs datasets, GRBE improves classification accuracy over comparison methods by +9.0173% to +31.951%, and the benefit grows as the train/test distribution shift becomes larger.","Rationale extraction quality improves substantially: on Spmotif, rationale-recovery AUC rises by up to +17.556% over the best baseline, and on MUTAG GRBE reaches 99.86% AUC.","The environment diversity measure increases across all datasets: GRBE yields 34 to 323 environment-subgraph clusters versus 2 to 157 for the representation-space baseline GREA.","Ablation studies show that removing either the contrastive refinement loss or the environment-augmentation loss degrades both classification and rationalization, indicating that the two modules reinforce each other.","On datasets with small distribution shifts, such as OGBG-Molbace and OGBG-Molbbbp, GRBE is slightly inferior to baselines, suggesting its advantage is specifically tied to the magnitude of distribution shift."],"supporting_citations":[{"why":"Supplies the representation-space environment augmentation baseline that GRBE argues is insufficiently diverse and compares against experimentally.","marker":"[13]"},{"why":"Establishes the rationale/environment decomposition and the invariant-rationale objective that GRBE builds on.","marker":"[12]"},{"why":"Provides the Spurious-Motifs dataset and the AUC-based rationale evaluation protocol used to measure rationalization quality.","marker":"[16]"},{"why":"Introduces stochastic attention masking for graph learning, the conceptual predecessor of PRSE's Bernoulli edge sampling.","marker":"[3]"},{"why":"Supplies the Gumbel-Softmax/Concrete relaxation trick that makes the discrete edge-sampling steps differentiable.","marker":"[32]"},{"why":"Provides the InfoNCE estimator used in the contrastive refinement loss of PRSE.","marker":"[38]"},{"why":"Motivates the need for original-space augmentation by arguing that representation-space environment augmentation can produce unfaithful samples.","marker":"[17]"},{"why":"Supports the use of the augmentation ratio $r_{\\mathrm{aug}}$ to balance the amount of generated data.","marker":"[37]"}],"fun_headline_variants":["Original-space environment mixing boosts OOD accuracy","Boosting environment diversity improves graph rationalization","Mixing environments in raw graph space lifts OOD performance","Splicing environment subgraphs in input space boosts OOD generalization","Diverse environments via graph-space mixing improve GNN rationalization"],"cache_read_input_tokens":21888,"weakest_assumption_plain":"The load-bearing premise is that the learned rationale subgraph alone determines the graph label, so every mixed graph can be assigned the label of the graph whose rationale was reused; if the rationale mask misses or adds even one edge, the augmented samples become mislabeled.","fun_headline_variants_meta":{"raw":{"variants":["Original-space environment mixing boosts OOD accuracy","Boosting environment diversity improves graph rationalization","Mixing environments in raw graph space lifts OOD performance","Splicing environment subgraphs in input space boosts OOD generalization","Diverse environments via graph-space mixing improve GNN rationalization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001079,"raw_usage":{"total_tokens":4504,"prompt_tokens":928,"completion_tokens":3576,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":544,"completion_tokens_details":{"reasoning_tokens":3498}},"tokens_in":544,"tokens_out":3576,"duration_ms":20538,"temperature":1.0,"reasoning_tokens":3498,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:38:55.596776+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run GRBE on Spurious-Motifs where ground-truth motif masks are known, then inspect the augmented graphs: if a non-trivial fraction of mixed graphs inherit a label that differs from the label determined by their ground-truth motif, the label-inheritance assumption is violated and the reported gains cannot be credited to clean environment diversity.","supporting_citations":[{"cited_title":"Graph rationalization with environment-based augmentations,","cited_arxiv_id":null,"evidence_quote":"Supplies the representation-space environment augmentation baseline that GRBE argues is insufficiently diverse and compares against experimentally."},{"cited_title":"Discovering invariant rationales for graph neural networks,","cited_arxiv_id":null,"evidence_quote":"Establishes the rationale/environment decomposition and the invariant-rationale objective that GRBE builds on."},{"cited_title":"Gn- nexplainer: Generating explanations for graph neural networks,","cited_arxiv_id":null,"evidence_quote":"Provides the Spurious-Motifs dataset and the AUC-based rationale evaluation protocol used to measure rationalization quality."},{"cited_title":"Interpretable and generalizable graph learning via stochastic attention mechanism,","cited_arxiv_id":null,"evidence_quote":"Introduces stochastic attention masking for graph learning, the conceptual predecessor of PRSE's Bernoulli edge sampling."},{"cited_title":"Categorical reparameterization with gumbel-softmax,","cited_arxiv_id":null,"evidence_quote":"Supplies the Gumbel-Softmax/Concrete relaxation trick that makes the discrete edge-sampling steps differentiable."},{"cited_title":"A simple framework for contrastive learning of visual representations,","cited_arxiv_id":null,"evidence_quote":"Provides the InfoNCE estimator used in the contrastive refinement loss of PRSE."},{"cited_title":"Does invariant graph learning via environment augmentation learn invariance?","cited_arxiv_id":null,"evidence_quote":"Motivates the need for original-space augmentation by arguing that representation-space environment augmentation can produce unfaithful samples."},{"cited_title":"Do generated data always help contrastive learning?","cited_arxiv_id":null,"evidence_quote":"Supports the use of the augmentation ratio $r_{\\mathrm{aug}}$ to balance the amount of generated data."}],"review_version":1}