{"id":"fd9e10a2-1fe5-4077-8f33-12e8f42eeb75","arxiv_id":"2506.19343","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"Adding a discrepancy-reconstruction loss that aligns embedding differences with raw feature differences improves graph self-supervised learning on heterophilic graphs.","lead":"This paper introduces DGMAE, a masked graph auto-encoder that learns node representations by reconstructing not only masked features but also the differences between connected nodes. The method reports large gains over existing graph self-supervised baselines on heterophilic graphs, where linked nodes often belong to different classes.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The implemented discrepancy target (Eq. 9) is not the normalized Laplacian signal (Eq. 4) on irregular graphs, so the paper's theoretical mechanism is unverified even though the reported gains could still be real.","rationale":"I agree with the reader's weakest assumption, and I do not see a stronger objection that would overturn the main empirical claim. The statistical-significance issue is worth noting (Cornell and Actor margins are small relative to the reported standard deviations), but several of the largest heterophilic gains (Squirrel, Chameleon, Crocodile, Roman) are far larger than the run-to-run noise, so a significance critique alone would not settle the core claim. The Eq. 4/Eq. 9 gap is more load-bearing because it affects the definition of the method rather than its evaluation: the paper repeatedly explains the method as reconstructing the normalized Laplacian discrepancy, and Appendix E derives the decoupling behavior from that identification. If the implemented target is a different, degree-biased weighted difference, the theoretical account does not describe the trained model. The empirical superiority could survive this correction, so a reject is not justified; the right outcome is the reader's conditional, requiring equation reconciliation, explicit normalization, and code verification. Hence no verdict change.","tokens_in":28711,"tokens_out":14919,"duration_ms":151024,"concrete_test":"Rerun DGMAE on Texas and Squirrel with the exact Laplacian target from Eq. 4, i.e., replace Eq. 9 by x_i^D = (1/C_i) Σ_{j in N(i)} 1/sqrt(d_i d_j)(x_i - x_j) with C_i = Σ_{j in N(i)} 1/sqrt(d_i d_j), keeping all hyperparameters fixed; if the Table 1 accuracies change substantially, the reported success is attributable to the unnormalized Eq. 9 target rather than to the claimed L_sym X discrepancy. As a prerequisite, verify from the released code which of the two targets is actually implemented.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing theoretical claim is that Eq. 9 reconstructs the high-pass discrepancy X_D = L_sym X defined in Eq. 4-5. With all m_ij=1, Eq. 9 expands to Σ_j 1/sqrt(d_i d_j)(x_i - x_j) = C_i x_i - Σ_j 1/sqrt(d_i d_j)x_j, where C_i = Σ_j 1/sqrt(d_i d_j). Eq. 4 is x_i - Σ_j 1/sqrt(d_i d_j)x_j. These coincide only when C_i=1, which holds for regular graphs but not for general irregular graphs (e.g., a star center has C_i=sqrt(d_i)). The Bernoulli factor m_ij in Eq. 8 does not repair the mismatch; its expectation scales the same expression by p_ij rather than correcting the normalization. Because Eq. 9 is the target used in the discrepancy loss Eq. 12, the paper's claim that the model reconstructs L_sym X, and the Appendix E analysis of Laplace sharpening, rest on an algebraic identity that is false on the benchmark graphs. This is an internal inconsistency, not a disagreement with external consensus. It does not disprove the empirical results, but it means the stated mechanism is unverified on the current equations.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DGMAE, a self-supervised masked graph auto-encoder that augments standard feature reconstruction with a second reconstruction branch targeting node-feature discrepancies. The discrepancy target is motivated as the high-pass signal X_D = L_sym X obtained from the symmetric normalized graph Laplacian, and an adaptive edge-sampling mechanism based on shared attention weights selects node pairs with large discrepancies. The authors evaluate on 17 node-classification datasets, node clustering, graph classification, and several ablations and sensitivity analyses, reporting state-of-the-art results on most benchmarks, especially on heterophilic graphs.","tokens_in":28981,"tokens_out":8529,"duration_ms":82466,"significance":"The reported empirical gains are substantial and the evaluation is broad, covering many datasets, multiple downstream tasks, standard deviations, ablations, and released code. If the proposed mechanism were exactly as claimed, the paper would be a solid contribution to self-supervised learning on heterophilic graphs. However, the derivation connecting the implemented discrepancy loss to the Laplacian high-pass signal is currently incorrect on irregular graphs, and an algebraic step in the theoretical analysis is also flawed. These issues do not necessarily invalidate the empirical results, but they mean the stated mechanism is not yet supported by the equations as written.","major_comments":[{"comment":"The assertion that Eq. (4) 'can be rewritten as' Eq. (9) is not correct for irregular graphs. With m_ij = 1 for all j, Eq. (9) expands to C_i x_i - \\sum_j x_j / \\sqrt(d_i d_j), where C_i = \\sum_j 1/\\sqrt(d_i d_j), whereas Eq. (4) is x_i - \\sum_j x_j / \\sqrt(d_i d_j). These two expressions coincide only when C_i = 1, which fails on general graphs (e.g., for the center of a star, C_i = \\sqrt(d_i)). The Bernoulli factor m_ij does not repair the mismatch, because its expectation multiplies both terms by p_ij rather than re-normalizing the self-coefficient. Since Eq. (9) is the target used in the discrepancy loss Eq. (12), the paper's claim that the model reconstructs X_D = L_sym X as defined in Eq. (5) is not established. Please either modify Eq. (9) so that it is exactly equivalent to Eq. (4) (for example by dividing by C_i or using a properly normalized self-term) or explicitly present Eq. (9) as a weighted approximation and adjust the theoretical discussion, including Appendix E, accordingly.","section":"Section 4.3.2, Eq. (9) vs. Eq. (4)"},{"comment":"The algebraic manipulation used to justify the 'pull closer / push away' interpretation is incorrect. From the preceding line, maximizing <z^D_i, x^D_i> with x^D_i = \\sum_{j \\in N'(i)} (x_i - x_j) gives <z^D_i, x_i> - \\sum_j <z^D_i, x_j>. The next line then claims this equals -1/2 ||z^D_i - x_i||^2 + \\sum_j 1/2 ||z^D_i - x_j||^2. The required identity <a,b> = -1/2 ||a-b||^2 is false; the correct identity is <a,b> = 1/2(||a||^2 + ||b||^2 - ||a-b||^2). Therefore the derived decoupling interpretation does not follow from the loss as written. In addition, the definition of x^D_i used here omits the normalization factors that appear in both Eq. (4) and Eq. (9), which is a further inconsistency in the theoretical narrative.","section":"Appendix E, Eq. (16)"},{"comment":"The reported statistics for Cora (2,078 nodes, 5,278 edges) and Roman-empire (32,927 nodes, 5,278 edges) do not match the standard versions of these datasets (Cora: 2,708 nodes, 5,429 edges; Roman-empire: 22,662 nodes, 32,927 edges, per the cited source [38]). Because the headline results in Section 5.4 are comparisons against baselines, the exact preprocessing or data source must be clarified, and the authors should confirm that all methods were evaluated on the same graphs. If these are typographical errors, they should be corrected; if a filtered or alternative variant was used, this must be stated explicitly and its effect on comparability discussed.","section":"Table 7, dataset statistics"}],"minor_comments":[{"comment":"The discrepancy loss is written as L_d = 1/|\\hat{V}| \\sum_{i \\notin \\hat{|V|}} ..., which is not well-formed; the summation set and the normalizer should be stated clearly (presumably the unmasked nodes).","section":"Eq. (12)"},{"comment":"The attention weight w_ij is normalized over N(i), so p_ij is not symmetric. The paper should state whether the Bernoulli edge mask m_ij is drawn independently for each ordered pair or is symmetrized, since Eq. (9) sums over j \\in N(i).","section":"Eqs. (6)-(9)"},{"comment":"The axes are not labeled and the caption does not define the quantity on the vertical axis; please add axis labels and a clear description of what is plotted.","section":"Figure 2"},{"comment":"There are several grammatical errors, e.g., 'three graph analytic including tasks' in the abstract; a careful proofread is needed.","section":"Abstract and Section 5"},{"comment":"The inner product is defined as <z^D_i, x^D_i> = x_i^T \\hat{z}_i / (||x_i^T|| ||\\hat{z}_i||), but the loss expression uses <z_i, x_i>; please make the notation consistent.","section":"Eq. (3)"}],"recommendation":"major_revision","confidential_remarks":"The Eq. (9) derivation gap and the Appendix E algebra error are fixable, but the dataset-statistics inconsistency in Table 7 needs to be resolved before the empirical claims can be fully assessed. I would like to see the revised version with these points addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, what's new: the idea of reconstructing a high-pass discrepancy signal (Laplacian-style) rather than only raw features is genuinely new among the cited GMAE baselines. The adaptive sampling of high-discrepancy edges via shared attention weights is a sensible extension of GCA. If the empirical gains hold up, this is a useful addition to the graph SSL toolbox.\n\nWhat the paper does well: extensive evaluation across 17 datasets, ablations, sensitivity analyses, and code release. The gains on heterophilic graphs are large—Texas 88.1 vs 82.7 for the best baseline, Squirrel 72.5 vs 64.8. The ablation shows that discrepancy reconstruction matters most on heterophilic graphs, consistent with the narrative. Training curves and complexity analysis are helpful.\n\nWhere the soft spots are: the derivation gap between Eq. 4 and Eq. 9 is real. Eq. 4 defines x_D_i = x_i - sum_j (1/sqrt(d_i d_j)) x_j. Eq. 9 defines x_D_i = sum_j m_ij / sqrt(d_i d_j) (x_i - x_j). With all m_ij=1, Eq. 9 expands to C_i x_i - sum_j (1/sqrt(d_i d_j)) x_j, where C_i = sum_j 1/sqrt(d_i d_j). These coincide only when C_i=1, which is not true for irregular graphs (e.g., a star center). The Bernoulli mask does not repair this; its expectation scales the same expression by p_ij. So the implemented target is not L_sym X, and the Appendix E analysis of Laplace sharpening rests on an incorrect premise. This doesn't sink the empirical results, but it means the paper's stated mechanism—that the model reconstructs the normalized Laplacian high-pass signal—is unverified. The authors need to either reconcile the equations or weaken the theoretical claim.\n\nA second, smaller issue: the definition of z_D in Eq. 11 as z_i - \\hat{z}_i is conceptually murky. Since \\hat{z}_i for an unmasked node depends on which nodes are masked, calling it the node's 'unique representation' is a stretch. The algebraic derivation in Appendix E also has an unstated normalization step. These are fixable with clearer writing.\n\nThe circularity burden is low: the target is a fixed function of raw features, not labels. The attention weights used for sampling are shared with the encoder, but evaluation uses external labels, so I don't see a circular fitting problem.\n\nBottom line: a solid empirical paper with a novel pretext task, but the theoretical framing is not yet accurate. A serious referee would catch the Eq. 4/Eq. 9 problem and ask for corrections. It deserves peer review, and if the authors fix the derivation and clarify the embedding discrepancy, it would be a good contribution.","headline":"Novel discrepancy-reconstruction pretext with strong heterophilic gains, but the paper's claimed Laplacian target doesn't match the implemented equation.","tokens_in":29580,"tokens_out":2577,"would_cite":false,"duration_ms":25510,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DGMAE, a masked graph auto-encoder that reconstructs neighbor-feature discrepancies, is reported to outperform state-of-the-art graph self-supervised methods on 16 of 17 node-classification benchmarks, with the largest gains on…","keywords":["graph masked auto-encoder","heterophilic graphs","discrepancy reconstruction","self-supervised graph representation learning","symmetric normalized graph Laplacian","high-pass graph signal","node classification","node clustering"],"falsifier":"Train DGMAE on a fixed heterophilic benchmark (for example Texas or Squirrel) with the exact target $X_D = L_{\\mathrm{sym}}X$ from Eq. 4 and with the sampled target from Eq. 9, keeping all hyperparameters and the Bernoulli masks fixed across runs. If the two variants give nearly identical accuracy, the discrepancy-reconstruction mechanism is robust to the definitional gap; if the exact-target variant is materially better, then the implemented loss approximates the claimed signal only imperfectly and the mechanism needs revision.","tokens_in":28468,"feed_emoji":"🕸️","tokens_out":14471,"duration_ms":136576,"temperature":0.7,"pith_summary":"Masked graph auto-encoders learn by hiding nodes and recovering them from neighbor context. This paper argues that this recover-the-context objective implicitly pushes connected nodes toward similar embeddings, which is a poor inductive bias on heterophilic graphs, where edges often connect different classes. The proposed alternative, DGMAE (Discrepancy-Aware Graph Mask Auto-Encoder), adds a second reconstruction target: the high-pass feature difference between a node and its neighbors, defined through the symmetric normalized graph Laplacian. The paper reports that this difference-aware objective achieves the best node-classification accuracy on 16 of 17 benchmarks, with its largest margins on heterophilic graphs, and also improves node clustering and graph classification. If the paper is right, the choice of pretext signal—not just the encoder architecture—controls whether self-supervised graph representations remain distinguishable.","feed_headline":"Node-difference auto-encoder beats 16 of 17 benchmarks","feed_subtitle":"A high-pass discrepancy target keeps connected-but-different classes apart, fixing a blind spot of masked graph auto-encoders.","key_machinery":"The load-bearing identity is $X_D = L_{\\mathrm{sym}}X = (I-\\tilde{A})X$ (Eq. 5), which turns neighbor-feature subtraction into multiplication by the symmetric normalized graph Laplacian and supplies the high-pass discrepancy target. The complementary mechanism is the masked/unmasked contextual discrepancy $z_i^D = z_i - \\hat{z}_i$ (Eq. 11), where $z_i$ comes from an encoder on the unmasked graph and $\\hat{z}_i$ from the masked branch; aligning this latent discrepancy to the raw feature discrepancy is what preserves node individuality. The adaptive discrepancy selection module converts the shared GAT attention weights into sampling probabilities $p_{ij} = \\min((1-w_{ij})p_c, p_\\tau)$ (Eq. 7), preferentially keeping edges whose representations are dissimilar. The two branches share one encoder and are combined as $\\mathcal{L}=(1-\\lambda)\\mathcal{L}_f+\\lambda\\mathcal{L}_d$ (Eq. 13), so the feature-reconstruction and discrepancy-reconstruction objectives are optimized together.","core_discovery":"At its core, the paper claims that reconstructing discrepancy information prevents the node confusion caused by feature reconstruction. The discrepancy target is $X_D = L_{\\mathrm{sym}}X = (I-\\tilde{A})X$ (Eq. 5), where $\\tilde{A}=D^{-1/2}AD^{-1/2}$; this is the high-pass filtered feature signal, and the paper treats it as a supervision signal rather than as an encoder input. In the embedding space, the paper measures contextual discrepancy as $z_i^D = z_i - \\hat{z}_i$ (Eq. 11), the difference between the encoder's unmasked representation and the masked decoder's reconstruction, and aligns it to $x_i^D$ with a scaled cosine error. An attention-based Bernoulli sampler selects edges whose learned attention weight is low, so the discrepancy target is biased toward pairs with large feature differences. On this basis the paper reports best node-classification accuracy on 16 of 17 benchmarks, with margin examples of Texas 88.11% versus 82.70% for the best baseline and Squirrel 72.47% versus 64.81%, together with gains in node clustering and graph classification.","pith_inferences":["Beyond the paper, the degree normalization in Eq. 4 can be tested in isolation: comparing a variant trained on the exact $L_{\\mathrm{sym}}X$ target with one trained on the sampled weighted-difference form of Eq. 9, on a fixed heterophilic dataset, would show whether the Laplacian normalization is load-bearing or incidental.","Beyond the paper, the strong performance under high mask ratios suggests the discrepancy target could act as an auxiliary objective in other sparse-context settings, such as node classification with missing features or few-shot labels, which the paper does not evaluate.","Beyond the paper, since the discrepancy target is feature-defined, a structural analogue (for example, differences between degree or positional encodings) could extend the idea to graphs without node attributes; the paper's own limitation section notes that featureless graphs are currently out of scope."],"forward_implications":["On heterophilic graphs, where connected nodes tend to carry different labels, the pretext objective should include a high-pass discrepancy term alongside feature reconstruction, since feature-only reconstruction drives representations of different classes together.","The discrepancy signal remains informative under heavy masking: on Roman, DGMAE retains about 73.5% accuracy at an 80% mask ratio, where the feature-reconstruction baseline falls to 38.2%.","On homophilic graphs the added branch does not hurt: DGMAE reaches 84.93% on Cora, 96.01% on Physics, and 51.56% on Flickr, matching or beating the compared self-supervised methods.","The benefit extends to larger heterophilic graphs: DGMAE reports ROC AUC 90.72% on Minesweeper versus 81.23% for GREET and 76.69% on Penn94, suggesting the approach scales.","Choosing which edges to include in the discrepancy target matters: sampling by reversed attention weights beats random, degree-based, and forward-attention sampling on Actor, Chameleon, and Squirrel."],"supporting_citations":[{"why":"It supplies the masked feature-reconstruction backbone and scaled cosine error loss that DGMAE extends into a discrepancy branch.","marker":"[7]"},{"why":"It establishes masked graph modeling via edge/path masking, representing the GMAE baseline whose consistency alignment the paper aims to correct.","marker":"[12]"},{"why":"It provides a heterophilic self-supervised baseline combining low- and high-pass views, and is the strongest runner-up on several heterophilic benchmarks.","marker":"[24]"},{"why":"It is cited as the reference for viewing the symmetric normalized Laplacian as a high-pass filter, motivating the discrepancy reconstruction target.","marker":"[29]"},{"why":"It is the GAT decoder that extracts common neighbor information; subtracting its output from the unmasked encoder output defines the contextual discrepancy.","marker":"[48]"},{"why":"It documents the alignment and uniformity behavior of masked auto-encoders, which the paper cites as evidence that feature reconstruction makes connected nodes too similar.","marker":"[52]"}],"fun_headline_variants":["High-pass discrepancy target makes graph auto-encoder SOTA on 16/17","Reconstruct node differences, not features: DGMAE wins 16 benchmarks","Discrepancy-aware masking fixes heterophily blind spot, beats baselines","Keep different nodes apart: DGMAE's high-pass target sets new SOTA","Graph auto-encoder that reconstructs discrepancy outperforms on 16/17"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the degree-normalized neighbor-subtraction signal $X_D=(I-\\tilde{A})X$ captures the semantic discrepancy that embeddings should preserve, and that the sampled weighted-difference form in Eq. 9 faithfully reproduces that signal—which it does only when each node's weights $\\sum_j 1/\\sqrt{d_i d_j}$ sum to 1, a condition that fails on most real graphs.","fun_headline_variants_meta":{"raw":{"variants":["High-pass discrepancy target makes graph auto-encoder SOTA on 16/17","Reconstruct node differences, not features: DGMAE wins 16 benchmarks","Discrepancy-aware masking fixes heterophily blind spot, beats baselines","Keep different nodes apart: DGMAE's high-pass target sets new SOTA","Graph auto-encoder that reconstructs discrepancy outperforms on 16/17"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000437,"raw_usage":{"total_tokens":2242,"prompt_tokens":985,"completion_tokens":1257,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":601,"completion_tokens_details":{"reasoning_tokens":1154}},"tokens_in":601,"tokens_out":1257,"duration_ms":9570,"temperature":1.0,"reasoning_tokens":1154,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:33:04.504622+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train DGMAE on a fixed heterophilic benchmark (for example Texas or Squirrel) with the exact target $X_D = L_{\\mathrm{sym}}X$ from Eq. 4 and with the sampled target from Eq. 9, keeping all hyperparameters and the Bernoulli masks fixed across runs. If the two variants give nearly identical accuracy, the discrepancy-reconstruction mechanism is robust to the definitional gap; if the exact-target variant is materially better, then the implemented loss approximates the claimed signal only imperfectly and the mechanism needs revision.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It documents the alignment and uniformity behavior of masked auto-encoders, which the paper cites as evidence that feature reconstruction makes connected nodes too similar."}],"review_version":2}