{"id":"2bbb3722-8e8d-4680-8c9d-45790652cb3a","arxiv_id":"2607.06546","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.5,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Under a denoising objective, linear attention is suboptimal; Graph Convolutional Attention matches idealized spectral attention on SBMs and improves graph denoising and diffusion in proportion to spectral diversity.","lead":"Linear attention can only learn an average spectral denoising filter, so it underperforms when graphs vary spectrally. Graph Convolutional Attention (GCA) adapts to each graph’s spectrum and improves denoising and diffusion, with gains tracking spectral diversity.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"The load-bearing optimality claim for GCA rests on asymptotic spiked-Wigner/SBM conditions that the paper never checks on the real graphs where gains are claimed.","rationale":"The reader’s weakest_assumption is exactly the right load-bearing concern: the optimality transfer from Spectral Attention to GCA is proved only in the asymptotic spiked-Wigner/SBM regime of Assumption 1 / Theorem 2 / Remark 2, and the softmax projection story needs equal blocks and small noise. The paper’s empirical diversity–gain correlation and DiGress ablations are consistent with the broader insight that spectrum-conditioned filters help, but they do not close the gap between the abstract spectral optimum and the practical GCA construction on real graphs. Because the core linear-vs-spectral analysis (Propositions 1–2, Theorem 1) is solid and the method is useful even without the asymptotic equality, the appropriate verdict remains CONDITIONAL rather than REJECT. No stronger internal inconsistency was found; the concern is scope of the optimality claim, not a contradiction inside the proofs under their stated hypotheses.","tokens_in":32236,"tokens_out":653,"duration_ms":6694,"concrete_test":"On each real dataset, estimate the top-k spectral gap of E[A] (or of the clean adjacency) versus the bulk edge of the residual Δ+E, and check whether the observed noisy eigenvalues produce a B matrix that is approximately diagonal and matches η_out(Λ̃) of Eq. 18 (or a learned pointwise polynomial of comparable degree). If the residual bulk is comparable to the community eigenvalues, or if B is far from diagonal/pointwise, the asymptotic matching assumption fails on the graphs where gains are claimed.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The strongest claim is that GCA matches the idealized spectral optimum (ℓ*_GCA = ℓ*_SA) and therefore inherits the spectral-diversity gap of Theorem 1. That equality is proved only under Assumption 1 + Theorem 2: fixed-rank community signal E[A], Δ negligible relative to its eigenvalues, Wigner (or edge-flip reduced to Wigner+bias) noise, and n→∞ so that E[ŨᵀAŨ|Λ̃] collapses to the pointwise outlier-shrinkage map η_out of Eq. 18. Softmax improvement (Theorem 3) further requires equal blocks and E[‖E‖_F^{2}] ≪ min(1, m^{2}/log^{2} k). The paper never verifies whether any of these hold for the real TUDataset graphs (PROTEINS, ENZYMES, COLLAB, DEEZER, IMDB) on which the diversity–gain correlation and generative claims are made. If those graphs are not well approximated by the spiked low-rank-plus-Wigner regime, the equality ℓ*_GCA = ℓ*_SA need not hold and the observed gains could be ordinary polynomial-filter expressivity rather than the claimed spectral-attention optimality. The reader correctly flags this as the weakest assumption; it is also the single most load-bearing one for the central theoretical claim.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper analyzes attention-based graph denoising under a Frobenius objective. It shows that linear attention (outer products of linear Q/K maps on noisy eigenvectors) is optimal only as the unconditional mean E[B] with B=ŨᵀAŨ, whereas spectral attention, which may depend arbitrarily on the noisy eigenvalues, achieves the conditional mean E[B|Λ̃]; the loss gap equals a spectral-diversity term SD(D_G) from the law of total variance (Props. 1–2, Thm. 1). Graph Convolutional Attention (GCA) realizes pointwise spectral filters via graph-filtered queries/keys and is permutation-equivariant; under an asymptotic spiked-Wigner/SBM regime (Assump. 1, Thm. 2), GCA matches spectral attention via outlier shrinkage η_out. Softmax is further argued to approximate projection onto the clean principal subspace and reduce eigenvector noise under equal-block, small-noise conditions (Lem. 2, Thm. 3). Empirically, replacing linear Q/K with GCA improves denoising and often diffusion metrics, with gains correlated to a kNN estimate of spectral diversity; DiGress ablations with GCA and R-PEARL match baseline quality while reducing inference cost on larger graphs.","tokens_in":32652,"tokens_out":1643,"duration_ms":28181,"significance":"If the analysis holds, the paper supplies a clean spectral account of why standard linear attention is limited for graph denoising and a concrete inductive bias (spectrum-adaptive, graph-filtered Q/K) that is both theoretically motivated and practically usable in diffusion models. Strengths include: (i) Propositions 1–2 and Theorem 1 follow standard conditional-mean / total-variance arguments and cleanly identify spectral diversity as the explained-variance gap; (ii) Theorem 2 ties GCA to classical spiked-Wigner outlier shrinkage with an explicit η_out; (iii) the predicted correlation between per-dataset gains and spectral diversity is a falsifiable empirical check that the authors attempt; (iv) code is released, and the DiGress + R-PEARL integration shows a practical path to avoid expensive eigendecomposition-based features. The contribution is a useful theory-to-architecture bridge for graph diffusion, even though the strongest GCA=SA equality is scoped to asymptotic SBMs.","major_comments":[{"comment":"§2.2, Theorem 2 / Remark 2: The claim that GCA matches the idealized spectral optimum (ℓ*_GCA=ℓ*_SA) is proved only under Assumption 1 and n→∞ spiked Wigner/SBM conditions (fixed-rank E[A], negligible Δ, Wigner or edge-flip-reduced noise, pointwise η_out in Eq. 18). The manuscript never checks whether real TUDataset graphs (PROTEINS, ENZYMES, COLLAB, DEEZER, IMDB) are even approximately in this regime, yet §4.3–4.4 and the abstract present GCA gains on those graphs as support for the spectral-attention story. Please either (a) add diagnostics (e.g., effective rank of E[A], bulk vs outlier spectrum, agreement of learned filters with η_out-like shrinkage) or (b) clearly separate in the main text what is proved for SBMs from what is a heuristic expressivity argument for general graphs, so readers do not over-read ℓ*_GCA=ℓ*_SA beyond its stated scope.","section":"§2.2, Theorem 2, Remark 2"},{"comment":"§2.2 and Lemma 1 vs §4.1 Eqs. (26)–(27): The equivariance and SA-matching analysis restrict H_Q^{(l)}, H_K^{(l)} to constant diagonals so that attention reduces to a pointwise spectral filter in F_GCA. Experiments primarily use the richer multi-tap filter-bank form (and LayerNorm). That is a reasonable design choice, but it means the implemented GCAT is not the object for which ℓ*_GCA=ℓ*_SA is proved. State this gap explicitly near the architecture definition and, if possible, ablate diagonal/scalar filters vs full banks on at least the synthetic SBMs where the theory applies.","section":"§2.2, Lemma 1; §4.1"},{"comment":"§4.4, Table 1 and Appendix A: Training-loss gains for GCAT are uniform, but structural MMDs worsen on COLLAB and DEEZER (and Orbit MMD on ENZYMES). DiGress ablations (Table 3) are single-run, early-stopped relative to the published 550k-step recipe, and the authors themselves call them exploratory. The main-text claim that GCA “matches standard graph-transformer performance” / improves diffusion should be tempered to match Appendix A: report uncertainty, avoid overstating parity, and discuss when lower NLL fails to translate into better samples.","section":"§4.4, Table 1; Appendix A, B.1"}],"minor_comments":[{"comment":"Notation: eA, eU, eΛ vs tildes in the abstract/theory mix with bA in §3; a short notation table would help.","section":"§2–3"},{"comment":"Figure 1c: report the exact correlation coefficient, noise model (Gaussian vs edge-flip), and k used for the diversity surrogate so the “strongly correlated” claim is reproducible from the figure alone.","section":"Figure 1c; Appendix B.5"},{"comment":"Theorem 3 / Assumption 2: equal-block and E[‖E‖_F²]≪min(1,m²/log²k) are strong; a one-sentence practical takeaway (when softmax is expected to help) would help non-theory readers.","section":"§3, Theorem 3"},{"comment":"Related work §1.1 / Appendix D: Specformer, PolyFormer, SAN, and PEARL are cited; a sharper sentence on how GCA differs (denoising objective + Q/K graph filters vs spectral features or filter parametrization for prediction) would reduce overlap concerns.","section":"§1.1; Appendix D.3"},{"comment":"Typos / polish: “Understanding Graph Denoising” appears as the supplementary title while the main title is different; arXiv id and “Preprint” headers are fine but unify titles. Occasional double spaces and “thediversity = 0” in B.5.","section":"Supplementary header; B.5"},{"comment":"Code link is given; ensure the released repo includes the spectral-diversity kNN estimator and DiGress ablation configs referenced in B.3–B.5 for full reproducibility.","section":"Abstract; Appendix B"}],"recommendation":"minor_revision","confidential_remarks":"The theoretical core (linear vs spectral attention and SD) is solid and publishable; the main risk is over-claiming that real-graph GCA gains instantiate the SBM optimality theorem. If the authors add a short scope paragraph and temper the DiGress language, this is appropriate for a solid ML theory+architecture venue. I would not reject on the SBM restriction alone—the paper already scopes Theorem 2 correctly in the abstract—but the current main-text narrative invites that misreading."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful core is a clean characterization: under Frobenius denoising loss, optimal linear attention is the unconditional mean E[B] (B = ŨᵀAŨ), while spectral attention gets the conditional mean E[B|Λ̃], and the gap is spectral diversity via total variance. That is standard math done carefully, and it gives a real diagnostic. GCA (graph-filter Q/K) is a practical equivariant realization; for spiked SBM/Wigner they show it asymptotically matches the spectral optimum via Benaych-Georges–Nadakuditi outlier shrinkage, and they give a softmax-as-approximate-projection story under equal blocks and small noise. Denoising experiments improve consistently and the gain tracks a kNN spectral-diversity surrogate. Code is linked.\n\nWhat is new relative to SAN/Specformer/PolyFormer/SignNet is not “spectral stuff in transformers,” but the denoising-loss analysis, the named gap, and the claim that GCA inherits that gap for community-like spectra. Propositions 1–2 and Theorem 1 are solid. Theorem 2 and the ℓ*_GCA = ℓ*_SA equality are load-bearing and rest on fixed-rank E[A], negligible Δ, Wigner (or edge-flip reduced to Wigner+bias), and n→∞. Softmax (Theorem 3) needs equal blocks and E[‖E‖_F²] ≪ min(1, m²/log²k). The paper never checks whether PROTEINS, ENZYMES, COLLAB, DEEZER, or IMDB look like that regime. So the observed real-data gains could be ordinary polynomial-filter expressivity rather than the claimed spectral-attention optimality. Generative and DiGress results are directionally consistent but mixed on structural MMDs, under-trained relative to published DiGress, and the authors themselves call the SOTA panel exploratory. Citation pattern is fair.\n\nThis is for people who care about graph diffusion denoisers and spectral inductive bias. The theory is honest about its assumptions; the soft spot is scope, not circularity. I would send it to peer review. Engage if you work on discrete graph diffusion or want a spectrum-adaptive Q/K baseline; treat the real-graph optimality story as a hypothesis, not a theorem.","headline":"Clean denoising-loss analysis of linear vs spectrum-conditioned attention, with a usable GCA architecture; the SBM optimality equality is asymptotic and untested on the real graphs where gains are claimed.","tokens_in":33315,"tokens_out":581,"would_cite":true,"duration_ms":7490,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Under a denoising loss, linear attention can only learn an average spectral filter over the training graphs; Graph Convolutional Attention adapts to each input spectrum and closes a gap equal to spectral diversity.","keywords":["graph denoising","graph diffusion","spectral attention","graph convolutional attention","spectral diversity","graph transformers","stochastic block models","eigenvector denoising"],"falsifier":"On synthetic SBMs with controlled block-size heterogeneity, plot GCA-versus-linear validation-loss gap against estimated spectral diversity; if the gap does not rise with diversity, or if on large spiked SBMs GCA fails to approach the theoretical spectral-attention loss, the central claim fails.","tokens_in":33105,"feed_emoji":"🕸️","tokens_out":915,"duration_ms":21428,"temperature":0.7,"pith_summary":"The paper argues that standard linear attention is the wrong default for graph denoising. Its optimal query-key map is only the unconditional average of clean graphs expressed in the noisy eigenbasis, so it cannot track graphs whose spectra differ across the dataset. Spectral Attention conditions that map on the noisy eigenvalues and is strictly better by a quantity the authors call spectral diversity. Graph Convolutional Attention (GCA) is a practical, permutation-equivariant version: queries and keys are built from graph-filtered features, so the attention pattern is a pointwise function of the spectrum. For large stochastic block models this matches the ideal spectral map, and the softmax that follows further denoises eigenvectors by roughly projecting onto the clean principal subspace. Empirically, swapping linear projections for GCA improves denoising and diffusion, with gains that track spectral diversity, and can match full graph-transformer diffusion performance without expensive hand-crafted spectral features.","feed_headline":"Linear attention averages spectra; GCA adapts and wins","feed_subtitle":"The loss gap is spectral diversity; filtered queries match the ideal on block models and speed diffusion.","key_machinery":"Spectral diversity SD(D_G) = E[‖E[B|Λ̃] − E[B]‖_F²] (with B = ŨᵀAŨ): the variance of the optimal spectral predictor. It is the exact improvement of spectral attention over linear attention, and GCA targets the same conditional map via pointwise spectral filters on the input eigenvalues.","core_discovery":"Under Frobenius denoising loss, the best linear-attention map is the unconditional mean of B = ŨᵀAŨ, while spectral attention achieves the conditional mean given the noisy eigenvalues; the loss gap is exactly the spectral diversity of the graph distribution. GCA implements that conditional filter with graph-filtered queries and keys, asymptotically matches spectral attention on spiked SBM-type models, and softmax adds further eigenvector denoising via approximate projection onto the clean eigenspace.","pith_inferences":["Spectral diversity is a practical dataset diagnostic for deciding when spectral inductive bias is worth the compute.","The same average-versus-conditional gap may apply to attention denoisers on other structured objects whose spectra vary (meshes, manifolds, point clouds).","As graph size grows, random-feature positional encodings plus GCA may become a default scalable recipe for graph diffusion because they avoid per-step eigendecomposition."],"forward_implications":["Replacing linear query-key projections with graph-filter banks should improve denoising whenever graphs vary spectrally across the training distribution.","Per-dataset denoising gains should correlate with a measurable spectral-diversity surrogate.","In discrete graph diffusion, GCA can stay competitive while dropping costly hand-crafted spectral node features and, with random-feature encodings, explicit eigendecomposition.","Softmax after attention is not only a normalizer: once eigenvalues are recovered it can further denoise eigenvectors by approximate projection onto the clean principal subspace."],"fun_headline_variants":["Linear attention averages spectra; GCA adapts via filtered queries","Spectral diversity sets the gap; GCA matches ideal on SBM graphs","Graph-filtered QK realizes conditional spectral denoising filter","Softmax projects noisy eigenvectors onto the clean eigenspace","GCA lifts denoising and diffusion without structural features"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"The proof that GCA matches ideal spectral attention assumes graphs look like large community models: a fixed low-rank community signal, within-block randomness small enough to ignore, and Wigner-type noise so the conditional map collapses to a simple pointwise outlier-shrinkage rule.","fun_headline_variants_meta":{"raw":{"variants":["Linear attention averages spectra; GCA adapts via filtered queries","Spectral diversity sets the gap; GCA matches ideal on SBM graphs","Graph-filtered QK realizes conditional spectral denoising filter","Softmax projects noisy eigenvectors onto the clean eigenspace","GCA lifts denoising and diffusion without structural features"]},"model":"grok-4.5","effort":"low","cost_usd":0.003204,"raw_usage":{"total_tokens":1172,"prompt_tokens":861,"num_sources_used":0,"completion_tokens":84,"cost_in_usd_ticks":32040000,"prompt_tokens_details":{"text_tokens":861,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":227,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":861,"tokens_out":84,"duration_ms":4811,"temperature":1.0,"reasoning_tokens":227,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-11T00:03:10.880101+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On synthetic SBMs with controlled block-size heterogeneity, plot GCA-versus-linear validation-loss gap against estimated spectral diversity; if the gap does not rise with diversity, or if on large spiked SBMs GCA fails to approach the theoretical spectral-attention loss, the central claim fails.","supporting_citations":[],"review_version":2}