{"id":"24a90879-1c9d-4b42-8ce8-f06790187441","arxiv_id":"2412.11739","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A gradient-scaling trick that updates filter and feature parameters at similar relative speeds improves training of spectral graph neural networks on heterophilic graphs.","lead":"This paper says spectral graph neural networks train better when the two types of parameters, filter coefficients and feature weights, are updated at matched speeds. The authors propose scaling each block's gradient by its parameter norm, and show accuracy gains on 18 graph datasets, especially on heterophilic graphs.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 17's proof applies Assumption 11 to block-Hessian eigenvalues, but that assumption only links full-Hessian eigenvalues at two nearby points; the needed ρΘ≥ρW step is unsupported, so the central reduction claim is not established.","rationale":"The reader correctly identifies Assumption 11 as the weak point of the theoretical claim, but the precise problem is more severe than the reader's statement. Table 8 does not actually show Assumption 11 failing for Cornell, Actor, and Squirrel: for those datasets the antecedent λmax(H_Ψ1) ≥ λmax(H_Ψ2) is false, so the implication is not violated. Chameleon is the clear violation. The deeper issue is that Theorem 17's proof uses Assumption 11 to justify a comparison between the two diagonal-block eigenvalues at a single point and the two GPNRs at the same point, while Assumption 11 only relates full-Hessian eigenvalues at two nearby points. This is a logical gap independent of the empirical check. The paper's empirical recipe may still be useful, and the released code allows the proposed test to be run, so a conditional verdict remains appropriate. The reader's conditional verdict is therefore unchanged, but the theoretical support should be revised or weakened until the missing implication is either proven or assumed explicitly.","tokens_in":20278,"tokens_out":6217,"duration_ms":55282,"concrete_test":"Instrument ChebNet training on Texas and Chameleon; at iterations t = 50, 100, ..., 500 compute λmax(H^t_ΘΘ) − λmax(H^t_WW) and ρ^t_Θ − ρ^t_W from finite-difference Hessian estimates. Count iterations where the first difference is positive and the second is negative. If any occur, the step at Eq. (26) is unsupported on that trajectory. In parallel, re-prove Theorem 17 with the amended assumption 'if λmax(H^t_ΘΘ) ≥ λmax(H^t_WW), then ρ^t_Θ ≥ ρ^t_W'; if the amended proof goes through, state that assumption explicitly and test it directly on all datasets.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that asymmetric learning reduces block condition numbers rests on Theorem 17. In the proof, Eq. (26) needs: if λmax(H^t_ΘΘ) ≥ λmax(H^t_WW), then ρ^t_Θ ≥ ρ^t_W, so that s^t_Θ/s^t_W ≤ 1. The paper invokes Assumption 11 for this step. But Assumption 11 is a statement about two nearby points Ψ and Ψ′: if λmax(H_Ψ) ≥ λmax(H_Ψ′), then ρ_Ψ ≥ ρ_Ψ′. Here λmax(H_Ψ) is the largest eigenvalue of the full Hessian at a point, not the largest eigenvalue of a diagonal block. Proposition 9 only supplies separate upper bounds ρ_Θ ≤ λmax(H_ΘΘ) and ρ_W ≤ λmax(H_WW); these cannot yield the cross-parameter implication needed at Eq. (26). Thus s^t_Θ/s^t_W ≤ 1 is not derived from the stated assumptions, and Theorem 17 does not follow as written. The empirical check in Table 8 is also not the right test: it compares full-Hessian eigenvalues and GPNRs at two noise-perturbed points, not the block-Hessian eigenvalues at one point. Moreover, on Chameleon Table 8 shows a true violation of Assumption 11 itself within that test, so even the stated assumptions are not met on all benchmark datasets.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies optimization of spectral GNNs whose parameters split into graph-filter parameters Θ and feature-transformation parameters W. It introduces a “block condition number” of the Hessian based on the largest eigenvalues of the two diagonal Hessian blocks, argues that spectral GNNs—especially on heterophilic graphs—are poorly conditioned in this sense, and proposes an asymmetric preconditioner that scales the Θ and W gradient blocks by the ratios of parameter norms to gradient norms. The authors claim (Theorem 17) that under Assumptions 7, 11, and 13 this preconditioning reduces the block condition number, and they report accuracy gains on 18 benchmark datasets with five spectral GNN baselines. The empirical study is broad and the method is simple to implement, but the theoretical argument as written has load-bearing gaps.","tokens_in":20658,"tokens_out":9726,"duration_ms":94028,"significance":"If the theoretical claim were established, the paper would provide a principled explanation for the common practice of using different effective learning rates for Θ and W in spectral GNNs, and it would offer a cheap training-time modification with clear practical payoff. The empirical contribution is substantial: 18 datasets, 5 spectral GNN architectures, code released, and consistent positive average gains that are especially large on small heterophilic graphs. However, the central theorem is not proven as stated: the proof uses an assumption in a regime it does not cover, the empirical validation tests a different quantity than the proof requires, and the preconditioned object H'=RH is not the Hessian of the actual transformed optimization problem. These issues are fixable in a revision, but they are central rather than cosmetic.","major_comments":[{"comment":"The paper states that preconditioning the gradient with R_t is equivalent to preconditioning the Hessian as H' = R_t H_t. This is not the standard equivalence. A block-diagonal positive scaling of the gradient corresponds, under a linear reparameterization, to a congruent transformation of the Hessian, H' = R_t^{1/2} H_t R_t^{1/2} (with the usual caveats about symmetrization), not to the left multiplication H' = R_t H_t. The matrix R_t H_t is generally non-symmetric, so the subsequent use of eigenvalue interlacing and the interpretation of the block condition number as a property of a Hessian are not justified. The authors should either prove the reduction for the Hessian that actually governs the preconditioned dynamics, or explicitly reframe the theorem as a statement about the diagonal blocks of R_t H_t without calling it the Hessian of the loss.","section":"§3, Eq. (9) and Theorem 17"},{"comment":"The decisive step in the proof of Theorem 17 is the implication: if λmax(H_t_{Θ,Θ}) ≥ λmax(H_t_{W,W}), then ρ_t_Θ ≥ ρ_t_W, which is then used to conclude s_t_Θ / s_t_W ≤ 1. The paper justifies this by invoking Assumption 11. But Assumption 11 is a statement about the full Hessian eigenvalues λmax(H_Ψ) and full-parameter GPNRs at two distinct nearby points Ψ and Ψ'. It does not relate the two diagonal-block eigenvalues λmax(H_Θ,Θ) and λmax(H_W,W) at a single point. Proposition 9 only provides separate upper bounds ρ_Θ ≤ λmax(H_Θ,Θ) and ρ_W ≤ λmax(H_W,W); these bounds cannot yield the required cross-parameter implication. Therefore Eq. (26) does not follow from the stated assumptions, and the reduction κ'(H'^t) ≤ κ'(H^t) is not established.","section":"Proof of Theorem 17, Eq. (26)"},{"comment":"The empirical validation of Assumption 11 in Table 8 is not a test of the assumption as used in Theorem 17. The table compares full-Hessian eigenvalues and full GPNRs at two noise-perturbed points, whereas the proof needs a monotonicity relation between the two diagonal block eigenvalues at the single current point. Moreover, even the stated full-Hessian version of Assumption 11 is violated on Chameleon: λmax(H_Ψ1) > λmax(H_Ψ2) but ρ_Ψ1 < ρ_Ψ2. Chameleon is one of the datasets where asymmetric learning gives the largest reported gains (Table 1), so the theoretical result does not cover a primary success case. The authors should either repair the theorem so it no longer requires this implication, or state and validate the actual block-level assumption used in Eq. (26).","section":"Table 8 and Assumption 11"},{"comment":"The role of Assumption 13 in the proof is to preserve the ordering of the two block eigenvalues after preconditioning, but the actual reduction κ'(H'^t) ≤ κ'(H^t) is obtained from Eq. (25) together with the upper bound s_t_Θ / s_t_W ≤ 1. Since s_t_Θ / s_t_W = ρ_t_W / ρ_t_Θ by Eq. (6), the theorem is essentially equivalent to assuming that the parameter block with the larger Hessian eigenvalue also has the larger GPNR. This property is the real content of the theorem, and it is not derived from Assumption 11 or from Proposition 9. The authors should make this dependence explicit and provide empirical evidence for the specific block-level GPNR ordering rather than for a different assumption.","section":"Assumption 13 and the structure of the proof"}],"minor_comments":[{"comment":"Equation (8) as printed, \"[∇ΘLS; ∇WLS] = R_t[∇ΘLS; ∇WLS]\", is false unless R_t is the identity; it should introduce a new symbol for the preconditioned gradient or use an assignment arrow, e.g., \"[∇̃Θ; ∇̃W] ← R_t[∇Θ; ∇W]\".","section":"§3, Eq. (8)"},{"comment":"In the displayed formulas for ρ_t_Θ and ρ_t_W, the denominators are written as ‖Θ‖2 and ‖W‖2 without the time index t; for consistency with Definition 5 and Eq. (5) they should be ‖Θ^t‖2 and ‖W^t‖2.","section":"Remark 6"},{"comment":"The last line of the proof states κ′(H′^t) ≤ κ(H^t); the right-hand side should be κ′(H^t), since the theorem compares block condition numbers.","section":"Proof of Theorem 17"},{"comment":"Several entries in Table 8 contain stray spaces (e.g., \"0 .016569\"), and the table caption should state explicitly which datasets violate Assumption 11; Chameleon is a violation, not merely a case where the assumption is untested.","section":"Table 8"},{"comment":"The claim that asymmetric learning \"consistently improves\" performance is stronger than the tables support: Table 2 contains negative deltas for ChebNetII on Cora, GPRGNN on Cora, and BernNet on Coauthor-Physics, and Table 6 contains negative deltas for ChebNetII on Questions and BernNet on Questions. The authors should qualify \"consistently\" or report a paired statistical test.","section":"Abstract and Q1"},{"comment":"The caption of Figure 2 refers to \"ChbNet,\" which is a typo for ChebNet.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The empirical study is the strongest part of this submission and would likely be publishable if the theory is either corrected or explicitly demoted to a heuristic explanation. The central proof issue is not a missing detail; Theorem 17 currently does not follow from the stated assumptions. I would not reject the paper on this basis, because a revised theorem with an honestly stated block-level assumption, properly validated on the exact quantities used in the proof, could restore the theoretical contribution. The authors should also be asked to make the connection between H' = R_t H_t and the actual optimization dynamics precise, since the present formulation is not the standard Hessian transformation under preconditioning."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This one is worth a look for the empirical result, not the theory. The authors propose asymmetric learning: before applying Adam, they rescale the Θ and W gradient blocks by the ratio of parameter norm to gradient norm, essentially a two-block LARS with exponential moving averages of parameter norms. They also introduce the block condition number—the ratio of the largest eigenvalues of the two diagonal Hessian blocks—as a diagnostic for why spectral GNNs are hard to optimize. On 18 datasets across five spectral models, the trick consistently gives a few points of accuracy, with the biggest gains on small heterophilic graphs (ChebNet on Texas +14, ChebNetII on Cornell +14). Code is out. That's a solid, reproducible empirical contribution.\n\nThe theory is where it falls apart. Theorem 17 claims the preconditioner reduces the block condition number. But H' = R H is not the Hessian of any reparameterized loss, so the optimization-landscape interpretation is hand-wavy. More seriously, the proof's key step needs the implication λmax(H_ΘΘ) ≥ λmax(H_WW) ⇒ ρ_Θ ≥ ρ_W. That is not what Assumption 11 says. Assumption 11 is about the full Hessian at two nearby points, not about diagonal blocks at one point. Proposition 9 only gives one-way upper bounds for each block, so you cannot get the cross-parameter ordering. The authors' own validation of Assumption 11 (Table 8) tests the wrong thing: it perturbs noise around a minimum and compares full Hessian eigenvalues, not block eigenvalues. And on Chameleon even that test shows a direct violation. On top of that, Assumption 13 is essentially the conclusion: it stipulates the exact ratio that makes the block condition number shrink. So the central theoretical claim is either unsupported or tautological.\n\nNone of this kills the empirical story. The method is cheap, easy to implement, and shows consistent gains, especially with sparse training labels. But the paper omits a natural baseline: LARS itself, or even just separate learning rates tuned per block. Given the update rule is recognizably LARS, that comparison is needed to know what the new framing buys you. The block condition number is a nice diagnostic, though, and might be worth keeping even if the proof is fixed or removed.\n\nTake it as a practical paper: the recipe works, the theory doesn't yet. A serious referee should send it back asking for the theorem to be corrected or downgraded, and for the LARS comparison. But it deserves referee time—the empirical result is reproducible and the method is simple enough to test in an afternoon.","headline":"Useful empirical trick with a broken proof; the LARS-like preconditioner consistently helps heterophilic GNN training, but the claimed block-condition-number reduction is not established.","tokens_in":21083,"tokens_out":3673,"would_cite":false,"duration_ms":37362,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Spectral GNNs train better when their two parameter blocks are updated at matched speeds; the paper proves this via a reduced block condition number and shows consistent accuracy gains on eighteen datasets.","keywords":["spectral graph neural networks","optimization landscape","Hessian block condition number","asymmetric learning","gradient preconditioning","heterophilic graphs","node classification","polynomial graph filters"],"falsifier":"Compute the block condition number at every iteration during ChebNet training on Texas with and without asymmetric learning. If $\\kappa'(H')$ is not smaller than $\\kappa'(H)$ in the early iterations while the method still improves final accuracy, the reduction in block condition number is not the mechanism explaining the gains.","tokens_in":20079,"feed_emoji":"⚖️","tokens_out":6936,"duration_ms":59863,"temperature":0.7,"pith_summary":"Spectral graph neural networks combine two very different parameter sets: the few coefficients $\\Theta$ of the polynomial graph filter and the many weights $W$ of the feature transformation. This paper argues that the mismatch between these blocks makes the loss landscape poorly conditioned, and that the damage is visible in a new quantity, the block condition number of the Hessian. It then proposes asymmetric learning, a training step that rescales the gradient for $\\Theta$ and $W$ separately so both parameter groups move at comparable speeds. The paper claims that this rescaling provably lowers the block condition number under stated assumptions, and that across eighteen benchmark datasets it consistently improves spectral GNN accuracy, with the largest gains on heterophilic graphs and small training sets. If true, it gives practitioners a drop-in training step that raises accuracy without changing model architecture.","feed_headline":"Rescaling two gradients fixes spectral GNN training","feed_subtitle":"An asymmetric preconditioner cuts the Hessian block condition number and lifts accuracy, mostly on heterophilic graphs.","key_machinery":"The engine of the argument is the block condition number of the Hessian, $\\kappa'(H)$, defined as the ratio of the larger to the smaller of the two largest eigenvalues of the diagonal blocks for the filter parameter $\\Theta$ and the feature parameter $W$. The method itself is the asymmetric preconditioner $R_t$: before each optimizer step, the gradient is rescaled block-wise by $s_{\\Theta}=\\|\\Theta\\|/\\|\\nabla_{\\Theta}L\\|$ and $s_W=\\|W\\|/\\|\\nabla_W L\\|$, so the gradient-parameter norm ratios of both blocks become 1. This rescaling acts as a Hessian preconditioner $H'_t=R_tH_t$, and Theorem 17 shows it shrinks $\\kappa'$ when the stated assumptions hold. The paper reads this as making the two parameter groups equally sensitive to updates, so neither block dominates the landscape.","core_discovery":"The paper's central claim is that the poor optimization of spectral GNNs is caused by unequal curvature in the two Hessian blocks for filter parameters $\\Theta$ and transformation parameters $W$, and that this can be fixed by asymmetric gradient preconditioning. Concretely, it defines the block condition number $\\kappa'(H)=\\max(\\lambda_{\\max}(H_{\\Theta,\\Theta}),\\lambda_{\\max}(H_{W,W}))/\\min(\\lambda_{\\max}(H_{\\Theta,\\Theta}),\\lambda_{\\max}(H_{W,W}))$, proves that applying the diagonal preconditioner $R_t=\\mathrm{diag}(s_{\\Theta} I_{d_\\Theta}, s_W I_{d_W})$ with $s_{\\Theta}=\\|\\Theta\\|/\\|\\nabla_{\\Theta}L\\|$ and $s_W=\\|W\\|/\\|\\nabla_W L\\|$ makes $\\kappa'(H'_t)\\leq \\kappa'(H_t)$ under Assumptions 7, 11, and 13, and reports consistent accuracy improvements when this rescaling is inserted before the optimizer on eighteen datasets. The empirical pattern it stresses is that heterophilic graphs have larger block condition numbers and receive the largest improvements, sometimes more than ten accuracy points on small graphs.","pith_inferences":["The same block-condition view could extend to other architectures with heterogeneous parameter groups, such as GNNs with separate encoder and classifier blocks; the paper does not test this.","The paper's own Table 8 shows Assumption 11 fails on Cornell, Actor, Chameleon, and Squirrel, so the proof's guarantee does not cover those datasets; the consistent accuracy gains there suggest the method may still work when the theorem does not apply, but the theoretical explanation would need a weaker assumption.","A direct test of the causal story would be to check whether the measured block condition number actually drops at every step on a dataset where the assumption holds; the paper provides empirical support on some datasets but not a per-iteration trace for all."],"forward_implications":["Asymmetric learning can be added to any optimizer, so existing spectral GNN baselines improve without architectural changes; the paper demonstrates this with Adam and five spectral models.","Heterophilic graphs, where the block condition number is largest, stand to gain the most; average improvements are about 3.08 accuracy points on six small heterophilic datasets and about 0.46 points on homophilic ones.","Small graphs with sparse training labels show the largest gains, with improvements above 10 points on Texas, Wisconsin, Cornell, and Chameleon, suggesting the method helps when gradient estimates are noisy.","Orthogonal polynomial-basis GNNs benefit more than non-orthogonal ones, consistent with the paper's view that interference among filter coefficients weakens a single uniform scaling."],"supporting_citations":[{"why":"Supplies ChebNet, the spectral GNN whose loss landscape is analyzed and the main experimental baseline.","marker":"(Defferrard, Bresson, and Vandergheynst 2016)"},{"why":"Supplies BernNet, a non-orthogonal spectral GNN baseline tested with and without asymmetric learning.","marker":"(He et al. 2021)"},{"why":"Supplies ChebNetII, an orthogonal-basis baseline, and illustrates the common practice of using separate learning rates for filter and feature parameters.","marker":"(He, Wei, and Wen 2022)"},{"why":"Supplies JacobiConv, an orthogonal-basis spectral GNN baseline used in the experiments.","marker":"(Wang and Zhang 2022b)"},{"why":"Supplies GPRGNN, a monomial-basis baseline, and the Cora/Citeseer/Pubmed data split adopted by the experiments.","marker":"(Chien et al. 2021)"},{"why":"Supplies Adam, the optimizer that asymmetric learning is combined with in the main experiments.","marker":"(Kingma and Ba 2014)"},{"why":"Supplies the standard condition-number definition that the block condition number extends.","marker":"(Golub and Van Loan 2013)"},{"why":"Supplies the interlacing theorem used to relate the block condition number to the full Hessian condition number.","marker":"(Horn and Johnson 2012)"},{"why":"Supplies the learning-rate-to-Hessian-eigenvalue relation that motivates the block-wise learning-rate analysis.","marker":"(Granziol, Zohren, and Roberts 2020)"},{"why":"Supplies the Weyl inequality used in the proof of Proposition 9.","marker":"(Tao 2012)"}],"fun_headline_variants":["Asymmetric gradient scaling tames spectral GNN optimization","Spectral GNNs train better with asymmetric gradients","Rescale gradients to cut spectral GNN Hessian gaps","Asymmetric updates fix hard heterophilic graph training","Dynamic asymmetric preconditioning improves spectral GNNs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the ordering of largest Hessian eigenvalues between two nearby points is mirrored by the ordering of gradient-parameter norm ratios; the paper's Table 8 shows this premise fails on Cornell, Actor, Chameleon, and Squirrel, so the proof's guarantee does not cover those cases.","fun_headline_variants_meta":{"raw":{"variants":["Asymmetric gradient scaling tames spectral GNN optimization","Spectral GNNs train better with asymmetric gradients","Rescale gradients to cut spectral GNN Hessian gaps","Asymmetric updates fix hard heterophilic graph training","Dynamic asymmetric preconditioning improves spectral GNNs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000904,"raw_usage":{"total_tokens":3906,"prompt_tokens":977,"completion_tokens":2929,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":593,"completion_tokens_details":{"reasoning_tokens":2854}},"tokens_in":593,"tokens_out":2929,"duration_ms":19984,"temperature":1.0,"reasoning_tokens":2854,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:37:47.249298+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the block condition number at every iteration during ChebNet training on Texas with and without asymmetric learning. If $\\kappa'(H')$ is not smaller than $\\kappa'(H)$ in the early iterations while the method still improves final accuracy, the reduction in block condition number is not the mechanism explaining the gains.","supporting_citations":[],"review_version":1}