{"id":"08f782fd-bb8d-4e27-b564-4b77977f8490","arxiv_id":"2502.01337","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"NPO, a transformer-based neural algebraic multigrid operator, learns to approximate the inverse of discretized PDE matrices and accelerates GMRES, though the evidence omits the AMG baseline and contains data inconsistencies.","lead":"A new neural network that mimics algebraic multigrid is trained to act as a preconditioner for Krylov solvers, reducing iteration counts on small Poisson, diffusion, and elasticity problems. The paper claims the learned preconditioner also generalizes to larger meshes than it was trained on, but key baselines and reproducibility details are missing.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline claim that NPO 'outperforms traditional methods' is untested because the AMG baseline used to generate the training data is absent from all comparisons; without it, the claimed significance is unsupported.","rationale":"The reader's weakest assumption was that the trained NAMG operators satisfy the multigrid Approximation Property and smoothing stability, making the theoretical convergence guarantee conditional. That is a valid concern, but it targets the theoretical framing rather than the paper's primary empirical assertion. My stress-test identifies a more direct problem: the empirical evaluation omits the AMG baseline that generated the training data. This omission undermines the central claim that NPO 'significantly reduces iteration counts and runtime' and 'outperforms traditional methods,' because AMG is the standard traditional preconditioner for the tested elliptic PDEs and is the natural reference point for a learned multigrid operator. The concern is concrete and testable: adding AMG to the comparison would either validate or refute the headline. The reader did mention the missing AMG baseline in the rationale, so we partially agree, but the reader's weakest_assumption field focused on theory. This stress-test elevates the missing baseline to the load-bearing concern because the paper's stated contribution is empirical superiority over classical methods. The verdict remains CONDITIONAL: the authors should add the AMG baseline, correct the impossible GPU-memory figures, and verify (or prove) the learned operators' multigrid properties, but the central idea is not disproven by this analysis.","tokens_in":14160,"tokens_out":2775,"duration_ms":25520,"concrete_test":"Re-run the experiments in Table 1 and Table 2 with the same GMRES (or CG) solver preconditioned by a standard AMG implementation (e.g., PyAMG or BoomerAMG), using identical tolerances (1e-10, 1e-6, 1e-4) and reporting both iteration counts and wall-clock time. Also include the AMG baseline in the resolution generalization experiment of Figure 4 for grid sizes 128, 1024, 2048, and 4096. If AMG matches or exceeds NPO in time or iterations, the central claim of outperforming traditional methods fails; if NPO genuinely beats AMG, the claim is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that NPO significantly reduces iteration counts and runtime relative to classical and neural methods. But Section 5.1.2 states that the training dataset is generated by running a baseline Krylov solver (CG or GMRES) preconditioned by Algebraic Multigrid (AMG), with tolerance 1e-10. This means the learned operator is effectively distilling AMG behavior. Yet Table 1 and Table 2 compare NPO only against Jacobi, Gauss-Seidel, SOR, and neural baselines; AMG is never included as a baseline in any experiment, including the resolution generalization study in Figure 4. AMG is the canonical classical preconditioner for the very elliptic problems considered (Poisson, diffusion, linear elasticity) and is known to achieve near-optimal iteration counts on such problems. If AMG converges in, say, 10-20 iterations on the 512 Poisson grid at 1e-10, then NPO's 184 iterations would be roughly an order of magnitude worse, directly contradicting the abstract's claim that NPO 'significantly reduces iteration counts' and 'outperforms traditional methods.' The omission is not a minor oversight; it removes the most relevant classical competitor from the evaluation. The theoretical guarantees in Section 4 are also conditional on unverified properties of the learned operators, but the empirical comparison against AMG is the load-bearing issue because the paper's primary evidence is experimental. Without an AMG baseline, the central claim that NPO beats classical solvers is unsubstantiated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces the Neural Preconditioning Operator (NPO), a learned preconditioner that combines an algebraic-multigrid-inspired restriction/prolongation structure with transformer-style attention and is trained with data, residual, and condition losses. The learned operator is applied as a left preconditioner for Krylov solvers (GMRES/CG). The authors claim two-grid convergence and spectrum-clustering guarantees for the NAMG operator, and report experiments on Poisson, diffusion, and linear elasticity problems on uniform and irregular meshes, showing lower iteration counts and runtimes than Jacobi, Gauss–Seidel, SOR, MLP, U-Net, FNO, Transolver, and M2NO, plus a resolution-generalization study up to 4096.","tokens_in":14497,"tokens_out":3496,"duration_ms":32584,"significance":"If the empirical comparison were complete, the reported result that NPO converges to 1e-10 on the 512 Poisson problem in 184 iterations and 0.623 s versus 513 iterations and 3.719 s for Jacobi would be a useful demonstration of a learned preconditioner for small elliptic systems. The paper is also valuable for articulating concrete losses (condition and residual) for training preconditioners and for combining multigrid ideas with neural attention. However, the significance is currently limited by two load-bearing issues: (i) the training data are generated with an AMG-preconditioned Krylov solver that is never included as a baseline, so the central claim of outperforming traditional methods is not tested against the most relevant classical competitor; and (ii) the theoretical guarantees in Section 4 are conditional on unverified multigrid properties of the learned operators. No code or data are provided, which also makes reproducibility harder to assess.","major_comments":[{"comment":"The dataset generation (Section 5.1.2) states that each system is solved with a baseline Krylov solver (CG or GMRES) preconditioned by Algebraic Multigrid (AMG) with tolerance 1e-10 and a cap of 100 iterations, and the recorded partial solutions and residuals are used to train M_theta. Yet AMG is absent from every experimental comparison: Tables 1 and 2 and Figure 4 compare NPO only against Jacobi, Gauss–Seidel, SOR, and neural baselines. Since NPO is trained to imitate the AMG-preconditioned solver's behavior, omitting that solver removes the most relevant classical competitor. Please add AMG-preconditioned CG/GMRES as a baseline in all experiments and report both iteration counts and wall-clock times. If AMG already converges in tens of iterations on the 512 Poisson problem, then NPO's 184 iterations would not support the abstract's claim that NPO 'significantly reduces iteration counts' relative to traditional methods. This is the load-bearing issue for the paper's primary empirical claim.","section":"§5.1.2, Tables 1–2, Figure 4"},{"comment":"The convergence guarantee in Theorem 4.1 and the spectrum-clustering guarantee in Theorem 4.3 are derived under the Approximation Property (Property 4.2) and smoothing stability for the restriction, prolongation, and smoothing operators. For the neural setting, these properties are assumed, not verified for the trained NPO. The proof of Theorem 4.3 in Appendix C.3 is a qualitative restatement of classical multigrid theory and does not establish quantitative eigenvalue bounds for the learned operator. Please either (a) verify the assumptions numerically for the trained networks, e.g., by estimating the approximation constant alpha and the smoothing factor nu on test matrices, or (b) state explicitly that the theoretical results apply only if the learned operators satisfy these properties. As written, the conclusion in Section 6 that 'theoretical analysis confirms that NAMG inherits two-grid convergence guarantees' is stronger than what is proven.","section":"§4, Appendix C.2–C.3"},{"comment":"The resolution-generalization claim—that NPO maintains robust convergence on grids as large as 4096—is supported only by an unlabeled plot. Figure 4 has no numeric values, so the reader cannot check the claimed iteration counts or the relative scaling of NPO versus the other methods. Please provide a table or labeled points with exact iteration counts (and ideally wall-clock times) for every method at resolutions 128, 1024, 2048, and 4096, including the training resolution 128. Note also that Table 5 reports training-phase efficiency only; it does not quantify solve-time scaling.","section":"§5.3, Figure 4"}],"minor_comments":[{"comment":"Appendix A contains unresolved reference placeholders '[? ? ]' and '[? ]' in the matrix reordering and multilevel methods discussion, and the section title 'Neural Preconditoner' has a typo ('Preconditoner' should be 'Preconditioner').","section":"Appendix A"},{"comment":"The text in Section 5.2 and the Figure 3 caption refer to a '512 × 512' grid, while Table 1 and the surrounding text describe the experiment as 'Grid=512' and '512 grid'. Please clarify whether the Poisson experiment is one-dimensional or two-dimensional, since this affects the interpretation of iteration counts and runtimes.","section":"Figure 3, Table 1, §5.2"},{"comment":"Table 1 has duplicated and misaligned column headers (e.g., 'ITERATION' and '1E-10' appear twice under each grid type). Reformat the table so each tolerance column is clearly labeled once.","section":"Table 1"},{"comment":"The training objective in Section 5.1.1 mentions only Data Loss and Residual Loss, but Section 3.3 defines a Condition Loss as well. It is unclear how the three losses are combined and weighted; the hyperparameter study in Table 4 also does not report loss-weight settings. Please state the full combined objective and the loss-weight values used.","section":"§5.1.1, §3.3, Table 4"},{"comment":"The heading 'Primary configurations for Nerual Preconditioning Operator' contains a typo ('Nerual' should be 'Neural').","section":"Appendix D.2"},{"comment":"Theorem 4.4 and its proof show that the attention aggregation can be viewed as a Monte-Carlo approximation of an integral operator. This is a representational observation rather than a quantitative convergence guarantee; labeling it a theorem may overstate its role in the paper's theoretical contributions.","section":"Theorem 4.4, Appendix C.4"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant and timely topic, and the proposed framework is sensible. However, the empirical evaluation omits the exact classical solver family used to generate the training data, which is the most natural baseline for a learned preconditioner. This is not a matter of experimental style but of the central claim's support: if AMG alone converges in far fewer iterations than NPO on these elliptic problems, the headline claims in the abstract and conclusion would need substantial revision. The theoretical section also needs to be reframed as conditional on assumptions that are either verified or explicitly stated as unverified. I would like to see the revised comparison before recommending acceptance, so major revision seems appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this one. First, the main empirical claim is untested: the paper trains on data generated by an AMG-preconditioned Krylov solver, but never compares against AMG itself. Second, the efficiency table in Appendix D contains GPU memory numbers that are physically impossible (280,320 MB for a 4096 matrix and 5.5 million MB for M2NO). Neither of these is a minor slip.\n\nWhat is actually new: the specific NAMG architecture, which inserts learned attention-based restriction and prolongation into a multigrid-style preconditioner and trains it with both condition and residual losses. That specific combination is not in the cited prior work. The paper is also clearly written and the ablation study is genuinely informative—removing the data loss, the AMG-style components, or the attention modules all degrade performance in sensible ways.\n\nThe soft spots are real. Section 5.1.2 says the dataset is produced by running CG or GMRES preconditioned by AMG at tolerance 1e-10. So NPO is essentially distilling AMG behavior. Yet Tables 1 and 2 only test against Jacobi, Gauss-Seidel, SOR, and neural baselines. AMG is the canonical classical preconditioner for the exact elliptic problems studied here. If AMG converges on the 512-grid Poisson problem in, say, 15–20 iterations, then NPO's 184 iterations is a regression, not an improvement, and the abstract's claim of outperforming traditional methods collapses. This is the load-bearing issue.\n\nThe theoretical section is also weaker than it looks. Theorems 4.1 and 4.3 assume the learned restriction and prolongation satisfy the Approximation Property and smoothing stability, but the proofs in Appendix C simply restate standard multigrid results and assert that the neural operators 'play a role analogous to' classical ones. No verification is offered for the trained networks. So the convergence guarantee is conditional on unverified assumptions.\n\nWho gets value from this? Someone working on learned preconditioners might take the NAMG architecture as a starting point, but only after the evaluation is redone. The paper deserves a serious referee because the idea is plausible and the problem matters, but it needs major revision: add the missing AMG baseline, correct the efficiency table, and either verify or appropriately soften the theoretical claims.","headline":"The central claim that NPO beats traditional solvers is untested because AMG, the solver used to generate the training data, never appears as a baseline; the appendix also contains physically impossible memory numbers.","tokens_in":15012,"tokens_out":2650,"would_cite":false,"duration_ms":25048,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["65F10","65N22","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"The NPO method learns a multigrid-style preconditioner that accelerates Krylov solvers for elliptic PDEs and transfers to grids far beyond its training resolution.","keywords":["neural preconditioning","algebraic multigrid","neural operators","Krylov subspace methods","PDE solvers","transformer attention","eigenvalue clustering","resolution generalization"],"falsifier":"Run the trained NAMG as a two-grid cycle on a Poisson problem at several grid sizes, starting from a random error, and measure the energy-norm contraction ratio per cycle and the largest eigenvalue spread of the preconditioned matrix $M A$; if the measured contraction ratio is not below 1 or becomes close to 1 as n grows, or if the eigenvalues of $M A$ are not clustered near 1, then the claimed convergence and spectral guarantees fail.","tokens_in":13961,"feed_emoji":"⚡","tokens_out":8147,"duration_ms":69782,"temperature":0.7,"pith_summary":"This paper proposes a way to speed up solving discretized PDEs: train a neural network to act as a preconditioner for Krylov iterative solvers, the class of linear-system solvers that build approximations in nested subspaces. The network, called the Neural Preconditioning Operator (NPO), learns a matrix $M$ that approximates the inverse of the system matrix $A$, so that the preconditioned problem $M A x = M b$ has eigenvalues clustered near 1 and converges in fewer iterations. The architecture blends algebraic multigrid with a transformer-based attention mechanism, so it reduces both high- and low-frequency errors on coarse and fine grids. The paper reports that on Poisson, diffusion, and linear elasticity problems, NPO converges in fewer iterations and less wall-clock time than classical and neural baselines, and that a model trained on a 128 resolution still works on grids up to 4096.","feed_headline":"Neural preconditioner cuts Poisson solve iterations by 64%","feed_subtitle":"Trained once at 128 resolution, it keeps converging on grids up to 4096 and outperforms classical and neural baselines.","key_machinery":"The load-bearing object is the NAMG (Neural Algebraic Multigrid) operator, which replaces the hand-built restriction and prolongation matrices of algebraic multigrid with learned attention weights. In the paper's notation, restriction is $R = A E$ and prolongation is $P = A E^T$, where $E$ contains softmax attention scores over graph neighborhoods; coarse features are refined by self-attention, then corrected back to the fine grid. The training objectives are the condition loss, which penalizes the norm of $(I - A M(A)) r$ over sampled residuals, and the residual loss, which penalizes the norm of $A M(A) b - b$; these steer $M$ toward $A^{-1}$ without forming the full matrix inverse. The theoretical part of the paper uses the standard two-grid contraction argument, whose contraction factor depends on the Approximation Property and smoothing stability, plus a spectral argument that $M A$ has eigenvalues in an interval near 1.","core_discovery":"The central claim is that a data-driven operator trained with condition, residual, and data losses can serve as an effective preconditioner for Krylov solvers across mesh types and PDE families. The learned operator is explicitly constructed as a Neural Algebraic Multigrid (NAMG), with learned attention-based restriction and prolongation, and the paper argues that this construction inherits the two-grid convergence guarantee and the eigenvalue-clustering property of classical multigrid, so that the preconditioned matrix $M A$ behaves close to the identity. The numerical evidence is that NPO reaches a relative residual of 1e-10 on a 512-grid Poisson problem in 184 iterations and 0.623 seconds versus 513 iterations and 3.719 seconds for Jacobi, and that it remains competitive on irregular meshes and on grids up to 4096 although training was done at 128.","pith_inferences":["If the spectral clustering claim holds for trained operators across PDE families, the same learned $M$ could be reused as a warm-start preconditioner inside nonlinear solves or time-stepping schemes that repeatedly solve nearby linear systems, where the reported generalization is most valuable.","The reported parameter count (0.14 MB) suggests NPO captures structural features of the discrete operator rather than memorizing a particular mesh or right-hand side; a test to confirm this is to train once on one domain shape and evaluate on a topologically different mesh with holes or reentrant corners.","Because the condition and residual losses are defined only through matrix-vector products with $A$, the framework could extend to implicit or matrix-free operators where $A$ is never explicitly assembled, broadening its reach beyond the symmetric positive-definite elliptic cases shown."],"forward_implications":["On the tested elliptic problems (Poisson, diffusion, linear elasticity), the preconditioned GMRES converges in fewer iterations and less time than Jacobi, Gauss–Seidel, SOR, and the listed neural baselines, at tolerances from 1e-4 to 1e-10.","A model trained only at resolution 128 retains useful preconditioning quality at 1024, 2048, and 4096, so the mesh resolution does not need to be a training parameter.","The theoretical analysis, conditional on the Approximation Property and smoothing stability of the learned operators, implies the two-grid iteration contracts the error in the energy norm at a rate independent of system size, and that the preconditioned system has a condition number near 1.","Ablations show that the NAMG structure, the system matrix $A$ as input, and the data loss each contribute materially; removing the full NAMG raises iteration count from 184 to 314, and removing the data loss from 184 to 206."],"supporting_citations":[{"why":"Supplies the classical two-grid convergence theory and the Approximation Property that the paper's Theorem 4.1 and Theorem 4.3 invoke.","marker":"[2]"},{"why":"Supplies the neural operator layer composition that the NPO framework uses as its base architecture.","marker":"[18]"},{"why":"Supplies the prior multigrid-augmented operator learning approach that NPO benchmarks against and extends.","marker":"[16]"},{"why":"Supplies the transformer-based PDE solver baseline that NPO compares against on the same meshes.","marker":"[28]"},{"why":"Supplies the learned-preconditioner setting of training $M$ toward $A^{-1}$ that motivates the condition and residual losses.","marker":"[15]"},{"why":"Supplies the related neural-operator-plus-Krylov baseline for comparison.","marker":"[23]"}],"fun_headline_variants":["Cuts 64% of Poisson solve iterations with neural preconditioner","Trained at 128, solves PDEs on grids up to 4096","Neural multigrid preconditioner beats classical baselines","Data-driven preconditioner speeds Krylov solvers on irregular meshes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof that NPO converges relies on the assumption, never verified numerically, that the learned attention-based restriction and prolongation operators satisfy the multigrid Approximation Property and smooth stably; if the trained network fails these conditions, the two-grid contraction guarantee does not apply to the actual learned preconditioner.","fun_headline_variants_meta":{"raw":{"variants":["Cuts 64% of Poisson solve iterations with neural preconditioner","Trained at 128, solves PDEs on grids up to 4096","Neural multigrid preconditioner beats classical baselines","Data-driven preconditioner speeds Krylov solvers on irregular meshes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000611,"raw_usage":{"total_tokens":2815,"prompt_tokens":892,"completion_tokens":1923,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":508,"completion_tokens_details":{"reasoning_tokens":1847}},"tokens_in":508,"tokens_out":1923,"duration_ms":13492,"temperature":1.0,"reasoning_tokens":1847,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T15:38:12.994964+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the trained NAMG as a two-grid cycle on a Poisson problem at several grid sizes, starting from a random error, and measure the energy-norm contraction ratio per cycle and the largest eigenvalue spread of the preconditioned matrix $M A$; if the measured contraction ratio is not below 1 or becomes close to 1 as n grows, or if the eigenvalues of $M A$ are not clustered near 1, then the claimed convergence and spectral guarantees fail.","supporting_citations":[{"cited_title":"Briggs, V .E","cited_arxiv_id":null,"evidence_quote":"Supplies the classical two-grid convergence theory and the Approximation Property that the paper's Theorem 4.1 and Theorem 4.3 invoke."},{"cited_title":"Stuart, and Anima Anandkumar","cited_arxiv_id":null,"evidence_quote":"Supplies the neural operator layer composition that the NPO framework uses as its base architecture."},{"cited_title":"M2NO: multiresolution operator learning with multiwavelet-based algebraic multigrid method","cited_arxiv_id":null,"evidence_quote":"Supplies the prior multigrid-augmented operator learning approach that NPO benchmarks against and extends."},{"cited_title":"Learning preconditioners for conjugate gradient PDE solvers","cited_arxiv_id":null,"evidence_quote":"Supplies the learned-preconditioner setting of training $M$ toward $A^{-1}$ that motivates the condition and residual losses."},{"cited_title":"Muravleva, Yuri M","cited_arxiv_id":null,"evidence_quote":"Supplies the related neural-operator-plus-Krylov baseline for comparison."}],"review_version":1}