Pith. sign in

REVIEW 35 references

A domain decomposition online-learning-enhanced nonlinear elimination preconditioner

T0 review · reviewed 2026-08-01 · deepseek-v4-flash

Pith's one-line read Thresholding a learned reconstruction of the residual, not the raw residual, makes nonlinear-elimination Newton solvers converge at Re up to 10,000.

desk verdict The learned-extraction idea is interesting and the experiments are extensive, but the central baseline comparison is confounded by the early-restart difference, so the robustness claim overreaches. read the letter →

arxiv 2607.17723 v1 pith:OSCZP2DB submitted 2026-07-20 math.NA cs.NA

classification math.NAcs.NA MSC 49M1568T0765N5576D05
keywords inexactNewtonnonlineareliminationpreconditionerprincipalcomponentanalysisautoencoderdomaindecompositionlid-drivencavityflowbadsubsetselectionresidualreconstruction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the unreliable step in nonlinear elimination preconditioning—choosing which unknowns to eliminate—can be fixed by learning the dominant structure of the residual online. During the stagnation phase of a Newton solve, residual snapshots are collected and an unsupervised extractor (PCA or an autoencoder) is trained to reconstruct the residual while suppressing oscillatory perturbations. Thresholding this reconstruction instead of the raw residual selects a compact, coherent bad subset. On lid-driven cavity flow up to Re=10,000, the method converges across a wide range of threshold and tolerance parameters, whereas the baseline NE preconditioner stagnates.

What carries the argument

The extraction operator G is the load-bearing object. It is learned online from residual snapshots collected during the stagnation window [k_s, k_t]. Two realizations are considered: a PCA-based linear operator (projection onto leading singular vectors plus the snapshot mean) and autoencoder neural networks (MLP and CNN). The reconstructed residual F = G(F) is used in place of the raw residual in the bad-subset selection criterion, and the same reconstruction is used in the restricted nonlinear solve. The operator is trained locally on each subdomain and assembled globally, so the learning step fits the existing domain-decomposition parallelism.

What would settle it

Construct a nonlinear problem with an exactly known slow subspace (e.g., a two-scale system) and compute the norm of the true slow residual component lying outside the kept modes of the learned extractor. If that norm is not small while the method still converges, or if a deliberately engineered high-frequency stiff mode is filtered out causing the solver to stagnate, the central separation claim fails.

Watch

Extended reading notes

Core claim

The central claim is that the bad subset for nonlinear elimination should be selected from a data-driven reconstruction G(F(X)) of the residual rather than from F(X) itself. The reconstruction is obtained by training an extractor on residual snapshots gathered during the stagnation phase. The reconstruction preserves the dominant 'slow' components responsible for Newton stagnation and filters out 'fast' oscillatory components; the selection criterion then becomes robust to the threshold and to the inner-solve tolerance. With local models trained independently per subdomain, the method converges for Re=10,000 cavity flow where the baseline NE preconditioner stagnates, and the selected bad sub

Load-bearing premise

The learned extractor must preserve exactly the residual components that drive Newton's slow convergence, while everything it discards must be irrelevant oscillation; this separation is asserted rather than proved.

Editorial extensions

If this is right

  • Baseline NE with raw-residual thresholding fails to converge at Re=7,500 and Re=10,000 for the tested parameters, while all three learned variants (PCA, MLP, CNN) converge.
  • The learned method converges for all 20 tested combinations of threshold rho and inner tolerance gamma at Re=10,000, whereas the baseline converges in only 8 of 20 cases.
  • Increasing the number of subdomains improves both parallelism and the quality of local extraction; with only two subdomains the CNN variant fails, while with four or more it converges robustly.
  • The learned extractors are not equivalent to fixed smoothing: Gaussian-smoothed residuals do not restore convergence, showing the adaptivity of the learned models is essential.
  • Earlier restart from the stagnation phase is more robust; restarting too late from a strongly stagnated trajectory degrades or destroys convergence.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable prediction follows: the kept modes of the learned extraction should contain the true slow directions; one could compare the learned bad subset with an oracle subset obtained from a known physical stiffness indicator and measure the overlap.
  • Because the method is algebraic and the residual reconstruction is problem-agnostic, it should transfer to other nonlinear PDE systems with localized stiffness, such as multiphase flow or contact problems, although the paper demonstrates only cavity flow.
  • The reliance on a low-dimensional bottleneck implies the slow components must be approximately low-rank; for problems where the stiff directions are spread across many modes or high-frequency components, the learned reconstruction could discard genuinely difficult parts, and the claimed robustness may not hold.
  • A fully online extension could retrain the extractor during the restarted Newton phase instead of only once at k_t, potentially adapting to residual structures that change after elimination.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Circularity Check

1 steps flagged · score 6.0 of 10

The restart-point residual reconstruction is in-sample (and for k_re=1 uses later Newton iterates), so the claimed 'learned' bad-subset selection is partly a retrospective fit rather than an independent prediction; the baseline comparison also varies the restart point, though that is a confound rather than circularity.

  1. fitted input called prediction [Algorithm 2 (Steps 2.1 and 3.1), Eq. (21), §3.4.1, §4.5]
    "Given an initial guess X0, ... the residual collection window [ks, kt], the restart step kre ∈ [ks, kt], and the bad subset threshold ρ. ... Step 2.1: For each subdomain Ωi, construct and train a local extraction model Gi ... from the collected residual snapshots. ... Step 3.1: Set Y0 = Xkre ... Eq. (21): Gi,PCA(R0_i F(X)) := P_i P_i^T (R0_i F(X) − ar F_i) + ar F_i."

    Because k_re ∈ [k_s,k_t], the restart point Y0 = X^{k_re} is one of the residual snapshots used in Step 2.1 to train the extractor. For PCA, Eq. (21) is the orthogonal projection of that very training snapshot onto the leading left-singular vectors of the snapshot matrix that contains it; the 'dominant structure' is therefore the least-squares reconstruction of the point on which the bad-subset selection is later made. In §4.5, with k_re=1 and k_t=9, the model is trained on F(X^2),...,F(X^9), i.e., on iterates that occur after the restart point, and is then applied at X^1. The claimed more-reliable, more-coherent subset is thus a retrospective in-sample fit, not an independent prediction; the advantage over raw thresholding is at least partly built into the training protocol.

full rationale

The paper's core contribution is experimental rather than theorem-based, so I looked for places where a numerical 'improvement' reduces to the way the training data and the test/application point coincide. The clearest such point is the restart mechanism: Algorithm 2 trains the extraction model on snapshots from [k_s,k_t] and then restarts at X^{k_re} with k_re inside that same window. For PCA, Eq. (21) reconstructs a training-point residual by projecting it onto the SVD subspace of the snapshot matrix containing that point; for the autoencoders, the same in-sample training applies. When k_re=1, the model is even trained on later iterates (X^2,...,X^{k_t}) before being used to 'identify' the structure at X^1, so the online-learning narrative masks future-data leakage. This is genuine partial circularity because the learned residual representation at the restart point is a fitted function of the very data it is then used to interpret. I do not treat the separate baseline issue as circularity: §4.1 claims the baseline is activated at the same k_re, while §4.5 says baseline NE starts from the ninth Newton iteration versus k_re=1 for DD-OLE-NE; that is a confounded experimental comparison and belongs under correctness/validity, not under the definitional-equivalence notion used here. Self-citations ([16], [17]) are related prior work by the same group but are not load-bearing or invoked as external uniqueness theorems; the PCA/autoencoder construction is stated and implemented in this paper. Score 6 reflects the in-sample/future-data leakage in the central extraction step, without claiming that every favorable convergence result is vacuous.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The method introduces no new physical entities; the burden lies in modeling assumptions about residual structure and in the in-sample use of the learned extractor. The central claim depends on several user-chosen parameters (rho, d, k_t, k_re, gamma_s^r, NN design) and on two unproven approximations: the slow/fast mode separation and the raw-Jacobian surrogate for the reconstructed Jacobian.

free parameters (5)
  • bad-subset threshold rho = 0.1, 0.2, 0.35, 0.4, 0.5 across experiments
    Defines the bad subset in Eq. (11); the paper studies sensitivity but must select a value per run.
  • PCA retained modes d = 5 for Re=7,500, 8 for Re=10,000 in Table 1
    The only PCA model parameter, chosen per test case; no sensitivity study for d is reported.
  • snapshot window length k_t and restart index k_re = k_t varies 3-9, k_re varies 1-7 in Tables 1 and 4
    Controls both training data and the point at which the learned correction is applied; strongly affects convergence and timing.
  • inner subspace tolerance gamma_s^r = 0.01, 0.05, 0.10, 0.20
    Stopping tolerance for the restricted nonlinear solve, Eq. (8); varied in sensitivity study.
  • NN training budget and architecture = epochs 150/300/500; various kernel/channel sequences
    The paper shows MLP fails at 150 epochs in Table 5, so the choice of training budget is consequential.
assumptions (3)
  • domain assumption Low-energy PCA/autoencoder modes are oscillatory perturbations, not part of the slow nonlinear dynamics.
    Used in Section 3.1 to justify thresholding the reconstructed residual in Eq. (11); no spectral gap or error bound is provided.
  • domain assumption The Jacobian of the original residual F' is a valid surrogate for the derivative of the reconstructed residual F'.
    Explicitly assumed in Section 3.4.3 ('F' serves as an approximation to F'') to avoid differentiating the extraction model; no error analysis is given.
  • domain assumption Residual snapshots from the stagnation window [k_s, k_t] are representative of the nonlinear imbalance at the restart point X^{k_re}.
    The training set in Eq. (14) and Phase 2/3 of Algorithm 2 relies on this representativeness, especially when k_re < k_t.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A domain decomposition online-learning-enhanced nonlinear elimination preconditioner." pith.science (2026). https://pith.science/paper/OSCZP2DB

@misc{pith2026260717723,
  author       = {Pith},
  title        = {Pith review of: A domain decomposition online-learning-enhanced nonlinear elimination preconditioner},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OSCZP2DB}},
  note         = {Machine review of arXiv:2607.17723}
}
read the original abstract

Nonlinearly preconditioned inexact Newton methods form an effective class of solvers for large-scale nonlinear algebraic systems arising from the discretization of partial differential equations. A central challenge in nonlinear elimination (NE) preconditioning is the reliable identification of the slowly converging components to be eliminated. Existing selection strategies often rely on problem-specific physical information or user-tuned thresholds applied directly to the raw nonlinear residual, which may contain irregular oscillatory structures near stagnation regions, making the selected bad subset highly sensitive to threshold parameters. In this work, we propose an online-learning-enhanced NE preconditioner that identifies the bad subset from the dominant structure of the nonlinear residual rather than from the raw residual itself. Residual snapshots are collected online during the stagnation phase of the current Newton solve, and an unsupervised extraction model is trained to capture the principal nonlinear imbalance. We consider both a linear extractor based on principal component analysis and nonlinear extractors based on autoencoder neural networks. Moreover, we integrate the approach into a parallel domain decomposition framework, which trains a local extraction model independently on each subdomain. The learned residual reconstruction is then used to define the bad subset and guide the nonlinear elimination process. Numerical experiments on lid-driven cavity flows at Reynolds numbers up to 10,000 show that the proposed method produces more reliable and coherent bad subsets, is robust with respect to both NE and learning parameters, and outperforms the baseline NE preconditioner in terms of the convergence.

Figures

Figures reproduced from arXiv: 2607.17723 by the authors.

Figure 1
Figure 1. A visual illustration of the residual F(Xk ) (left column, with the threshold level ρ∥F(Xk )∥∞ shown in blue) and the selected bad components RbF(Xk ) (right column) for thresholds ρ = 0.1, 0.2 and 0.3 (top to bottom). A small threshold leaves highly oscillatory protrusions in the bad subset, while a large threshold discards part of the dominant structure. raw residual with its reconstruction in the selection criter… view at source ↗
Figure 2
Figure 2. A schematic illustration of the proposed online-learning-based extraction of the dominant residual [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. A typical residual curve obtained using IN when the method fails to converge. The part in the [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Subdomain-wise extraction models used in the proposed DD-OLE-NE preconditioner. The [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Streamlines of the lid-driven cavity flow obtained by using the DD-OLE-NE preconditioned inexact [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Centerline velocity profiles compared with the results of Ghia et al. [34] at [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Nonlinear residual histories for baseline NE, DD-OLE-NE(PCA), DD-OLE-NE(MLP), and DD [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Comparison of the selected bad subset and the corresponding restricted vorticity residual surfaces [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Parameter sensitivity at Re = 10,000 for DD-OLE-NE(CNN) and baseline NE over the threshold ρ and the inner Newton relative tolerance γ s r . Converged runs are annotated with the Newton iteration count, following the NIres(NItot) convention for DD-OLE-NE(CNN) and using…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references

  1. [1]

    Cai, X.C., Li, X.: Inexact newton methods with restricted additive schwarz based nonlinear elimination for problems with high local nonlinearity. SIAM J. Sci. Comput.33, 746–762 (2011)

  2. [2]

    Hwang, F.N., Su, Y.C., Cai, X.C.: A parallel adaptive nonlinear elimination preconditioned inexact newton method for transonic full potential equation. Comput. Fluids110, 96–107 (2015)

  3. [3]

    In: Domain Decomposition Methods in Science and Engineering XXVI,Lecture Notes in Computational Science and Engineering, vol

    Klawonn, A., Lanser, M., Uran, M.: Adaptive nonlinear elimination in nonlinear FETI-DP methods. In: Domain Decomposition Methods in Science and Engineering XXVI,Lecture Notes in Computational Science and Engineering, vol. 145, pp. 337–345. Springer, Cham (2022)

  4. [4]

    Yang, H., Yang, C., Sun, S.: Active-set reduced-space methods with nonlinear elimination for two-phase flow problems in porous media. SIAM J. Sci. Comput.38, B593–B618 (2016)

  5. [5]

    Yang, H., Sun, S., Yang, C.: Nonlinearly preconditioned semismooth newton methods for variational inequality solution of two-phase flow in porous media. J. Comput. Phys.332, 1–20 (2017)

  6. [6]

    Yang, H., Hwang, F.N.: An adaptive nonlinear elimination preconditioned inexact newton algorithm for highly local nonlinear multicomponent pde systems. Appl. Numer. Math.133, 100–115 (2018)

  7. [7]

    In: Domain Decomposition Methods in Science and Engineering XXVII,Lecture Notes in Computational Science and Engineering, vol

    Klawonn, A., Lanser, M.: Efficient adaptive elimination strategies in nonlinear FETI-DP methods in combination with adaptive spectral coarse spaces. In: Domain Decomposition Methods in Science and Engineering XXVII,Lecture Notes in Computational Science and Engineering, vol. 149, pp. 295–302. Springer, Cham (2024)

  8. [8]

    In: Domain Decomposition Methods in Science and Engineering XXVIII, Lecture Notes in Computational Science and Engineering, vol

    Liang, C.W., Hwang, F.N.: Nonlinear elimination preconditioned space-time solution algorithms for hyperbolic PDE problems. In: Domain Decomposition Methods in Science and Engineering XXVIII, Lecture Notes in Computational Science and Engineering, vol. 155, pp. 101–108. Springer, Cham (2026)

Show all 35 references
  1. [9]

    Huang, J., Yang, C., Cai, X.C.: A nonlinearly preconditioned inexact newton algorithm for steady lattice boltzmann equations. SIAM J. Sci. Comput.38, A1701–A1724 (2016)

  2. [10]

    Luo, L., Shiu, W.S., Chen, R., Cai, X.C.: A nonlinear elimination preconditioned inexact newton method for blood flow problems in human artery with stenosis. J. Comput. Phys.399, 108926 (2019)

  3. [11]

    Luo, L., Cai, X.C., Yan, Z., Xu, L., Keyes, D.E.: A multilayer nonlinear elimination preconditioned inexact newton method for steady-state incompressible flow problems in three dimensions. SIAM J. Sci. Comput.42, B1404–B1428 (2020)

  4. [12]

    Luo, L., Cai, X.C., Keyes, D.E.: Nonlinear preconditioning strategies for two-phase flows in porous media discretized by a fully implicit discontinuous galerkin method. SIAM J. Sci. Comput.43, S317–S344 (2021) 22

  5. [13]

    Sirovich, L.: Turbulence and the dynamics of coherent structures. I. coherent structures. Q. Appl. Math. 45(3), 561–571 (1987)

  6. [14]

    Cambridge University Press, Cambridge (1996)

    Holmes, P., Lumley, J.L., Berkooz, G.: Turbulence, Coherent Structures, Dynamical Systems and Symmetry. Cambridge University Press, Cambridge (1996)

  7. [15]

    AIAA J.55(12), 4013–4041 (2017)

    Taira, K., Brunton, S.L., Dawson, S.T.M., Rowley, C.W., Colonius, T., McKeon, B.J., Schmidt, O.T., Gordeyev, S., Theofilis, V., Ukeiley, L.S.: Modal analysis of fluid flows: An overview. AIAA J.55(12), 4013–4041 (2017)

  8. [16]

    Luo, L., Cai, X.C.: PINL : Preconditioned inexact Newton with learning capability for nonlinear system of equations. SIAM J. Sci. Comput.45(2), A849–A871 (2023)

  9. [17]

    Gong, Y., Luo, L., Cai, X.C.: Stagnation-shortening inexact Newton method based on unsupervised learning for highly nonlinear algebraic problems and applications. SIAM J. Sci. Comput.46(5), S513–S534 (2024)

  10. [18]

    Schmid, P.J.: Dynamic mode decomposition of numerical and experimental data. J. Fluid Mech.656, 5–28 (2010)

  11. [19]

    Towne, A., Schmidt, O.T., Colonius, T.: Spectral proper orthogonal decomposition and its relationship to dynamic mode decomposition and resolvent analysis. J. Fluid Mech.847, 821–867 (2018)

  12. [20]

    Kramer, M.A.: Nonlinear principal component analysis using autoassociative neural networks. AlChE J. 37(2), 233–243 (1991)

  13. [21]

    Science 313(5786), 504–507 (2006)

    Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)

  14. [22]

    Murata, T., Fukami, K., Fukagata, K.: Nonlinear mode decomposition with convolutional neural networks for fluid dynamics. J. Fluid Mech.882, A13 (2020)

  15. [23]

    Fukami, K., Nakamura, T., Fukagata, K.: Convolutional neural network based hierarchical autoencoder for nonlinear mode decomposition of fluid field data. Phys. Fluids32(9), 095110 (2020)

  16. [24]

    Lee, K., Carlberg, K.T.: Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders. J. Comput. Phys.404, 108973 (2020)

  17. [25]

    Luo, L., Zhang, Q., Wang, X.P., Cai, X.C.: A parallel finite element method for 3d two-phase moving contact line problems in complex domains. J. Sci. Comput.72(3), 1119–1145 (2017)

  18. [26]

    Shadid, J.N., Tuminaro, R.S., Walker, H.F.: An inexact newton method for fully coupled solution of the Navier–Stokes equations with heat and mass transport. J. Comput. Phys.137, 155–185 (1997)

  19. [27]

    SIAM, Philadelphia (1996)

    Dennis, J.E., Schnabel, R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. SIAM, Philadelphia (1996)

  20. [28]

    Liu, L., Hwang, F.N., Luo, L., Cai, X.C., Keyes, D.E.: A nonlinear elimination preconditioned inexact newton algorithm. SIAM J. Sci. Comput.44(3), A1579–A1605 (2022)

  21. [29]

    Li, J., Cai, X.C.: Summation pollution of principal component analysis and an improved algorithm for location-sensitive data. Numer. Linear Algebra Appl.28(5), e2370 (2021)

  22. [30]

    Jagtap, A.D., Kharazmi, E., Karniadakis, G.E.: Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems. Comput. Methods Appl. Mech. Eng.365, 113028 (2020)

  23. [31]

    Jagtap, A.D., Karniadakis, G.E.: Extended physics-informed neural networks (XPINNs): A generalized space-time domain-decomposition-based deep learning framework for nonlinear partial differential equations. Commun. Comput. Phys.28(5), 2002–2041 (2020) 23

  24. [32]

    Shukla, K., Jagtap, A.D., Karniadakis, G.E.: Parallel physics-informed neural networks via domain decomposition. J. Comput. Phys.447, 110683 (2021)

  25. [33]

    Prudencio, E.E., Byrd, R., Cai, X.C.: Parallel full space SQP lagrange-newton-krylov-schwarz algorithms for PDE-constrained optimization problems. SIAM J. Sci. Comput.27, 1305–1328 (2006)

  26. [34]

    Ghia, U., Ghia, K.N., Shin, C.T.: High- Re solutions for incompressible flow using the Navier–Stokes equations and a multigrid method. J. Comput. Phys.48, 387–411 (1982)

  27. [35]

    In: International conference on machine learning, pp

    Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F., Bengio, Y., Courville, A.: On the spectral bias of neural networks. In: International conference on machine learning, pp. 5301–5310. PMLR (2019) 24

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.