REVIEW 35 references
A domain decomposition online-learning-enhanced nonlinear elimination preconditioner
T0 review · reviewed 2026-08-01 · deepseek-v4-flash
Pith's one-line read Thresholding a learned reconstruction of the residual, not the raw residual, makes nonlinear-elimination Newton solvers converge at Re up to 10,000.
desk verdict The learned-extraction idea is interesting and the experiments are extensive, but the central baseline comparison is confounded by the early-restart difference, so the robustness claim overreaches. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The extraction operator G is the load-bearing object. It is learned online from residual snapshots collected during the stagnation window [k_s, k_t]. Two realizations are considered: a PCA-based linear operator (projection onto leading singular vectors plus the snapshot mean) and autoencoder neural networks (MLP and CNN). The reconstructed residual F = G(F) is used in place of the raw residual in the bad-subset selection criterion, and the same reconstruction is used in the restricted nonlinear solve. The operator is trained locally on each subdomain and assembled globally, so the learning step fits the existing domain-decomposition parallelism.
What would settle it
Construct a nonlinear problem with an exactly known slow subspace (e.g., a two-scale system) and compute the norm of the true slow residual component lying outside the kept modes of the learned extractor. If that norm is not small while the method still converges, or if a deliberately engineered high-frequency stiff mode is filtered out causing the solver to stagnate, the central separation claim fails.
Extended reading notes
Core claim
The central claim is that the bad subset for nonlinear elimination should be selected from a data-driven reconstruction G(F(X)) of the residual rather than from F(X) itself. The reconstruction is obtained by training an extractor on residual snapshots gathered during the stagnation phase. The reconstruction preserves the dominant 'slow' components responsible for Newton stagnation and filters out 'fast' oscillatory components; the selection criterion then becomes robust to the threshold and to the inner-solve tolerance. With local models trained independently per subdomain, the method converges for Re=10,000 cavity flow where the baseline NE preconditioner stagnates, and the selected bad sub
Load-bearing premise
The learned extractor must preserve exactly the residual components that drive Newton's slow convergence, while everything it discards must be irrelevant oscillation; this separation is asserted rather than proved.
Editorial extensions
If this is right
- Baseline NE with raw-residual thresholding fails to converge at Re=7,500 and Re=10,000 for the tested parameters, while all three learned variants (PCA, MLP, CNN) converge.
- The learned method converges for all 20 tested combinations of threshold rho and inner tolerance gamma at Re=10,000, whereas the baseline converges in only 8 of 20 cases.
- Increasing the number of subdomains improves both parallelism and the quality of local extraction; with only two subdomains the CNN variant fails, while with four or more it converges robustly.
- The learned extractors are not equivalent to fixed smoothing: Gaussian-smoothed residuals do not restore convergence, showing the adaptivity of the learned models is essential.
- Earlier restart from the stagnation phase is more robust; restarting too late from a strongly stagnated trajectory degrades or destroys convergence.
Reading between the lines
- A testable prediction follows: the kept modes of the learned extraction should contain the true slow directions; one could compare the learned bad subset with an oracle subset obtained from a known physical stiffness indicator and measure the overlap.
- Because the method is algebraic and the residual reconstruction is problem-agnostic, it should transfer to other nonlinear PDE systems with localized stiffness, such as multiphase flow or contact problems, although the paper demonstrates only cavity flow.
- The reliance on a low-dimensional bottleneck implies the slow components must be approximately low-rank; for problems where the stiff directions are spread across many modes or high-frequency components, the learned reconstruction could discard genuinely difficult parts, and the claimed robustness may not hold.
- A fully online extension could retrain the extractor during the restarted Newton phase instead of only once at k_t, potentially adapting to residual structures that change after elimination.
Editorial analysis
A structured set of objections, weighed in public.
Circularity Check
The restart-point residual reconstruction is in-sample (and for k_re=1 uses later Newton iterates), so the claimed 'learned' bad-subset selection is partly a retrospective fit rather than an independent prediction; the baseline comparison also varies the restart point, though that is a confound rather than circularity.
-
fitted input called prediction
[Algorithm 2 (Steps 2.1 and 3.1), Eq. (21), §3.4.1, §4.5]
"Given an initial guess X0, ... the residual collection window [ks, kt], the restart step kre ∈ [ks, kt], and the bad subset threshold ρ. ... Step 2.1: For each subdomain Ωi, construct and train a local extraction model Gi ... from the collected residual snapshots. ... Step 3.1: Set Y0 = Xkre ... Eq. (21): Gi,PCA(R0_i F(X)) := P_i P_i^T (R0_i F(X) − ar F_i) + ar F_i."
Because k_re ∈ [k_s,k_t], the restart point Y0 = X^{k_re} is one of the residual snapshots used in Step 2.1 to train the extractor. For PCA, Eq. (21) is the orthogonal projection of that very training snapshot onto the leading left-singular vectors of the snapshot matrix that contains it; the 'dominant structure' is therefore the least-squares reconstruction of the point on which the bad-subset selection is later made. In §4.5, with k_re=1 and k_t=9, the model is trained on F(X^2),...,F(X^9), i.e., on iterates that occur after the restart point, and is then applied at X^1. The claimed more-reliable, more-coherent subset is thus a retrospective in-sample fit, not an independent prediction; the advantage over raw thresholding is at least partly built into the training protocol.
full rationale
The paper's core contribution is experimental rather than theorem-based, so I looked for places where a numerical 'improvement' reduces to the way the training data and the test/application point coincide. The clearest such point is the restart mechanism: Algorithm 2 trains the extraction model on snapshots from [k_s,k_t] and then restarts at X^{k_re} with k_re inside that same window. For PCA, Eq. (21) reconstructs a training-point residual by projecting it onto the SVD subspace of the snapshot matrix containing that point; for the autoencoders, the same in-sample training applies. When k_re=1, the model is even trained on later iterates (X^2,...,X^{k_t}) before being used to 'identify' the structure at X^1, so the online-learning narrative masks future-data leakage. This is genuine partial circularity because the learned residual representation at the restart point is a fitted function of the very data it is then used to interpret. I do not treat the separate baseline issue as circularity: §4.1 claims the baseline is activated at the same k_re, while §4.5 says baseline NE starts from the ninth Newton iteration versus k_re=1 for DD-OLE-NE; that is a confounded experimental comparison and belongs under correctness/validity, not under the definitional-equivalence notion used here. Self-citations ([16], [17]) are related prior work by the same group but are not load-bearing or invoked as external uniqueness theorems; the PCA/autoencoder construction is stated and implemented in this paper. Score 6 reflects the in-sample/future-data leakage in the central extraction step, without claiming that every favorable convergence result is vacuous.
Assumptions & free parameters
free parameters (5)
- bad-subset threshold rho =
0.1, 0.2, 0.35, 0.4, 0.5 across experiments
- PCA retained modes d =
5 for Re=7,500, 8 for Re=10,000 in Table 1
- snapshot window length k_t and restart index k_re =
k_t varies 3-9, k_re varies 1-7 in Tables 1 and 4
- inner subspace tolerance gamma_s^r =
0.01, 0.05, 0.10, 0.20
- NN training budget and architecture =
epochs 150/300/500; various kernel/channel sequences
assumptions (3)
- domain assumption Low-energy PCA/autoencoder modes are oscillatory perturbations, not part of the slow nonlinear dynamics.
- domain assumption The Jacobian of the original residual F' is a valid surrogate for the derivative of the reconstructed residual F'.
- domain assumption Residual snapshots from the stagnation window [k_s, k_t] are representative of the nonlinear imbalance at the restart point X^{k_re}.
Cite this review
Pith. "Pith review of A domain decomposition online-learning-enhanced nonlinear elimination preconditioner." pith.science (2026). https://pith.science/paper/OSCZP2DB
@misc{pith2026260717723,
author = {Pith},
title = {Pith review of: A domain decomposition online-learning-enhanced nonlinear elimination preconditioner},
year = {2026},
howpublished = {\url{https://pith.science/paper/OSCZP2DB}},
note = {Machine review of arXiv:2607.17723}
}
read the original abstract
Nonlinearly preconditioned inexact Newton methods form an effective class of solvers for large-scale nonlinear algebraic systems arising from the discretization of partial differential equations. A central challenge in nonlinear elimination (NE) preconditioning is the reliable identification of the slowly converging components to be eliminated. Existing selection strategies often rely on problem-specific physical information or user-tuned thresholds applied directly to the raw nonlinear residual, which may contain irregular oscillatory structures near stagnation regions, making the selected bad subset highly sensitive to threshold parameters. In this work, we propose an online-learning-enhanced NE preconditioner that identifies the bad subset from the dominant structure of the nonlinear residual rather than from the raw residual itself. Residual snapshots are collected online during the stagnation phase of the current Newton solve, and an unsupervised extraction model is trained to capture the principal nonlinear imbalance. We consider both a linear extractor based on principal component analysis and nonlinear extractors based on autoencoder neural networks. Moreover, we integrate the approach into a parallel domain decomposition framework, which trains a local extraction model independently on each subdomain. The learned residual reconstruction is then used to define the bad subset and guide the nonlinear elimination process. Numerical experiments on lid-driven cavity flows at Reynolds numbers up to 10,000 show that the proposed method produces more reliable and coherent bad subsets, is robust with respect to both NE and learning parameters, and outperforms the baseline NE preconditioner in terms of the convergence.
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[1]
Cai, X.C., Li, X.: Inexact newton methods with restricted additive schwarz based nonlinear elimination for problems with high local nonlinearity. SIAM J. Sci. Comput.33, 746–762 (2011)
2011
-
[2]
Hwang, F.N., Su, Y.C., Cai, X.C.: A parallel adaptive nonlinear elimination preconditioned inexact newton method for transonic full potential equation. Comput. Fluids110, 96–107 (2015)
2015
-
[3]
In: Domain Decomposition Methods in Science and Engineering XXVI,Lecture Notes in Computational Science and Engineering, vol
Klawonn, A., Lanser, M., Uran, M.: Adaptive nonlinear elimination in nonlinear FETI-DP methods. In: Domain Decomposition Methods in Science and Engineering XXVI,Lecture Notes in Computational Science and Engineering, vol. 145, pp. 337–345. Springer, Cham (2022)
2022
-
[4]
Yang, H., Yang, C., Sun, S.: Active-set reduced-space methods with nonlinear elimination for two-phase flow problems in porous media. SIAM J. Sci. Comput.38, B593–B618 (2016)
2016
-
[5]
Yang, H., Sun, S., Yang, C.: Nonlinearly preconditioned semismooth newton methods for variational inequality solution of two-phase flow in porous media. J. Comput. Phys.332, 1–20 (2017)
2017
-
[6]
Yang, H., Hwang, F.N.: An adaptive nonlinear elimination preconditioned inexact newton algorithm for highly local nonlinear multicomponent pde systems. Appl. Numer. Math.133, 100–115 (2018)
2018
-
[7]
In: Domain Decomposition Methods in Science and Engineering XXVII,Lecture Notes in Computational Science and Engineering, vol
Klawonn, A., Lanser, M.: Efficient adaptive elimination strategies in nonlinear FETI-DP methods in combination with adaptive spectral coarse spaces. In: Domain Decomposition Methods in Science and Engineering XXVII,Lecture Notes in Computational Science and Engineering, vol. 149, pp. 295–302. Springer, Cham (2024)
2024
-
[8]
In: Domain Decomposition Methods in Science and Engineering XXVIII, Lecture Notes in Computational Science and Engineering, vol
Liang, C.W., Hwang, F.N.: Nonlinear elimination preconditioned space-time solution algorithms for hyperbolic PDE problems. In: Domain Decomposition Methods in Science and Engineering XXVIII, Lecture Notes in Computational Science and Engineering, vol. 155, pp. 101–108. Springer, Cham (2026)
2026
Show all 35 references
-
[9]
Huang, J., Yang, C., Cai, X.C.: A nonlinearly preconditioned inexact newton algorithm for steady lattice boltzmann equations. SIAM J. Sci. Comput.38, A1701–A1724 (2016)
2016
-
[10]
Luo, L., Shiu, W.S., Chen, R., Cai, X.C.: A nonlinear elimination preconditioned inexact newton method for blood flow problems in human artery with stenosis. J. Comput. Phys.399, 108926 (2019)
2019
-
[11]
Luo, L., Cai, X.C., Yan, Z., Xu, L., Keyes, D.E.: A multilayer nonlinear elimination preconditioned inexact newton method for steady-state incompressible flow problems in three dimensions. SIAM J. Sci. Comput.42, B1404–B1428 (2020)
2020
-
[12]
Luo, L., Cai, X.C., Keyes, D.E.: Nonlinear preconditioning strategies for two-phase flows in porous media discretized by a fully implicit discontinuous galerkin method. SIAM J. Sci. Comput.43, S317–S344 (2021) 22
2021
-
[13]
Sirovich, L.: Turbulence and the dynamics of coherent structures. I. coherent structures. Q. Appl. Math. 45(3), 561–571 (1987)
1987
-
[14]
Cambridge University Press, Cambridge (1996)
Holmes, P., Lumley, J.L., Berkooz, G.: Turbulence, Coherent Structures, Dynamical Systems and Symmetry. Cambridge University Press, Cambridge (1996)
1996
-
[15]
AIAA J.55(12), 4013–4041 (2017)
Taira, K., Brunton, S.L., Dawson, S.T.M., Rowley, C.W., Colonius, T., McKeon, B.J., Schmidt, O.T., Gordeyev, S., Theofilis, V., Ukeiley, L.S.: Modal analysis of fluid flows: An overview. AIAA J.55(12), 4013–4041 (2017)
2017
-
[16]
Luo, L., Cai, X.C.: PINL : Preconditioned inexact Newton with learning capability for nonlinear system of equations. SIAM J. Sci. Comput.45(2), A849–A871 (2023)
2023
-
[17]
Gong, Y., Luo, L., Cai, X.C.: Stagnation-shortening inexact Newton method based on unsupervised learning for highly nonlinear algebraic problems and applications. SIAM J. Sci. Comput.46(5), S513–S534 (2024)
2024
-
[18]
Schmid, P.J.: Dynamic mode decomposition of numerical and experimental data. J. Fluid Mech.656, 5–28 (2010)
2010
-
[19]
Towne, A., Schmidt, O.T., Colonius, T.: Spectral proper orthogonal decomposition and its relationship to dynamic mode decomposition and resolvent analysis. J. Fluid Mech.847, 821–867 (2018)
2018
-
[20]
Kramer, M.A.: Nonlinear principal component analysis using autoassociative neural networks. AlChE J. 37(2), 233–243 (1991)
1991
-
[21]
Science 313(5786), 504–507 (2006)
Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)
2006
-
[22]
Murata, T., Fukami, K., Fukagata, K.: Nonlinear mode decomposition with convolutional neural networks for fluid dynamics. J. Fluid Mech.882, A13 (2020)
2020
-
[23]
Fukami, K., Nakamura, T., Fukagata, K.: Convolutional neural network based hierarchical autoencoder for nonlinear mode decomposition of fluid field data. Phys. Fluids32(9), 095110 (2020)
2020
-
[24]
Lee, K., Carlberg, K.T.: Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders. J. Comput. Phys.404, 108973 (2020)
2020
-
[25]
Luo, L., Zhang, Q., Wang, X.P., Cai, X.C.: A parallel finite element method for 3d two-phase moving contact line problems in complex domains. J. Sci. Comput.72(3), 1119–1145 (2017)
2017
-
[26]
Shadid, J.N., Tuminaro, R.S., Walker, H.F.: An inexact newton method for fully coupled solution of the Navier–Stokes equations with heat and mass transport. J. Comput. Phys.137, 155–185 (1997)
1997
-
[27]
SIAM, Philadelphia (1996)
Dennis, J.E., Schnabel, R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. SIAM, Philadelphia (1996)
1996
-
[28]
Liu, L., Hwang, F.N., Luo, L., Cai, X.C., Keyes, D.E.: A nonlinear elimination preconditioned inexact newton algorithm. SIAM J. Sci. Comput.44(3), A1579–A1605 (2022)
2022
-
[29]
Li, J., Cai, X.C.: Summation pollution of principal component analysis and an improved algorithm for location-sensitive data. Numer. Linear Algebra Appl.28(5), e2370 (2021)
2021
-
[30]
Jagtap, A.D., Kharazmi, E., Karniadakis, G.E.: Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems. Comput. Methods Appl. Mech. Eng.365, 113028 (2020)
2020
-
[31]
Jagtap, A.D., Karniadakis, G.E.: Extended physics-informed neural networks (XPINNs): A generalized space-time domain-decomposition-based deep learning framework for nonlinear partial differential equations. Commun. Comput. Phys.28(5), 2002–2041 (2020) 23
2002
-
[32]
Shukla, K., Jagtap, A.D., Karniadakis, G.E.: Parallel physics-informed neural networks via domain decomposition. J. Comput. Phys.447, 110683 (2021)
2021
-
[33]
Prudencio, E.E., Byrd, R., Cai, X.C.: Parallel full space SQP lagrange-newton-krylov-schwarz algorithms for PDE-constrained optimization problems. SIAM J. Sci. Comput.27, 1305–1328 (2006)
2006
-
[34]
Ghia, U., Ghia, K.N., Shin, C.T.: High- Re solutions for incompressible flow using the Navier–Stokes equations and a multigrid method. J. Comput. Phys.48, 387–411 (1982)
1982
-
[35]
In: International conference on machine learning, pp
Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F., Bengio, Y., Courville, A.: On the spectral bias of neural networks. In: International conference on machine learning, pp. 5301–5310. PMLR (2019) 24
2019
Reviewed August 1, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.