A modified Anderson acceleration with sharp linear convergence rate predictions and application to incompressible flows
Pith reviewed 2026-05-19 22:19 UTC · model grok-4.3
The pith
Modified Anderson acceleration using nonlinear residuals gives sharp linear convergence predictions for Navier-Stokes Picard iterations.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
For the Picard iteration applied to the Navier-Stokes equations, the AAg method converges linearly with a rate that equals the gain of the least-squares problem constructed from the nonlinear residual; the bound is sharp for any fixed depth and is inherited from the gain-controlled contraction factor.
What carries the argument
The AAg variant, which defines the Anderson least-squares problem using the nonlinear residual rather than the fixed-point residual, so that the optimization gain directly supplies the contraction factor in the convergence bound.
If this is right
- The linear rate can be predicted a priori from the gain of each least-squares solve, allowing reliable performance estimates before running the iteration.
- An adaptive depth strategy follows immediately from monitoring the same gain quantity.
- Numerical tests demonstrate that the method outperforms both standard Anderson acceleration and nonlinear GMRES on incompressible flow benchmarks.
- The theory holds for any depth parameter without additional restrictions.
Where Pith is reading between the lines
- The same gain-based rate control may apply to other nonlinear iterations that admit a residual-based least-squares acceleration step.
- The explicit rate formula could be used to compare AAg against other acceleration families on a common footing.
- Testing the adaptive rule on time-dependent or three-dimensional flows would check whether the gain remains a reliable indicator outside the steady-state setting examined here.
Load-bearing premise
The analysis assumes that the gain of the least-squares problem built from the nonlinear residual directly sets the contraction factor of the overall iteration.
What would settle it
Compute the observed convergence rate on a standard cavity or channel flow and compare it to the rate predicted from the measured gain value at each step; systematic mismatch would falsify the sharp prediction.
Figures
read the original abstract
In this work, we extend a modified Anderson acceleration proposed in [Y. He, arXiv:2603.25983, 2026] to accelerate the Picard iteration for the Navier-Stokes equations. In this variant of Anderson acceleration, named AAg, the nonlinear residual--rather than the standard fixed-point iteration residual--is used to define the associated least-squares problem. We establish a convergence analysis for this method with any depth that shows how AAg accelerates convergence through the gain of the optimization problem, and obtain a sharp prediction of its linear convergence rate (a feature that is not part of the known theory for classical Anderson acceleration). Additionally, motivated by this sharp convergence prediction, we introduce an adaptive strategy that automatically selects the depth parameter. Results of several numerical experiments are given that illustrate the new theory and also demonstrate the effectiveness of the proposed adaptive approach. Comparisons of AAg to usual AA and nonlinear GMRES are also provided.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper extends the modified Anderson acceleration (AAg) from prior work to the Picard iteration for the incompressible Navier-Stokes equations. In AAg the nonlinear residual (rather than the fixed-point residual) defines the least-squares problem. The central claims are a convergence analysis valid for any depth that links acceleration to the gain of this optimization problem, a sharp prediction of the resulting linear convergence rate, an adaptive strategy for choosing the depth parameter motivated by the rate prediction, and supporting numerical experiments on several test problems that also compare AAg to classical Anderson acceleration and nonlinear GMRES.
Significance. If the convergence analysis and sharp-rate claim hold for the Navier-Stokes Picard map, the work would supply a theoretical feature absent from standard Anderson acceleration theory and could guide practical parameter selection in incompressible-flow solvers. The numerical comparisons and adaptive strategy are useful but secondary to the analysis.
major comments (2)
- The convergence analysis (abstract and §3) asserts that the optimization gain directly controls the contraction factor and yields a sharp linear rate. It is not evident that the specific properties of the Navier-Stokes Picard map (convective term, divergence-free constraint, inf-sup stability) have been re-established from first principles; the link appears inherited from the referenced prior AAg paper without a self-contained derivation for this setting.
- §4 (rate prediction): the claim that the predicted rate is sharp and independent of the target convergence factor requires explicit verification that the gain is computed from quantities independent of the contraction factor itself; otherwise the prediction risks circularity.
minor comments (2)
- Notation for the nonlinear residual and the gain should be introduced with a clear distinction from the classical Anderson acceleration residual.
- The adaptive depth strategy is motivated by the rate prediction; a brief remark on its robustness when the gain estimate is noisy would strengthen the presentation.
Simulated Author's Rebuttal
We thank the referee for the careful reading and constructive comments on our manuscript. We address each major comment below with clarifications and proposed revisions to strengthen the presentation of the analysis.
read point-by-point responses
-
Referee: The convergence analysis (abstract and §3) asserts that the optimization gain directly controls the contraction factor and yields a sharp linear rate. It is not evident that the specific properties of the Navier-Stokes Picard map (convective term, divergence-free constraint, inf-sup stability) have been re-established from first principles; the link appears inherited from the referenced prior AAg paper without a self-contained derivation for this setting.
Authors: Section 3 provides a self-contained convergence analysis for AAg applied to the Picard iteration of the incompressible Navier-Stokes equations. While the general framework originates from the referenced prior work, the derivation is adapted and re-established specifically for this setting: we verify that the nonlinear residual least-squares problem respects the divergence-free constraint and accounts for the convective term under inf-sup stable discretizations. Theorem 3.1 and its corollaries explicitly link the optimization gain to the contraction factor for arbitrary depth. To make these adaptations more transparent, we will add a remark in §3 summarizing the first-principles verification of the NS-specific properties. revision: yes
-
Referee: §4 (rate prediction): the claim that the predicted rate is sharp and independent of the target convergence factor requires explicit verification that the gain is computed from quantities independent of the contraction factor itself; otherwise the prediction risks circularity.
Authors: The gain in §4 is computed exclusively from the least-squares minimization over the history of nonlinear residuals, which are quantities observed from prior iterates and do not depend on the contraction factor or any target rate. The sharpness claim is supported by an explicit construction showing the bound is attained in certain cases. The prediction is therefore non-circular and provides an a posteriori estimate usable for the adaptive strategy. We will revise §4 to include an explicit statement and short verification confirming that the gain computation is independent of the contraction factor. revision: yes
Circularity Check
Convergence analysis for AAg on Navier-Stokes Picard iteration is derived independently
full rationale
The manuscript extends the AAg variant (nonlinear residual in the least-squares problem) from the referenced prior work and then establishes its own convergence analysis for arbitrary depth on the Navier-Stokes equations. This analysis explicitly connects acceleration to the optimization gain and supplies a sharp linear rate prediction that is stated to be absent from classical Anderson theory. Because the central derivation is presented as newly established for the incompressible-flow setting (including the specific Picard map), and no equation or claim in the abstract reduces the rate bound to a fitted parameter or to an unverified self-citation by construction, the logical chain remains self-contained. Numerical experiments further provide external illustration of the predicted rates.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
H. An, X. Jia, and H. Walker. Anderson acceleration and application to the three-temperature energy equations. Journal of Computational Physics , 347:1–19, 2017
work page 2017
-
[2]
D. G. Anderson. Iterative procedures for nonlinear integral equations. J. Assoc. Comput. Mach. , 12(4):547–560, 1965
work page 1965
-
[3]
A. Z. Atanasov, B. Uekermann, C. Pachajoa, H. Bungartz, and P. Neumann. Steady-state Anderson accelerated coupling of Lattice Boltzmann and Navier-Stokes solvers. Comput., 4:38, 2016
work page 2016
-
[4]
N. A. Barnafi and M. L. Pasini. Two-level sketching alternating Anderson acceleration for complex physics applications. arXiv preprint arXiv:2505.08587 , 2025
work page internal anchor Pith review arXiv 2025
-
[5]
M. Benzi and M. Olshanskii. An augmented Lagrangian-based approach to the Oseen problem. SIAM J. Sci. Comput. , 28:2095–2113, 2006
work page 2095
-
[6]
W. Bian and X. Chen. Anderson acceleration for nonsmooth fixed point problems. SIAM Journal on Numerical Analysis, 60(5):2565–2591, 2022
work page 2022
-
[7]
C. Brezinski, M. Redivo-Zaglia, and Y. Saad. Shanks sequence transformations and Anderson acceler- ation. Siam Review , 60(3):646–669, 2018. 17
work page 2018
-
[8]
P. N. Brown and Y. Saad. Hybrid Krylov methods for nonlinear systems of equations. SIAM Journal on Scientific and Statistical Computing , 11(3):450–481, 1990
work page 1990
-
[9]
P. N. Brown and Y. Saad. Convergence theory of nonlinear Newton–Krylov algorithms. SIAM Journal on Optimization , 4(2):297–330, 1994
work page 1994
-
[10]
P. R. Brune, M. G. Knepley, B. F. Smith, and X. Tu. Composing scalable nonlinear algebraic solvers. siam REVIEW , 57(4):535–565, 2015
work page 2015
-
[11]
T. Charnyi, T. Heister, M. Olshanskii, and L. Rebholz. On conservation laws of Navier-Stokes Galerkin discretizations. Journal of Computational Physics , 337:289–308, 2017
work page 2017
-
[12]
K. Chen and C. Vuik. Composite Anderson acceleration method with two window sizes and optimized damping. International Journal for Numerical Methods in Engineering , 123(23):5964–5985, 2022
work page 2022
- [13]
-
[14]
H. De Sterck and Y. He. On the asymptotic linear convergence speed of Anderson acceleration, Nesterov acceleration, and nonlinear GMRES. SIAM Journal on Scientific Computing , 43(5):S21–S46, 2021
work page 2021
- [15]
-
[16]
H. Fang and Y. Saad. Two classes of multisecant methods for nonlinear acceleration. Numer. Linear Algebra Appl., 16(3):197–221, 2009
work page 2009
-
[17]
X. Feng, M. P. Laiu, and T. Strohmer. Convergence analysis of the alternating Anderson–Picard method for nonlinear fixed-point problems. SIAM Journal on Scientific Computing , pages S436–S461, 2025
work page 2025
-
[18]
M. Filippini, P. Alotto, and A. Giust. Anderson acceleration for electromagnetic nonlinear problems. Compel, 38(5):1493–1506, 2019
work page 2019
- [19]
-
[20]
A. Fu, J. Zhang, and S. Boyd. Anderson accelerated Douglas-Rachford splitting. SIAM Journal on Scientific Computing , 42(6):A3560–A3583, 2020
work page 2020
-
[21]
V. Girault and P.-A. Raviart. Finite element methods for Navier–Stokes equations: Theory and algo- rithms. Springer-Verlag, 1986
work page 1986
-
[22]
C. Greif and Y. He. Convergence properties of nonlinear GMRES applied to linear systems. SIAM Journal on Matrix Analysis and Applications , 47(1):330–352, 2026
work page 2026
-
[23]
W. W. Hager and H. Zhang. A survey of nonlinear conjugate gradient methods. Pacific Journal of Optimization, 2(1):35–58, 2006
work page 2006
-
[24]
E. Hawkins and L. Rebholz. On the choice of optimization norm for Anderson acceleration of the Picard iteration for Navier-Stokes equations. Numerical Methods for Partial Differential Equations , in revision, 2026
work page 2026
-
[25]
E. Hawkins, L. Rebholz, and D. Vargun. Removing splitting/modeling error in projection/penalty meth- ods for Navier-Stokes simulations with continuous data assimilation. Communications in Mathematical Research, 40(1):1–29, 2024. 18
work page 2024
-
[26]
H. He, Z. Tang, S. Zhao, Y. Saad, and Y. Xi. nlTGCR: A class of nonlinear acceleration procedures based on conjugate residuals. SIAM Journal on Matrix Analysis and Applications , 45(1):712–743, 2024
work page 2024
-
[27]
Y. He. Convergence analysis for nonlinear GMRES. IMA Journal of Numerical Analysis , 2026
work page 2026
-
[28]
Y. He. Convergence analysis of an alternating nonlinear GMRES on linear systems. Numerical Linear Algebra with Applications , 33(1):e70065, 2026
work page 2026
- [29]
- [30]
- [31]
- [32]
-
[33]
T. Heister and G. Rapin. Efficient augmented Lagrangian-type preconditioning for the Oseen problem using grad-div stabilization. Int. J. Numer. Meth. Fluids , 71:118–134, 2013
work page 2013
-
[34]
V. Henson et al. Multigrid methods nonlinear problems: an overview. Computational Imaging, 5016:36– 48, 2003
work page 2003
-
[35]
V. John, A. Linke, C. Merdon, M. Neilan, and L. G. Rebholz. On the divergence constraint in mixed finite element methods for incompressible flows. SIAM Review , 59(3):492–544, 2017
work page 2017
-
[36]
W. Layton. An Introduction to the Numerical Analysis of Viscous Incompressible Flows . SIAM, Philadel- phia, 2008
work page 2008
-
[37]
M. Olshanskii and L. Rebholz. Approximating a branch of solutions to the Navier-Stokes equations by reduced-order modeling. Journal of Computational Physics , 524:1–14, 2025
work page 2025
-
[38]
C. W. Oosterlee and T. Washio. Krylov subspace acceleration of nonlinear multigrid with application to recirculating flows. SIAM Journal on Scientific Computing , 21(5):1670–1690, 2000
work page 2000
-
[39]
Y. Peng, B. Deng, J. Zhang, F. Geng, W. Qin, and L. Liu. Anderson acceleration for geometry opti- mization and physics simulation. ACM Trans. Graph., 37(4), 2018
work page 2018
-
[40]
S. Pollock and L. Rebholz. Anderson acceleration for contractive and noncontractive operators. IMA Journal of Numerical Analysis , 41(4):2841–2872, 2021
work page 2021
-
[41]
S. Pollock and L. Rebholz. Filtering for Anderson acceleration. SIAM Journal on Scientific Computing , 45(4):A1571–A1590, 2023
work page 2023
-
[42]
S. Pollock and L. Rebholz. Anderson Acceleration for Numerical PDEs . Society for Industrial and Applied Mathematics, Philadelphia, 2025
work page 2025
-
[43]
S. Pollock, L. Rebholz, X. Tu, and M. Xiao. Analysis of the Picard-Newton iteration for the Navier- Stokes equations: global stability and quadratic convergence. Journal of Scientific Computing , 14(25):1– 23, 2025
work page 2025
-
[44]
S. Pollock, L. Rebholz, and M. Xiao. Anderson-accelerated convergence of Picard iterations for incom- pressible Navier-Stokes equations. SIAM Journal on Numerical Analysis , 57:615– 637, 2019. 19
work page 2019
-
[45]
W. Rodi. Comparison of LES and RANS calculations of the flow around bluff bodies. Journal of Wind Engineering and Industrial Aerodynamics , 69-71:55–75, 1997. Proceedings of the 3rd International Colloqium on Bluff Body Aerodynamics and Applications
work page 1997
-
[46]
Y. Saad. Acceleration methods for fixed-point iterations. Acta Numerica, 34:805–890, 2025
work page 2025
-
[47]
A. Sohankar, L. Davidson, and C. Norberg. Large eddy simulation of flow past a square cylinder: Comparison of different subgrid scale models. Journal of Fluids Engineering , 122(1):39–47, 11 1999
work page 1999
-
[48]
Z. Tang, T. Xu, H. He, Y. Saad, and Y. Xi. Anderson acceleration with truncated Gram–Schmidt. SIAM Journal on Matrix Analysis and Applications , 45(4):1850–1872, 2024
work page 2024
-
[49]
R. Temam. Navier-Stokes equations . Elsevier, North-Holland, 1991
work page 1991
-
[50]
J. M. Tempestti, S. Kim, B. Lindsey, and A. Veneziani. A pseudo-spectral method for wall shear stress estimation from Doppler ultrasound imaging in coronary arteries. Cardiovascular Engineering and Technology, 15(6):647–666, 2024
work page 2024
-
[51]
A. Toth, C. Kelley, S. Slattery, S. Hamilton, K. Clarno, and R. Pawlowski. Analysis of Anderson acceleration on a simplified neutronics/thermal hydraulics system. Proceedings of the ANS MC2015 Joint International Conference on Mathematics and Computation (M &C), Supercomputing in Nuclear Applications (SNA) and the Monte Carlo (MC) Method , ANS MC2015 CD:1...
work page 2015
- [52]
-
[53]
H. F. Walker and P. Ni. Anderson acceleration for fixed-point iterations. SIAM J. Numer. Anal. , 49(4):1715–1735, 2011
work page 2011
-
[54]
D. Wang, Y. He, and H. De Sterck. On the asymptotic linear convergence speed of Anderson acceleration applied to ADMM. Journal of Scientific Computing , 88(2):38, 2021
work page 2021
-
[55]
Q. Wang, W. Li, W. Bao, and X. Gao. Nonlinear Kaczmarz algorithms and their convergence. Journal of Computational and Applied Mathematics , 399:113720, 2022
work page 2022
-
[56]
T. Washio and C. W. Oosterlee. Krylov subspace acceleration for nonlinear multigrid schemes. Electron. Trans. Numer. Anal , 6(271-290):3, 1997
work page 1997
- [57]
- [58]
-
[59]
K. Wong and A. Baker. A 3D incompressible Navier-Stokes velocity-vorticity weak form finite element algorithm. International Journal for Numerical Methods in Fluids , 38:99–123, 2002
work page 2002
-
[60]
M. Xiao. Superlinear convergence of Anderson accelerated Newton’s method for solving stationary Navier-Stokes equations. Numerical Methods for Partial Differential Equations , 39:3089–3107, 2023
work page 2023
-
[61]
Y. Yang. Anderson acceleration for seismic inversion. Geophysics, 86(1):R99–R108, 2021. 20 Appendix In the Appendix, we give proofs of identities used in the analysis of Section 3. Proof of Lemma 3.1. This proof uses the definition of uk+1 from AAg, that Pk+1 j=k−m+1 αj = 1 , and appro- priately grouping terms. For the first identity, expanding uk+1 and u...
work page 2021
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.