Pith. sign in

REVIEW 5 major objections 5 minor 38 references

Expansive Natural Neural Gradient Flows for Energy Minimization

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read When a neural natural-gradient flow stalls on a convex energy, the paper claims, expanding the network with a layer aligned to the ideal Hilbert-space gradient restores descent to the target loss, and its experiments show random…

desk verdict A competent, honest methods paper whose headline expansion experiment is confounded by selection budget; the core idea is worth a serious referee but not acceptance as is. read the letter →

arxiv 2507.13475 v1 pith:LT5HK574 submitted 2025-07-17 math.OC cs.NAmath.NA

Wolfgang Dahmen , Wuchen Li , Yuankai Teng , Zhu Wang This is my paper · ORCID
classification math.OCcs.NAmath.NA MSC 68T0790C2665K10
keywords NaturalgradientflowNeuralmatrixHilbert-spaceenergyminimizationNetworkexpansionGradient-alignedinitializationModelreductionPhysics-informednetworksLossstagnation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that a stalled neural-network optimization can be rescued by expanding the network, provided the new parameters are initialized to align with the gradient of the underlying energy in the ambient Hilbert space. The setting is minimization of strictly convex, usually quadratic, energies over a Hilbert space, covering supervised learning, elliptic PDEs, and parametric model reduction. The paper argues that natural gradient steps on the neural manifold stall when the tangent space is nearly perpendicular to the ideal Hilbert-space gradient, and that inserting a layer whose initial weights are chosen using computable projections of that gradient restores descent toward the target energy. Experiments on oscillatory function fitting, a one-dimensional diffusion problem, and parametric Burgers flow report that gradient-aligned expansion reaches the prescribed loss tolerance where random initialization of the new layer fails to do so in the harder cases.

What carries the argument

The central object is the neural flow matrix $G_{f,\Theta}(\theta) = ((\partial_{\theta_i} f(\theta), \partial_{\theta_j} f(\theta))_H)_{i,j=1}^D$, the Gramian of the tangent directions with respect to the ambient Hilbert-space inner product, viewed as a generalized Wasserstein-2 type metric on mapping spaces; its singular or ill-conditioned form is handled by Levenberg-Marquardt style regularization $\lambda$ tied to the largest diagonal entry. The expansion mechanism is the second main piece: on loss stagnation the network inserts a layer before the output, and the new weights are chosen by sampling random candidates and solving an auxiliary alignment problem, using the fact that projections of $\nabla_H E$ can be evaluated even though the full gradient cannot. The new tangent directions are meant to restore the energy-decay conditions expressed through $c_4$ and $c_5$.

What would settle it

Construct a quadratic energy where stagnation is manufactured by taking the regularization $\lambda$ so large that the singular-value factor $c_5$ is tiny while the alignment ratio $\|P_{T(\theta)}\nabla_H E\|/\|\nabla_H E\|$ stays near $1$; the expansion algorithm, triggered only by stagnation, should fail to improve the loss. Measuring the alignment ratio on the Burgers example before and after expansion would also settle the claim, since it predicts a jump after gradient-aligned expansion but not after random expansion.

Watch

Extended reading notes

Core claim

The central claim is that minimizing $E(f(\theta))$ over a neural network class is a projected version of the ideal gradient flow in $H$, and the obstruction to convergence is the alignment constant $c_4$ in $\|P_{T(\theta)}\nabla_H E(f(\theta))\|_H \ge c_4 \|\nabla_H E(f(\theta))\|_H$. When $c_4$ is small, the tangent space of the neural manifold contains no good downhill direction, so natural gradient flow saturates even though the ambient energy is convex and benign. The paper proposes inserting a new layer before the output layer and initializing its weights so that the expanded tangent space contains a component of the Hilbert-space gradient, which is observable through inner products $(v - Rg, h)_H$ for quadratic energies. The experiments show that this gradient-aligned expansion lets training continue to the target loss, whereas randomly initialized expansion can exhaust the iteration budget without reaching tolerance.

Load-bearing premise

The expansion trigger assumes that a stagnating loss is caused by poor alignment between the network's tangent space and the true gradient, because expansion is activated by stagnation and only repairs alignment.

Editorial extensions

If this is right

  • If the expansion claim is right, training can start from a small network and grow only when needed, so target accuracy can be reached without committing to an over-parametrized model from the start.
  • Assembling the flow matrix in the correct ambient inner product rather than a convenient $\ell^2$ or $L^2$ default should make NGF produce accuracy in the model-compliant norm, such as $H^1_0$ for Poisson-type problems.
  • When expansion is triggered, gradient-aligned initialization should beat random initialization, because the expansion specifically repairs tangent-space alignment rather than merely adding capacity.
  • On the tested convex-energy tasks, NGF reaches the tolerance in far fewer iterations than Adam, so the cost of assembling flow matrices is offset by fast convergence in the hybrid last-two-layers version.
  • The old network is embedded in the expanded one by setting new weights to zero, so expansion can only add representational capability and never destroys the current approximation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The loss-stagnation trigger is a proxy for small alignment $c_4$; a more direct estimate of $\|P_{T(\theta)}\nabla_H E(f(\theta))\|_H$ at stagnation could distinguish alignment failure from the other two failure modes the paper names: too-small regularized singular values and over-large linearization steps.
  • Because the alignment step samples only $K=20$ random candidate layers, its success rate is probably improvable with a deterministic subspace search, more samples, or a few SGD steps on the auxiliary problem, as the paper's own discussion of more elaborate minimizers suggests.
  • For non-quadratic energies the projection trick is no longer directly available, but the same alignment idea could be carried into saddle-point or proximal formulations where the dual maximizer supplies the needed projection direction.
  • The experiments are one-dimensional toy problems; whether alignment-driven expansion survives high-dimensional parametric PDEs and deeper ResNets is the natural next test, since flow-matrix assembly scales with the width of the trained layers.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper develops a neural gradient flow (NGF) framework for minimizing convex, mostly quadratic, energy functionals in a Hilbert space, using a regularized Gramian (the neural flow matrix) built from the ambient inner product. It proposes an adaptive network expansion strategy that is triggered by loss saturation and aims to align the tangent space of the expanded neural manifold with the ideal Hilbert-space gradient direction, exploiting that projections of that gradient can be evaluated. The paper contains two elementary lemmas on the regularized least-squares update, a perturbation estimate for tangent-plane energy decay, two algorithms, and experiments on supervised learning, a 1D Poisson problem, and a parametrized Burgers model-reduction task.

Significance. If the central claims were fully supported, the paper would make a useful contribution to natural-gradient-type optimization for PDE- and learning-based energy minimization, especially its insistence on assembling the flow matrix in the correct Hilbert-space inner product. Lemmas 1 and 2 are clean and correct, the algorithms are described in sufficient detail to be re-implemented, and the authors are candid about several limitations, including the tangent-plane versus manifold gap and the heuristic nature of the stagnation trigger. The main theoretical estimate, however, depends on an unpublished reference and contains an incompletely stated condition, and the headline empirical evidence for the alignment mechanism is confounded by unequal random-search budgets. The conceptual idea is promising, but the manuscript in its current form does not yet substantiate its central claim.

major comments (5)
  1. [Section 3.4, Proposition 1 and Lemma 3] Proposition 1, which provides the contraction estimate (47) used in the proof of Lemma 3, is not proved in the manuscript; the proof is deferred to reference [16], which is listed as "in preparation." Since Lemma 3 is the paper's main quantitative guarantee for the regularized NGF step, the central theoretical support is not verifiable from the manuscript alone. The authors should either include a complete proof in an appendix or replace the reference with a published or otherwise available source.
  2. [Section 3.4, Lemma 3 (Eqs. (60)-(67))] The proof of Lemma 3 states "assuming without loss of generality that 2c5c4^2 < 1," but this is not without loss of generality: since c5 and c4 can both be close to 1, 2c5c4^2 can exceed 1, in which case the displayed bound (66) involves the square root of a negative number. The lemma should state 2c5c4^2 < 1 as an explicit hypothesis or give a separate argument for the complementary case. In addition, the contraction factor in (62) appears to contain sqrt(1 - c5c4^2) while the proof and the step-size condition (63) use sqrt(1 - 2c5c4^2); please correct this inconsistency.
  3. [Section 3.6 and Section 4.3, Table 9] The headline comparison between gradient-aligned and random expansion initialization is confounded. As described in Section 3.6 and the experimental setup, the gradient-aligned arm draws K = 20 random candidate settings for the new hidden layer, computes the corresponding output-layer parameters via (87), and selects the candidate with the lowest loss via (86). The random-initialization arm appears to use a single random draw. Thus the reported advantage in Table 9 (4 expansions to reach tolerance versus 6 expansions without success) could be due to selection over 20 candidates rather than to alignment with the Hilbert-space gradient. To support the alignment claim in the Abstract and Section 1.2(c), the random arm should use the same K = 20 selection budget, or the gradient-aligned arm should be tested with K = 1, and ideally multiple seeds should be reported.
  4. [Algorithm 2 and Section 3.4] The expansion trigger in Algorithm 2 is loss saturation, but Section 3.4 states that a small value of c4 is only "one possible reason" for stagnation, listing a too-small c5 and an overly large linearization step as other causes. The algorithm does not diagnose which cause is operative, so expansions may be triggered in regimes where the proposed alignment mechanism is not the limiting factor. The random-initialization runs in Tables 5, 7, and 9, where expansion sometimes fails to reach tolerance, are consistent with this limitation. The authors should either add a diagnostic based on the estimates (69)-(70) or explicitly present the stagnation trigger as a heuristic without claiming it isolates the alignment failure.
  5. [Section 3.4, Lemma 3 and Algorithm 1] Lemma 3 certifies energy decay along the tangent-plane direction f(θ) - τg(θ), not along the manifold update f(θ + Δθ) used in Algorithm 1. The paper concedes this gap in the discussion surrounding (42)-(43), but the consequence is that the theoretical results do not yet provide a convergence guarantee for the implemented NGF or expansive algorithm; the support for the algorithmic claims is therefore primarily experimental. This is acceptable for an exploratory methods paper only if the framing is adjusted accordingly, for example by stating in the conclusions that the theory currently addresses the tangent-plane model rather than the full manifold dynamics.
minor comments (5)
  1. [Abstract] The abstract promises a "generalized Wasserstein-2 metric" and a "full diffeomorphism mapping space," but these concepts are not defined or used in the body of the paper; please either develop them in Section 2 or remove them from the abstract.
  2. [Table 2] The caption of Table 2 says "SL-Case I (k = 10)", but the table reports the k = 10 experiments, which are Case II in Section 4.1; please correct the caption.
  3. [References] The reference list contains a LaTeX artifact immediately after [18] ("@articleUrban, author =L. Ernst..."); this entry should be cleaned up or removed.
  4. [Sections 2.4, 3.7, and 4] There are several typos: "arive" in Section 2.4, "satuation" and "expandion" in Section 3.7, and "an expansdion occurs" in the experimental setup of Section 4.
  5. [Section 4.3 and Tables 5, 7, 9] The expansive experiments appear to report single runs; given the stochastic nature of the initialization and the K = 20 selection, presenting results over multiple random seeds would make the empirical claims substantially more robust.

Circularity Check

1 steps flagged · score 2.0 of 10

No definitional circularity; one load-bearing proof is deferred to an in-preparation self-citation, and the headline alignment experiment is confounded by K=20 selection, which is a validity issue rather than circularity.

  1. self citation load bearing [Section 3.4, Proposition 1]
    "The claim under (a) follows from standard arguments; (b) can be derived along similar lines. We refer to [16] for a proof."

    Proposition 1(b) supplies the contraction estimate (47) that Lemma 3 then converts into the paper's central rationale: when c4 is small, the admissible step-size collapses and expansion is needed. The proof is not given here but deferred to [16], an in-preparation manuscript by two of the present authors, so the quantitative core of the 'alignment controls decay' argument currently rests on a self-citation of unpublished work. This is load-bearing self-citation rather than a definitional reduction; the result is standard and can be verified independently, so it does not make the whole derivation circular.

full rationale

The paper's formal derivation (Lemmas 1–3) is largely self-contained except that Proposition 1(b), the contraction estimate on which Lemma 3's c6 bound rests, has its proof deferred to [16], an in-preparation work by authors Dahmen and Wang. That is a load-bearing self-citation, but the proposition is a standard convex-optimization estimate, and the rest of the derivation uses it as an external lemma rather than assuming its conclusion. The expansion algorithm is not circular: c4 is defined through projection norms, the trigger is loss stagnation, and the alignment construction solves least-squares problems (83)–(87) using computable projections of the Hilbert-space gradient; success is reported as an empirical outcome. The headline random-versus-gradient-aligned comparison is confounded because the gradient-aligned arm selects the best of K=20 random candidates by loss (86) while the random arm appears to use one draw, so the experiment does not isolate alignment; this is a validity threat, not a definitional reduction. No fitted constant is renamed as a prediction, and no known result is merely relabeled. Overall circularity is low.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

No new particles or fields. The paper's load-bearing postulates are the stagnation-to-misalignment heuristic, the tangent-plane-to-manifold transfer, and the quadrature-accuracy assumption, plus a large set of hand-tuned hyperparameters.

free parameters (6)
  • regularization rule lambda(G_mm) = lambda_j = 5e-6 * 10^j, x_j = 10^(j-1) for j=1..6 (SL/PDE); lambda_j = 1e-8 * 10^j for MOR (Eq. 92)
    Piecewise function of max diagonal entry of G, hand-chosen per experiment; affects step size and stability.
  • initial learning rate gamma0 = 10 (NGF), 5e-3 (Adam)
    Starting point for Armijo backtracking in Algorithm 1; chosen per experiment.
  • Armijo constant c = 2e-4
    Line search parameter in Eq. 90.
  • number of alignment candidates K = 20
    Random candidates for gradient-aligned initialization (Section 3.6); larger K improves alignment probability but costs more.
  • saturation tolerances = tau_sa = 1e-7, tau_sr = 5e-3 for NGF; tau_sa = 1e-8, tau_sr = 5e-4 for Adam (most runs)
    Trigger expansion and early stopping (Eqs. 89, 91).
  • expansion width m = not stated explicitly (equal to hidden width 15 in experiments)
    Number of neurons added per expansion.
assumptions (5)
  • domain assumption Energy E is alpha-strictly convex and beta-smooth (assumption preceding Proposition 1, Section 3.4)
    Holds for the quadratic energies in Section 2; needed for the decay estimates.
  • domain assumption Tangent-plane energy decay (Lemma 3) is a reliable proxy for manifold decay
    Paper admits this is generally not an implication without second-order information (Section 3.4).
  • ad hoc to paper Loss stagnation implies poor gradient-tangent alignment c4
    Used to trigger expansion (Algorithm 2); paper admits c4 small is only one possible cause (Section 3.4).
  • domain assumption Monte Carlo / quadrature approximations of Hilbert inner products are sufficiently accurate
    Used to assemble G and evaluate projections; no error analysis is supplied.
  • standard math Proposition 1 (convex gradient descent decay) holds as stated
    Cited to [16] (Dahmen and Wang, in preparation), not proved in this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Expansive Natural Neural Gradient Flows for Energy Minimization." pith.science (2026). https://pith.science/paper/LT5HK574

@misc{pith2026250713475,
  author       = {Pith},
  title        = {Pith review of: Expansive Natural Neural Gradient Flows for Energy Minimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LT5HK574}},
  note         = {Machine review of arXiv:2507.13475}
}
read the original abstract

This paper develops expansive gradient dynamics in deep neural network-induced mapping spaces. Specifically, we generate tools and concepts for minimizing a class of energy functionals in an abstract Hilbert space setting covering a wide scope of applications such as PDEs-based inverse problems and supervised learning. The approach hinges on a Hilbert space metric in the full diffeomorphism mapping space, which could be viewed as a generalized Wasserstein-2 metric. We then study a projection gradient descent method within deep neural network parameterized sets. More importantly, we develop an adaptation and expanding strategy to step-by-step enlarge the deep neural network structures. In particular, the expansion mechanism aims to enhance the alignment of the neural manifold induced natural gradient direction as well as possible with the ideal Hilbert space gradient descent direction leveraging the fact that we can evaluate projections of the Hilbert space gradient. We demonstrate the efficacy of the proposed strategy for several simple model problems for energies arising in the context of supervised learning, model reduction, or inverse problems. In particular, we highlight the importance of assembling the neural flow matrix based on the inner product for the ambient Hilbert space. The actual algorithms are the simplest specifications of a broader spectrum based on a correspondingly wider discussion, postponing a detailed analysis to forthcoming work.

Figures

Figures reproduced from arXiv: 2507.13475 by the authors.

Figure 1
Figure 1. SL-Case I (k = 5): Neural network of fixed depth L trained using Adam optimizer. (a) History of training loss, where the dark line indicates the prescribed loss tolerance; (b) Final approximation on the test set. When switching to NGF for optimization, the training process converges and reaches the tolerance for all tested depths, as shown in [PITH_FULL_IMAGE:figures/full_fig_p026_1.png] view at source ↗
Figure 2
Figure 2. SL-Case I (k = 5): Neural network of fixed depth L trained using NGF optimizer. (a) History of training loss, where the dark line indicates the prescribed loss tolerance; (b) Final approximation on the test set. We summarize the total number of iterations, final training loss values, and the L2 approximation errors on the test set in [PITH_FULL_IMAGE:figures/full_fig_p027_2.png] view at source ↗
Figure 3
Figure 3. SL-Case II (k = 10): Neural network of fixed depth L trained using Adam optimizer. (a) History of training loss, where the dark line indicates the prescribed loss tolerance; (b) Final approximation on the test set. 0 100 200 300 400 500 600 700 800 900 1000 1e-06 1e-05 1e-04 1e-03 1e-02 1e-01 1e+00 1e+01 L=2 L=3 L=4 (a) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 -1.5 -1 -0.5 0 0.5 1 1.5 2 Target L=2 L=3 L=4 (b) [PITH_… view at source ↗
Figures from the paper (16 more)
Figure 4
Figure 4. Figure 4: SL-Case II (k = 10): Neural network of fixed depth L trained using NGF optimizer. (a) History of training loss, where the dark line indicates the prescribed loss tolerance; (b) Final approximation on the test set. The final loss values and approximation errors are comp…
Figure 5
Figure 5. Figure 5: shows the training loss histories for two expansive NN models, each expanded once at iteration 13 under the two initialization methods. While using random initial￾ization narrowly misses the convergence tolerance by the maximum iteration limit, the gradient-aligned ini…
Figure 6
Figure 6. Figure 6: , which displays the model outputs over the test set both before the expansion and at training termination. 0 500 1000 1500 2000 2500 3000 1e-05 1e-04 1e-03 1e-02 1e-01 1e+00 1e+01 epoch 13 (a) 0 20 40 60 80 100 120 140 160 1e-06 1e-05 1e-04 1e-03 1e-02 1e-01 1e+00 epo…
Figure 7
Figure 7. Figure 7: PDE-Case I (k = 5): (a) Training loss history for NNs of fixed depths L, along with the exact Ritz energy (dark line); (b) Final NN approximations on the test set. All models were trained using the Adam optimizer. 0 100 200 300 400 500 600 700 800 900 1000 -120 -100 -8…
Figure 8
Figure 8. Figure 8: PDE-Case I (k = 5): (a) Training loss history for NNs of fixed depths L, along with the exact Ritz energy (dark line); (b) Final NN approximations on the test set. All models were trained using the NGF optimizer. L Adam NGF Nite Loss L2 error H1 error Nite Loss L2 erro…
Figure 9
Figure 9. Figure 9: PDE-Case I (k = 5): expansive NN with initial depth L0 = 2, comparing (a) random vs. (b) gradient-aligned initialization. Vertical lines indicate expansion events (dashed) and NGF-to-Adam transitions (dash￾dot). The dark horizontal line indicates the exact Ritz energy.…
Figure 10
Figure 10. Figure 10: PDE-Case I (k = 5): (a) Approximation of the expansive NN model on the test set before the first expansion; (b) Final approximation on the test set. The history of training loss using Adam is shown in [PITH_FULL_IMAGE:figures/full_fig_p032_10.png]
Figure 11
Figure 11. Figure 11: PDE-Case II (k = 10): (a) Training loss history for NNs of fixed depths L, along with the exact Ritz energy (dark line); (b) Final NN approximations on the test set. All models were trained using the Adam optimizer. 0 100 200 300 400 500 600 700 800 900 1000 -400 -300…
Figure 12
Figure 12. Figure 12: PDE-Case II (k = 10): (a) Training loss history for NNs of fixed depths L, along with the exact Ritz energy (dark line); (b) Final NN approximations on the test set. All models were trained using the NGF optimizer. L Adam NGF Nite Loss L2 error H1 error Nite Loss L2 e…
Figure 13
Figure 13. Figure 13: PDE-Case II (k = 10): expansive NN with initial depth L0 = 2, comparing (a) random vs. (b) gradient-aligned initialization. Ver￾tical lines indicate expansion events (dashed) and NGF-to-Adam transi￾tions (dash-dot). The dark horizontal line indicates the exact Ritz en…
Figure 14
Figure 14. Figure 14: PDE-Case II (k = 10): (a)-(b) Approximation of the ex￾pansive NN model on the test set before the first and second expansion, respectively; (b) Final approximation on the test set. 4.3. MOR for Parametrized Burgers. Next, we consider solving u(x, t; µ) from a parametr…
Figure 15
Figure 15. Figure 15: Full order simulations of the parametrized Burgers equation at (a) µ = 0.0174 and (b) µ = 0.0296, respectively. at 11 uniformly spaced parameter samples from the prescribed interval. The numerical solutions on the grids are collected once every 20 time steps, which yi…
Figure 16
Figure 16. Figure 16: MOR: Neural network of fixed depth L trained using Adam optimizer. (a) History of training loss; (b) Approximate solutions at two test parameters [PITH_FULL_IMAGE:figures/full_fig_p035_16.png]
Figure 17
Figure 17. Figure 17: MOR: Neural network of fixed depth L trained using NGF optimizer. (a) History of training loss, where the dark line indicates the prescribed loss tolerance; (b) Approximate solutions at two test parameters. For both optimizers, the training iterations, final loss valu…
Figure 18
Figure 18. Figure 18: MOR: Expansive NN with initial depth L0 = 2, comparing (a) random vs. (b) gradient-aligned initialization. Vertical lines indicate expansion events (dashed) and NGF-to-Adam transitions (dash-dot). Initalization Expansive NN with L0 = 2 Nite Nexp Final loss L2 error Ra…
Figure 19
Figure 19. Figure 19: MOR: Approximations of the expansive NN model for test parameters µ = 0.0174 (first row) and µ = 0.0296 (second row), where (a) and (d) show pre-first-expansion, (b) and (e) pre-third-expansion, and (c) and (f) final results. 5. Summary and Conclusions We have present…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 32 canonical work pages

  1. [16]

    Dahmen, Z

    W. Dahmen, Z. Wang, Variationally correct residual regression via saddle point formulations, in preparation

    work page

  2. [1]

    S. Amari. Natural Gradient Works Efficiently in Learning. Neural Computation , 10 (2): 251–276, 1998

    work page 1998

  3. [2]

    Ambrosio, N

    L. Ambrosio, N. Gigli and G. Savare. Gradient Flows in Metric Spaces and in the Space of Probability Measures, 2008

    work page 2008

  4. [3]

    N. Ay, J. Jost, H. V. Lˆ e, and L. Schwachh¨ ofer.Information Geometry, 2017

    work page 2017

  5. [4]

    Bachmayr, W

    M. Bachmayr, W. Dahmen, M. Oster, Variationally Correct Neural Residual Regression for Parametric PDEs: On the Viability of Controlled Accuracy, preprint May 31, 2024, http://arxiv.org/abs/2405.20065

    arXiv 2024

  6. [5]

    Barron, A

    A. Barron, A. Cohen, W. Dahmen and R. Devore. Approximation and learning by greedy algorithms. Annals of statistics , 64–94, 2008

    work page 2008

  7. [6]

    Bruna, B

    J. Bruna, B. Peherstorfer, E. Vanden-Eijnden, Neural Galerkin schemes with active learning for high- dimensional evolution equations, Journal of Computational Physics 496 (2024), 112588

    work page 2024

  8. [7]

    Berman, B

    J. Berman, B. Peherstorfer, Randomized Sparse Neural Galerkin Schemes for Solving Evolution Equa- tions with Deep Networks, 37th Conference on Neural Information Processing Systems (NeurIPS 2023) EXPANSIVE NATURAL NEURAL GRADIENT FLOWS 39

    work page 2023

Show all 38 references
  1. [8]

    Bertoluzza and E

    S. Bertoluzza and E. Burman and C. He, t W AN discretization of PDEs: Best approximation, stabi- lization and essential boundary conditions, =arXiv preprint arXiv:2307.05012v2 [math.NA] Jan. 29, 2024,

    2024 arXiv

  2. [9]

    Binev, A

    P. Binev, A. Cohen, W. Dahmen, R. DeVore, V. Temlyakov, Universal algorithms for learning theory - Part I : piecewise constant functions, Journal of Machine Learning Research (JMLR), 6(2005), 1297–1321

    2005

  3. [10]

    Binev, A

    P. Binev, A. Cohen, W. Dahmen, R. DeVore, Classification algorithms using adaptive partitioning, Annals of Statistics, 42 (No. 6)(2014), 2141-2163

    2014

  4. [11]

    Bao and X

    G. Bao and X. Ye and Y. Zang and H. Zhou, Numerical solution of inverse problems by weak adver- sarial networks, 36, No 11 (2020), 115003, https://dx.doi.org/10.1088/1361-6420/abb447

    2020 doi

  5. [12]

    Berrone and C

    S. Berrone and C. Canuto and M. Pintore, Variational Physics Informed Neural Networks: the Role of Quadratures and Test Functions, Journal of Scientific Computing, 92 No. 3 (2022), 100, https://doi.org/10.1007/s10915-022-01950-4

    2022 doi

  6. [13]

    Cayci, Gauss-newton dynamics for neural networks: A Riemannian optimization perspective, arXiv:2412.14031v3 [math.OC], December 2024

    S. Cayci, Gauss-newton dynamics for neural networks: A Riemannian optimization perspective, arXiv:2412.14031v3 [math.OC], December 2024

    2024

  7. [14]

    J. A. Carrillo, D. Matthes, and M.T. Wolfram. Lagrangian schemes for Wasserstein gradient flows. Handbook of Numerical Analysis , 22: 271–311, 2021

    2021

  8. [15]

    T. M. Cover and J. A. Thomas. Elements of Information Theory. Wiley Series in Telecommunications. Wiley, New York, 1991

    1991

  9. [17]

    Dahmen, M

    W. Dahmen, M. Wang, Z. Wang. Nonlinear Reduced DNN Models for State Estimation. Communi- cations in Computational Physics , 32 (No 1), 1–40, 2022

    2022

  10. [18]

    DeVore, B

    R. DeVore, B. Hanin, G. Petrova, Neural Network Approximation, Acta Numerica, Volume 30 , May 2021,pp. 327 – 444. DOI: https://doi.org/10.1017/S0962492921000052 @articleUrban, author =L. Ernst and K. Urban, title = A certified wavelet-based physics-informed neural network for ...

    2021 arXiv

  11. [19]

    Chen and J

    F. Chen and J. Huang and C. Wang and H. Yang, Friedrichs Learning: Weak Solutions of Partial Differential Equations via Deep Learning, SIAM Journal on Scientific Computing, 45, No. 3 (2023), A1271-A1299, https://doi.org/10.1137/22M1488405,

    2023 doi

  12. [20]

    Goodfellow, Y

    I. Goodfellow, Y. Bengio and A. Courville. Deep Learning. MIT Press, 2016

    2016

  13. [21]

    Goodfellow, J

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville and Y. Bengio. Generative Adversarial Nets. NIPS, 2014

    2014

  14. [22]

    Jnini, Anas and Vella, Flavio and Zeinhofer, Marius, Gauss-Newton Natural Gradient Descent for Physics-Informed Computational Fluid Dynamics, arXiv preprint arXiv:2402.10680, 2024

    2024 arXiv

  15. [23]

    Jordan, D

    R. Jordan, D. Kinderlehrer, and F. Otto. The Variational Formulation of the Fokker–Planck Equation. SIAM Journal on Mathematical Analysis , Vol. 29, Iss. 1, 1998

    1998

  16. [24]

    W. Li. Hessian metric via transport information geometry. Journal of Mathematical Physics , 2021

    2021

  17. [25]

    W. Li. Transport information geometry: Riemannian calculus on probability simplex. Information Geometry, 5, 161-–207, 2020

    2020

  18. [26]

    W. Li. Transport information Bregman divergences. Information Geometry, 4, 435—470, 2021

    2021

  19. [27]

    Li and G

    W. Li and G. Montufar. Natural gradient via optimal transport. Information Geometry, 1, 181–214, 2018

    2018

  20. [28]

    A. Lin, W. Li, S. Osher, and G. Montufar. Wasserstein Proximal of GANs. Geometric Science of Information, pp 524–533, 2021

    2021

  21. [29]

    Ollivier

    Y. Ollivier. Riemannian metrics for neural networks I: Feedforward networks. Information and Infer- ence, 108–153, 2015

    2015

  22. [30]

    F. Otto. The geometry of dissipative evolution equations: the porous medium equation. Communica- tions in Partial Differential Equations , 26(1-2):101–174, 2001

    2001

  23. [31]

    Taylor and D

    J. Taylor and D. Pardo and I. Muga, A Deep Fourier Residual method for solving PDEs using Neural Networks, Computer Methods in Applied Mechanics and Engineering, 405 (2023), 115850, https://doi.org/10.1016/j.cma.2022.115850, 40 DAHMEN, LI, TENG, AND W ANG

    2023

  24. [32]

    Ruthotto, S

    L. Ruthotto, S. Osher, W. Li, L. Nurbekyan, S. Fung. A machine learning framework for solving high- dimensional mean field game and mean field control problems. Proceedings of the National Academy of Sciences, 2020

    2020

  25. [33]

    Kovachki, Z

    N. Kovachki, Z. Li, B. Liu, K. Azizzadenesheli, K. Bhattacharya, A. Stuart, A. Anandkumar. Neural Operator: Learning Maps Between Function Spaces With Applications to PDEs, Journal of Machine Learning Research, 24 (2023), 1–97

    2023

  26. [34]

    J. Lu, E. Tadmor, A. Zenginoglu, Swarm-based gradient descent method for non-convex optimiza- tion, Communications of the American Mathematical Society, Volume 4, Issue 17, (2024), 787–822, https://doi.org/10.1090/cams/42

    2024 doi

  27. [35]

    C. Villani. Optimal Transport: Old and New , 2009

    2009

  28. [36]

    Yarotsky, Error bounds for approximations with deep ReLu networks, Neural Networks 94, (2017), 103–114

    D. Yarotsky, Error bounds for approximations with deep ReLu networks, Neural Networks 94, (2017), 103–114

    2017

  29. [37]

    Zhang, W

    L. Zhang, W. E, and L. Wang. Monge-Amp` ere Flow for Generative Modeling.arXiv:1809.10188, 2018

    2018 arXiv

  30. [38]

    Zang and G

    Y. Zang and G. Bao and X. Ye and H. Zhou, Weak adversarial networks for high- dimensional partial differential equations, Journal of Computational Physics, 411 (2020), 109409, https://doi.org/10.1016/j.jcp.2020.109409, Email address : wolfgang.anton.dahmen@googlemail.com Depar...

    2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.