Pith. sign in

REVIEW 10 cited by

Challenges in Training PINNs: A Loss Landscape Perspective

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2402.01868 v2 pith:PAF737H7 submitted 2024-02-02 cs.LG math.OCstat.ML

Pratik Rathore , Weimu Lei , Zachary Frangella , Lu Lu , Madeleine Udell This is my paper · ORCID
classification cs.LGmath.OCstat.ML
keywords losspinnstrainingadamdifferentiall-bfgspinnchallenges
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

This paper explores challenges in training Physics-Informed Neural Networks (PINNs), emphasizing the role of the loss landscape in the training process. We examine difficulties in minimizing the PINN loss function, particularly due to ill-conditioning caused by differential operators in the residual term. We compare gradient-based optimizers Adam, L-BFGS, and their combination Adam+L-BFGS, showing the superiority of Adam+L-BFGS, and introduce a novel second-order optimizer, NysNewton-CG (NNCG), which significantly improves PINN performance. Theoretically, our work elucidates the connection between ill-conditioned differential operators and ill-conditioning in the PINN loss and shows the benefits of combining first- and second-order optimization methods. Our work presents valuable insights and more powerful optimization strategies for training PINNs, which could improve the utility of PINNs for solving difficult partial differential equations.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Cosmo-SPINN: Fuzzy Dark Matter Simulations with Physics-Informed Generative Networks

    astro-ph.CO 2026-07 conditional novelty 6.0 of 10

    Physics-informed generative U-Nets evolve and super-resolve fuzzy dark matter fields under Schrödinger–Poisson constraints with far less supervised data than pure data-driven baselines.

  2. PIKS: Universal Physics-Informed Kernel Methods

    stat.ML 2026-07 accept novelty 6.0 of 10

    PIKS is universally consistent for linear differential constraints: with universal kernels it recovers both the target and the physics residual in the misspecified regime, with rates under source conditions.

  3. Material-agnostic temperature field prediction for metal additive manufacturing via a parametric PINN framework

    cs.LG 2026-04 unverdicted novelty 6.0 of 10

    One physics-informed network trained on the heat equation, not labeled data, predicts laser-scan temperature fields for unseen alloys including copper with ~1% relative error.

  4. Optimizing Rank for High-Fidelity Implicit Neural Representations

    cs.CV 2025-12 conditional novelty 6.0 of 10

    Muon, an orthogonalizing optimizer, improves INR fidelity across images, audio, shapes, CT, super-resolution, and NeRF, with plain ReLU MLPs gaining up to ~9 dB PSNR.

  5. A Sketch-and-Project Analysis of Subsampled Natural Gradient Algorithms

    cs.LG 2025-08 conditional novelty 6.0 of 10

    For linear least squares, SNGD and SPRING are proved equivalent to accelerated regularized Kaczmarz methods, yielding the first fast rates and first SPRING guarantee; the general quadratic analysis holds under strong ...

  6. BWLer: Barycentric Weight Layer Elucidates a Precision-Conditioning Tradeoff for PINNs

    cs.LG 2025-06 conditional novelty 6.0 of 10

    Adding a barycentric interpolation layer to PINNs lifts their precision ceiling, achieving up to 1e-13 relative error on smooth PDEs, while exposing a tradeoff between accuracy and loss conditioning.

  7. SPINN: Advancing Cosmological Simulations of Fuzzy Dark Matter with Physics Informed Neural Networks

    astro-ph.CO 2025-06 conditional novelty 6.0 of 10

    A physics-informed neural network (SPINN) solves the Schrödinger-Poisson equations for fuzzy dark matter collapse in 1D and 3D, matching a spectral solver on a sinusoidal test case.

  8. Semi-Explicit Neural DAEs: Learning Long-Horizon Dynamical Systems with Algebraic Constraints

    cs.LG 2025-05 conditional novelty 5.0 of 10

    Manifold projection at each ODE step enforces algebraic constraints in neural ODEs, producing near-zero constraint violation and competitive long-horizon state accuracy on six benchmarks.

  9. Guaranteeing Conservation of Integrals with Projection in Physics-Informed Neural Networks

    cs.LG 2025-11 reject novelty 4.0 of 10

    A projection layer can enforce linear and quadratic integral conservation in PINNs, but the quadratic projection formula as printed omits the discretization factor and therefore does not satisfy its own constraint.

  10. Physics-Informed Neural Networks with Hard Nonlinear Equality and Inequality Constraints

    cs.LG 2025-07 conditional novelty 4.0 of 10

    KKT-Hardnet enforces hard nonlinear equality and inequality constraints in neural network outputs via a differentiable KKT projection layer, reducing constraint violations to near machine precision.

Pith tools