pith. sign in

arxiv: 2606.06314 · v1 · pith:WBE4V2HQnew · submitted 2026-06-04 · 🧮 math.NA · cs.LG· cs.NA· stat.ML

DAS-PINNs for high-dimensional partial differential equations: extending deep adaptive sampling to spacetime domains

Anshima Singh , David J. Silvester This is my paper

Pith reviewed 2026-06-28 00:08 UTC · model grok-4.3

classification 🧮 math.NA cs.LGcs.NAstat.ML
keywords physics-informed neural networksdeep adaptive samplingnormalizing flowshigh-dimensional PDEsspacetime domainsadaptive collocationresidual distribution
0
0 comments X

The pith

A normalizing flow learns the PDE residual distribution to generate adaptive collocation points across unified spacetime domains.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper extends deep adaptive sampling methods for physics-informed neural networks to time-dependent high-dimensional PDEs by treating space and time as a single domain. A normalizing flow model is trained to capture the distribution of the PDE residual and then produces new collocation points focused on regions the network finds hardest to approximate. This replaces uniform sampling and avoids any explicit time marching or moving meshes. A sympathetic reader would care because high-dimensional problems with localized or moving features quickly make uniform points ineffective, and the method lets the residual itself drive sampling in both space and time. The approach is demonstrated on benchmarks that include sharp moving fronts in two dimensions and localized structures up to eight spatial dimensions.

Core claim

A normalising flow neural network model effectively learns the distribution induced by the PDE residual and generates new collocation points concentrated in regions where the solution is most difficult to learn. Unlike conventional adaptive strategies that require explicit time stepping or moving meshes, high-residual regions are automatically identified and tracked across both space and time, driven purely by the PDE residual distribution.

What carries the argument

Normalizing flow neural network that learns the distribution induced by the PDE residual to produce new collocation points

If this is right

  • High-residual regions are tracked across space and time without explicit time marching or moving meshes.
  • The method applies to problems with sharp and moving features in two spatial dimensions.
  • Localized structures can be resolved in up to eight spatial dimensions.
  • Uniform collocation sampling is replaced by residual-driven adaptive sampling in a single spacetime domain.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Fewer total collocation points may suffice for the same accuracy once sampling concentrates on difficult regions.
  • The same residual-driven flow idea could be tested on other mesh-free PDE solvers beyond PINNs.
  • Performance on problems whose residual distribution changes rapidly in time would reveal how often the flow must be retrained.

Load-bearing premise

The normalizing flow can learn the PDE residual distribution in the unified spacetime domain accurately enough to generate useful adaptive samples.

What would settle it

On a benchmark problem, the training error of the PINN remains the same or higher when collocation points are chosen by the flow compared with uniform sampling.

Figures

Figures reproduced from arXiv: 2606.06314 by Anshima Singh, David J. Silvester.

Figure 1
Figure 1. Figure 1: Moving Gaussian peak problem (4.2): evolution of the mean square error (Emse), relative L2 error (E2), and L∞ error (E∞) per epoch. The initial stage is followed by four adaptive stages. Dashed vertical lines mark the start of each adaptive stage. The solution network and KRnet are trained with ntrain = 2000 collocation points and batch size 500, using a learning rate of 10−4 . The solution network is trai… view at source ↗
Figure 2
Figure 2. Figure 2: Moving Gaussian peak problem (4.2): exact solution (top row), predicted solution (middle row), and pointwise error (bottom row) at four time levels t ∈ {0, 0.25, 0.50, 1.00}. near the peak trajectory, reflecting the residual-driven nature of the adaptive sampling. As the training advances, the point distribution spreads further across the domain, indicating that the residual is becoming more uniform—consis… view at source ↗
Figure 3
Figure 3. Figure 3: Moving Gaussian peak problem (4.2): spatial distribution of collocation points at each stage of DAS-PINNs. The training set grows from 2000 points in the initial stage to 10,000 points by the final adaptive stage [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Moving Gaussian peak problem (4.2): temporal histogram of collocation points at each stage of DAS-PINNs. 9 [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Rotation problem (4.4): exact solution (top row), predicted solution (middle row), and pointwise error (bottom row) at four time levels t ∈ {0, 0.25, 0.50, 1.00} [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Rotation problem (4.4): spatial distribution of collocation points at each stage of DAS-PINNs. The dashed line shows the circular trajectory of the peak centre [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Rotation problem (4.4): temporal histogram of newly added collocation points at each adaptive stage of DAS-PINNs [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Nonlinear problem (4.6): exact solution (top) and predicted solution (bottom) at five time levels t ∈ {0, 0.25, 0.50, 0.75, 1.00}. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Nonlinear problem (4.6): pointwise error at five time levels t ∈ {0, 0.25, 0.50, 0.75, 1.00} [PITH_FULL_IMAGE:figures/full_fig_p015_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Nonlinear problem (4.6): spatial distribution of collocation points at each stage of DAS-PINNs. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: High-dimensional parabolic problem (4.8) d = 6: exact solution (top row), DAS-PINNs predicted solution (middle row), and PINNs with uniform sampling (bottom row) at four time levels t ∈ {0, 0.25, 0.50, 1.00}, shown as a 2D slice along (x3, x5) [PITH_FULL_IMAGE:figures/full_fig_p017_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: High-dimensional parabolic problem (4.8) d = 8: exact solution (top row), DAS-PINNs predicted solution (middle row), and PINNs with uniform sampling (bottom row) at four time levels t ∈ {0, 0.25, 0.50, 1.00}, shown as a 2D slice along (x4, x5). 18 [PITH_FULL_IMAGE:figures/full_fig_p018_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: High-dimensional parabolic problem (4.8) d = 8: spatial distribution of newly added collocation points at each stage of DAS-PINNs, projected onto (x4, x5). The star marks the peak location at the origin. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: High-dimensional hyperbolic problem (4.11)): evolution of the loss components—residual loss, PDE loss, boundary loss, and initial condition loss—per epoch. Dashed vertical lines mark the start of each adaptive stage. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: High-dimensional hyperbolic problem (4.11)): exact solution (top row), predicted solution (middle row), and pointwise error (bottom row) at four time levels t ∈ {0, 0.25, 0.50, 1.00}, shown as a 2D slice along (x2, x5). 23 [PITH_FULL_IMAGE:figures/full_fig_p023_15.png] view at source ↗
read the original abstract

Time-dependent high-dimensional partial differential equations (PDEs) with spatially localised and dynamically evolving solutions pose a fundamental challenge for physics-informed neural networks (PINNs), as uniform collocation sampling becomes increasingly ineffective in high-dimensional spatiotemporal domains. In this work, a deep adaptive sampling framework for PINNs is extended to the time-dependent setting by treating space and time as a unified domain without any explicit time marching. A normalising flow neural network model effectively learns the distribution induced by the PDE residual and generates new collocation points concentrated in regions where the solution is most difficult to learn. Unlike conventional adaptive strategies that require explicit time stepping or moving meshes, high-residual regions are automatically identified and tracked across both space and time, driven purely by the PDE residual distribution. The effectiveness of the proposed strategy is assessed on a range of benchmark problems, from sharp and moving features in two spatial dimensions to localised structures in up to eight spatial dimensions.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 0 minor

Summary. The paper claims to extend deep adaptive sampling (DAS) for physics-informed neural networks (PINNs) to time-dependent high-dimensional PDEs by treating the spacetime domain as unified (without explicit time marching or moving meshes). A normalizing flow is used to learn the distribution induced by the PDE residual and to generate adaptive collocation points concentrated in high-residual regions. Effectiveness is assessed via benchmarks involving sharp/moving features in 2D and localized structures up to 8 spatial dimensions plus time.

Significance. If the central claim holds, the work could advance adaptive collocation strategies for PINNs on evolutionary PDEs in high dimensions by automating residual-driven sampling over a single spacetime domain. The approach avoids conventional time-stepping requirements, which is a practical strength for problems with dynamically evolving localized features. No machine-checked proofs, parameter-free derivations, or open reproducible code are referenced.

major comments (2)
  1. [Abstract] Abstract: The assertion that the normalizing flow 'effectively learns' the residual distribution and produces useful adaptive samples lacks any referenced quantitative diagnostics (KL divergence, effective sample size, coverage of high-residual regions, or direct comparison to ground-truth residual histograms) for the 8D+time cases. This is load-bearing because downstream PINN accuracy improvements could result from increased total collocation points or optimizer choices rather than faithful approximation of the (potentially multimodal, non-stationary) residual distribution.
  2. [Numerical experiments] Numerical experiments section: No ablation studies or isolated metrics are supplied that separate the normalizing flow's density estimation quality from other implementation factors, leaving the weakest assumption (accurate learning of the residual distribution in unified spacetime without time marching) untested directly.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their thoughtful review and constructive suggestions. We address each major comment below, providing clarifications and indicating where revisions will strengthen the manuscript.

read point-by-point responses

  1. Referee: [Abstract] The assertion that the normalizing flow 'effectively learns' the residual distribution and produces useful adaptive samples lacks any referenced quantitative diagnostics (KL divergence, effective sample size, coverage of high-residual regions, or direct comparison to ground-truth residual histograms) for the 8D+time cases. This is load-bearing because downstream PINN accuracy improvements could result from increased total collocation points or optimizer choices rather than faithful approximation of the (potentially multimodal, non-stationary) residual distribution.

    Authors: We acknowledge that the manuscript does not include explicit quantitative diagnostics such as KL divergence or direct histogram comparisons for the normalizing flow in the 8D+time cases. The effectiveness is instead shown indirectly through benchmark results demonstrating lower errors with adaptive sampling versus uniform sampling at fixed total collocation point counts. In high dimensions, direct ground-truth residual histograms are computationally intractable. We will add KL divergence, effective sample size, and coverage metrics for the 2D cases in the revised version; for higher dimensions the downstream performance on moving and localized features provides supporting evidence that the improvements stem from residual-driven adaptation rather than point count or optimizer alone. revision: partial

  2. Referee: [Numerical experiments] No ablation studies or isolated metrics are supplied that separate the normalizing flow's density estimation quality from other implementation factors, leaving the weakest assumption (accurate learning of the residual distribution in unified spacetime without time marching) untested directly.

    Authors: The current numerical section compares the complete DAS-PINNs method against uniform-sampling PINNs on problems with sharp moving features (2D) and localized structures (up to 8D+time), with the same total collocation budget. While this does not isolate the flow's density estimation in an ablation, the consistent gains on non-stationary problems support that the unified spacetime sampling tracks high-residual regions without explicit time marching. We agree dedicated ablations would strengthen the presentation and will add them for the lower-dimensional benchmarks in revision, including comparisons of learned versus uniform distributions. revision: yes

Circularity Check

0 steps flagged

No circularity: method effectiveness evaluated on external benchmarks

full rationale

The paper extends a normalizing-flow-based adaptive sampling procedure to spacetime PINN collocation by training the flow on the PDE residual and sampling from the learned density. This construction is not self-definitional because the claim that the resulting points improve solution accuracy is tested on independent benchmark problems (sharp moving features in 2D up to localized structures in 8D+time) rather than being true by algebraic identity or by re-using the same fitted quantity. No load-bearing self-citation chain, uniqueness theorem, or ansatz imported from prior author work is invoked to force the central result; the derivation therefore remains self-contained against external numerical evidence.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract provides insufficient detail to identify specific free parameters, axioms, or invented entities; all fields left empty pending full text.

pith-pipeline@v0.9.1-grok · 5699 in / 1048 out tokens · 22852 ms · 2026-06-28T00:08:52.268339+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

20 extracted references · 7 canonical work pages · 1 internal anchor

  1. [1]

    D. F. Griffiths, J. W. Dold, D. J. Silvester, Essential Partial Differential Equations, Springer, Heidelberg, 2015

    2015

  2. [2]

    Raissi, P

    M. Raissi, P. Perdikaris, G. E. Karniadakis, Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys. 378 (2019) 686--707. https://doi.org/10.1016/j.jcp.2018.10.045 doi:10.1016/j.jcp.2018.10.045

    work page doi:10.1016/j.jcp.2018.10.045 2019

  3. [3]

    J. Han, A. Jentzen, W. E, Solving high-dimensional partial differential equations using deep learning, Proc. Natl. Acad. Sci. U.S.A. 115 (34)) (2018) 8505--8510. https://doi.org/10.1073/pnas.1718942115 doi:10.1073/pnas.1718942115

    work page doi:10.1073/pnas.1718942115 2018

  4. [4]

    K. Tang, X. Wan, C. Yang, D AS - PINN s: a deep adaptive sampling method for solving high-dimensional partial differential equations, J. Comput. Phys. 476 (2023) Paper No. 111868, 26. https://doi.org/10.1016/j.jcp.2022.111868 doi:10.1016/j.jcp.2022.111868

    work page doi:10.1016/j.jcp.2022.111868 2023

  5. [5]

    X. Wang, K. Tang, J. Zhai, X. Wan, C. Wang, Deep adaptive sampling for surrogate modeling without labeled data, Journal of Scientific Computing 101 (77) (2024). https://doi.org/10.1007/s10915-024-02711-1 doi:10.1007/s10915-024-02711-1

    work page doi:10.1007/s10915-024-02711-1 2024

  6. [6]

    B. Xu, H. Yu, J. Zhai, K. Tang, X. Wan, Moving sample method for solving time-dependent partial differential equations, arXiv preprint arXiv:2601.18575 (2026). https://doi.org/10.48550/arXiv.2601.18575 doi:10.48550/arXiv.2601.18575

    work page doi:10.48550/arxiv.2601.18575 2026

  7. [7]

    A. G. Baydin, B. A. Pearlmutter, A. A. Radul, J. M. Siskind, Automatic differentiation in machine learning: a survey, J. Mach. Learn. Res. 18 (2017) Paper No. 153, 43

    2017

  8. [8]

    K. Tang, J. Zhai, X. Wan, C. Yang, Adversarial adaptive sampling: Unify PINN and optimal transport for the approximation of PDE s, Proceedings of ICLR 2024 (2024)

    2024

  9. [9]

    K. Tang, X. Wan, Q. Liao, Deep density estimation via invertible block-triangular mapping , Theoretical and Applied Mechanics Letters 10 (3) (2020) 143--148. https://doi.org/10.1016/j.taml.2020.01.023 doi:10.1016/j.taml.2020.01.023

    work page doi:10.1016/j.taml.2020.01.023 2020

  10. [10]

    D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980 (2014). https://doi.org/10.48550/arXiv.1412.6980 doi:10.48550/arXiv.1412.6980

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.1412.6980 2014

  11. [11]

    and Perdikaris, P

    Raissi, M. and Perdikaris, P. and Karniadakis, G. E. , TITLE =. J. Comput. Phys. , FJOURNAL =. 2019 , PAGES =

    2019

  12. [12]

    Baydin, Atilim Gunes and Pearlmutter, Barak A and Radul, Alexey Andreyevich and Siskind, Jeffrey Mark , TITLE =. J. Mach. Learn. Res. , FJOURNAL =. 2017 , PAGES =

    2017

  13. [13]

    Tang, Kejun and Wan, Xiaoliang and Yang, Chao , TITLE =. J. Comput. Phys. , FJOURNAL =. 2023 , PAGES =

    2023

  14. [14]

    Adversarial adaptive sampling: Unify

    Tang, Kejun and Zhai, Jiayu and Wan, Xiaoliang and Yang, Chao , journal=. Adversarial adaptive sampling: Unify

  15. [15]

    arXiv preprint arXiv:1412.6980 , year=

    Adam: A method for stochastic optimization , author=. arXiv preprint arXiv:1412.6980 , year=

    Pith/arXiv arXiv

  16. [16]

    arXiv preprint arXiv:2601.18575 , year=

    Moving sample method for solving time-dependent partial differential equations , author=. arXiv preprint arXiv:2601.18575 , year=

    arXiv

  17. [17]

    2020 , publisher=

    Tang, Kejun and Wan, Xiaoliang and Liao, Qifeng , journal=. 2020 , publisher=

    2020

  18. [18]

    Journal of Scientific Computing , volume=

    Deep Adaptive Sampling for Surrogate Modeling Without Labeled Data , author=. Journal of Scientific Computing , volume=. 2024 , publisher=

    2024

  19. [19]

    and Dold, John W

    Griffiths, David F. and Dold, John W. and Silvester, David J. , TITLE =. 2015 , PAGES =

    2015

  20. [20]

    and Jentzen, A and E, W

    Han,J. and Jentzen, A and E, W. , title=. Proc. Natl. Acad. Sci. U.S.A. , volume=. 2018 , DOI=

    2018