Recognition: unknown
Uncertainty Quantification in PINNs for Turbulent Flows: Bayesian Inference and Repulsive Ensembles
Pith reviewed 2026-05-10 06:16 UTC · model grok-4.3
The pith
Bayesian PINNs provide the most consistent uncertainty estimates for reconstructing turbulent flows from sparse data.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
By incorporating Hamiltonian Monte Carlo sampling with a tempered multi-component likelihood in Bayesian PINNs and enforcing diversity in function space for repulsive ensembles, the framework shows that Bayesian PINNs deliver the most consistent uncertainty estimates across all inferred quantities while function-space repulsive ensembles provide a computationally efficient approximation with competitive accuracy for primary flow variables.
What carries the argument
The combination of Hamiltonian Monte Carlo sampling with a tempered multi-component likelihood in Bayesian PINNs together with repulsive deep ensembles that enforce diversity in function space.
If this is right
- Bayesian PINNs should be preferred when reliable uncertainty quantification is required across all flow variables in data-driven RANS modeling.
- Repulsive ensembles can serve as a practical substitute when computational resources limit the use of full Bayesian sampling but primary flow accuracy remains essential.
- Likelihood tempering and function-space diversity are the mechanisms that drive improved calibration in these PDE-constrained settings.
- The hierarchy of test cases supports extension of the same uncertainty methods to other sparse-data turbulent reconstruction tasks.
Where Pith is reading between the lines
- The same probabilistic extensions could be tested on inverse problems governed by other PDEs outside fluid dynamics to check whether the consistency-efficiency trade-off persists.
- Hybrid approaches that combine repulsive ensembles with partial Bayesian sampling might retain most of the calibration benefit at reduced cost.
- Function-space diversity techniques may also reduce over in standard deep ensembles for non-physics machine-learning tasks.
Load-bearing premise
The tempered multi-component likelihood and enforced ensemble diversity improve uncertainty calibration for PDE-constrained inverse problems without introducing systematic biases that are not captured by the chosen test cases.
What would settle it
Empirical coverage rates of the uncertainty intervals falling significantly outside the nominal levels on a new turbulent flow dataset with different sparsity, noise, or Reynolds number would falsify the claim of consistent calibration.
Figures
read the original abstract
Physics-informed neural networks (PINNs) have emerged as a promising framework for solving inverse problems governed by partial differential equations (PDEs), including the reconstruction of turbulent flow fields from sparse data. However, most existing PINN formulations are deterministic and do not provide reliable quantification of epistemic uncertainty, which is critical for ill-posed problems such as data-driven Reynolds-averaged Navier-Stokes (RANS) modeling. In this work, we develop and systematically evaluate a set of probabilistic extensions of PINNs for uncertainty quantification in turbulence modeling. The proposed framework combines (i) Bayesian PINNs with Hamiltonian Monte Carlo sampling and a tempered multi-component likelihood, (ii) Monte Carlo dropout, and (iii) repulsive deep ensembles that enforce diversity in function space. Particular emphasis is placed on the role of ensemble diversity and likelihood tempering in improving uncertainty calibration for PDE-constrained inverse problems. The methods are assessed on a hierarchy of test cases, including the Van der Pol oscillator and turbulent flow past a circular cylinder at Reynolds numbers Re=3,900 (direct numerical simulation data) and Re = 10,000 (experimental particle image velocimetry data). The results demonstrate that Bayesian PINNs provide the most consistent uncertainty estimates across all inferred quantities, while function-space repulsive ensembles offer a computationally efficient approximation with competitive accuracy for primary flow variables. These findings provide quantitative insight into the trade-offs between accuracy, computational cost, and uncertainty calibration in physics-informed learning, and offer practical guidance for uncertainty quantification in data-driven turbulence modeling.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript develops and evaluates probabilistic extensions of physics-informed neural networks (PINNs) for uncertainty quantification in inverse problems for turbulent flows. It introduces Bayesian PINNs using Hamiltonian Monte Carlo with a tempered multi-component likelihood, Monte Carlo dropout, and repulsive deep ensembles enforcing function-space diversity. These are assessed on a test hierarchy consisting of the Van der Pol oscillator and cylinder flow at Re=3900 (DNS data) and Re=10,000 (PIV data). The central claim is that Bayesian PINNs deliver the most consistent uncertainty estimates across inferred quantities, while repulsive ensembles provide a computationally efficient approximation with competitive accuracy on primary flow variables.
Significance. If the empirical ranking holds under broader validation, the work supplies practical guidance on accuracy-cost-calibration trade-offs for physics-informed learning in data-driven turbulence modeling, particularly for ill-posed RANS-type inverse problems where epistemic uncertainty matters.
major comments (1)
- [Test cases and results] The evaluation hierarchy (Van der Pol oscillator plus cylinder at Re=3900 DNS and Re=10,000 PIV) primarily exercises data sparsity and moderate nonlinearity. It does not include regimes in which PDE residual errors dominate or in which tempering materially shifts posterior mass on secondary quantities such as Reynolds stresses or pressure. Consequently, the claim that Bayesian PINNs yield the 'most consistent' uncertainty estimates (and that repulsive ensembles are competitive) rests on test cases that may not expose systematic mis-calibration introduced by the tempered likelihood or enforced diversity; this is load-bearing for the central ranking.
minor comments (2)
- [Abstract] The abstract states comparative results on standard test cases yet supplies no quantitative metrics, error bars, or ablation details on tempering/diversity parameters; the full manuscript should make these explicit in the results section.
- [Methods] Clarify the precise definition and implementation of the 'tempered multi-component likelihood' and the function-space repulsion term (including any hyperparameters) so that the reported calibration behavior can be reproduced.
Simulated Author's Rebuttal
We thank the referee for the constructive feedback and positive assessment of the manuscript's significance. We address the major comment on the evaluation hierarchy below, providing clarification on our test case rationale while agreeing to strengthen the discussion of limitations.
read point-by-point responses
-
Referee: [Test cases and results] The evaluation hierarchy (Van der Pol oscillator plus cylinder at Re=3900 DNS and Re=10,000 PIV) primarily exercises data sparsity and moderate nonlinearity. It does not include regimes in which PDE residual errors dominate or in which tempering materially shifts posterior mass on secondary quantities such as Reynolds stresses or pressure. Consequently, the claim that Bayesian PINNs yield the 'most consistent' uncertainty estimates (and that repulsive ensembles are competitive) rests on test cases that may not expose systematic mis-calibration introduced by the tempered likelihood or enforced diversity; this is load-bearing for the central ranking.
Authors: We appreciate this observation on the scope of our test cases. The hierarchy was selected to span increasing levels of complexity relevant to turbulent flow inverse problems: from the nonlinear Van der Pol oscillator (exercising the tempered multi-component likelihood) to cylinder flows at Re=3900 (DNS) and Re=10,000 (PIV), where data sparsity, turbulence effects, and PDE-data balance are central. In these cases, the inverse problem inherently involves non-negligible PDE residuals due to the approximate RANS-like modeling and sparse observations, and our results demonstrate that tempering improves calibration for both primary variables and secondary quantities such as Reynolds stresses. Nevertheless, we agree that additional regimes with more dominant PDE errors or explicit sensitivity analysis of tempering on posterior mass could further substantiate the ranking. We will revise the manuscript to expand the discussion of these limitations, include a dedicated paragraph on potential mis-calibration risks from tempering and diversity enforcement, and add a forward-looking statement on future validation in more challenging regimes. revision: partial
Circularity Check
No significant circularity; claims rest on empirical comparisons with independent DNS and PIV benchmarks
full rationale
The paper proposes probabilistic extensions of PINNs (Bayesian HMC with tempered likelihood, MC dropout, repulsive ensembles) and evaluates them on a hierarchy of test problems including the Van der Pol oscillator and cylinder flow at Re=3900 (DNS) and Re=10000 (PIV). The central claims concern relative consistency of uncertainty estimates and computational trade-offs; these are supported by direct numerical comparisons against external data rather than by any derivation that reduces to a fitted parameter, self-citation chain, or redefinition of inputs. No load-bearing step equates a reported uncertainty or ranking to its own construction by definition. Minor self-citations of prior PINN work are present but do not carry the empirical ranking results.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption PINNs can be trained to solve ill-posed inverse problems for turbulent flows governed by RANS equations
Forward citations
Cited by 2 Pith papers
-
Posterior Concentration of Bayesian Physics-Informed Neural Networks for Elliptic PDEs
Bayesian PINNs for elliptic PDEs have posteriors that contract around the true solution at near-optimal rates, with the prior adapting automatically to unknown smoothness.
-
Functional-prior-based Bayesian PDE-constrained inversion using PINNs
Presents fpBPINN framework using FPI-BPINN and fParVI-PINN to enable functional priors in Bayesian PINN-based PDE inversion, with random Fourier features aiding Gaussian prior representation.
Reference graph
Works this paper leans on
-
[1]
D. C. Wilcox, Turbulence Modeling for CFD, DCW Industries, 2006
2006
-
[2]
S. B. Pope, Turbulent Flows, Cambridge University Press, 2000
2000
-
[3]
P. A. Durbin, Some recent developments in turbulence closure modeling, Annual Review of Fluid Mechanics 50 (2018) 77–103
2018
-
[4]
P. R. Spalart, Detached-eddy simulation, Annual Review of Fluid Mechan- ics 41 (2009) 181–202
2009
-
[5]
P. Moin, K. Mahesh, Direct numerical simulation: A tool in turbulence research, Annual Review of Fluid Mechanics 30 (1998) 539–578
1998
-
[6]
P.Sagaut, LargeEddySimulationforIncompressibleFlows, Springer, 2006
2006
-
[7]
Duraisamy, G
K. Duraisamy, G. Iaccarino, H. Xiao, Turbulence modeling in the age of data, Annual Review of Fluid Mechanics 51 (2019) 357–377
2019
-
[8]
J. Ling, A. Kurzawski, J. Templeton, Reynolds averaged turbulence mod- elling using deep neural networks, Journal of Fluid Mechanics 807 (2016) 155–166
2016
-
[9]
M. Emory, J. Larsson, G. Iaccarino, Modeling of structural uncertainties in Reynolds-averaged Navier–Stokes closures, Physics of Fluids 25 (11) (2013) 110822.doi:10.1063/1.4824659
-
[10]
G. Iaccarino, A. A. Mishra, S. Ghili, Eigenspace perturbations for un- certainty estimation of single-point turbulence closures, Physical Review Fluids 2 (2) (2017) 024605.doi:10.1103/PhysRevFluids.2.024605
-
[11]
M. Raissi, P. Perdikaris, G. E. Karniadakis, Physics-informed neural net- works: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, Journal of Computational Physics 378 (2019) 686–707.doi:10.1016/j.jcp.2018.10.045
-
[12]
Shukla, P
K. Shukla, P. C. Di Leoni, J. Blackshire, D. Sparkman, G. E. Karniadakis, Physics-informed neural network for ultrasound nondestructive quantifica- tion of surface breaking cracks, Journal of Nondestructive Evaluation 39 (3) (2020) 61
2020
-
[13]
Shukla, A
K. Shukla, A. D. Jagtap, G. E. Karniadakis, Parallel physics-informed neu- ral networks via domain decomposition, Journal of Computational Physics 447 (2021) 110683
2021
-
[14]
Patel, V
Y. Patel, V. Mons, O. Marquet, G. Rigas, Turbulence model augmented physics-informed neural networks for mean-flow reconstruction, Physical Review Fluids 9 (3) (2024) 034605. 32
2024
-
[15]
H. Eivazi, M. Tahani, P. Schlatter, R. Vinuesa, Physics-informed neural networks for the Reynolds-averaged Navier–Stokes equations, Journal of Computational Physics 452 (2022) 110912.doi:10.1016/j.jcp.2021. 110912
-
[16]
Z.Zhang, K.Shukla, Z.Wang, A.Morales, T.Käufer, S.Salauddin, N.Wal- ters, D. Barrett, K. Ahmed, M. Triantafyllou, et al., Turbulence closure in rans and flow inference around a cylinder using pinns and sparse experi- mental data, arXiv preprint arXiv:2510.06049 (2025)
-
[17]
A. F. Psaros, X. Meng, Z. Zou, L. Guo, G. E. Karniadakis, Uncertainty quantification in scientific machine learning: Methods, metrics, and com- parisons, Journal of Computational Physics 477 (2023) 111902
2023
-
[18]
L.Yang, X.Meng, G.E.Karniadakis, B-PINNs: Bayesianphysics-informed neural networks for forward and inverse pde problems with noisy data, Journal of Computational Physics 425 (2021) 109913
2021
-
[19]
Z. Zou, X. Meng, G. E. Karniadakis, Uncertainty quantification for noisy inputs–outputs in physics-informed neural networks and neural operators, Computer Methods in Applied Mechanics and Engineering 433 (2025) 117479
2025
-
[20]
H. Liu, Z. Wang, R. Deng, S. Wang, X. Meng, C. Xu, S. Cai, Flow recon- struction with uncertainty quantification from noisy measurements based on bayesian physics-informed neural networks, Physics of Fluids 36 (11) (2024)
2024
-
[21]
X. Meng, L. Yang, Z. Mao, J. del Águila Ferrandis, G. E. Karniadakis, Learning functional priors and posteriors from data and physics, Journal of Computational Physics 457 (2022) 111073
2022
-
[22]
Z. Zou, G. E. Karniadakis, Multi-head physics-informed neural networks for learning functional priors and uncertainty quantification, Journal of Computational Physics 531 (2025) 113947
2025
-
[23]
X. Meng, H. Babaee, G. E. Karniadakis, Multi-fidelity Bayesian neural networks: Algorithms and applications, Journal of Computational Physics 438 (2021) 110361
2021
-
[24]
Aikawa, N
Y. Aikawa, N. Ueda, T. Tanaka, Improving the efficiency of training physics-informed neural networks using active learning, New Generation Computing 42 (4) (2024) 739–760
2024
-
[25]
Z. Zou, T. Meng, P. Chen, J. Darbon, G. E. Karniadakis, Leveraging vis- cous Hamilton–Jacobi PDEs for uncertainty quantification in scientific ma- chine learning, SIAM/ASA Journal on Uncertainty Quantification 12 (4) (2024) 1165–1191. 33
2024
-
[26]
Linka, A
K. Linka, A. Schäfer, X. Meng, Z. Zou, G. E. Karniadakis, E. Kuhl, Bayesian physics informed neural networks for real-world nonlinear dynam- icalsystems, ComputerMethodsinAppliedMechanicsandEngineering402 (2022) 115346
2022
-
[27]
R. M. Neal, Bayesian Learning for Neural Networks, Vol. 118 of Lecture Notes in Statistics, Springer, 1996.doi:10.1007/978-1-4612-0745-0
-
[28]
R. M. Neal, MCMC using Hamiltonian dynamics, in: Handbook of Markov Chain Monte Carlo, Chapman and Hall/CRC, 2011, pp. 113–162
2011
-
[29]
M. D. Hoffman, A. Gelman, The No-U-Turn sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo, Journal of Machine Learning Research 15 (47) (2014) 1593–1623
2014
-
[30]
Y. Gal, Z. Ghahramani, Dropout as a Bayesian approximation: Repre- senting model uncertainty in deep learning, in: Proceedings of the 33rd International Conference on Machine Learning (ICML), 2016
2016
-
[31]
Lakshminarayanan, A
B. Lakshminarayanan, A. Pritzel, C. Blundell, Simple and scalable predic- tive uncertainty estimation using deep ensembles, in: Advances in Neural Information Processing Systems (NeurIPS), 2017
2017
-
[32]
D’Angelo, V
F. D’Angelo, V. Fortuin, Repulsive deep ensembles are Bayesian, in: Ad- vances in Neural Information Processing Systems (NeurIPS), 2021
2021
-
[33]
arXiv preprint arXiv:2505.17308
P. Pilar, M. Heinonen, N. Wahlström, Repulsive Ensembles for Bayesian Inference in Physics-informed Neural Networks, arXiv preprint arXiv:2505.17308 (2025)
-
[34]
Zhang, L
D. Zhang, L. Lu, L. Guo, G. E. Karniadakis, Quantifying total uncer- tainty in physics-informed neural networks for solving forward and inverse stochastic problems, Journal of Computational Physics 397 (2019) 108850
2019
-
[35]
Wenzel, K
F. Wenzel, K. Roth, B. S. Veeling, J. Świątkowski, L. Tran, S. Mandt, J. Snoek, T. Salimans, R. Jenatton, S. Nowozin, How good is the Bayes posterior in deep neural networks really?, in: Proceedings of the 37th In- ternational Conference on Machine Learning (ICML), 2020
2020
-
[36]
Aitchison, A statistical theory of cold posteriors in deep neural networks, in: International Conference on Learning Representations (ICLR), 2021
L. Aitchison, A statistical theory of cold posteriors in deep neural networks, in: International Conference on Learning Representations (ICLR), 2021
2021
- [37]
-
[38]
D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, in: International Conference on Learning Representations (ICLR), 2015. 34
2015
-
[39]
D. C. Liu, J. Nocedal, On the limited memory BFGS method for large scale optimization, Mathematical Programming 45 (1–3) (1989) 503–528. doi:10.1007/BF01589116
-
[40]
Z. Zou, Z. Wang, G. Em Karniadakis, Learning and discovering multiple solutionsusingphysics-informedneuralnetworkswithrandominitialization and deep ensemble, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 481 (2325) (2025)
2025
-
[41]
Z. Zou, X. Meng, G. E. Karniadakis, Correcting model misspecification in physics-informed neural networks (PINNs), Journal of Computational Physics 505 (2024) 112918
2024
-
[42]
Kuleshov, N
V. Kuleshov, N. Fenner, S. Ermon, Accurate uncertainties for deep learn- ing using calibrated regression, in: Proceedings of the 35th International Conference on Machine Learning (ICML), 2018
2018
-
[43]
J. Yao, W. Pan, S. Ghosh, F. Doshi-Velez, Quality of uncertainty quantifi- cation for Bayesian neural network inference, in: Workshop on Uncertainty and Robustness in Deep Learning, ICML, 2019
2019
-
[44]
Z. Zou, X. Meng, A. F. Psaros, G. E. Karniadakis, NeuralUQ: A compre- hensivelibraryforuncertaintyquantificationinneuraldifferentialequations and operators, SIAM Review 66 (1) (2024) 161–190
2024
-
[45]
Fischer, S
P. Fischer, S. Kerkemeier, M. Min, Y.-H. Lan, M. Phillips, T. Rathnayake, E. Merzari, A. Tomboulides, A. Karakus, N. Chalmers, et al., NekRS, a GPU-accelerated spectral element Navier-Stokes solver, Parallel Comput- ing 114 (2022) 102982
2022
-
[46]
D. Fan, G. Jodin, T. Consi, L. Bonfiglio, Y. Ma, L. Keyes, G. E. Karni- adakis, M. S. Triantafyllou, A robotic intelligent towing tank for learning complex fluid-structure dynamics, Science Robotics 4 (36) (2019) eaay5063
2019
-
[47]
Westerweel, F
J. Westerweel, F. Scarano, Universal outlier detection for PIV data, Ex- periments in Fluids 39 (6) (2005) 1096–1100. 35
2005
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.