pith. sign in

arxiv: 2206.00578 · v2 · submitted 2022-06-01 · ⚛️ physics.comp-ph

Extending OpenKIM with an Uncertainty Quantification Toolkit for Molecular Modeling

Yonatan Kurniawan (1) , Cody L. Petrie (1) , Mark K. Transtrum (1) , Ellad B. Tadmor (2) , Ryan S. Elliott (2) , Daniel S. Karls (2) , Mingjian Wen (3) ((1) Department of Physics , Astronomy
show 11 more authors
Brigham Young University Provo United States (2) Department of Aerospace Engineering Mechanics University of Minnesota Minneapolis (3) Energy Technologies Area Lawrence Berkeley National Laboratory Berkeley United States)
This is my paper

Pith reviewed 2026-05-24 11:28 UTC · model grok-4.3

classification ⚛️ physics.comp-ph
keywords uncertainty quantificationinteratomic potentialsMarkov chain Monte Carlomolecular modelingparameter fittingfunctional form inadequacyatomistic simulations
0
0 comments X

The pith

Adjusting sampling temperature in parallel-tempered Markov chain Monte Carlo estimates uncertainty from interatomic potential functional form inadequacy.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces tools for quantifying uncertainty in atomistic simulations that rely on interatomic potentials. It addresses two distinct sources: variation in parameter values and inadequacy in the chosen functional form. The approach implements parallel-tempered Markov chain Monte Carlo and tunes the sampling temperature to isolate the contribution from functional form inadequacy. This matters because it supplies a concrete way to judge when simulation predictions for energies and forces can be trusted. The demonstration applies the method to predictions in a diamond atomic configuration.

Core claim

The central claim is that parallel-tempered Markov chain Monte Carlo, when the sampling temperature is adjusted, provides an estimate of the uncertainty in atomic energy and force predictions that arises specifically from the functional form of the interatomic potential, separate from uncertainty due to parameter values.

What carries the argument

Parallel-tempered Markov chain Monte Carlo with adjustable sampling temperature, which isolates uncertainty due to interatomic potential functional form inadequacy.

If this is right

  • Uncertainty from functional form can be quantified separately from parameter uncertainty in interatomic potential models.
  • The resulting estimates help determine the reliability of energy and force predictions in atomistic simulations.
  • Practitioners receive guidance on subtleties that arise when applying these uncertainty tools.
  • Interatomic potential developers obtain recommendations for incorporating such quantification during model construction.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The temperature-adjustment technique could be tested on other sampling algorithms beyond parallel-tempered Markov chain Monte Carlo to check generality.
  • Explicit separation of functional-form uncertainty might encourage systematic comparison of multiple potential forms during model selection.
  • The estimates could be cross-checked against discrepancies observed when predictions are compared to higher-accuracy reference calculations.

Load-bearing premise

That adjusting the sampling temperature in parallel-tempered Markov chain Monte Carlo isolates and reliably estimates uncertainty arising specifically from inadequacy of the interatomic potential functional form rather than other sources.

What would settle it

A calculation that compares the method's estimated functional-form uncertainty against the actual prediction differences obtained when the same data are refit with a qualitatively different functional form would test whether the temperature adjustment captures the intended source.

Figures

Figures reproduced from arXiv: 2206.00578 by (2) Department of Aerospace Engineering, (3) Energy Technologies Area, Astronomy, Berkeley, Brigham Young University, Cody L. Petrie (1), Daniel S. Karls (2), Ellad B. Tadmor (2), Lawrence Berkeley National Laboratory, Mark K. Transtrum (1), Mechanics, Mingjian Wen (3) ((1) Department of Physics, Minneapolis, Provo, Ryan S. Elliott (2), United States, United States), University of Minnesota, Yonatan Kurniawan (1).

Figure 1
Figure 1. Figure 1: UQ framework implemented in KLIFF. The UQ process starts with the [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Example Python script for constructing a model using KLIFF. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Example Python script for using the run_mcmc function to perform sampling. The convergence of the parameter chains is assessed by calculating Rˆp . In our implementation, this is realized by the function kliff.uq.rhat, demonstrated by the listing in [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Example Python script for using kliff.uq.rhat function to compute Rˆp as a convergence assessment tool. model and MCMC setup in these scripts. We then present and discuss the sampling results and some of the subtleties associated with their interpretation. This example is based on the Stillinger–Weber potential in OpenKIM [65], [66]. This is a cluster potential originally introduced to model silicon [67]. … view at source ↗
Figure 6
Figure 6. Figure 6: Marginal distributions of the MCMC samples (the projection of the joint distribution onto a single parameter axis) of the SW potential at several [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Distribution of cost at each sampling temperature. [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
read the original abstract

Atomistic simulations are an important tool in materials modeling. Interatomic potentials (IPs) are at the heart of such molecular models, and the accuracy of a model's predictions depends strongly on the choice of IP. Uncertainty quantification (UQ) is an emerging tool for assessing the reliability of atomistic simulations. The Open Knowledgebase of Interatomic Models (OpenKIM) is a cyberinfrastructure project whose goal is to collect and standardize the study of IPs to enable transparent, reproducible research. Part of the OpenKIM framework is the Python package, KIM-based Learning-Integrated Fitting Framework (KLIFF), that provides tools for fitting parameters in an IP to data. This paper introduces a UQ toolbox extension to KLIFF. We focus on two sources of uncertainty: variations in parameters and inadequacy of the functional form of the IP. Our implementation uses parallel-tempered Markov chain Monte Carlo (PTMCMC), adjusting the sampling temperature to estimate the uncertainty due to the functional form of the IP. We demonstrate on a Stillinger--Weber potential that makes predictions for the atomic energies and forces for silicon in a diamond configuration. Finally, we highlight some potential subtleties in applying and using these tools with recommendations for practitioners and IP developers.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 2 minor

Summary. The paper extends the KLIFF fitting framework in OpenKIM with a UQ toolbox that uses parallel-tempered Markov chain Monte Carlo (PTMCMC) to quantify two uncertainty sources in interatomic potentials: parameter variations and functional-form inadequacy. The key innovation is adjusting the PTMCMC sampling temperature to isolate the latter; this is demonstrated by fitting a Stillinger-Weber potential to atomic energies and forces for silicon in the diamond structure and reporting resulting uncertainty bands on predictions.

Significance. If the temperature-adjustment procedure can be shown to isolate functional-form error, the toolkit would be a useful addition to OpenKIM for practitioners who need quantitative reliability estimates on IP-driven simulations. The work builds directly on an existing open-source fitting infrastructure and provides a concrete demonstration, which are positive features.

major comments (1)
  1. [Methods] Methods (PTMCMC temperature scaling): the manuscript states that raising the sampling temperature estimates uncertainty attributable specifically to IP functional-form inadequacy, yet provides neither a derivation separating this source from finite-data effects, parameter correlations, or noise-model misspecification nor a controlled comparison (e.g., against a more flexible reference model or synthetic data with injected functional error). The silicon SW demonstration reports only the resulting bands.
minor comments (2)
  1. [Abstract] Abstract: the description of the PTMCMC procedure is too terse to convey how temperature adjustment is implemented or validated.
  2. [Results/Demonstration] Figure captions and text should explicitly state the number of chains, temperature ladder, and convergence diagnostics used in the PTMCMC runs.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their thoughtful review of our manuscript. Below we address the major comment raised.

read point-by-point responses

  1. Referee: [Methods] Methods (PTMCMC temperature scaling): the manuscript states that raising the sampling temperature estimates uncertainty attributable specifically to IP functional-form inadequacy, yet provides neither a derivation separating this source from finite-data effects, parameter correlations, or noise-model misspecification nor a controlled comparison (e.g., against a more flexible reference model or synthetic data with injected functional error). The silicon SW demonstration reports only the resulting bands.

    Authors: We agree that the manuscript would benefit from a more detailed explanation of how the temperature scaling isolates functional-form uncertainty. While the approach builds on established methods in Bayesian model averaging and tempered MCMC for handling model discrepancy, we did not provide an explicit derivation or synthetic validation in the original submission. In the revised version, we will include a derivation in the Methods section showing how the effective posterior at elevated temperature incorporates an additional variance term attributable to model inadequacy, and we will discuss the assumptions regarding separation from parameter correlations and noise misspecification. Additionally, we will add a brief discussion of the limitations and suggest that the reported bands represent an upper bound on the combined uncertainties. We believe this addresses the concern while maintaining the focus of the paper on the toolkit implementation. revision: yes

Circularity Check

0 steps flagged

No significant circularity; standard PTMCMC application to KLIFF with temperature adjustment for UQ

full rationale

The paper extends an existing fitting framework (KLIFF) by implementing parallel-tempered MCMC and using temperature scaling to probe functional-form uncertainty, then demonstrates the resulting bands on a Stillinger-Weber silicon model. No load-bearing step equates a claimed prediction or uniqueness result to a fitted parameter or self-citation by construction. The method description invokes a standard statistical technique without re-deriving it from the target quantities, and the demonstration reports outputs rather than asserting that any output is forced by the inputs. This is self-contained against external benchmarks and receives the default non-circularity finding.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The approach rests on standard Bayesian sampling assumptions and the applicability of PTMCMC to interatomic potential parameter spaces; no new entities or fitted constants are introduced in the abstract.

axioms (1)
  • standard math PTMCMC sampling converges to the target posterior distribution when temperatures are adjusted appropriately
    Invoked as the core mechanism for estimating both parameter and functional-form uncertainty.

pith-pipeline@v0.9.0 · 5844 in / 1175 out tokens · 26747 ms · 2026-05-24T11:28:40.299458+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

72 extracted references · 72 canonical work pages · 4 internal anchors

  1. [1]

    The art and science of an analytic potential,

    D. W. Brenner, “The art and science of an analytic potential,” physica status solidi (b) , vol. 217, no. 1, pp. 23–40, 2000

    work page 2000

  2. [2]

    The potential of atomistic simulations and the Knowledgebase of Interatomic Models,

    E. B. Tadmor, R. S. Elliott, J. P. Sethna, R. E. Miller, and C. A. Becker, “The potential of atomistic simulations and the Knowledgebase of Interatomic Models,” JOM, vol. 63, no. 7, pp. 17–17, Jul 2011

    work page 2011

  3. [3]

    Roadmap on multiscale materials modeling,

    E. v. d. Giessen, P. A. Schultz, N. Bertin, V . V . Bulatov, W. Cai, G. Cs´anyi, S. M. Foiles, M. G. D. Geers, C. Gonz ´alez, M. H ¨utter, W. K. Kim, D. M. Kochmann, J. LLorca, A. E. Mattsson, J. Rottler, A. Shluger, R. B. Sills, I. Steinbach, A. Strachan, and E. B. Tadmor, “Roadmap on multiscale materials modeling,” Modelling and Simulation in Materials ...

    work page 2020

  4. [4]

    LeSar, Introduction to computational materials science

    R. LeSar, Introduction to computational materials science. Cambridge, England: Cambridge University Press, Apr. 2013

    work page 2013

  5. [5]

    The atomic simulation environment – a Python library for working with atoms,

    A. H. Larsen, J. J. Mortensen, J. Blomqvist, I. E. Castelli, R. Christensen, M. Dułak, J. Friis, M. N. Groves, B. Hammer, C. Hargus, E. D. Hermes, P. C. Jennings, P. B. Jensen, J. Kermode, J. R. Kitchin, E. L. Kolsbjerg, J. Kubal, K. Kaasbjerg, S. Lysgaard, J. B. Maronsson, T. Maxson, T. Olsen, L. Pastewka, A. Peterson, C. Rostgaard, J. Schiøtz, O. Sch ¨u...

    work page 2017

  6. [6]

    Lammps - a flexible simulation tool for particle- based materials modeling at the atomic, meso, and continuum scales,

    A. P. Thompson, H. M. Aktulga, R. Berger, D. S. Bolintineanu, W. M. Brown, P. S. Crozier, P. J. in ’t Veld, A. Kohlmeyer, S. G. Moore, T. D. Nguyen, R. Shan, M. J. Stevens, J. Tranchida, C. Trott, and S. J. Plimpton, “Lammps - a flexible simulation tool for particle- based materials modeling at the atomic, meso, and continuum scales,” Computer Physics Comm...

    work page 2022

  7. [7]

    Bayesian calibration of computer models,

    M. C. Kennedy and A. O’Hagan, “Bayesian calibration of computer models,” Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol. 63, no. 3, p. 425464, 2001

    work page 2001

  8. [8]

    Interatomic potentials from first- principles calculations: The force-matching method,

    F. Ercolessi and J. B. Adams, “Interatomic potentials from first- principles calculations: The force-matching method,” EPL (Europhysics Letters), vol. 26, no. 8, p. 583, Jun 1994

    work page 1994

  9. [9]

    Transferability of empirical potentials and the Knowledgebase of Interatomic Models (KIM),

    D. S. Karls, “Transferability of empirical potentials and the Knowledgebase of Interatomic Models (KIM),” Ph.D., University of Minnesota, United States – Minnesota, Apr 2016. [Online]. Available: http://www.proquest.com/docview/1822512470/ abstract/22AAC5589C4D49E6PQ/1

    work page arXiv 2016

  10. [10]

    Uncertainty quantifi- cation and propagation of errors of the Lennard-Jones 12-6 parameters for n-alkanes,

    R. A. Messerly, T. A. Knotts, and W. V . Wilding, “Uncertainty quantifi- cation and propagation of errors of the Lennard-Jones 12-6 parameters for n-alkanes,” The Journal of Chemical Physics , vol. 146, no. 19, p. 194110, May 2017

    work page 2017

  11. [11]

    Quantifying parameter sensitivity and uncertainty for interatomic potential design: Application to saturated hydrocarbons,

    M. A. Tschopp, B. Chris Rinderspacher, S. Nouranian, M. I. Baskes, S. R. Gwaltney, and M. F. Horstemeyer, “Quantifying parameter sensitivity and uncertainty for interatomic potential design: Application to saturated hydrocarbons,” ASCE-ASME J Risk and Uncert in Engrg Sys Part B Mech Engrg , vol. 4, no. 1, Mar 2018. [Online]. Available: https://asmedigital...

    work page 2018

  12. [12]

    Multiobjective genetic training and uncertainty quantification of reactive force fields,

    A. Mishra, S. Hong, P. Rajak, C. Sheng, K.-i. Nomura, R. K. Kalia, A. Nakano, and P. Vashishta, “Multiobjective genetic training and uncertainty quantification of reactive force fields,” npj Computational Materials, vol. 4, no. 11, pp. 1–7, Aug 2018

    work page 2018

  13. [13]

    Bayesian, frequentist, and information geometry approaches to parametric uncertainty quan- tification of classical empirical interatomic potentials,

    Y . Kurniawan, C. L. Petrie, K. J. Williams, M. K. Transtrum, E. B. Tadmor, R. S. Elliott, D. S. Karls, and M. Wen, “Bayesian, frequentist, and information geometry approaches to parametric uncertainty quan- tification of classical empirical interatomic potentials,” arXiv preprint arXiv:2112.10851, 2021

    work page arXiv 2021

  14. [14]

    Bayesian ensemble approach to error estimation of interatomic poten- tials,

    S. L. Frederiksen, K. W. Jacobsen, K. S. Brown, and J. P. Sethna, “Bayesian ensemble approach to error estimation of interatomic poten- tials,” Physical Review Letters , vol. 93, no. 16, p. 165501, Oct 2004

    work page 2004

  15. [15]

    Bayesian uncertainty quantification and propagation in molecular dynamics sim- ulations: A high performance computing framework,

    P. Angelikopoulos, C. Papadimitriou, and P. Koumoutsakos, “Bayesian uncertainty quantification and propagation in molecular dynamics sim- ulations: A high performance computing framework,” The Journal of Chemical Physics, vol. 137, no. 14, p. 144103, Oct 2012

    work page 2012

  16. [16]

    Uncertainty quantification in MD simulations. part II: Bayesian inference of force-field parameters,

    F. Rizzi, H. N. Najm, B. J. Debusschere, K. Sargsyan, M. Salloum, H. Adalsteinsson, and O. M. Knio, “Uncertainty quantification in MD simulations. part II: Bayesian inference of force-field parameters,” Multiscale Modeling & Simulation , vol. 10, no. 4, pp. 1460–1492, Jan 2012

    work page 2012

  17. [17]

    Uncertainty quantification in MD simulations of concentration driven ionic flow through a silica nanopore. II. uncertain potential parameters,

    F. Rizzi, R. E. Jones, B. J. Debusschere, and O. M. Knio, “Uncertainty quantification in MD simulations of concentration driven ionic flow through a silica nanopore. II. uncertain potential parameters,” The Journal of Chemical Physics , vol. 138, no. 19, p. 194105, May 2013

    work page 2013

  18. [18]

    Uncertainty prediction in molecular simulations using ab initio derived force fields

    M. Cools-Ceuppens and T. Verstraelen, “Uncertainty prediction in molecular simulations using ab initio derived force fields.” Ph.D. dis- sertation, Ghent University, 2017. [Online]. Available: https://lib.ugent. be/fulltxt/RUG01/002/366/973/RUG01-002366973 2017 0001 AC.pdf

    work page 2017

  19. [19]

    Uncertainty quan- tification confirms unreliable extrapolation toward high pressures for united-atom Mie λ-6 force field,

    R. A. Messerly, M. R. Shirts, and A. F. Kazakov, “Uncertainty quan- tification confirms unreliable extrapolation toward high pressures for united-atom Mie λ-6 force field,” The Journal of Chemical Physics , vol. 149, no. 11, p. 114109, Sep 2018

    work page 2018

  20. [20]

    Quantifying uncertainty in thermophysical properties of walls by means of Bayesian inversion,

    L. De Simon, M. Iglesias, B. Jones, and C. Wood, “Quantifying uncertainty in thermophysical properties of walls by means of Bayesian inversion,” Energy and Buildings , vol. 177, pp. 220–245, Oct 2018

    work page 2018

  21. [21]

    Uncertainty quantification in non-equilibrium molecular dynamics simulations of thermal transport,

    M. V ohra, A. Y . Nobakht, S. Shin, and S. Mahadevan, “Uncertainty quantification in non-equilibrium molecular dynamics simulations of thermal transport,” International Journal of Heat and Mass Transfer , vol. 127, pp. 297–307, Dec 2018

    work page 2018

  22. [22]

    Discovering the active subspace for efficient UQ of molecular dynamics simulations of phonon transport in silicon,

    M. V ohra and S. Mahadevan, “Discovering the active subspace for efficient UQ of molecular dynamics simulations of phonon transport in silicon,” International Journal of Heat and Mass Transfer , vol. 132, pp. 577–586, Apr 2019

    work page 2019

  23. [23]

    Uncertainty analysis and estimation of robust AIREBO parameters for graphene,

    G. Dhaliwal, P. B. Nair, and C. V . Singh, “Uncertainty analysis and estimation of robust AIREBO parameters for graphene,” Carbon, vol. 142, pp. 300–310, Feb 2019

    work page 2019

  24. [24]

    Uncertainty and sensitivity analysis of mechanical and thermal properties computed through Embedded Atom Method potential,

    ——, “Uncertainty and sensitivity analysis of mechanical and thermal properties computed through Embedded Atom Method potential,” Com- putational Materials Science , vol. 166, pp. 30–41, Aug 2019

    work page 2019

  25. [25]

    Uncertainty quantification for classical effective potentials: an extension to potfit,

    S. Longbottom and P. Brommer, “Uncertainty quantification for classical effective potentials: an extension to potfit,” Modelling and Simulation in Materials Science and Engineering , vol. 27, no. 4, p. 044001, Apr 2019

    work page 2019

  26. [26]

    A KIM-compliant potfit for fitting sloppy interatomic potentials: application to the EDIP model for silicon,

    M. Wen, J. Li, P. Brommer, R. S. Elliott, J. P. Sethna, and E. B. Tad- mor, “A KIM-compliant potfit for fitting sloppy interatomic potentials: application to the EDIP model for silicon,” Modelling and Simulation in Materials Science and Engineering , vol. 25, no. 1, p. 014001, Nov 2016

    work page 2016

  27. [27]

    A force-matching stillinger-weber potential for MoS 2: Pa- rameterization and fisher information theory based sensitivity analysis,

    M. Wen, S. N. Shirodkar, P. Plech ´a˘c, E. Kaxiras, R. S. Elliott, and E. B. Tadmor, “A force-matching stillinger-weber potential for MoS 2: Pa- rameterization and fisher information theory based sensitivity analysis,” Journal of Applied Physics , vol. 122, no. 24, p. 244301, Dec 2017

    work page 2017

  28. [28]

    Exper- imental design and model reduction in systems biology,

    J. E. Jeong, Q. Zhuang, M. K. Transtrum, E. Zhou, and P. Qiu, “Exper- imental design and model reduction in systems biology,” Quantitative Biology, vol. 6, no. 4, pp. 287–306, Dec 2018

    work page 2018

  29. [29]

    The limitations of model-based experimental design and parameter estimation in sloppy systems,

    A. White, M. Tolman, H. D. Thames, H. R. Withers, K. A. Mason, and M. K. Transtrum, “The limitations of model-based experimental design and parameter estimation in sloppy systems,” PLOS Computational Biology, vol. 12, no. 12, p. e1005227, Dec 2016

    work page 2016

  30. [30]

    Bridging mechanistic and phenomeno- logical models of complex biological systems,

    M. K. Transtrum and P. Qiu, “Bridging mechanistic and phenomeno- logical models of complex biological systems,” PLOS Computational Biology, vol. 12, no. 5, p. e1004915, May 2016

    work page 2016

  31. [31]

    B. K. Mannakee, A. P. Ragsdale, M. K. Transtrum, and R. N. Gutenkunst, Sloppiness and the Geometry of Parameter Space , ser. Studies in Mechanobiology, Tissue Engineering and Biomaterials. Springer International Publishing, 2016, vol. 17, pp. 271–299. [Online]. Available: http://link.springer.com/10.1007/978-3-319-21296-8 11

    work page doi:10.1007/978-3-319-21296-8 2016

  32. [32]

    Perspective: Sloppiness and emergent theories in physics, biology, and beyond,

    M. K. Transtrum, B. B. Machta, K. S. Brown, B. C. Daniels, C. R. Myers, and J. P. Sethna, “Perspective: Sloppiness and emergent theories in physics, biology, and beyond,” The Journal of Chemical Physics , vol. 143, no. 1, p. 010901, Jul 2015

    work page 2015

  33. [33]

    Optimal experiment selection for pa- rameter estimation in biological differential equation models,

    M. K. Transtrum and P. Qiu, “Optimal experiment selection for pa- rameter estimation in biological differential equation models,” BMC Bioinformatics, vol. 13, no. 1, p. 181, Jul 2012

    work page 2012

  34. [34]

    Information geometry approach to verification of dynamic models in power systems,

    M. K. Transtrum, A. T. Sari ´c, and A. M. Stankovi ´c, “Information geometry approach to verification of dynamic models in power systems,” IEEE Transactions on Power Systems, vol. 33, no. 1, pp. 440–450, 2017

    work page 2017

  35. [35]

    Sloppiness, modeling, and evolution in biochemical networks,

    R. Gutenkunst, “Sloppiness, modeling, and evolution in biochemical networks,” Ph.D. dissertation, Cornell University, Aug 2007, accepted: 2007-08-29T17:25:45Z. [Online]. Available: https://ecommons.cornell. edu/handle/1813/8206

    work page 2007

  36. [36]

    Parameter space compression underlies emergent theories and predictive models,

    B. B. Machta, R. Chachra, M. K. Transtrum, and J. P. Sethna, “Parameter space compression underlies emergent theories and predictive models,” Science, vol. 342, no. 6158, pp. 604–607, Nov 2013

    work page 2013

  37. [37]

    Sloppy-model universality class and the Vandermonde matrix,

    J. J. Waterfall, F. P. Casey, R. N. Gutenkunst, K. S. Brown, C. R. Myers, P. W. Brouwer, V . Elser, and J. P. Sethna, “Sloppy-model universality class and the Vandermonde matrix,” Phys. Rev. Lett. , vol. 97, p. 150601, Oct 2006. [Online]. Available: https://link.aps.org/doi/10.1103/PhysRevLett.97.150601

    work page doi:10.1103/physrevlett.97.150601 2006

  38. [38]

    The parameter uncertainty inflation fallacy,

    P. Pernot, “The parameter uncertainty inflation fallacy,” The Journal of Chemical Physics, vol. 147, no. 10, p. 104102, Sep 2017

    work page 2017

  39. [39]

    emcee: The MCMC Hammer

    D. Foreman-Mackey, D. W. Hogg, D. Lang, and J. Goodman, “emcee: The mcmc hammer,” Publications of the Astronomical Society of the Pacific, vol. 125, no. 925, pp. 306–312, Mar 2013, arXiv: 1202.3665

    work page internal anchor Pith review Pith/arXiv arXiv 2013

  40. [40]

    Chaospy: An open source tool for designing methods of uncertainty quantification,

    J. Feinberg and H. P. Langtangen, “Chaospy: An open source tool for designing methods of uncertainty quantification,” Journal of Computa- tional Science, vol. 11, pp. 46–57, Nov 2015

    work page 2015

  41. [41]

    Easyvvuq: A library for verification, validation and uncertainty quantification in high performance computing,

    R. A. Richardson, D. W. Wright, W. Edeling, V . Jancauskas, J. Lakhlili, and P. V . Coveney, “Easyvvuq: A library for verification, validation and uncertainty quantification in high performance computing,” Journal of Open Research Software, vol. 8, no. 11, p. 11, Apr 2020

    work page 2020

  42. [42]

    Potfit: effective potentials from ab initio data,

    P. Brommer and F. G ¨ahler, “Potfit: effective potentials from ab initio data,” Modelling and Simulation in Materials Science and Engineering, vol. 15, no. 3, pp. 295–304, mar 2007. [Online]. Available: https://doi.org/10.1088/0965-0393/15/3/008

    work page doi:10.1088/0965-0393/15/3/008 2007

  43. [43]

    Force-matched embedded- atom method potential for niobium,

    M. R. Fellinger, H. Park, and J. W. Wilkins, “Force-matched embedded- atom method potential for niobium,” Physical Review B, vol. 81, no. 14, p. 144119, Apr 2010

    work page 2010

  44. [44]

    Highly optimized tight-binding model of silicon,

    T. J. Lenosky, J. D. Kress, I. Kwon, A. F. V oter, B. Edwards, D. F. Richards, S. Yang, and J. B. Adams, “Highly optimized tight-binding model of silicon,” Physical Review B , vol. 55, no. 3, pp. 1528–1544, Jan 1997

    work page 1997

  45. [45]

    Perspective: Machine learning potentials for atomistic simu- lations,

    J. Behler, “Perspective: Machine learning potentials for atomistic simu- lations,” The Journal of Chemical Physics , vol. 145, no. 17, p. 170901, Nov 2016

    work page 2016

  46. [46]

    Improvements to the Levenberg-Marquardt algorithm for nonlinear least-squares minimization

    M. K. Transtrum and J. P. Sethna, “Improvements to the Levenberg- Marquardt algorithm for nonlinear least-squares minimization,” arXiv:1201.5885 [physics] , Jan 2012, arXiv: 1201.5885. [Online]. Available: http://arxiv.org/abs/1201.5885

    work page internal anchor Pith review Pith/arXiv arXiv 2012

  47. [47]

    Why are nonlinear fits so challenging?

    M. K. Transtrum, B. B. Machta, and J. P. Sethna, “Why are nonlinear fits so challenging?” Physical Review Letters, vol. 104, no. 6, p. 060201, Feb 2010, arXiv: 0909.3884

    work page internal anchor Pith review Pith/arXiv arXiv 2010

  48. [48]

    Christensen, T

    R. Christensen, T. Bligaard, and K. W. Jacobsen, 3 - Bayesian error estimation in density functional theory , ser. Elsevier Series in Mechanics of Advanced Materials. Woodhead Publishing, Jan 2020, pp. 77–91. [Online]. Available: https://www.sciencedirect.com/science/ article/pii/B9780081029411000031

    work page 2020

  49. [49]

    Bayesian error estimation in density- functional theory,

    J. J. Mortensen, K. Kaasbjerg, S. L. Frederiksen, J. K. Nørskov, J. P. Sethna, and K. W. Jacobsen, “Bayesian error estimation in density- functional theory,” Physical Review Letters , vol. 95, no. 21, p. 216401, Nov 2005

    work page 2005

  50. [50]

    Construction of new electronic density functionals with error estimation through fitting,

    V . Petzold, T. Bligaard, and K. W. Jacobsen, “Construction of new electronic density functionals with error estimation through fitting,” Topics in Catalysis, vol. 55, no. 5, p. 402417, Jun 2012

    work page 2012

  51. [51]

    Parallel tempering: Theory, applications, and new perspectives,

    D. J. Earl and M. W. Deem, “Parallel tempering: Theory, applications, and new perspectives,” Physical Chemistry Chemical Physics (Incorpo- rating Faraday Transactions), vol. 7, p. 3910, 2005

    work page 2005

  52. [52]

    Robust bayesian inference via coarsen- ing,

    J. W. Miller and D. B. Dunson, “Robust bayesian inference via coarsen- ing,” Journal of the American Statistical Association , vol. 114, no. 527, p. 11131125, Jul 2019

    work page 2019

  53. [53]

    Dynamic temperature selection for parallel-tempering in Markov chain Monte Carlo simulations

    W. V ousden, W. M. Farr, and I. Mandel, “Dynamic temperature selec- tion for parallel-tempering in markov chain monte carlo simulations,” Monthly Notices of the Royal Astronomical Society , vol. 455, no. 2, pp. 1919–1937, Jan 2016, arXiv: 1501.05823

    work page internal anchor Pith review Pith/arXiv arXiv 1919

  54. [54]

    Inference from iterative simulation using multiple sequences,

    A. Gelman and D. B. Rubin, “Inference from iterative simulation using multiple sequences,” Statistical Science, vol. 7, no. 4, pp. 457–472, Nov 1992, zbl: 06853057

    work page 1992

  55. [55]

    General methods for monitoring conver- gence of iterative simulations,

    S. P. Brooks and A. Gelman, “General methods for monitoring conver- gence of iterative simulations,” Journal of Computational and Graphical Statistics, vol. 7, no. 4, pp. 434–455, Dec 1998

    work page 1998

  56. [56]

    Revisiting the Gelman–Rubin diagnostic,

    D. Vats and C. Knudson, “Revisiting the Gelman–Rubin diagnostic,” Statistical Science, vol. 36, no. 4, pp. 518–529, 2021

    work page 2021

  57. [57]

    Types of kim content

    “Types of kim content.” [Online]. Available: https://openkim.org/doc/ repository/kim-content/

    work page

  58. [58]

    Knowledgebase of Interatomic Models (KIM) application programming interface (API),

    R. S. Elliott and E. B. Tadmor, “Knowledgebase of Interatomic Models (KIM) application programming interface (API),” https://openkim.org/ kim-api, 2011

    work page 2011

  59. [59]

    Simulation codes compatible with the KIM API

    “Simulation codes compatible with the KIM API.” [Online]. Available: https://openkim.org/projects-using-kim/

    work page

  60. [60]

    KLIFF: A framework to develop physics-based and machine learning interatomic potentials,

    M. Wen, Y . Afshar, R. S. Elliott, and E. B. Tadmor, “KLIFF: A framework to develop physics-based and machine learning interatomic potentials,” Computer Physics Communications , vol. 272, p. 108218, Mar 2022

    work page 2022

  61. [61]

    KLIFF git repository

    “KLIFF git repository.” [Online]. Available: https://github.com/openkim/ kliff

    work page

  62. [62]

    SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python,

    P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C. J. Carey,˙I. Polat, Y . Feng, E. W. Moore, J. VanderPlas, D. Laxalde, J. Perktold, R. Cimrman, I. Henrik- se...

    work page 2020

  63. [63]

    Colabfit: Collaborative development of data-driven interatomic potentials for predictive molecular simulations

    “Colabfit: Collaborative development of data-driven interatomic potentials for predictive molecular simulations.” [Online]. Available: https://colabfit.org/

    work page

  64. [64]

    KLIFF uq

    Y . Kurniawan, “KLIFF uq.” [Online]. Available: https://gitlab.com/ yonatank93/kliff uq

    work page

  65. [65]

    Stillinger- Weber (SW) Model Driver v005,

    M. Wen, Y . Afshar, F. H. Stillinger, and T. A. Weber, “Stillinger- Weber (SW) Model Driver v005,” OpenKIM, https://doi.org/10.25950/ dd263fe3, 2021

    work page 2021

  66. [66]

    Stillinger-Weber potential for Si due to Stillinger and Weber (1985) v006,

    A. K. Singh, F. H. Stillinger, and T. A. Weber, “Stillinger-Weber potential for Si due to Stillinger and Weber (1985) v006,” OpenKIM, https://doi.org/10.25950/dd263fe3, 2021

    work page doi:10.25950/dd263fe3 1985

  67. [67]

    Computer simulation of local order in condensed phases of silicon,

    F. H. Stillinger and T. A. Weber, “Computer simulation of local order in condensed phases of silicon,” Physical Review B , vol. 31, pp. 5262– 5271, Apr 1985

    work page 1985

  68. [69]

    EDIP model for Si developed by Justo et al. (1998) v002,

    D. S. Karls, “EDIP model for Si developed by Justo et al. (1998) v002,” OpenKIM, https://doi.org/10.25950/545ca247, 2018

    work page doi:10.25950/545ca247 1998

  69. [70]

    In- teratomic potential for silicon defects and disordered phases,

    J. a. F. Justo, M. Z. Bazant, E. Kaxiras, V . V . Bulatov, and S. Yip, “In- teratomic potential for silicon defects and disordered phases,” Physical Review B, vol. 58, pp. 2539–2550, Aug 1998

    work page 1998

  70. [71]

    Information geometry for multiparameter models: New per- spectives on the origin of simplicity,

    K. N. Quinn, M. C. Abbott, M. K. Transtrum, B. B. Machta, and J. P. Sethna, “Information geometry for multiparameter models: New per- spectives on the origin of simplicity,” arXiv preprint arXiv:2111.07176, 2021

    work page arXiv 2021

  71. [72]

    Model reduction by manifold boundaries,

    M. K. Transtrum and P. Qiu, “Model reduction by manifold boundaries,” Physical Review Letters , vol. 113, no. 9, p. 098701, Aug 2014

    work page 2014

  72. [73]

    Maximum likelihood, profile likelihood, and penalized likelihood: A primer,

    S. R. Cole, H. Chu, and S. Greenland, “Maximum likelihood, profile likelihood, and penalized likelihood: A primer,” American Journal of Epidemiology, vol. 179, no. 2, pp. 252–260, Jan 2014

    work page 2014