Recognition: unknown
Fidelity of Machine Learned Potentials: Quantitative Assessment for Protonated Oxalate
Pith reviewed 2026-05-10 13:45 UTC · model grok-4.3
The pith
Two different machine-learned potentials for protonated oxalate agree closely on vibrational energies, infrared spectra, and hydrogen tunneling splittings.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Two widely applied machine-learned potential energy surfaces, one based on permutationally invariant polynomial regression and the other on the PhysNet neural network approach, are trained on the same dataset for protonated oxalate. When these surfaces are used in VSCF/VCI calculations of vibrational energies and wavefunctions, in direct comparison of infrared spectra, and in tunneling splitting calculations via ring polymer instanton theory, diffusion Monte Carlo, and the Q_im path method, the two surfaces yield results in excellent agreement with each other.
What carries the argument
Stress tests consisting of VSCF/VCI vibrational calculations, ring polymer instanton theory, diffusion Monte Carlo simulations, and the Q_im path method that evaluate the potentials on roughly one billion points across the 15-dimensional space.
If this is right
- Either potential can be used interchangeably for computing spectroscopic properties and tunneling rates in this system.
- Agreement on these quantities indicates that both approaches faithfully capture the potential beyond the training set points.
- The stress-test protocol provides a practical way to check fidelity for other machine-learned surfaces in high-dimensional spaces.
- The results support the use of either regression method for similar polyatomic molecules where extensive dynamical calculations are needed.
Where Pith is reading between the lines
- If similar agreement appears for other molecules, the need to maintain multiple independent machine-learned surfaces for cross-validation could decrease.
- The protocol could be extended to larger systems or to properties such as reaction rates that also sample distant regions of configuration space.
- Disagreement on only one of the three tunneling methods would point to a specific limitation in how a given potential handles rare but important geometries.
Load-bearing premise
The selected stress tests are sufficient to reveal any meaningful differences in how the two potentials represent the underlying surface throughout the full 15-dimensional space.
What would settle it
A clear numerical discrepancy, larger than method-specific uncertainties, appearing in the tunneling splitting values or in the vibrational energy levels computed from the two potentials.
Figures
read the original abstract
There has been a veritable explosion of methods and software to perform machine-learned regression on datasets of electronic energies and forces to develop high-dimensional machine learned potential energy surfaces (ML-PESs). A major, but not deeply-studied aspect is how well different ML-PESs represent the same dataset on which they are trained, beyond the standard fitting precision metrics. Here, this is examined in detail using several ''stress tests'', for two widely applied machine-learned potential approaches. One is based on permutationally invariant polynomial (PIP) linear least square regression and the other is the message-passing neural network PhysNet approach. These potentials and dipole moment surfaces are used in VSCF/VCI calculations of vibrational energies and wavefunctions. The energies from the two PESs are directly compared as are the IR spectra. In addition, tunneling splittings for the hydrogen transfer between two equivalent structures are reported from using three methods: ring polymer instanton theory, diffusion Monte Carlo simulations, and the $Q_{im}$ path method. These calculations require the evaluation of on the order of one billion energies that are widely dispersed in the 15-dimensional configurational space. The two PESs yield results for these quantities in excellent agreement with each other.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript examines the fidelity of two machine-learned potential energy surfaces (ML-PESs) for protonated oxalate (a 15-dimensional system) trained on the same electronic structure dataset: a permutationally invariant polynomial (PIP) linear least-squares model and the PhysNet message-passing neural network. Beyond standard fitting metrics, the authors apply multiple stress tests—VSCF/VCI calculations of vibrational energies and IR spectra, plus tunneling splittings obtained independently via ring-polymer instanton theory, diffusion Monte Carlo, and the Q_im path method—requiring ~10^9 energy evaluations widely dispersed across configuration space. The central claim is that the two PESs (and associated dipole surfaces) produce results in excellent quantitative agreement with each other on all tested observables.
Significance. If the reported agreement holds, the work provides concrete evidence that distinct ML architectures can faithfully represent the same underlying PES for high-dimensional quantum dynamics properties, including tunneling, without being limited by fitting precision alone. The multi-method cross-validation (~10^9 points) and use of independent quantum methods constitute a genuine advance over typical ML-PES validation. This strengthens confidence in deploying ML-PESs for spectroscopy and reaction dynamics in complex molecular systems.
minor comments (3)
- [Abstract/Introduction] The abstract and introduction would benefit from a brief explicit statement of the training-set size, energy range, and force RMSE values for both models to allow immediate comparison with the stress-test results.
- [VSCF/VCI calculations] In the VSCF/VCI section, clarify whether the same vibrational basis and mode-coupling scheme were used for both PESs or whether any re-optimization was performed; this would strengthen the direct comparability of the reported energies and spectra.
- [Figures and Tables] Figure captions for the IR spectra and tunneling-splitting tables should include the precise number of points sampled in each method and the coordinate ranges covered, to substantiate the claim of wide dispersion in the 15D space.
Simulated Author's Rebuttal
We thank the referee for the positive assessment of our manuscript and for recommending acceptance. The referee's summary correctly captures the central claim and the extensive stress-testing performed across vibrational properties and tunneling splittings.
Circularity Check
No significant circularity detected
full rationale
The paper's central claim is that two independently trained ML-PESs (PIP linear least-squares and PhysNet neural network) produce excellent agreement on VSCF/VCI vibrational energies/IR spectra and on tunneling splittings computed via three external methods (ring-polymer instanton, DMC, Q_im path). These stress tests evaluate ~10^9 points dispersed across the 15D space and are not derived from the PES fitting procedure itself. No load-bearing step reduces the agreement result to a fitted parameter, self-definition, or self-citation chain; the multi-method cross-validation supplies independent evidence. Minor self-citations to prior PIP or PhysNet work exist but are not invoked to justify the fidelity conclusion.
Axiom & Free-Parameter Ledger
axioms (2)
- domain assumption VSCF/VCI calculations accurately reflect differences in PES quality for vibrational energies and wavefunctions
- domain assumption Ring polymer instanton theory, diffusion Monte Carlo, and Q_im path method provide reliable tunneling splittings from a given PES
Reference graph
Works this paper leans on
-
[1]
1250.13 1.70018 1241.07 27.9534 90.8098 3.45034 1213.17 29.7036 49.193 5.20051 1167.59 31.4538 20.038 6.9507 1106.19 33.2041 3.61036 8.70089 1031.33 34.2042 0. 10.4511 945.749 42.7052 135.185 12.2013 852.367 51.2561 541.933 13.9515 754.143 59.8069 1165.57 15.7017 653.949 68.3575 1958.58 17.452 554.487 76.9081 2889.29 19.2022 458.221 85.4587 3937.51 20.952...
2041
-
[2]
Meuwly, WIREs Computational Molecular Science9, e1386 (2019)
M. Meuwly, WIREs Computational Molecular Science9, e1386 (2019)
2019
-
[3]
S.ManzhosandT.J.Carrington,Chem.Rev.121,10187 (2021)
2021
-
[5]
O. T. Unke, S. Chmiela, H. E. Sauceda, M. Gastegger, I. Poltavsky, K. T. Schütt, A. Tkatchenko, and K.-R. Müller, Chem. Rev.121, 10142 (2021)
2021
-
[6]
D. Lu, H. Wang, M. Chen, L. Lin, R. Car, E. Weinan, W. Jia, and L. Zhang, Comput. Phys. Commun.259, 107624 (2021)
2021
-
[7]
Zhang, J
L. Zhang, J. Han, H. Wang, R. Car, and W. E, Phys. Rev. Lett.120, 143001 (2018)
2018
-
[8]
O. T. Unke, M. Stöhr, S. Ganscha, T. Unterthiner, H. Maennel, S. Kashubin, D. Ahlin, M. Gastegger, L. Medrano Sandonas, J. T. Berryman,et al., Sci. Adv. 10, eadn4397 (2024)
2024
-
[9]
V. Zaverkin, M. Ferraz, F. Alesiani, and M. Niepert, arXiv preprint arXiv:2508.10841 (2025)
-
[10]
Musaelian, S
A. Musaelian, S. Batzner, A. Johansson, L. Sun, C. J. Owen, M. Kornbluth, and B. Kozinsky, Nat. Comm.14, 579 (2023)
2023
-
[11]
Kozinsky, A
B. Kozinsky, A. Musaelian, A. Johansson, and S. Batzner, inProceedings of the International Con- ference for High Performance Computing, Networking, Storage and Analysis, SC ’23 (Association for Comput- ing Machinery, New York, NY, USA, 2023)
2023
-
[12]
Kadau, T
K. Kadau, T. C. Germann, and P. S. Lomdahl, Int. J. Mod. Phys. C17, 1755 (2006)
2006
-
[13]
T. C. Germann and K. Kadau, Int. J. Mod. Phys. C19, 1315 (2008)
2008
-
[14]
Shibuta, S
Y. Shibuta, S. Sakane, E. Miyoshi, S. Okita, T. Takaki, and M. Ohno, Nat. Comm.8, 10 (2017)
2017
-
[15]
Casalino, C
L. Casalino, C. Seitz, J. Lederhofer, Y. Tsybovsky, I. A. Wilson, M. Kanekiyo, and R. E. Amaro, ACS Cent. Sci. 8, 1646 (2022)
2022
-
[16]
Santos, S
K. Santos, S. Moore, T. Oppelstrup, A. Sharifian, I. Sharapov, A. Thompson, D. Z. Kalchev, D. Perez, R. Schreiber, S. Pakin,et al., inSC24: International Conference for High Performance Computing, Network- ing, Storage and Analysis(IEEE, 2024) pp. 1–13
2024
-
[17]
Ugarte La Torre, J
D. Ugarte La Torre, J. Jung, and Y. Sugita, inPro- ceedings of the International Conference on High Per- formance Computing in Asia-Pacific Region(2025) pp. 1–12
2025
-
[18]
Lek Chaton and M
K. Lek Chaton and M. Meuwly, J. Comput. Chem.46, e70162 (2025)
2025
-
[19]
S.-L. J. Lahey and C. N. Rowley, Chem. Sci.11, 2362 (2020)
2020
-
[20]
Gastegger, K
M. Gastegger, K. T. Schütt, and K.-R. Müller, Chem. Sci.12, 11473 (2021)
2021
-
[21]
T. J. Inizan, T. Plé, O. Adjoua, P. Ren, H. Gökcan, O. Isayev, L. Lagardère, and J.-P. Piquemal, Chem. Sci. 14, 5438 (2023)
2023
-
[22]
K. Song, S. Käser, K. Töpfer, L. I. Vazquez-Salazar, and M. Meuwly, J. Chem. Phys.159(2023)
2023
-
[23]
Zinovjev, J
K. Zinovjev, J. Chem. Theory Comput.19, 1888 (2023)
2023
-
[24]
Zinovjev, L
K. Zinovjev, L. Hedges, R. Montagud Andreu, C. Woods, I. Tuñón, and M. W. van der Kamp, J. Chem. Theory Comput.20, 4514 (2024)
2024
-
[25]
Kalayan, I
J. Kalayan, I. Ramzan, C. D. Williams, R. A. Bryce, and N. A. Burton, J. Comput. Chem.45, 1143 (2024)
2024
-
[26]
Poltavsky, A
I. Poltavsky, A. Charkin-Gorbulin, M. Puleva, G. Fon- seca, I. Batatia, N. J. Browning, S. Chmiela, M. Cui, J. T. Frank, S. Heinen, B. Huang, S. Käser, A. Kabylda, D. Khan, C. Müller, A. J. A. Price, K. Riedmiller, K. Töpfer, T. W. Ko, M. Meuwly, M. Rupp, G. Csányi, O. A. von Lilienfeld, J. T. Margraf, K.-R. Müller, and A. Tkatchenko, Chem. Sci.16, 3720 (2025)
2025
-
[27]
Poltavsky, M
I. Poltavsky, M. Puleva, A. Charkin-Gorbulin, G. Fon- seca, I. Batatia, N. J. Browning, S. Chmiela, M. Cui, J. T. Frank, S. Heinen, B. Huang, S. Käser, A. Kabylda, D. Khan, C. Müller, A. J. A. Price, K. Riedmiller, K. Töpfer, T. W. Ko, M. Meuwly, M. Rupp, G. Csányi, O. A. von Lilienfeld, J. T. Margraf, K.-R. Müller, and A. Tkatchenko, Chem. Sci.16, 3738 (2025)
2025
-
[28]
J. Xia, Y. Zhang, and B. Jiang, Chem. Soc. Rev. 10.1039/D5CS00104H (2025), dOI: 10.1039/D5CS00104H
-
[29]
J. M. Bowman, C. Qu, R. Conte, A. Nandi, P. L. Hous- ton, and Q. Yu, J. Chem. Phys.162, 180901 (2025)
2025
-
[30]
Käser, L
S. Käser, L. I. Vazquez-Salazar, M. Meuwly, and K. Töpfer, Digit. Discov.2, 28 (2023)
2023
-
[31]
Hwang, S
W. Hwang, S. L. Austin, A. Blondel, E. D. Boittier, S. Boresch, M. Buck, J. Buckner, A. Caflisch, H.-T. Chang, X. Cheng, Y. K. Choi, J.-W. Chu, M. F. Crow- ley, Q. Cui, A. Damjanovic, Y. Deng, M. Devereux, X. Ding, M. F. Feig, J. Gao, D. R. Glowacki, J. E. I. Gonzales, M. B. Hamaneh, E. D. Harder, R. L. Hayes, J. Huang, Y. Huang, P. S. Hudson, W. Im, S. M...
2024
-
[32]
M. E. Taylor and P. Stone, J. Mach. Learn. Res.10, 1633 (2009)
2009
-
[33]
S. J. Pan and Q. Yang, IEEE Trans. Knowl. Data Eng. 22, 1345 (2009)
2009
-
[34]
J. S. Smith, B. T. Nebgen, R. Zubatyuk, N. Lubbers, C. Devereux, K. Barros, S. Tretiak, O. Isayev, and A. E. Roitberg, Nat. Commun.10, 1 (2019)
2019
-
[35]
Käser and M
S. Käser and M. Meuwly, Phys. Chem. Chem. Phys.24, 5269 (2022)
2022
-
[36]
Käser, J
S. Käser, J. O. Richardson, and M. Meuwly, J. Chem. Theory Comput.18, 6840 (2022)
2022
-
[37]
Werner, P
H.-J. Werner, P. J. Knowles, G. Knizia, F. R. Manby, M. Schütz, P. Celani, W. Györffy, D. Kats, T. Korona, R. Lindh, A. Mitrushenkov, G. Rauhut, K. R. Shamasun- dar, T.B.Adler, R.D.Amos, S.J.Bennie, A.Bernhards- son, A. Berning, D. L. Cooper, M. J. O. Deegan, A. J. Dobbyn, F. Eckert, E. Goll, C. Hampel, A. Hesselmann, G. Hetzer, T. Hrenar, G. Jansen, C. K...
2019
- [38]
-
[39]
Andreichev, S
V. Andreichev, S. Käser, E. L. Bocanegra, M. Salik, M. A. Johnson, and M. Meuwly, Phys. Chem. Chem. Phys. (2025)
2025
-
[40]
B. J. Braams and J. M. Bowman, Int. Rev. Phys. Chem. 28, 577 (2009)
2009
-
[41]
Xie and J
Z. Xie and J. M. Bowman, J. Chem. Theory Comput.6, 26 (2010)
2010
-
[42]
Msa software with gradients,https://github.com/ szquchen/MSA-2.0(2019), accessed: 2019-01-20
2019
-
[43]
P. L. Houston, C. Qu, Q. Yu, R. Conte, A. Nandi, J. K. Li, and J. M. Bowman, J. Chem. Phys.158, 044109 (2023)
2023
-
[44]
Ramakrishnan, P
R. Ramakrishnan, P. Dral, M. Rupp, and O. A. von Lilienfeld, J. Chem. Theory Comput.11, 2087 (2015)
2087
-
[45]
Nandi, C
A. Nandi, C. Qu, P. L. Houston, R. Conte, and J. M. Bowman, J. Chem. Phys.154, 051102 (2021)
2021
-
[46]
Nandi, G
A. Nandi, G. Laude, S. S. Khire, N. D. Gurav, C. Qu, R. Conte, Q. Yu, S. Li, P. L. Houston, S. R. Gadre, J. O. Richardson, F. A. Evangelista, and J. M. Bowman, J. Amer. Chem. Soc.145, 9655 (2023)
2023
-
[47]
J. M. Bowman, C. Qu, R. Conte, A. Nandi, P. L. Hous- ton, and Q. Yu, J. Chem. Theory and Comput.19, 1 (2023)
2023
-
[48]
Liu and J
Y. Liu and J. Li, J. Phys. Chem. Letts.13, 4729 (2022)
2022
-
[49]
P. L. Houston, C. Qu, Q. Yu, P. Pandey, R. Conte, A. Nandi, J. M. Bowman, and S. G. Kukolich, J. Chem. Theory Comput.20, 1821 (2024)
2024
-
[50]
Huang, B
X. Huang, B. J. Braams, and J. M. Bowman, J. Chem. Phys.122, 044308 (2005)
2005
-
[52]
Griewank and A
A. Griewank and A. Walther,Evaluating Derivatives: Principles and Techniques of Algorithmic Differentia- tion, 2nd ed. (Society for Industrial and Applied Mathe- matics (SIAM), Philadelphia, PA, 2008)
2008
-
[53]
Adam: A Method for Stochastic Optimization
D. Kingma and J. Ba, arXiv preprint arXiv:1412.6980 (2014)
work page internal anchor Pith review Pith/arXiv arXiv 2014
-
[54]
S. J. Reddi, S. Kale, and S. Kumar, On the convergence of adam and beyond (2019), arXiv:1904.09237 [cs.LG]
work page Pith review arXiv 2019
-
[55]
O. T. Unke and M. Meuwly, J. Chem. Theory Comput. 15, 3678 (2019)
2019
-
[56]
Boittier, K
E. Boittier, K. Töpfer, M. Devereux, and M. Meuwly, Journal of Chemical Theory and Computation20, 8088 (2024)
2024
- [57]
-
[58]
Käser, D
S. Käser, D. Koner, A. S. Christensen, O. A. von Lilien- feld, and M. Meuwly, J. Phys. Chem. A124, 8853 (2020)
2020
-
[59]
164, 061103 (2026)
C.Qu, P.L.Houston,andJ.M.Bowman,J.Chem.Phys. 164, 061103 (2026)
2026
- [60]
-
[61]
Nandi, G
A. Nandi, G. Laude, S. S. Khire, N. D. Gurav, C. Qu, R. Conte, Q. Yu, S. Li, P. L. Houston, S. R. Gadre,et al., J. Am. Chem. Soc.145, 9655 (2023)
2023
-
[62]
C. Qu, A. Nandi, P. L. Houston, and J. M. Bowman, J. Phys. Chem. Axx, xx (2026)
2026
-
[63]
J. M. Bowman, S. Carter, and X. Huang, Int. Rev. Phys. Chem.22, 533 (2003)
2003
-
[64]
Yu and et al.,MULTIMODE, the n-mode Represen- tation of the Potential and Illustrations to IR Spectra of Glycine and Two Protonated Water Clusters., edited by J
Q. Yu and et al.,MULTIMODE, the n-mode Represen- tation of the Potential and Illustrations to IR Spectra of Glycine and Two Protonated Water Clusters., edited by J. M. Bowman (Vibrational Dynamics of Molecules, World Scientific, 2022)
2022
-
[65]
C. Qu, T. C. Allison, P. L. Houston, R. Conte, A. Nandi, and J. M. Bowman, J. Chem. Phys.xxx, xxxx (2026), in revision
2026
-
[66]
J. B. Anderson, J. Chem. Phys.63, 1499 (1975)
1975
-
[67]
Kosztin, B
I. Kosztin, B. Faber, and K. Schulten, Am. J. Phys.64, 633 (1996)
1996
-
[68]
J. B. Anderson, J. Chem. Phys.65, 4121 (1976)
1976
-
[69]
Y. Wang, B. J. Braams, J. M. Bowman, S. Carter, and D. P. Tew, J. Chem. Phys.128, 224314 (2008)
2008
-
[70]
J. O. Richardson and S. C. Althorpe, J. Chem. Phys. 134, 054109 (2011)
2011
-
[71]
J. O. Richardson, Int. Rev. Phys. Chem.37, 171 (2018)
2018
-
[72]
J. O. Richardson, Intern. Rev. Phys. Chem.37, 171 (2018)
2018
-
[73]
J. E. Lawrence, J. Dušek, and J. O. Richardson, J. Chem. Phys.159, 014111 (2023)
2023
-
[74]
Kamarchik, Y
E. Kamarchik, Y. Wang, and J. Bowman, J. Phys. Chem. A113, 7556 (2009)
2009
-
[75]
D. T. Colbert and W. H. Miller, J. Chem. Phys.96, 1982 (1992)
1982
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.