{"id":"f35ef1db-ccc5-49d0-a7c1-4aa1b780fc14","arxiv_id":"2505.12447","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"HORM provides 1.84 million DFT Hessians for reactive organic molecules, and Hessian-informed training reduces Hessian errors and improves transition state search in benchmarked MLIPs.","lead":"This paper introduces HORM, a dataset of 1.84 million molecular Hessian matrices computed with density functional theory for reactive geometries from two existing reaction datasets. It reports that training machine learning interatomic potentials with Hessian information improves Hessian accuracy and transition state search success, especially for direct-force models.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 200x TS-search gain rests on a near-zero baseline (3 vs 684 intended TSs) and an unpublished, overlapping-author workflow [29]; without independent reproduction the headline ratio is not robust.","rationale":"The reader's weakest assumption identifies the unpublished TS workflow and the near-zero baseline as the most load-bearing concern, and I agree. The Hessian-supervision improvements in energy, force, and Hessian prediction are well supported by the tables, so the core dataset contribution is not in doubt. However, the headline '200 times' claim is the paper's most striking result and it depends entirely on an external, overlapping-author workflow and a baseline of 3 intended TSs. A concrete independent reproduction would settle whether the ratio is a real effect or a pipeline artifact. Since the paper already receives a conditional verdict from the reader, my stress-test does not change that verdict; it reinforces the condition that the TS-search workflow must be made public and independently verified. I also note that the dataset and code are not released, which is an additional barrier to verification, but the primary concern remains the TS-search evaluation.","tokens_in":12014,"tokens_out":4318,"duration_ms":47094,"concrete_test":"Obtain the trained EquiformerV2 E-F and E-F-H checkpoints (or retrain them using the paper's hyperparameters in Table A.2) and rerun the TS search on the same validation reactions with an independent, openly documented workflow—e.g., PyGSM for initial guesses followed by a standard saddle-point optimizer and a separate IRC verification, rather than reference [29]. Report absolute intended-TS counts and 95% bootstrap confidence intervals for both models. If the E-F baseline remains near 3 and E-F-H remains above 600, the 200x claim is robust to the workflow; if the E-F-H count drops substantially or the E-F count rises, the reported ratio is an artifact of the unpublished pipeline.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that Hessian-informed training improves TS search is anchored in Section 5.2, where EquiformerV2 rises from 3 intended TSs under E-F training to 684 under E-F-H training, described as a 200x improvement. This ratio is fragile for two reasons. First, the E-F baseline is 3 successes; with such a small denominator, a few borderline cases or a small change in convergence thresholds can move the ratio by an order of magnitude. Second, the evaluation uses the authors' 'recently developed end-to-end TS search workflow' (reference [29], a ChemRxiv preprint by overlapping authors), which is not described in this paper. The 'intended TS' definition in Section 2.3 depends on IRC verification matching reactants and products, but the actual implementation, thresholds, initial guess generation, and failure handling are external to this manuscript. The paper also does not state the total number of test reactions, so the reader cannot judge whether 684 represents a high absolute success rate. The Hessian-prediction improvements in Tables 1 and 2 are internally consistent and support the dataset's value, but the headline TS-search claim is the least secure part of the paper. Additionally, the abstract's 'up to 63% reduction in Hessian MAE' is inconsistent with the 97% and 93% reductions reported in Tables 1 and 2, which further reduces confidence in the reported numbers.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript introduces HORM, a dataset of 1,836,206 Hessian matrices computed at the ωB97X/6-31G(d) level for reactive configurations sampled from Transition1x (1,776,206 geometries) and RGD1 (60,000 geometries). The authors propose an energy-force-Hessian (E-F-H) training loss with stochastic row sampling and compare it with standard energy-force (E-F) training for AlphaNet, LEFTNet (autograd and direct-force variants), and EquiformerV2. They report consistent reductions in Hessian and eigenvalue MAE on the in-distribution Transition1x validation set and the out-of-distribution RGD1 set, reduced Hessian asymmetry for direct-force models, and large gains in an end-to-end transition state search benchmark, most prominently an increase from 3 to 684 intended TSs for EquiformerV2. The paper claims up to 200 times higher TS search success rates and up to 63% reduction in Hessian MAE.","tokens_in":12317,"tokens_out":4426,"duration_ms":44837,"significance":"The dataset fills a real gap: off-equilibrium Hessians of reactive systems are scarce, and HORM is an order-of-magnitude larger resource than Hessian-QM9, with diverse non-equilibrium geometries. The central methodological result—that adding Hessian supervision with only one or two sampled rows per Hessian improves second-order accuracy, Hessian symmetry, and TS-related performance across four model variants and on out-of-distribution data—is plausible and, if confirmed, practically valuable. The paper deserves credit for benchmarking multiple architectures, including the OOD split, and for reporting Hessian asymmetry errors. However, the headline TS-search claim is not yet robustly supported because it depends on a near-zero baseline and an external, unpublished workflow.","major_comments":[{"comment":"The headline 'up to 200 times' improvement rests on an E-F baseline of only 3 intended TSs for EquiformerV2. With such a small denominator, the ratio is highly sensitive to convergence thresholds, a few borderline cases, or stochasticity. The paper also does not report the total number of test reactions, so the reader cannot judge whether 684 represents a high absolute success rate. In addition, the evaluation relies entirely on the authors' end-to-end TS search workflow in reference [29], an unpublished ChemRxiv preprint by overlapping authors; the workflow's initial-guess generation, GSM settings, convergence criteria, and IRC verification thresholds are not described in this manuscript. Please provide the test-set size, describe the workflow (or validate it independently), and report success counts with a stable, non-degenerate baseline before using the 200x ratio as a headline claim.","section":"Section 5.2, Figs. 2a-b"},{"comment":"The abstract states 'up to 63% reduction in the Hessian mean absolute error,' but Tables 1 and 2 report reductions of 97% (EquiformerV2, HORM-Transition1x validation) and 93% (EquiformerV2, HORM-RGD1) in Hessian MAE relative to the E-F baseline, and Section 5.1 explicitly states those numbers. Please reconcile the abstract with the tables; if 63% refers to a different metric, model subset, or aggregation, say so explicitly.","section":"Abstract vs. Tables 1 and 2"},{"comment":"The loss weights α, β, and γ are never specified. Table A.2 reports NHR, learning rate, and batch size but omits the weights that control the trade-off among energy, force, and Hessian terms. Because the main conclusions depend on these weights, omitting them hinders reproduction and sensitivity analysis. Relatedly, the claim that sampling one row (autograd models) or two rows (direct-force models) per Hessian per epoch is sufficient is an assumption; no ablation over NHR or γ is provided. Please report the weights and add a small sensitivity study, even for a single representative model.","section":"Equations (2)-(3) and Table A.2"},{"comment":"The claim in Section 5.1 that Hessian supervision improves performance across 'nearly all evaluation metrics' is not fully supported by the tables: LEFTNet-df force MAE on the Transition1x validation set worsens from 0.029 to 0.044 eV/Å under E-F-H training (Table 1), and the corresponding OOD force MAE shows only a marginal change. This is a reporting inconsistency worth addressing, since the text presents force behavior as essentially neutral or improved.","section":"Tables 1 and A.1"}],"minor_comments":[{"comment":"The word 'parathensis' appears in the caption text; it should be 'parentheses'.","section":"Table captions, Tables 1-2"},{"comment":"The notation H^(j)_i and F^(j)_φ is introduced in the text but could be defined immediately after the equation for clarity, particularly the meaning of the superscript j as the selected Hessian row or force entry.","section":"Section 4, Eq. (3)"},{"comment":"No data or code availability statement is provided. Since HORM is the central contribution, a repository URL or a clear statement about how to obtain the dataset and training/evaluation code should be added.","section":"Data availability"},{"comment":"The TS search evaluation reports no uncertainty or multiple-seed statistics for the intended-TS counts; given the 3->684 ratio is a headline result, at least a repeated-run or bootstrap-style measure would help.","section":"Section 5.2"}],"recommendation":"major_revision","confidential_remarks":"The dependence on unpublished, overlapping-author reference [29] for the central TS evaluation is the main risk. If the workflow cannot be described or independently validated within the paper, the 200x claim should be de-emphasized and the TS results treated as preliminary. The missing loss weight values and the abstract/table inconsistency are additional reproducibility concerns that should be fixed before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Good dataset paper, weaker headline. HORM—1.84M Hessians on off-equilibrium reactive geometries—is a genuinely useful resource, roughly 40x larger than Hessian-QM9 and covering geometries that matter for TS work. The benchmark evidence that adding Hessian supervision improves Hessian and eigenvalue MAE consistently across four models and both ID/OOD splits is solid, and the effect is largest for direct-force models like EquiformerV2, where Hessian MAE drops by ~97%. That is the paper's real result, and it is convincing enough on its face.\n\nThe stochastic row-sampling training scheme is borrowed from Amin et al., and the authors say so; the adaptation to their loss is straightforward and sensible. The Hessian asymmetry analysis is a nice extra: it quantifies how Hessian supervision fixes non-conservative force predictions. The limitations section is honest about small molecules, C/H/O/N coverage, and lack of long-timescale MD testing.\n\nThe soft spots are real but mostly addressable. First, the abstract claims 'up to 63% reduction in Hessian MAE' while Tables 1 and 2 show 97% and 93% reductions for EquiformerV2. That inconsistency should be fixed before publication. Second, no dataset or code release is mentioned, which matters for a resource paper; I hope that is just a preprint artifact. Third, there are no uncertainty estimates or multiple seeds, so we don't know if the improvements are within noise—though the consistency across architectures makes me think they are not.\n\nThe bigger concern is the TS search evaluation. The '200x' claim rests on EquiformerV2 going from 3 to 684 intended TSs, which is a near-zero baseline: three successes is essentially failure, and a small change in thresholds could move the ratio by an order of magnitude. The evaluation also depends on an unpublished end-to-end TS workflow (ref [29], overlapping authors) that is not described here, and the paper never states the total number of test reactions, so we can't judge the absolute success rate. The Hessian-prediction results stand on their own; the TS-search headline is the weakest link.\n\nWho is this for? Anyone building reactive MLIPs or needing off-equilibrium Hessian data. It deserves a serious peer review, but the editor should insist on dataset release, a fix for the abstract/table mismatch, and either an appendix describing the TS workflow or a clear statement of the test set size and success criteria.","headline":"A useful reactive Hessian dataset and consistent evidence that Hessian supervision helps, but the 200x TS-search headline is fragile.","tokens_in":12854,"tokens_out":2686,"would_cite":true,"duration_ms":25170,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that 1.84 million Hessians, plus a row-sampled Hessian loss, are what reactive machine-learning potentials need to find transition states reliably.","keywords":["Hessian database","machine learning interatomic potentials","transition state search","second-order derivatives","reactive molecular dynamics","stochastic row sampling","omegaB97x/6-31G(d)","direct-force models"],"falsifier":"Run the same trained E-F and E-F-H models through an independent, transparent transition-state search pipeline, such as NEB or GSM followed by eigenvector-following refinement with IRC verification, and compare intended-TS counts; if E-F-H does not clearly outperform E-F, or if E-F already recovers hundreds of intended TSs, the reported 3-to-684 result will not reproduce.","tokens_in":1621,"feed_emoji":"🧪","tokens_out":3518,"duration_ms":76231,"temperature":0.7,"pith_summary":"This paper introduces HORM, a dataset of 1.84 million Hessian matrices for reactive molecular geometries at the $\\omega$B97X/6-31G(d) level, and argues that training machine-learning interatomic potentials with second-derivative supervision makes them substantially better at finding transition states. The authors show that adding a Hessian loss term, computed cheaply by randomly sampling a few rows of the Hessian, improves Hessian prediction, restores near-symmetry to force-derived Hessians, and lifts the number of correctly recovered transition states, most dramatically for direct-force models, with EquiformerV2 going from 3 to 684 intended transition states. A sympathetic reader would care because transition-state searches are the bottleneck in automated reaction exploration, and this is a practical recipe for making reactive MLIPs usable for that task.","feed_headline":"1.84M Hessians make ML potentials find transition states","feed_subtitle":"Adding second-derivative training cuts Hessian error by up to 97% and lifts TS search success from 3 to 684 cases.","key_machinery":"HORM itself is the load-bearing object: 1,836,206 Hessian matrices recomputed at $\\omega$B97X/6-31G(d) from Transition1x and RGD1 geometries, spanning non-equilibrium regions of the potential energy surface. The accompanying training method augments the standard energy-force loss with a Hessian-matching term in which a small random subset of Hessian rows (one for autograd models, two for direct-force models) is compared against the derivative of the model's predicted forces, computed by batched vector-Jacobian products so the cost is $O(s)$ rather than $O(N^2)$. This targeted supervision is what lets the models learn curvature and recover symmetric Hessians without paying the full computational price.","core_discovery":"The paper's central claim is that explicit Hessian supervision during MLIP training, enabled by a dataset large enough to provide off-equilibrium second derivatives, is what makes reactive machine-learning potentials reliable for transition-state optimization. On the HORM-Transition1x validation set, adding the Hessian loss cuts Hessian mean absolute error by up to 97% (EquiformerV2: 2.231 to 0.075 eV/Å$^2$) and eigenvalue error by up to 99%, while in the end-to-end TS search benchmark the same model's number of intended transition states rises from 3 under energy-force training to 684 under energy-force-Hessian training. The authors interpret this as evidence that second-order information enforces force consistency and correct local curvature, overcoming a known limitation of direct-force architectures.","pith_inferences":["The 3-to-684 jump suggests the energy-force-trained EquiformerV2 was essentially failing at the TS search task, so the '200 times' ratio may overstate the practical gain over a reasonable baseline; a fairer comparison would match training budgets and report success rates per reaction.","Because HORM covers mostly C/H/O/N systems with under eight heavy atoms, the observed gains are a proof-of-concept for small-molecule reactivity; extending the same recipe to P/S/halogens or larger fragments is the natural next test and the authors say they are working toward it.","Stochastic row sampling with just one or two Hessian rows per structure implies that full-Hessian supervision is unnecessary; an adaptive scheme that samples rows near the largest predicted curvature could make the method even cheaper while retaining the symmetry-enforcing benefit.","The Hessian dataset could also serve as a benchmark for evaluating whether a model learns physically consistent curvature, independent of its downstream TS search performance."],"forward_implications":["Direct-force MLIPs, which are fast but normally produce non-conservative forces and asymmetric Hessians, become viable for transition-state search once trained with Hessian supervision.","Training on HORM improves out-of-distribution generalization: on the RGD1 subset, EquiformerV2 energy MAE drops by 45% and Hessian MAE by 93% relative to energy-force-only training.","Autograd-based models also gain, with in-distribution Hessian and eigenvalue MAEs reduced by up to 59% and 78%, even though their force errors barely change.","Barrier-height prediction improves by up to 10%, while TS geometry RMSD changes little, indicating that the Hessian supervision primarily corrects local curvature rather than the overall geometry.","The dataset and row-sampling loss provide a template for training future reactive MLIPs that need second-order information without paying the full $O(N^2)$ Hessian cost."],"supporting_citations":[{"why":"Supplies the Transition1x geometries and reaction split that form the bulk of HORM.","marker":"[36]"},{"why":"Supplies the RGD1 reaction pathways sampled along IRC trajectories for the complementary HORM subset.","marker":"[37]"},{"why":"The GPU quantum-chemistry engine used to recompute energies, forces, and Hessians at the stated level for every HORM geometry.","marker":"[43]"},{"why":"Provides the stochastic row-sampling and vector-Jacobian product technique the Hessian loss is built on.","marker":"[42]"},{"why":"Defines the end-to-end transition-state search workflow and intended-TS metric used to evaluate the trained MLIPs.","marker":"[29]"},{"why":"The Hessian-QM9 equilibrium database used as the scale and coverage baseline, with 41,645 equilibrium Hessians.","marker":"[38]"},{"why":"Documents the non-conservative-force and asymmetric-Hessian problem in direct-force models that motivates the Hessian supervision.","marker":"[30]"},{"why":"EquiformerV2 architecture used as the direct-force model that shows the largest gains from Hessian training.","marker":"[22]"}],"fun_headline_variants":["1.84M Hessians make MLIPs find 200x more transition states","HORM's 1.84M Hessians cut Hessian error 97%","Hessian training: 684 transition states vs 3 without","HORM: largest Hessian DB for reactive MLIPs","97% lower error, 200x more TS with Hessian loss"],"cache_read_input_tokens":14976,"weakest_assumption_plain":"The headline TS-search gain rests on the separate end-to-end TS search workflow cited as [29] and its IRC-based definition of an intended TS; if that workflow's success criteria are biased, or if the energy-force baseline is essentially a failed model, the 200x comparison is not a fair measure.","fun_headline_variants_meta":{"raw":{"variants":["1.84M Hessians make MLIPs find 200x more transition states","HORM's 1.84M Hessians cut Hessian error 97%","Hessian training: 684 transition states vs 3 without","HORM: largest Hessian DB for reactive MLIPs","97% lower error, 200x more TS with Hessian loss"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000808,"raw_usage":{"total_tokens":3560,"prompt_tokens":972,"completion_tokens":2588,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":588,"completion_tokens_details":{"reasoning_tokens":2489}},"tokens_in":588,"tokens_out":2588,"duration_ms":16984,"temperature":1.0,"reasoning_tokens":2489,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:34:00.382647+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same trained E-F and E-F-H models through an independent, transparent transition-state search pipeline, such as NEB or GSM followed by eigenvector-following refinement with IRC verification, and compare intended-TS counts; if E-F-H does not clearly outperform E-F, or if E-F already recovers hundreds of intended TSs, the reported 3-to-684 result will not reproduce.","supporting_citations":[{"cited_title":"Transition1x-a dataset for building generalizable reactive machine learning potentials.Scientific Data, 9(1):779, 2022","cited_arxiv_id":null,"evidence_quote":"Supplies the Transition1x geometries and reaction split that form the bulk of HORM."},{"cited_title":"Comprehensive exploration of graphically defined reaction spaces","cited_arxiv_id":null,"evidence_quote":"Supplies the RGD1 reaction pathways sampled along IRC trajectories for the complementary HORM subset."},{"cited_title":"Harnessing machine learning to enhance transition state search with interatomic potentials and generative models","cited_arxiv_id":null,"evidence_quote":"Defines the end-to-end transition-state search workflow and intended-TS metric used to evaluate the trained MLIPs."},{"cited_title":"Hessian qm9: A quantum chemistry database of molecular hessians in implicit solvents","cited_arxiv_id":null,"evidence_quote":"The Hessian-QM9 equilibrium database used as the scale and coverage baseline, with 41,645 equilibrium Hessians."}],"review_version":1}