{"id":"326d13fa-5367-4714-b35f-db8a9811d838","arxiv_id":"2411.19629","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"OpenQDC packages 37 public QM datasets, over 250 QM methods, and 393 million geometries into a unified Python library with normalization tools and initial architecture benchmarks.","lead":"This paper introduces OpenQDC, a Python library that consolidates 37 quantum-mechanical datasets with over 400 million molecular geometries for training machine learning interatomic potentials. It is a bid to give MLIP researchers a standardized, one-line-access data resource similar to what ImageNet did for computer vision.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Computed isolated-atom energies are unvalidated and load-bearing: any error corrupts the atomization-energy normalization that is a core promise of OpenQDC; the paper gives no method or checks to verify them.","rationale":"The reader's weakest assumption is the same one I would flag: computed isolated-atom energies. It is more load-bearing than the split bug because the paper's central contribution is a trustworthy, standardized data resource, not the preliminary benchmark. The split bug is admitted and affects only Table 2, whereas if reference energies are wrong, every user who applies the library's atomization normalization trains on incorrectly shifted labels. There is no internal evidence to refute or confirm this: the paper never describes the calculation, gives no validation numbers, and does not provide a data manifest. This is an omission of support, not a disagreement with external consensus. The resource may still be correct; the concern is testable. A conditional acceptance requiring code release with a commit hash and validation of these references is appropriate, consistent with the reader's verdict. I do not see grounds to reject: the datasets are sourced from existing works, the library architecture is described, and the claimed scale is plausible. The paper needs a verification pass before the correctness risk is retired.","tokens_in":18844,"tokens_out":3649,"duration_ms":32495,"concrete_test":"Check the openQDC repository at the stated URL with a pinned commit. Independently recompute isolated-atom energies for a representative subset of levels of theory (e.g., GFN2-xTB, ωB97X/6-31G*, ωB97X-D/def2-SVP, B3LYP/6-31G*, PBE0+MBD) with specified spin multiplicities and convergence criteria, using the same QM packages as the original datasets where possible. Compare to the values shipped in openQDC for every element that appears in those datasets. Then use the shipped references to compute atomization energies for 50–100 molecules from QM7/QM9 and compare to the published atomization energies in those datasets. If any reference differs from the independent recomputation by more than 1 kcal/mol per atom, or if QM7/QM9 atomization energies are systematically off, the normalization pipeline is corrupted and the harmonization claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that OpenQDC standardizes QM data for MLIP training, and Section 4 states that for datasets lacking metadata, the authors 'computed these values for all QM methods in the repository' (isolated atom energies). This is a load-bearing preprocessing step: these reference energies underlie the physical, size-extensive atomization-energy normalization, and any error propagates into every energy label and benchmark built on that normalization. The manuscript does not state how these energies were computed: no QM package, no basis set or spin-state settings, no convergence criteria, no treatment of charged or open-shell atoms, no list of which methods were computed versus taken from originals, and no comparison against known atomization energies (e.g., QM7/QM9) or against original dataset metadata. Given that some source datasets do contain atomization energies or reference metadata, one would expect a validation table. Without it, a silent sign or magnitude error in these references cannot be detected by users, undermining the 'meticulously preprocessed, standardized' claim. The admitted split bug in Table 2 is a separate, self-reported flaw in the benchmark contribution; the dataset resource itself should be evaluated on the correctness of the preprocessing, of which the isolated-atom energies are the least documented part.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces OpenQDC, a Python library and dataset repository that consolidates 37 existing quantum-mechanical datasets, spanning over 250 QM methods and roughly 400 million geometries, into a standardized, programmatically accessible format for machine-learned interatomic potential (MLIP) training. The authors describe the data storage and loading infrastructure, unit handling, QM-method notation standardization, and energy normalization based on isolated-atom reference energies. They also report baseline MLIP experiments with SchNet, DimeNet, and TorchMD-Net on a subset of these datasets, presenting a leaderboard of mean absolute errors. The paper explicitly acknowledges a bug in the dataset splitting that contaminated some baseline results, and notes that corrected results are forthcoming.","tokens_in":19116,"tokens_out":2852,"duration_ms":26140,"significance":"If the repository and preprocessing are correct, OpenQDC would be a valuable community resource: it offers a single, large, multi-fidelity corpus of QM data with a clean Python interface, which could lower the barrier for MLIP research and support multi-task and transfer-learning studies. The paper is also honest about its limitations, including the known split bug. However, the scientific value of the benchmark results is currently weakened by that bug, and the central preprocessing step of computing isolated-atom energies for all QM methods is undocumented and unvalidated. The manuscript therefore has a plausible and potentially impactful core, but the evidence presented is not yet sufficient for its claims.","major_comments":[{"comment":"The manuscript states that most datasets lacked isolated-atom energies and that the authors 'computed these values for all QM methods in the repository.' These values underpin the atomization-energy normalization that the paper advertises as a physics-motivated, size-extensive approach, so an error in any one of them propagates into every energy label and derived benchmark. The paper does not report the computational protocol (software, basis set, spin-state treatment, charge handling, convergence criteria), does not list which methods were computed versus taken from the original sources, and provides no validation against known reference values (e.g., QM7/QM9 atomization energies or original dataset metadata). Please add this protocol and a validation table, or state explicitly which datasets rely on externally provided reference energies.","section":"Section 4 (The OpenQDC Library), paragraph on metadata and isolated atom energies"},{"comment":"The paper admits that 'a bug affected the dataset splits: some were random rather than molecule-based as intended,' yet Table 2 reports numerical MAE values and the surrounding text draws comparative conclusions, such as SchNet being 'consistently and significantly outperformed' by TorchMDNet and DimeNet and SchNet being more competitive on interaction datasets. These conclusions are not supported by the current, contaminated results. The authors should either remove the benchmark table and defer all architectural comparisons until the corrected splits are available, or clearly label Table 2 as preliminary and confine the discussion to the corrected experiments.","section":"Section 5.2 (Results) and Table 2"},{"comment":"The abstract and Table 1 present exact counts of geometries, energy labels, and force labels, but the paper provides no checksums, hash-based integrity verification, or scripts that reproduce these statistics from the downloaded source archives. Given that the central claim is the creation of a trustworthy, standardized data commons, adding a reproducibility appendix with dataset-manifest hashes and a count-validation script would materially strengthen the paper; currently these counts are unverifiable from the manuscript alone.","section":"Section 4 (Dataset Storage) and Section 3 (Table 1)"}],"minor_comments":[{"comment":"The text contains unresolved placeholder references 'Appendix ??' for dataset visualizations; these should be fixed or removed before publication.","section":"Section 3.1 and Section 3.2"},{"comment":"The table heading contains a typo, 'min-manximum', and the column header 'Atom Min/-Max' is unclear; please use 'Atom Min/Max' and correct the typo.","section":"Section 3, Table 1"},{"comment":"The word 'significently' should be 'significantly'.","section":"Section 5.2"},{"comment":"The phrase 'website (to be release upon acceptance)' is grammatically incorrect; it should read 'to be released upon acceptance.'","section":"Section 2 (Related works)"},{"comment":"The statement in Section 5.1 that 'For all datasets, we performed a molecule/system-based split' is directly contradicted by the split-bug disclosure in Section 5.2; please reconcile these statements in the revised manuscript.","section":"Section 5.1 and Section 5.2"},{"comment":"The sentence 'Accordingly, we will only compare different architectures and discuss differences based on chemical space only after correcting the splits' suggests the results in Table 2 should be explicitly labeled as provisional; currently the text draws preliminary conclusions despite this caveat.","section":"Section 5.2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's main risk is not circularity or overreach in the dataset collection claim, but rather the unvalidated isolated-atom energies and the contaminated benchmark splits. Both are fixable with additional documentation, validation experiments, and corrected results. I would not reject on these grounds, but the paper should not be accepted in its current form because the benchmark leaderboard is explicitly unreliable and the core normalization premise lacks verification. The authors' openness about the split bug is commendable and should be preserved in the revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"OpenQDC is a useful aggregation effort, and the paper is worth reading for that, not for the benchmarks. The authors have pulled together 37 QM datasets, roughly 400M geometries, into one Python package with unified unit handling, a QM-method ontology, and energy normalization utilities. That specific integration for bio-organic MLIP training is not in QCArchive or ColabFit, which are broader or more materials-focused. If the code works, that is a genuine resource for the community. Credit also for openly reporting the split bug in the benchmarks; that suggests honesty.\n\nThe soft spots are real. The isolated atom energies computed for methods that lacked them are a load-bearing preprocessing step, and the paper gives no details: no software, basis, spin treatment, or validation against known values. An error there would silently propagate into every atomization-energy normalized label and the benchmarks built on it. The stress-test note is fair. Second, the benchmark section is currently unusable for comparing architectures because some splits were random rather than molecule-based; the authors say they will retrain, but until then Table 2 is a placeholder. No error bars either. Third, a minor inconsistency: Table 1 lists ANI-1 at 22M geometries while the text says 17.2M, and PubChemQC PM6 at 189M vs 221M in text. Probably a different accounting, but it should be resolved.\n\nThe resource idea holds up; the issues are fixable. I would send this to a serious referee for a datasets-and-benchmarks venue, with major revisions required: a tagged code release, corrected molecule-based splits, multi-seed results, and documentation and validation of the isolated atom energies. After that, it becomes a citable contribution. For now I'd not rely on the normalization or the benchmark numbers.","headline":"A genuinely useful dataset aggregation effort whose benchmark section is currently invalid and whose isolated-atom energy normalization lacks the documentation and validation needed to trust it.","tokens_in":19629,"tokens_out":2879,"would_cite":false,"duration_ms":25261,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"OpenQDC collects 37 quantum-mechanical datasets, standardizes them into one Python-accessible format for training machine-learning interatomic potentials, and reports baseline results showing that established architectures often miss…","keywords":["quantum chemistry datasets","machine learning interatomic potentials","dataset standardization","energy normalization","multi-fidelity quantum methods","memory-mapped storage","molecular dynamics"],"falsifier":"Recompute isolated-atom energies for a sample of the QM methods in the repository using an independent electronic-structure code and compare them against the stored values; a disagreement beyond numerical tolerance, say greater than 1 meV per atom, would propagate directly into the energy labels users train on. Checking a dataset's distance-unit metadata against its original publication would similarly confirm whether the conversions that feed radial cutoffs are trustworthy.","tokens_in":1511,"feed_emoji":"🧪","tokens_out":2134,"duration_ms":46384,"temperature":0.7,"pith_summary":"The paper claims to consolidate 37 published quantum-mechanical datasets, covering over 400 million molecular geometries and more than 250 distinct QM method/basis-set combinations, into one programmatically accessible, ML-ready repository. It argues this removes a major barrier for machine-learning interatomic potential (MLIP) development, since QM data were previously scattered across repositories with inconsistent formats, units, and missing metadata. The work supports the repository with a Python library that standardizes loading, unit conversion, QM-method naming, and energy normalization, and with baseline experiments on SchNet, DimeNet, and TorchMD-Net. A careful reader should care because the claim is that a single library now gives the MLIP community the kind of standardized data access that has driven progress in other machine-learning fields.","feed_headline":"OpenQDC puts 400M quantum geometries behind one Python API","feed_subtitle":"37 QM datasets and 250+ methods standardized for machine-learning force fields, with energy normalization built in.","key_machinery":"The machinery is a memory-mapped storage layout plus a QM-method registry. Geometries, atomic charges, positions, per-geometry index ranges, energy labels, and force labels are stored in memory-mapped arrays that allow indexing and batching without loading data into RAM. QM methods are normalized into Python enums composed of a functional, a basis set, and a correction method, which standardizes naming across datasets and enables studying basis-set or correction effects. For energy normalization, the library pre-computes per-atom-type mean energies via linear or ridge regression and a per-atom residual scale, and it stores computed isolated-atom energies so users can convert potential energies to atomization energies while preserving extensivity.","core_discovery":"The central discovery is that the fragmented landscape of QM datasets can be unified at scale: 37 datasets, hundreds of millions of geometries, 70 atom types, and 250+ quantum methods are re-exposed through one interface with harmonized metadata. The paper further claims that normalization matters physically: converting potential energies to atomization energies by subtracting computed isolated-atom energies preserves size extensivity, while naive Z-scoring violates it. On the benchmark side, the paper reports that well-known architectures such as SchNet, DimeNet, and TorchMD-Net frequently fail to reach chemical accuracy of 1 kcal/mol on these datasets, indicating headroom for new architectures.","pith_inferences":["The paper's own admission that a bug affected dataset splits, making some random rather than molecule-based, implies the reported benchmark numbers should be treated as provisional until corrected re-runs appear.","If the computed isolated-atom energies are independently validated, the resulting harmonized multi-method repository would enable principled delta-learning and transfer-learning studies across levels of theory, which the paper only gestures toward.","The standardized method enum of functional, basis set, and correction opens a direct route to systematic studies of basis-set convergence and functional-family effects on learned potentials, going beyond what any single dataset paper provides.","One testable extension is to train two MLIPs on the same dataset with and without atomization-energy normalization and measure transfer error on larger molecules; the extensivity argument predicts the normalized model degrades less."],"forward_implications":["Any of the 37 datasets can be loaded in one line of Python with user-specified energy, distance, and array formats, while original units are preserved for conversion.","Multi-fidelity training is supported because the same molecular families appear at multiple levels of theory, such as QM9 and MultiXCQM9 with its 228 DFT variants.","Energy normalization via isolated-atom subtraction lets models trained on one dataset be applied to different system sizes without breaking size extensivity.","Interaction-energy datasets such as DES370K, DES5M, Splinter, Metcalf, X40, and L7 provide a dedicated track for training models of drug-target, drug-drug, and protein-protein interactions.","Baseline results show SchNet consistently behind the other two architectures on potential-energy datasets while being competitive on interaction datasets, suggesting different architectural biases for the two task families."],"supporting_citations":[{"why":"Provides the ANI-1 dataset, one of the foundational large-scale off-equilibrium QM datasets that OpenQDC consolidates.","marker":"[27]"},{"why":"Supplies QM9, a widely used benchmark dataset that the repository integrates into its potential-energy collection.","marker":"[56]"},{"why":"Supplies GEOM, a 37-million-conformer dataset whose scale drives the repository's claimed coverage.","marker":"[85]"},{"why":"Provides QMugs, a bioactive-molecule dataset with both semi-empirical and DFT labels that OpenQDC includes.","marker":"[97]"},{"why":"Supplies Spice, a dataset of drug-like molecules and peptides with ωB97M-D3(BJ) labels used in the potential-energy collection.","marker":"[101]"},{"why":"Provides PubChemQC PM6, the single largest dataset in the collection with 221 million geometries.","marker":"[58]"},{"why":"Supplies PubChemQC B3LYP/6-31G*//PM6, the DFT-labeled counterpart covering the same PubChem chemical space.","marker":"[95]"},{"why":"Provides DES370K and DES5M, gold-standard dimer interaction energies that anchor the interaction-energy collection.","marker":"[111]"},{"why":"Defines the SchNet architecture that the paper benchmarks, and its GDML and ISO17 data are integrated as datasets.","marker":"[70]"},{"why":"Defines the TorchMD-Net architecture used as one of the three baseline models in the benchmark experiments.","marker":"[116]"}],"fun_headline_variants":["One Python API unifies 400M quantum geometries for MLIPs","37 QM datasets, 400M geometries, one open access point","OpenQDC: Standardizing 400M quantum geometries for ML force fields","Unify 37 QM datasets into one Python API for MLIPs","400M geometries across 37 datasets now one Python package"],"cache_read_input_tokens":21888,"weakest_assumption_plain":"The computed isolated-atom energies and unit metadata for QM methods whose original datasets lacked them are correct; if they are wrong, the atomization-energy normalization and every energy benchmark built on it are wrong.","fun_headline_variants_meta":{"raw":{"variants":["One Python API unifies 400M quantum geometries for MLIPs","37 QM datasets, 400M geometries, one open access point","OpenQDC: Standardizing 400M quantum geometries for ML force fields","Unify 37 QM datasets into one Python API for MLIPs","400M geometries across 37 datasets now one Python package"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000463,"raw_usage":{"total_tokens":2271,"prompt_tokens":859,"completion_tokens":1412,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":475,"completion_tokens_details":{"reasoning_tokens":1317}},"tokens_in":475,"tokens_out":1412,"duration_ms":8868,"temperature":1.0,"reasoning_tokens":1317,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:58:27.720654+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute isolated-atom energies for a sample of the QM methods in the repository using an independent electronic-structure code and compare them against the stored values; a disagreement beyond numerical tolerance, say greater than 1 meV per atom, would propagate directly into the energy labels users train on. Checking a dataset's distance-unit metadata against its original publication would similarly confirm whether the conversions that feed radial cutoffs are trustworthy.","supporting_citations":[{"cited_title":"Geom, energy-annotated molecular conforma- tions for property prediction and molecular generation","cited_arxiv_id":null,"evidence_quote":"Supplies GEOM, a 37-million-conformer dataset whose scale drives the repository's claimed coverage."},{"cited_title":"Qmugs, quantum mechanical properties of drug-like molecules","cited_arxiv_id":null,"evidence_quote":"Provides QMugs, a bioactive-molecule dataset with both semi-empirical and DFT labels that OpenQDC includes."},{"cited_title":"Spice, a dataset of drug-like molecules and peptides for training machine learning potentials","cited_arxiv_id":null,"evidence_quote":"Supplies Spice, a dataset of drug-like molecules and peptides with ωB97M-D3(BJ) labels used in the potential-energy collection."},{"cited_title":"Pubchemqc pm6: Data sets of 221 million molecules with optimized molecular geometries and electronic properties","cited_arxiv_id":null,"evidence_quote":"Provides PubChemQC PM6, the single largest dataset in the collection with 221 million geometries."},{"cited_title":"Quantum chemical benchmark databases of gold-standard dimer interaction energies.Scientific data, 8(1):55, 2021","cited_arxiv_id":null,"evidence_quote":"Provides DES370K and DES5M, gold-standard dimer interaction energies that anchor the interaction-energy collection."}],"review_version":1}