{"id":"8cfdda93-9289-47ce-93c1-b9c256f46273","arxiv_id":"2412.06064","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"IDLe, a multi-task training strategy with fidelity-specific prediction heads on a shared representation, matches high-fidelity NNP energy accuracy while using up to 50x less high-fidelity QM data.","lead":"Implicit Delta Learning (IDLe) trains neural network potentials with mostly cheap quantum-mechanics labels, using a shared network with separate output heads for cheap and expensive energy levels. The authors report matching high-fidelity training accuracy while using up to 50 times less expensive data, and describe 11 million new semi-empirical calculations.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 50x data-efficiency headline counts only newly added target-set HF labels and omits the source-set HF labels that IDLe also trains on; under total-HF accounting the ratio can be far smaller.","rationale":"The reader's weakest assumption was the mutual-information premise between LF and HF labels. That is a reasonable transferability hypothesis, but the manuscript already provides empirical support for it across several datasets, and the OOD results are the paper's strongest evidence. The more immediate threat to the central claim is that the headline 50x number is not measuring what the abstract says it measures. Section 4.4 states that all methods use all HF labels of A, while Section 4.5 defines NIDLe as 1% of the total available HF labels. For the Spice transfer experiment, these two statements are inconsistent: the IDLe model that reaches the reported accuracy uses roughly all of A's HF labels plus 1% of B's, not 1% of the combined set. Because the abstract's strongest quantitative hook is the 'up to 50x less high-fidelity data' factor and it is taken directly from Table 3's Spice row, this accounting issue is load-bearing. A re-analysis using explicit counts can settle it without new training runs. If the numbers confirm the narrow reading, the paper remains a useful but more modest contribution about incremental data efficiency; if they do not, the headline claim must be rewritten. Either way the appropriate verdict is conditional, consistent with the reader's overall assessment.","tokens_in":16097,"tokens_out":10973,"duration_ms":106502,"concrete_test":"Recompute the Spice row of Table 3 using explicit label counts from the Figure 3 runs: record N_HF(A), N_HF(B), N_LF(A), N_LF(B), and the exact direct-learning B-fraction that matches IDLe at 1% of B. Compute R_total = (N_HF(A) + N_direct_B) / (N_HF(A) + 0.01 * N_HF(B)) and compare it with the reported 50x; also recompute the CPU-time ratio including HF-A generation in the IDLe column. If R_total drops below 5x, the headline should be rephrased as incremental-label efficiency, not total high-fidelity data reduction.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central 50x claim rests on Table 3 and on the definition in Section 4.5 that NIDLe is '1% of the total available HF labels' while Ndirect is the HF-label count at which direct learning matches IDLe. In the SpiceV1->2 experiment (Section 4.4), however, the setup is explicitly 'we train with all LF labels of A and B, all HF labels of A and vary the amount of HF labels of B.' Thus the IDLe model whose performance is plotted actually trains on 100% of the source-set A (SpiceV1) HF labels plus 1% of the target-set B (SpiceV1->2) HF labels, along with all LF labels from both sets. The direct-learning baseline is trained on all HF labels of A plus the indicated fraction of B. Consequently, taking NIDLe as 1% of the combined dataset 'SpicevL2' undercounts the HF data IDLe uses by roughly the fraction of HF labels that reside in A. If A and B are comparable in size, the true total-HF-label ratio is approximately (A + 0.5B)/(A + 0.01B), which is near 1.4 for A ≈ B, not 50. The abstract's 'up to 50x less high-fidelity data' is therefore only supported under the much narrower reading '50x fewer newly computed high-fidelity labels for an already-labeled source dataset,' a caveat that the paper does not state. This is an accounting inconsistency in the headline metric, not merely missing error bars.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces IDLe (Implicit Delta Learning), an end-to-end multi-task training strategy for neural network potentials in which a shared backbone is augmented with fidelity-specific prediction heads for high- and low-fidelity QM energies. Training minimizes a masked multi-fidelity MSE loss (Eq. 2) over all available labels, so LF labels from methods such as GFN2-xTB, DFTB3, and PM6 can substitute for expensive HF labels without increasing inference cost. Experiments on QMugs, QM7-X, ANI1-ccx, and Spice compare IDLe with direct learning, explicit delta learning, and fine-tuning across DFT and CCSD(T) target levels, and the authors report up to 50x HF data efficiency and release a large set of newly computed LF single-point energies.","tokens_in":16398,"tokens_out":7072,"duration_ms":70278,"significance":"If the reported results hold, IDLe is a practically valuable contribution: it is a simple, end-to-end way to exploit cheap semi-empirical labels while avoiding the inference-time LF calculation required by explicit delta learning. The breadth of the benchmark suite, spanning four datasets with multiple HF/LF method pairs, is a strength, and the release of millions of new semi-empirical energies is a useful community resource. However, the headline data-efficiency number is computed with an accounting that undercounts the HF labels actually used by IDLe, and the experimental section does not report seed variance or goodness-of-fit statistics for the scaling-law claims. These issues mean the central claim, as stated in the abstract, is not yet established at the reported magnitude, although the underlying method remains plausible and the problems appear addressable.","major_comments":[{"comment":"The 50x data-efficiency ratio is not computed under a consistent accounting of high-fidelity labels. In the SpiceV1->2 OOD experiment, the setup is explicitly that IDLe trains on all LF labels of A and B, all HF labels of A, and a varying fraction of HF labels of B (Section 4.4). Yet Section 4.5 defines N_IDLe as 1% of the total available HF labels of the combined dataset and N_direct as the HF-label count at which direct learning matches IDLe. This definition omits the source-set HF labels of A. If A and B are comparable in size, the true total-HF-label ratio is approximately (A + 0.5B)/(A + 0.01B), which is close to 1.4 rather than 50. The abstract's 'up to 50x less high-fidelity data' is therefore only valid under the much narrower reading '50x fewer newly computed HF labels for an already labeled source dataset.' Please redefine N_IDLe to include all HF labels actually used by IDLe, or explicitly state and prominently caveat the narrower interpretation, and recompute the ratios in Table 3 accordingly.","section":"Section 4.5, Table 3; Section 4.4; Abstract"},{"comment":"No seed variance or repeated runs are reported; all curves appear to be single training runs without error bars, confidence intervals, or significance tests. Statements such as 'IDLe and fine-tuning approaches reach chemical accuracy with 4-6x fewer DFT labels' and the relative ordering of IDLe versus fine-tuning and delta-learning are therefore not statistically supported. Please provide multiple seeds (at least 3-5) for the key comparisons and report mean plus/minus standard deviation, or an equivalent uncertainty quantification, for the MAE curves that underlie the efficiency ratios in Table 3.","section":"Section 4.1, Figures 1-4"},{"comment":"The statement that the model 'perfectly follows power-law scaling' is unsupported by the reported evidence. No goodness-of-fit statistic, confidence interval, or residual analysis is given for the fitted exponents, and the curves contain at most five evaluation points; the segment-wise slopes in Table 5 are computed from two-point segments. Please report fit quality (e.g., R^2 and standard error of beta) and either add more evaluation points or soften the claim to 'approximately follows' power-law scaling.","section":"Section 4.5, Figure 5 and Appendix A.6"},{"comment":"Please state explicitly whether the normalization constants mu_{j,k} and sigma_j are estimated on the training split only. If they are estimated on the full dataset, including the test set, then test-set statistics leak into training, which would make all reported MAEs optimistic and would require rerunning the experiments with train-only statistics. If they are train-only, this should be stated in the text so that readers do not have to infer it.","section":"Equation (3), Section 4.1"}],"minor_comments":[{"comment":"The sentence 'as discussed in Section 4.3' at the start of Section 4.3 appears to reference the preceding IID results and should presumably refer to Section 4.2.","section":"Section 4.3"},{"comment":"The caption says 'OOD performance on SpiceV2->1' but the text and experimental setup refer to SpiceV1->2; the direction of the transfer should be corrected.","section":"Figure 3 caption"},{"comment":"The claimed '~11 million single point energy computations' does not obviously match Table 2: summing the conformer counts times the number of added LF labels per dataset gives roughly 17.9 million computations, or 8.7 million conformers. Please reconcile the stated number in the abstract and contributions with the dataset table.","section":"Abstract and Table 2"},{"comment":"The heading 'CPU Time [d]' is ambiguous because the columns contain dimensionless ratios, not CPU days; rename to something like 'Data ratio' and 'Compute-time ratio'.","section":"Table 3"},{"comment":"The data-efficiency ratios are said to be 'inferred from Figures 1, 2, and 3'; please provide the corresponding MAE values in a table so that the ratios are reproducible without reading values off log-scale plots.","section":"Table 3 and Section 4.5"},{"comment":"Capitalization of the method name is inconsistent (IDLe versus IDLE); please unify.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The core idea is sound and the empirical direction is valuable, but the headline 50x claim is currently misleading under total-HF-label accounting, and the lack of seed variance and fit statistics makes the quantitative comparisons hard to evaluate. These issues are addressable with a revised metric definition, additional experiments or clarifications, and more careful statistical reporting. If the 50x claim is retained without correcting the accounting, I would not be able to recommend acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Let me get to the point: the central 'up to 50x less HF data' claim does not survive careful accounting of the OOD experiment. In the SpiceV1->2 setup (Section 4.4), IDLe trains on all HF labels of SpiceV1 plus 1% of the target set, and the direct-learning baseline also trains on all HF labels of SpiceV1. So the definition of NIDLe in Section 4.5 as '1% of the total available HF labels' undercounts what IDLe actually uses. If the two splits are comparable in size, the true HF-to-HF ratio is close to 1.4, not 50. The 50x claim only holds under the narrow reading '50x fewer newly computed HF labels for an already-labeled source dataset.' That is still useful, but it is not what the abstract and Table 3 imply.\n\nThe method itself is worth taking seriously. A shared backbone with fidelity-specific heads, trained end-to-end on LF labels plus a small HF subset, is a natural and underexplored combination. The systematic comparison across QMugs, QM7-X, ANI1-ccx, and Spice is more thorough than most multi-fidelity papers, and the OOD evaluation on chemical transferability and molecule-size extrapolation is genuinely informative. The promise of 11 million semi-empirical single-point energies is a real community resource if released. The scaling-law analysis, though overclaimed, identifies a saturation regime worth exploring.\n\nSoft spots beyond the accounting issue: no seed variance is reported, so we cannot tell whether the gap between IDLe and fine-tuning is real or noise. The 'perfectly follows power-law scaling' lacks any fit statistics. There is no code or data URL in the preprint, only a promise. Training is energy-only, so the practical MD impact is unproven; the authors acknowledge this in the limitations section. The mutual-information hypothesis (Section 4.3) is plausible but untested. The citation pattern is fine—related work is well covered.\n\nWho this is for: anyone working on multi-fidelity NNP training, dataset generation, or transfer learning for interatomic potentials. The method and dataset are worth engaging with, but the headline should be corrected.\n\nRecommendation: yes, it deserves a serious referee. The method is novel enough and the dataset valuable enough. The referee should ask for a corrected data-efficiency definition, seed variance, and code/data release.","headline":"Good method, useful dataset, but the 50x data-efficiency claim is inflated by an accounting error in the OOD setup.","tokens_in":16959,"tokens_out":4433,"would_cite":true,"duration_ms":38470,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Implicit delta learning (IDLe) claims neural network potentials can reach the same accuracy as single high-fidelity quantum baselines while using up to 50 times less high-fidelity data, by training fidelity-specific heads on a shared…","keywords":["neural network potentials","multi-fidelity learning","implicit delta learning","semi-empirical quantum chemistry","data efficiency","transfer learning","molecular dynamics","tight-binding methods"],"falsifier":"A direct test: take a dataset with LF labels and a small set of HF labels, train IDLe, and then remove the correlation between LF and HF energies on a held-out test set (e.g., by permuting LF energies among geometries). If the HF head still performs as well as a model trained with correlated labels, the shared-latent mechanism is not the source of the gains; if it degrades to direct-learning level, the mutual-information premise is confirmed.","tokens_in":15876,"feed_emoji":"⚛️","tokens_out":5659,"duration_ms":46450,"temperature":0.7,"pith_summary":"High-fidelity quantum chemistry labels are the bottleneck for training neural network potentials. This paper proposes Implicit $\\Delta$ Learning (IDLe), an end-to-end multi-task training scheme in which one shared latent representation of a molecule feeds several prediction heads, each decoding energies at a different level of theory: cheap semi-empirical methods (PM6, GFN2-xTB, DFTB3) plus the expensive target (DFT or CCSD(T)). Because semi-empirical labels cost a tiny fraction of high-fidelity ones, IDLe can be trained on many more geometries for the same budget. The paper reports that IDLe matches the accuracy of directly trained high-fidelity models with up to 50x fewer high-fidelity labels, and that inference uses only the high-fidelity head, so the runtime cost is unchanged. If the claim holds, generating new training data for neural network potentials becomes far cheaper, widening their chemical coverage.","feed_headline":"A multi-task network cuts costly quantum training data by 50x","feed_subtitle":"Implicit delta learning uses cheap semi-empirical labels to match DFT and CCSD(T) accuracy with 4-50x fewer expensive labels.","key_machinery":"The central object is the shared latent representation with fidelity-specific heads, trained by minimizing a multi-fidelity multi-task mean-squared-error loss over all available labels. The mechanism: LF heads force the backbone to build a representation good enough to reproduce cheap energies across a large chemical space; the HF head then decodes the expensive energy from that same representation, so geometries that never saw an HF label still benefit from the representation shaped by LF labels. Because only the HF head is used at inference, IDLe avoids the extra LF computation that explicit delta learning pays at simulation time.","core_discovery":"IDLe is a multi-task learning strategy that replaces the explicit energy-difference target of classical delta learning with an implicit one: instead of predicting the gap between a low-fidelity (LF) and a high-fidelity (HF) energy, the network predicts both energies from the same latent representation, with the LF head trained on abundant cheap labels and the HF head trained on fewer expensive ones. The authors show that this shared-latent decoding transfers information from LF to HF: on the SpiceV1 to SpiceV2 transfer, IDLe trained with 2.5% of the DFT labels plus GFN2-xTB and PM6 labels reaches the same accuracy as a direct-learning model trained on 100% of the DFT labels; on ANI1-ccx, IDLe with only 2.5% of the CCSD(T) labels plus GFN2-xTB, PM6, and DFT labels nearly matches the 100%-CCSD(T) baseline. The paper also provides ~11 million new semi-empirical single-point energies on existing datasets, and a scaling study showing power-law gains when high-fidelity data are increased and a saturation regime when only low-fidelity data are added.","pith_inferences":["If IDLe's data efficiency transfers to force training (energies plus forces), the practical cost reduction for MD-grade potentials may be larger than the energy-only numbers here, because force labels are the most expensive part of many NNP datasets.","The mutual-information explanation suggests a cheap diagnostic before launching a new LF/HF campaign: compute the LF-HF correlation on a small probe set; low correlation would predict that IDLe gives little advantage over direct learning for that pair.","The saturation regime when adding LF-only conformers indicates a specific resource-allocation rule: spend the next budget on HF labels for existing conformers rather than on more LF labels, until the shared representation improves.","The 11M semi-empirical labels released here could make IDLe a drop-in pre-training scheme for other architectures (equivariant GNNs, MACE-style models), not just the TorchMDNet backbone used in the paper."],"forward_implications":["Training NNPs for DFT or CCSD(T) level accuracy can require 4-50x less high-fidelity data when cheap semi-empirical labels are available, reducing the cost of building new datasets.","IDLe works out-of-distribution: with no additional high-fidelity labels it matches a 100%-HF direct-learning baseline on small and medium chemical shifts, and it is the most efficient user of the few HF labels added under larger shifts.","IDLe is less sensitive than explicit delta learning to the quality of the LF method, and can combine several LF methods (e.g., GFN2-xTB plus PM6) to improve the HF head.","The scaling results imply that for a fixed LF dataset, adding HF labels follows a power law whose exponent grows with LF data size, while adding LF labels beyond the HF set shows diminishing returns."],"supporting_citations":[{"why":"Introduces explicit delta learning, the baseline and conceptual contrast for IDLe.","marker":"[20]"},{"why":"Supplies the graph-neural-network architecture used for all models in the study.","marker":"[5]"},{"why":"Provides the QMugs dataset with DFT and GFN2-xTB labels used in IID and size-extrapolation experiments.","marker":"[23]"},{"why":"Provide the Spice and Spice 2.0 datasets used for the out-of-distribution chemical transfer experiments.","marker":"[24,25]"},{"why":"Provides the ANI1-ccx dataset with CCSD(T) labels, the hardest-fidelity test case.","marker":"[27]"},{"why":"Provides the QM7-X dataset with DFTB3 labels for the second IID benchmark.","marker":"[26]"},{"why":"Defines GFN2-xTB, one of the two semi-empirical low-fidelity methods whose labels IDLe leverages.","marker":"[17]"},{"why":"Defines PM6, the other semi-empirical low-fidelity method used alongside GFN2-xTB.","marker":"[19]"},{"why":"Describes the transfer-learning/fine-tuning baseline that IDLe is compared against and that motivates the end-to-end design.","marker":"[32]"}],"fun_headline_variants":["Implicit delta learning cuts QM data needs by 50x","Multi-task network matches DFT accuracy with 2.5% of labels","Cheap QM labels, full accuracy: the IDLe trick","Shared latent trains NNPs with 50x less expensive data","Semi-empirical data powers high-fidelity NNP accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole method rests on the assumption that a cheap low-fidelity calculation and an expensive high-fidelity calculation on the same molecule share enough mutual information that a model trained mostly on cheap labels can predict the expensive energy on molecules it has never seen with expensive labels.","fun_headline_variants_meta":{"raw":{"variants":["Implicit delta learning cuts QM data needs by 50x","Multi-task network matches DFT accuracy with 2.5% of labels","Cheap QM labels, full accuracy: the IDLe trick","Shared latent trains NNPs with 50x less expensive data","Semi-empirical data powers high-fidelity NNP accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000211,"raw_usage":{"total_tokens":1432,"prompt_tokens":977,"completion_tokens":455,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":593,"completion_tokens_details":{"reasoning_tokens":364}},"tokens_in":593,"tokens_out":455,"duration_ms":4436,"temperature":1.0,"reasoning_tokens":364,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:02:41.372117+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test: take a dataset with LF labels and a small set of HF labels, train IDLe, and then remove the correlation between LF and HF energies on a held-out test set (e.g., by permuting LF energies among geometries). If the HF head still performs as well as a model trained with correlated labels, the shared-latent mechanism is not the source of the gains; if it degrades to direct-learning level, the mutual-information premise is confirmed.","supporting_citations":[{"cited_title":"Big data meets quantum chemistry approximations: the δ-machine learning approach","cited_arxiv_id":null,"evidence_quote":"Introduces explicit delta learning, the baseline and conceptual contrast for IDLe."},{"cited_title":"TorchMD-Net 2.0: Fast Neural Network Potentials for Molecular Simulations","cited_arxiv_id":"2402.17660","evidence_quote":"Supplies the graph-neural-network architecture used for all models in the study."},{"cited_title":"Robert A., and Tkatchenko Alexandre","cited_arxiv_id":null,"evidence_quote":"Provides the QM7-X dataset with DFTB3 labels for the second IID benchmark."},{"cited_title":"Gfn2-xtb—an accurate and broadly parametrized self-consistent tight-binding quantum chemical method with multipole electrostatics and density-dependent dispersion contributions","cited_arxiv_id":null,"evidence_quote":"Defines GFN2-xTB, one of the two semi-empirical low-fidelity methods whose labels IDLe leverages."},{"cited_title":"Optimization of parameters for semiempirical methods v: Modification of nddo approximations and application to 70 elements","cited_arxiv_id":null,"evidence_quote":"Defines PM6, the other semi-empirical low-fidelity method used alongside GFN2-xTB."},{"cited_title":"Smith, Benjamin T","cited_arxiv_id":null,"evidence_quote":"Describes the transfer-learning/fine-tuning baseline that IDLe is compared against and that motivates the end-to-end design."}],"review_version":1}