{"id":"aac17065-f35f-4930-bd48-8784dce3d584","arxiv_id":"2501.02059","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Retraining molecular property predictors on active-learning batches of DFT-computed molecules enables a genetic generative model to generate molecules with properties beyond the training data and more stable molecules.","lead":"This paper shows that generative models for molecule design fail mainly because their property predictors cannot handle molecules unlike those seen in training. Adding an active-learning loop that retrains the predictors on quantum chemistry results lets the generator find denser and more stable molecules beyond the training range.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1 comparison is not budget-matched and lacks a passive-retraining control, so the causal claim that active learning—rather than 100x more DFT labels—enables extrapolation is unproven.","rationale":"The reader's weakest assumption is DFT accuracy; that is a real external-validity concern, but it is secondary to the internal comparison flaw. The headline claim is causal: active learning enables extrapolation. The evidence in Table 1 compares an active pipeline with ~52k DFT labels and a retrained predictor against baselines with 500 labels and an unretrained predictor. This confound must be resolved before asking whether the DFT labels themselves are accurate enough. The missing passive-retraining control is directly implementable from the paper's own pipeline and would settle whether active selection, rather than simply more DFT-labeled data in the generated chemical space, is responsible for the improvement. The existing Figure S12 comparison (290k CCDC densities) does not address this because it uses a different distribution and does not retrain on generated molecules. The reported ablation without the Stability-Prediction MPNN addresses stability only. Therefore the paper should remain CONDITIONAL pending this control, but not be rejected: the active-learning mechanism may well be real, and the missing experiment is easy to run.","tokens_in":15818,"tokens_out":7763,"duration_ms":79071,"concrete_test":"Add a passive-retraining control: use the 51,453 DFT-labeled molecules generated in active-learning iterations 1-3, retrain the density and heat-of-formation MPNNs once on the 10k set plus this pool, train the Stability-Prediction MPNN on the same pool, then run JANUS with the same full objective for 200 generations and select the top 500 molecules for DFT evaluation. Compute the Table 1 metrics. If this passive control reaches a top multi-property score of at least 5.24, a top density of at least 2.014 g/cc, or at least 17 SOTA molecules, then iterative active selection is not the cause of the extrapolation and the central claim fails. If it falls short, the active-learning claim survives; repeat the same control with REINVENT as a further check.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract's central claim is 'only our active learning approach generates molecules with properties that extrapolate beyond the training data.' For that comparison to attribute success to active learning, the active and non-active pipelines must be matched in oracle budget and in what the property predictor has seen. They are not. Per Methods and Table 2, active-learning iterations 1-3 evaluate 980, 2,433, and 48,040 molecules with DFT, plus 500 in iteration 4, for about 51,953 total DFT labels. JANUS and REINVENT are capped at 500 DFT evaluations and use MPNN0, trained only on the 10k set. The final active run uses MPNN3, retrained on all previous DFT labels. Thus the comparison conflates three things: active selection, a roughly 100x larger oracle budget, and retraining on generated molecules. The only reported ablation, 'JANUS with retrained MPNN predictors but without the Stability-Prediction MPNN,' addresses stability (0.4% stable) and does not report the top multi-property score, so it does not control for the extrapolation claim. Without a passive-retraining control—one batch of the same 51,453 DFT labels from iterations 1-3 used to retrain the MPNN once, then a non-active JANUS or REINVENT run using that predictor—we cannot distinguish 'active learning enables extrapolation' from 'more data and retraining enables extrapolation.' The paper's Figure S12 shows that large unrelated training sets do not fix extrapolation, but that is not the same as retraining on the generated-molecule labels.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a closed-loop molecular design pipeline in which a genetic algorithm (JANUS) is coupled to DFT calculations and to message-passing neural network (MPNN) predictors for density and solid heat of formation. After each generation round, the MPNNs are retrained on DFT-labeled molecules, and in the final round a Stability-Prediction MPNN filters candidates before DFT evaluation. The authors report that this active-learning pipeline generates molecules whose DFT-computed multi-property scores, densities, and stability fractions exceed those of JANUS and REINVENT baselines limited to 500 DFT evaluations, and they argue that the main bottleneck in generative molecular design is the property predictor rather than the generator.","tokens_in":16088,"tokens_out":8783,"duration_ms":87398,"significance":"If the central claim is established, the paper would make a useful practical contribution: it would show that iteratively retraining property predictors on quantum-chemical feedback can improve extrapolation in generative molecular design, and it would identify predictor generalization as a key bottleneck. The manuscript has several strengths: the evaluations use hold-out sets, the DFT pipeline is described in detail, retraining comparisons with ensembles and foundation models are reported (Figure S12), and the data and code are promised to be available. The main weakness is that the headline comparison is not oracle-budget-matched and lacks a passive-retraining control, so the causal attribution to active learning is not yet supported. The abstract also overstates the exclusivity of the extrapolation result relative to the paper's own Table 1.","major_comments":[{"comment":"The headline comparison is not budget-matched. The abstract and Table 1 compare JANUS with active learning to JANUS and REINVENT limited to 500 DFT evaluations, yet the active-learning pipeline uses 980 + 2,433 + 48,040 + 500 = 51,953 DFT labels across the four iterations. Moreover, the baselines use MPNN0 trained only on the 10k set, while the active-learning final run uses MPNN3 retrained on all prior DFT labels. The comparison therefore conflates active selection, a roughly 100x larger oracle budget, and retraining. To support the claim that active learning specifically enables extrapolation, please add a passive-retraining control that uses the same 51,453 labels from iterations 1-3 to retrain the MPNN once and then runs JANUS or REINVENT non-actively with that predictor, or alternatively restrict the active-learning pipeline to a 500-label total budget.","section":"Methods, Active Learning Procedure and Table 2"},{"comment":"The abstract's claim that 'only our active learning approach generates molecules with properties that extrapolate beyond the training data' is contradicted by Table 1 itself: REINVENT achieves a top DFT heat of formation of 417 kcal/mol, exceeding the training-set maximum of 387 kcal/mol, and REINVENT also has 12/500 SOTA molecules that exceed the training-data Pareto front. If the intended claim is only about the multi-property score (Equation 1) or about simultaneous extrapolation in both properties, the wording should be corrected to say that explicitly. As written, the exclusivity claim is too strong and should be revised.","section":"Abstract and Table 1"},{"comment":"The causal attribution to active learning is not established because the only reported ablation, 'JANUS with retrained MPNN predictors but without the Stability-Prediction MPNN,' addresses only the stability metric (0.4% stable) and does not report the top multi-property score or extrapolation metrics. This ablation does not control for whether the improvement comes from the iterative active selection or simply from retraining on a large set of DFT-labeled generated molecules. In addition, the Methods state that iterations 1-3 select molecules randomly from the filtered generated list, so the procedure is closer to iterative random data collection and retraining than to active selection; the authors should either acknowledge this discrepancy or implement an active selection strategy and show that it matters.","section":"Results, 'Active-Learning Enables Extrapolation' and Discussion"},{"comment":"The paper validates DFT-computed densities against experimental densities (Figure S1) but does not validate the solid heat-of-formation values obtained with the Byrd-Rice method, whose sublimation parameters a, b, c are fitted to experimental data. Since the headline multi-property score (Equation 1) and the top heat-of-formation numbers in Table 1 depend on this pipeline, please add a comparison of the computed solid heats of formation against experimental values for a held-out set of molecules, or otherwise quantify the uncertainty in the sublimation fit. Without this validation, the reader cannot assess whether the reported extrapolation in heat of formation is a real molecular-property improvement or an artifact of the simulator on out-of-distribution CHNO molecules.","section":"Methods, High-Throughput DFT Calculations, and Figure S1"},{"comment":"The hold-out protocol for the models evaluated in Figure 4a is not stated clearly. The Methods say that after each active-learning iteration the MPNNs are retrained on all molecules that passed through DFT, while the Figure 4 caption says the models are evaluated against a 10% hold-out test set from each active-learning iteration. For Figure S12 the authors explicitly state that data leakage is prevented by training on 80% and holding out 10%, but the main text does not state whether the same leakage-free protocol was used for the Figure 4a results. Please clarify this; if the retrained models were evaluated on molecules used in their training, the reported 75-83% RMSE reductions and the 7%-to-86% precision improvement would be optimistic.","section":"Figure 4a and Methods, Message-Passing Neural Network"}],"minor_comments":[{"comment":"The manuscript text contains numerous OCR-like artifacts (e.g., 'Ac#ve,' 'opImized,' 'ANer,' 'I' in place of 't') in the supplied version; a clean, proofread version should be provided.","section":"Throughout"},{"comment":"The text says 'In Figure 4c, we compare the fraction of stable generated molecules with REINVENT and JANUS,' but the stability comparison appears in Figure 5c; the citation should be corrected.","section":"Results, Stability paragraph"},{"comment":"The footnote for 'Top molecules' is incomplete ('three standard deviations above the training data' is cut off) and the footnote for 'SOTA molecules' should define the Pareto-front criterion more explicitly; please complete the definitions.","section":"Table 1 footnotes"},{"comment":"The phrase 'reaching up to 0.44 standard deviations beyond the training data range' is unclear; it should say that the best active-learning multi-property score exceeds the best training-set score by 0.44 standard-deviation units.","section":"Abstract and Results"},{"comment":"The Data Availability statement lists molecules from the first three active-learning iterations but does not mention the 500 generated molecules from iteration 4; those should be included or the omission should be justified for reproducibility.","section":"Data Availability"}],"recommendation":"major_revision","confidential_remarks":"The main risk is that the headline comparison is not budget-matched and the abstract overstates the exclusivity of the extrapolation result. A passive-retraining control is feasible because the DFT labels already exist, so I would request it in revision rather than rejecting outright. The heat-of-formation validation is also needed to establish that the extrapolation is a molecular-property result rather than a simulation artifact."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper credibly shows that the property predictor, not the generator, is the bottleneck for extrapolation in molecular generative design, and that iterative retraining on DFT-labeled generated molecules fixes it. The RMSE drops 75-83%, precision for top molecules jumps from 7% to 86%, and adding a stability classifier triples the rate of stable molecules. Those are real, carefully measured results, with hold-out tests, parity plots, and comparisons against larger training sets and foundation models. I believe the core empirical observation: MPNNs trained only on the 10k set do not generalize to generated molecules, and retraining on generated-molecule DFT labels does.\n\nThe soft spot is the headline comparison. Table 1 says \"only our active learning approach\" extrapolates, but the active run consumes roughly 52,000 DFT calculations while JANUS and REINVENT get 500. There is no passive-retraining control that gives the same 52,000 labels to the baselines. So the causal claim conflates three things: active selection, 100x more oracle budget, and retraining on generated molecules. The first three iterations also use random sampling, not uncertainty-based acquisition, so calling it \"active learning\" oversells the selection strategy. Figure S12 shows that larger unrelated training sets do not fix extrapolation, which is suggestive, but it is not the same as retraining on the generated-molecule labels. Without that control, you cannot attribute the gains to active learning versus simply more relevant data.\n\nTwo smaller issues. The Data Availability statement covers only the first three iterations; the 500 final-iteration molecules and their DFT values are missing, which limits reproducibility. Table 1 also reports no error bars, so single-molecule metrics like top score 5.24 vs 4.80 should be read with caution. The DFT ground truth is itself an approximation, especially solid heats of formation from Byrd-Rice parameters fitted to experiment; that is a shared limitation across all methods, not a fatal flaw.\n\nThis paper is for anyone building closed-loop molecular discovery pipelines or benchmarking generative models. It deserves a serious referee but needs revision: a budget-matched passive-retraining control, release of the final-iteration data, and more cautious framing of the active-learning claim. I would send it to peer review and push on those points.","headline":"Real, useful demonstration that retraining property predictors on DFT-labeled generated molecules fixes extrapolation, but the headline causal claim about active learning is undercut by a 100x oracle-budget mismatch.","tokens_in":16641,"tokens_out":1833,"would_cite":false,"duration_ms":20986,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that the property predictor, not the generator, is what limits extrapolation in molecular generative models, and that active learning on DFT feedback corrects the problem.","keywords":["active learning","molecular generative models","extrapolation","property prediction","density functional theory","message-passing neural networks","inverse molecular design","thermodynamic stability"],"falsifier":"Measure the crystal density and heat of formation experimentally for the top-scoring generated molecules, such as those with DFT density above 2 g/cc or multi-property scores above 5. If the measured values do not exceed the best training-molecule values of 1.963 g/cc and score 4.80, or if they scatter widely from the DFT predictions, the extrapolation claim fails. A cheaper version is to recompute the top candidates with a higher-level ab initio method and check whether the same molecules still beat the training Pareto front.","tokens_in":15603,"feed_emoji":"🧪","tokens_out":7111,"duration_ms":64858,"temperature":0.7,"pith_summary":"Molecular generative models promise to design molecules better than any in their training set, but in practice they plateau. This paper argues the bottleneck is not the generator but the property predictor that steers it: regression models trained on known molecules misjudge out-of-distribution candidates, so the generator gets bad feedback. The authors build a closed loop in which generated molecules are validated by density functional theory, and the property predictors are retrained on those results. After four iterations, this active-learning pipeline is the only one that generates molecules whose density and heat of formation exceed the training data's best values, and it triples the fraction of thermodynamically stable molecules. If correct, the result suggests that cheap retraining on physics-based feedback matters as much as better generation algorithms.","feed_headline":"Active learning lets molecule generators beat their training data","feed_subtitle":"A closed-loop retraining loop on quantum chemistry produces denser, more stable candidates that plain models miss.","key_machinery":"The load-bearing mechanism is the closed-loop active-learning pipeline: generate candidate molecules with the JANUS genetic algorithm scored by a multi-property objective that combines z-scores of predicted density and solid heat of formation; validate a batch of candidates with a high-throughput DFT pipeline that computes relaxed geometry, density, heat of formation via the Byrd-Rice method, and thermodynamic stability; retrain three Chemprop message-passing neural networks on the accumulated DFT labels, one for each property plus a stability classifier; and feed the retrained predictors back into JANUS. The critical identity is that the retrained predictors now have training coverage in exactly the chemical regions the generator explores, so their scores become trustworthy enough to steer extrapolation.","core_discovery":"On its own terms, the paper's central discovery is that extrapolation in molecular generative design is an active-learning problem, not a generation problem. Using JANUS as the generator and a message-passing neural network as the property predictor, it shows that predictors trained only on the 10,206-molecule CSD-derived dataset have near-zero precision for identifying top-performing generated molecules, because the generated molecules are far from the training distribution. Retraining the predictors on DFT-computed density, solid heat of formation, and stability labels collected from generated molecules cuts prediction error by up to 19x, improves top-molecule precision to 86%, and enables generation of molecules that exceed the training Pareto front: a top density of 2.014 g/cc versus 1.963 g/cc for the best training molecule, and a top multi-property score of 5.24 versus 4.80. Only the active-learning variant generated any such extrapolating molecules; JANUS and REINVENT without active learning, at the same 500-molecule oracle budget, did not.","pith_inferences":["A natural next test is whether uncertainty-based selection of which generated molecules to send to DFT shrinks the oracle budget; the paper did not use advanced sampling, so this is an open question.","If the predictor-bottleneck result generalizes, it reframes research priorities: improving property-predictor generalization, whether by active learning or other means, may yield larger gains than inventing new generative architectures.","The paper's evidence covers only CHNO molecules with N-O bonds; a strong cross-check would be running the same active-learning loop on drug-like chemical space or inorganic solids to see whether the property predictor remains the limiting factor."],"forward_implications":["Only the active-learning pipeline generates molecules that beat the training-data Pareto front in density and heat of formation; JANUS and REINVENT without active learning, at the same 500-molecule oracle budget, do not.","Retraining property predictors on DFT labels from generated molecules reduces extrapolation error by up to 19x and raises precision for identifying top multi-property molecules from 7% to 86%.","Conditioning generation on a DFT-stability classifier raises the fraction of thermodynamically stable generated molecules to 22%, 3.5x higher than the best baseline.","Property predictors trained only on known molecules severely misrank out-of-distribution candidates, so generative models guided by such predictors inherit their blind spots.","Adding active learning to the JANUS generator produces a larger performance gain than switching from JANUS to REINVENT, suggesting that property-predictor generalization may matter as much as generator choice."],"supporting_citations":[{"why":"Supplies the JANUS genetic algorithm used as the generative model in all pipeline runs and baselines.","marker":"[17]"},{"why":"Supplies the Chemprop message-passing neural network implementation used for density, heat-of-formation, and stability prediction.","marker":"[44]"},{"why":"Supplies the Tartarus benchmark and the REINVENT baseline setup against which active learning is compared.","marker":"[21]"},{"why":"Supplies the REINVENT reinforcement-learning agent used as the second baseline generative model.","marker":"[47]"},{"why":"Supplies the Byrd-Rice method for converting DFT gas-phase energies into solid heats of formation, a core label in the active-learning loop.","marker":"[37]"},{"why":"Supplies the 10k dataset of CHNO molecules from the Cambridge Structural Database used as the initial training set and screening baseline.","marker":"[38]"},{"why":"Supplies the NWChem quantum-chemistry code used for DFT geometry relaxation, density, stability, and energy calculations.","marker":"[50]"},{"why":"Supplies the 500-molecule oracle budget constraint that limits all generative models in the comparison.","marker":"[41]"}],"fun_headline_variants":["Active learning pushes molecule generators beyond training data","Active learning lets molecule generators extrapolate","Active learning enables molecular extrapolation","Closed-loop active learning boosts molecular generation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything hinges on DFT being an accurate oracle for molecules far outside the training set: if the calculated densities, solid heats of formation, and stability labels are systematically wrong for out-of-distribution CHNO molecules, then the reported extrapolation is an artifact of the simulator rather than a real molecular property.","fun_headline_variants_meta":{"raw":{"variants":["Active learning pushes molecule generators beyond training data","Active learning lets molecule generators extrapolate","Active learning enables molecular extrapolation","Closed-loop active learning boosts molecular generation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00055,"raw_usage":{"total_tokens":2606,"prompt_tokens":908,"completion_tokens":1698,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":524,"completion_tokens_details":{"reasoning_tokens":1647}},"tokens_in":524,"tokens_out":1698,"duration_ms":13341,"temperature":1.0,"reasoning_tokens":1647,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:15:17.432831+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the crystal density and heat of formation experimentally for the top-scoring generated molecules, such as those with DFT density above 2 g/cc or multi-property scores above 5. If the measured values do not exceed the best training-molecule values of 1.963 g/cc and score 4.80, or if they scatter widely from the DFT predictions, the extrapolation claim fails. A cheaper version is to recompute the top candidates with a higher-level ab initio method and check whether the same molecules still beat the training Pareto front.","supporting_citations":[],"review_version":1}