REVIEW 2 major objections 5 minor 1 cited by
Challenging reaction prediction models to generalize to novel chemistry
T0 review · 2 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read Random splits make reaction-prediction models look about 10 accuracy points better than they are, and the paper's split panel shows which reaction classes the models can and cannot extrapolate to.
desk verdict A careful empirical demonstration that random splits flatter reaction predictors; the NameRxn label-noise caveat is real but does not sink the central conclusion. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying object is a sequence of split constructions over a cleaned, deduplicated version of the Pistachio patent reaction database, each evaluated with the same encoder-decoder transformer (BART architecture with SMILES tokenization, closely related to the Molecular Transformer). Three split mechanisms do the principal work: document- and author-based splits, which stop a patent's series of closely related scope or structure-activity reactions from straddling the train/test boundary; a ladder of time cutoffs from 1996 to 2020 paired with held-out test sets for each year, making extrapolation distance a measurable axis; and NameRxn reaction-class splits, which hold out complete classes (Grignard ester, Heck, Chloro Suzuki, Triflyloxy Suzuki, and all Suzuki) and contrast them with a baseline given 1,000 in-class examples, separating a class's intrinsic difficulty from its extrapolation difficulty. A diagnostic supplement computes fingerprint distances from each test reaction to its five nearest training neighbors in reactant space and in reaction-transform space, which distinguishes splits that shift the substrate distribution from splits that shift the transform distribution and correlates with the observed difficulty.
What would settle it
Re-run the All Suzuki holdout with an independent, mechanism-based classifier that identifies Suzuki couplings by bond changes and catalyst rather than by the vendor label; if the model still scores near 51% top-1 after every mechanistically equivalent reaction is provably absent from training, the extrapolation claim stands, whereas a sharp accuracy jump after purging mislabeled or unclassified examples would show the result was partly leakage.
Extended reading notes
Core claim
The central claim, on the paper's own terms, is that accuracy figures reported on standard in-distribution benchmarks are over-optimistic, and that a panel of out-of-distribution splits—by document, by author, by time, and by reaction class—gives a more faithful measure of how reaction predictors behave in deployment and whether they can extrapolate to novel chemistry. A BART-based encoder-decoder transformer, close in design to the Molecular Transformer, reaches 65% top-1 accuracy on a random reaction-level split of Pistachio patent data but only 58% on a document-level split and 55% on an author-level split. On time-based splits, accuracy declines as the extrapolation distance grows, yet a model trained only on reactions reported through 1996 still makes correct top-1 predictions some 25 years out; on a static held-out set of Buchwald–Hartwig reactions its top-1 accuracy climbs from 5% to above 60% as later training cutoffs add more in-class examples. On reaction-type splits, held-out Chloro Suzuki and Triflyloxy Suzuki classes are extrapolated with less than a 10-point drop (74% and 83% top-1), largely because other Suzuki reactions remain in training, whereas removing the whole Suzuki family cuts All Suzuki accuracy to 51%. Grignard ester and Heck holdouts are far harder (10% and 7% top-1), and the paper shows these failures are partly compositional and representational: iterating the Grignard double addition as two single steps raises accuracy from 4.5% to 49.7%, and ignoring stereochemistry and then regiochemistry raises the Heck extrapolation score from 7% to 42%.
Load-bearing premise
Every reaction-type result assumes NameRxn tagging is complete, so that removing the named classes plus all uncategorized reactions strips every training example of the held-out transform; if reactions that are mechanistically the same survive in training under another label, the reported extrapolation accuracies such as 51% top-1 on all Suzuki are inflated.
Editorial extensions
If this is right
- Random-split leaderboard numbers overstate deployment accuracy by roughly 7–10 top-1 points, so future reaction-prediction papers should report document- and author-level splits alongside them.
- Time-based evaluation bounds a model's useful lifetime: accuracy decays with the gap between training cutoff and test year, but the non-zero accuracy of a 1996-trained model decades later shows future reactions are partly predictable from past chemistry.
- Extrapolation to an unseen reaction class works when a sibling family stays in training (Chloro and Triflyloxy Suzuki) and largely fails when the whole family is removed (All Suzuki, 51% top-1), which means claims about reaction discovery must be tested with full-family holdouts.
- Much of the observed failure is recoverable: iterating the Grignard double-addition prediction as two single steps raises accuracy from 4.5% to 49.7%, and evaluation that ignores stereochemistry and then regiochemistry raises Heck extrapolation from 7% to 42%.
- The three split families answer different questions—retrospective deployment, prospective use, and reaction discovery—so they should be reported together rather than treated as interchangeable benchmarks.
Reading between the lines
- The split construction is author- and document-dense by design, so the 10-point gap probably understates the inflation of random splits on datasets with broader author diversity, such as academic literature or high-throughput experimentation collections.
- The Grignard double-addition result suggests a general design principle the paper leaves implicit: reaction predictors may gain more from a step-decomposition objective, in which the model predicts one elementary transform and feeds the product forward, than from more training data.
- The nearest-neighbor fingerprint distances offer a cheap pre-registration rule for future benchmarks: large reaction-transform distances should predict behavior like the Grignard ester, Heck, and All Suzuki cases, while large reactant distances should predict behavior like the document and author splits.
- A natural next experiment not performed here is to run the same split panel on graph-based or pretrained-language-model reaction predictors; the ordering of which classes extrapolate well could serve as a fingerprint of a model's inductive bias.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper evaluates a BART-based encoder-decoder transformer for reaction product prediction on a cleaned, deduplicated subset of the Pistachio patent dataset, comparing several out-of-distribution (OOD) split strategies against the standard random split. It reports that random splits overestimate top-1 accuracy (65%) relative to document-based (58%) and author-based (55%) splits; that time-based splits show accuracy declining with extrapolation distance from the training cutoff; and that reaction-type splits based on NameRxn classes give mixed evidence about zero-shot extrapolation, with Suzuki subtypes being easier than Grignard ester or Heck reactions. The authors further analyze why certain classes are harder, such as the need for multi-step reasoning in Grignard double additions and stereochemical/regiochemical sensitivity in Heck reactions, and they provide a fingerprint-distance analysis of the distribution shifts induced by each split type. The paper explicitly positions itself as a benchmark and evaluation study, releasing code for data cleaning and model training.
Significance. If the findings hold, the paper makes a useful and timely contribution by quantifying how much standard random splits overstate reaction-prediction performance and by proposing a suite of complementary OOD splits (document, author, time, reaction-type) that better reflect deployment. The careful dataset cleaning, the controlled training-size comparisons, and the fingerprint-based analysis of split-induced distribution shifts are strengths, as is the public release of code. The time-split protocol with multiple cutoffs and the Buchwald–Hartwig case study are particularly nice. However, the reaction-type extrapolation results—which underpin the paper's statements about reaction discovery—depend on the accuracy and completeness of NameRxn labels, and the paper does not validate those labels. The study also uses a single model and single training runs, so the quantitative accuracy gaps should be interpreted with appropriate caution.
major comments (2)
- [S1.4, Fig. 4, Table S2.4] The reaction-type splits, especially the 'All Suzuki' zero-shot evaluation, are not robust to NameRxn label noise. The paper removes only uncategorized reactions (NameRxn '0.0') before forming splits, explicitly acknowledging this is to 'avoid inadvertently training on reactions that might be similar to those that we are trying to exclude.' However, NameRxn covers only 74% of the cleaned dataset (stated in S1.4), so a Suzuki reaction mislabeled as, say, a generic cross-coupling or another named class would remain in the 1M-reaction training set. This could inflate the reported extrapolation accuracy (e.g., 51% top-1 for All Suzuki) and weaken the paper's conclusion that models possess some ability to extrapolate to unseen reaction types. The same issue affects the Buchwald–Hartwig test set in S1.3, which is constructed from NameRxn codes 1.3.1–1.3.4 and 1.9.43; BH reactions mislabeled under other codes could appear in the training sets of the time-split models. To support the central claims of Section 2.3 and Section 2.2.1, the authors should validate NameRxn assignments on a representative sample, or use an independent reaction classifier (e.g., template matching or mechanistic rules) to detect and remove reactions of the held-out type from the training set, then re-evaluate. A sensitivity analysis that relaxes the definition of the held-out class would also help establish how much the results depend on label completeness.
- [Section 2.1, Fig. 2, Table S2.2; Section 2.2, Fig. 3] The headline accuracy numbers are reported from single training runs with no uncertainty intervals. The gap between random, document, and author splits (65%, 58%, 55%) is large enough that it is probably real, but the time-split results in Figure 3A show substantial inter-year variability, and the paper itself attributes part of this to 'variability from the random selection of test sets.' Without multiple seeds, standard errors, or confidence intervals, it is difficult to assess whether smaller differences—for example between the document- and author-based splits, or between adjacent time cutoffs—are meaningful. Since the paper's central claim is that random splits overestimate performance, and since it makes quantitative claims such as 'the drop of ~10% accuracy,' the authors should provide at least bootstrap confidence intervals over the test set, or ideally repeat training with a few seeds for the main splits. This would not change the qualitative direction of the results, but it would make the reported magnitudes more interpretable.
minor comments (5)
- [Section 2.1, first paragraph] The phrase 'a model trained and evaluated on an on reactions split' is grammatically awkward and should be rephrased, e.g., 'a model trained and evaluated on a reaction-level (random) split.'
- [Figure 3A caption] The description of the dashed line is confusing: the caption says 'the dashed line indicates model performance when the model is extrapolating,' but in the figure the dashed line appears to be a boundary separating interpolative and extrapolative regions. Please clarify what the dashed line represents and how the reader is to interpret the region to its right.
- [Figure 5B and 5C] The small SMILES structures and the labels 'True' and 'False' are not self-explanatory; the reader has to infer that 'True' is the correct product and 'False' is an incorrect model prediction. A legend or a sentence in the caption would help.
- [S1.6] The reaction fingerprint is defined as the difference between product and reactant fingerprints, but the text then refers to 'cosine distance' between these difference vectors; it would be clearer to state explicitly that the cosine distance is computed on these difference vectors.
- [S1.4] The paper notes that the NameRxn system is not the only possible classification scheme and that different schemes may lead to different results; this is a good caveat, but it would be even more informative to include one such alternative (e.g., RXNO or template-based classes) for at least one held-out class, to test the sensitivity of the extrapolation conclusions to the choice of taxonomy.
Circularity Check
No significant circularity: the paper's claims are empirical measurements from explicitly defined dataset splits, with no fitted parameter renamed as a prediction and no load-bearing self-citation chain.
full rationale
The paper's central results are direct empirical comparisons: accuracies under reaction-, document-, and author-based splits (Section 2.1), time-based extrapolation curves (Section 2.2), and held-out reaction-class accuracies (Section 2.3). Each split is defined by external metadata or the NameRxn classification system, not by model predictions or by quantities fitted to the evaluation outcome. Hyperparameters are tuned on validation sets and held fixed for the OOD evaluations, which is standard practice and does not make the reported accuracies circular. The Buchwald–Hartwig and NameRxn test sets are constructed by selecting reactions with specified NameRxn codes, and the reported accuracies are simply measured on those sets; no step derives a prediction from its own input by construction. Self-citations (e.g., references to prior work by the same authors on generative models, retrosynthesis, or reaction discovery) appear only as background, tooling, or baseline context, and none is invoked as a uniqueness theorem or as the justification for a forbidden alternative. The reviewer-identified concern about NameRxn label noise potentially inflating the 'All Suzuki' zero-shot accuracy is a data-quality and external-validity limitation, not a circularity: the split definition does not assume the labels are correct, and the model does not contribute to those labels. Overall, the derivation chain is self-contained and the findings stand as empirical evidence independent of the paper's own assumptions.
Assumptions & free parameters
free parameters (4)
- Dataset cleaning thresholds =
Heavy atoms >=5; product heavy atoms >=2; token length <=800; etc.
- Split size budgets =
1M train, 100k test, 30k/10k/2k validation, 3k test per year, 1k baseline OOD sample
- Nearest-neighbor count k =
5
- Model hyperparameters =
Selected by HPO over 100 trials per task (e.g., learning rate 1e-5 to 1e-2, encoder/decoder layers 2-12)
assumptions (5)
- domain assumption Pistachio 2022Q4 US grants reactions and their metadata (author, document, year, NameRxn class) are a faithful sample of real reaction chemistry.
- domain assumption The cleaning and deduplication steps in S1.1 remove all identical reactions across training and test splits.
- domain assumption NameRxn class labels are accurate enough that removing a class and the uncategorized reactions removes essentially all training examples of that transform.
- domain assumption Morgan fingerprint cosine distance is a valid proxy for chemical and reaction similarity.
- domain assumption A single BART-style Transformer is representative of current reaction predictors for drawing qualitative conclusions.
Cite this review
Pith. "Pith review of Challenging reaction prediction models to generalize to novel chemistry." pith.science (2026). https://pith.science/paper/GHPD3SEF
@misc{pith2026250106669,
author = {Pith},
title = {Pith review of: Challenging reaction prediction models to generalize to novel chemistry},
year = {2026},
howpublished = {\url{https://pith.science/paper/GHPD3SEF}},
note = {Machine review of arXiv:2501.06669}
}
read the original abstract
Deep learning models for anticipating the products of organic reactions have found many use cases, including validating retrosynthetic pathways and constraining synthesis-based molecular design tools. Despite compelling performance on popular benchmark tasks, strange and erroneous predictions sometimes ensue when using these models in practice. The core issue is that common benchmarks test models in an in-distribution setting, whereas many real-world uses for these models are in out-of-distribution settings and require a greater degree of extrapolation. To better understand how current reaction predictors work in out-of-distribution domains, we report a series of more challenging evaluations of a prototypical SMILES-based deep learning model. First, we illustrate how performance on randomly sampled datasets is overly optimistic compared to performance when generalizing to new patents or new authors. Second, we conduct time splits that evaluate how models perform when tested on reactions published in years after those in their training set, mimicking real-world deployment. Finally, we consider extrapolation across reaction classes to reflect what would be required for the discovery of novel reaction types. This panel of tasks can reveal the capabilities and limitations of today's reaction predictors, acting as a crucial first step in the development of tomorrow's next-generation models capable of reaction discovery.
Figures
Figures from the paper (3 more)
Forward citations
Cited by 1 Pith paper
-
Retro-Rank-In: A Ranking-Based Approach for Inorganic Materials Synthesis Planning
A pairwise ranking model over a shared embedding space outperforms prior fixed-vocabulary classifiers for inorganic retrosynthesis, especially when ranking many candidate precursor sets.
Reference graph
Works this paper leans on
-
[1]
Reagent prediction with a molecular transformer improves reaction data quality
Mikhail Andronov, Varvara Voinarovska, Natalia Andronova, Michael Wand, Djork-Arné Clevert, and Jürgen Schmidhuber. Reagent prediction with a molecular transformer improves reaction data quality. Chemical Science, 14(12):3235–3246, 2023. doi:10.1039/D2SC06798F
-
[2]
ASKCOS (Automated System for Knowledge-based Continuous Organic Synthesis), 2019
ASKCOS Team. ASKCOS (Automated System for Knowledge-based Continuous Organic Synthesis), 2019. URL https://askcos.mit.edu/
2019
-
[3]
Learning to Split for Automatic Bias Detection
Yujia Bao and Regina Barzilay. Learning to split for automatic bias detection. arXiv [cs.LG] , 2022. doi:10.48550/arXiv.2204.13749
work page Pith review arXiv doi:10.48550/arxiv.2204.13749 2022
-
[4]
Non-autoregressive electron redistribution modeling for reaction prediction
Hangrui Bi, Hengyi Wang, Chence Shi, Connor W Coley, Jian Tang, and Hongyu Guo. Non-autoregressive electron redistribution modeling for reaction prediction. In Proceedings of the 38th International Conference on Machine Learning, volume 139, pages 904–913. PMLR, 2021
2021
-
[5]
Discovery of novel chemical reactions by deep generative recurrent neural network
William Bort, Igor I Baskin, Timur Gimadiev, Artem Mukanov, Ramil Nugmanov, Pavel Sidorov, Gilles Marcou, Dragos Horvath, Olga Klimchuk, Timur Madzhidov, and Alexandre Varnek. Discovery of novel chemical reactions by deep generative recurrent neural network. Scientific reports, 11(1):3178, 2021. doi:10.1038/s41598- 021-81889-y
doi:10.1038/s41598- 2021
-
[6]
A generative model for electron paths
John Bradshaw, Matt J Kusner, Brooks Paige, Marwin H S Segler, and José Miguel Hernández-Lobato. A generative model for electron paths. In International Conference on Learning Representations 2019, 2019
2019
-
[7]
A model to search for synthesizable molecules
John Bradshaw, Brooks Paige, Matt J Kusner, Marwin H S Segler, and José Miguel Hernández-Lobato. A model to search for synthesizable molecules. In Advances in Neural Information Processing Systems 32, pages 7937–7949. Curran Associates, Inc., 2019
2019
-
[8]
Barking up the right tree: an approach to search over molecule synthesis DAGs
John Bradshaw, Brooks Paige, Matt J Kusner, Marwin H S Segler, and José Miguel Hernández-Lobato. Barking up the right tree: an approach to search over molecule synthesis DAGs. In Advances in Neural Information Processing Systems 33. Curran Associates Inc., 2020
2020
Show all 108 references
-
[9]
Analysis of the reactions used for the preparation of drug candidate molecules
John S Carey, David Laffan, Colin Thomson, and Mike T Williams. Analysis of the reactions used for the preparation of drug candidate molecules. Organic & biomolecular chemistry , 4(12):2337–2347, 2006. doi:10.1039/b602413k
2006 doi
- [10]
- [11]
-
[12]
SCUBIDOO: A large yet screenable and easily searchable database of computationally created chemical compounds optimized toward high likelihood of synthetic tractability
F Chevillard and P Kolb. SCUBIDOO: A large yet screenable and easily searchable database of computationally created chemical compounds optimized toward high likelihood of synthetic tractability. Journal of chemical information and modeling, 55(9):1824–1835, 2015. doi:10.1021/a...
2015 doi
-
[13]
ChemBERTa: Large-scale self-supervised pretraining for molecular property prediction
Seyone Chithrananda, Gabriel Grand, and Bharath Ramsundar. ChemBERTa: Large-scale self-supervised pretraining for molecular property prediction. In Machine Learning for Molecules Workshop at NeurIPS 2020,
2020
-
[14]
A graph-convolutional neural network model for the prediction of chemical reactivity
Connor W Coley, Wengong Jin, Luke Rogers, Timothy F Jamison, Tommi S Jaakkola, William H Green, Regina Barzilay, and Klavs F Jensen. A graph-convolutional neural network model for the prediction of chemical reactivity. Chemical science, 10(2):370–377, 2019. doi:10.1039/C8SC04228D
2019 doi
-
[15]
Graph transformation policy network for chemical reaction prediction
Kien Do, Truyen Tran, and Svetha Venkatesh. Graph transformation policy network for chemical reaction prediction. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, page 750–760, 2019. doi:10.1145/3292500.3330958
2019
-
[16]
Deep learning for chemical reaction prediction.Molecular Systems Design & Engineering, 3(3):442–452, 2018
David Fooshee, Aaron Mood, Eugene Gutman, Mohammadamin Tavakoli, Gregor Urban, Frances Liu, Nancy Huynh, David Van Vranken, and Pierre Baldi. Deep learning for chemical reaction prediction.Molecular Systems Design & Engineering, 3(3):442–452, 2018. doi:10.1039/C7ME00107J
2018 doi
-
[17]
The 25th anniversary of the Buchwald–Hartwig amination: Development, applications, and outlook
Paola A Forero-Cortés and Alexander M Haydl. The 25th anniversary of the Buchwald–Hartwig amination: Development, applications, and outlook. Organic process research & development , 23(8):1478–1483, 2019. doi:10.1021/acs.oprd.9b00161
2019 doi
-
[18]
Amortized tree generation for bottom-up synthesis planning and synthesizable molecular design
Wenhao Gao, Rocío Mercado, and Connor W Coley. Amortized tree generation for bottom-up synthesis planning and synthesizable molecular design. In International Conference on Learning Representations 2022, 2022
2022
-
[19]
Holistic chemical evaluation reveals pitfalls in reaction prediction models
Victor Sabanza Gil, Andrés M Bran, Malte Franke, Remi Schlama, J Luterbacher, and Philippe Schwaller. Holistic chemical evaluation reveals pitfalls in reaction prediction models. arXiv [physics.chem-ph], 2023. doi:10.48550/arXiv.2312.09004
-
[20]
Deep Learning
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. The MIT Press, 2016
2016
- [21]
-
[22]
Reaction prediction and synthesis design
Jonathan M Goodman. Reaction prediction and synthesis design. In Applied Chemoinformatics, pages 86–105. Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim, Germany, 2018. doi:10.1002/9783527806539.ch4b
2018 doi
-
[23]
Learning to navigate the synthetically accessible chemical space using reinforcement learning
Sai Krishna Gottipati, Boris Sattarov, Sufeng Niu, Yashaswi Pathak, Haoran Wei, Shengchao Liu, Karam M J Thomas, Simon Blackburn, Connor W Coley, Jian Tang, Sarath Chandar, and Yoshua Bengio. Learning to navigate the synthetically accessible chemical space using reinforcement ...
2020
-
[24]
Dataset bias in the natural sciences: A case study in chemical reaction prediction and synthesis design
Ryan-Rhys Griffiths, Philippe Schwaller, and Alpha Lee. Dataset bias in the natural sciences: A case study in chemical reaction prediction and synthesis design. ChemRxiv, 2018. doi:10.26434/chemrxiv.7366973.v1
2018 doi
-
[25]
In search of lost domain generalization
Ishaan Gulrajani and David Lopez-Paz. In search of lost domain generalization. In International Conference on Learning Representations 2020, 2020
2020
-
[26]
Palladium-catalyzed aromatic aminations with in situ generated aminostannanes
Anil S Guram and Stephen L Buchwald. Palladium-catalyzed aromatic aminations with in situ generated aminostannanes. Journal of the American Chemical Society, 116(17):7901–7902, 1994. doi:10.1021/ja00096a059
1994 doi
-
[27]
A baseline for detecting misclassified and out-of-distribution examples in neural networks
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In International Conference on Learning Representations 2017, 2017
2017
-
[28]
Reaction planning: prediction of new organic reactions
Rainer Herges. Reaction planning: prediction of new organic reactions. Journal of Chemical Information and Computer Sciences, 30(4):377–383, 1990. doi:10.1021/ci00068a006
1990 doi
-
[29]
Molecular design in synthetically accessible chemical space via deep reinforcement learning
Julien Horwood and Emmanuel Noutahi. Molecular design in synthetically accessible chemical space via deep reinforcement learning. ACS omega, 5(51):32984–32994, 2020. doi:10.1021/acsomega.0c04153. 12
2020 doi
-
[30]
LoRA: Low-rank adaptation of large language models
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Represen- tations 2022, 2022
2022
-
[31]
Chemformer: A pre-trained transformer for computational chemistry
Ross Irwin, Spyridon Dimitriadis, Jiazhen He, and Esben Jannik Bjerrum. Chemformer: A pre-trained transformer for computational chemistry. Machine Learning: Science and Technology , 3(1):015022, 2022. doi:10.1088/2632-2153/ac3ffb
2022 doi
-
[32]
Transformer performance for chemical reactions: Analysis of different predictive and evaluation scenarios
Fernando Jaume-Santero, Alban Bornet, Alain Valery, Nona Naderi, David Vicente Alvarez, Dimitrios Proios, Anthony Yazdani, Colin Bournez, Thomas Fessard, and Douglas Teodoro. Transformer performance for chemical reactions: Analysis of different predictive and evaluation scenar...
1914 doi
-
[33]
Predicting organic reaction outcomes with Weisfeiler-Lehman network
Wengong Jin, Connor W Coley, Regina Barzilay, and Tommi Jaakkola. Predicting organic reaction outcomes with Weisfeiler-Lehman network. In Advances in Neural Information Processing Systems 30, pages 2607–2616. Curran Associates, Inc., 2017
2017
-
[34]
Latent biases in machine learning models for predicting binding affinities using popular data sets
Ganesh Chandan Kanakala, Rishal Aggarwal, Divya Nayar, and U Deva Priyakumar. Latent biases in machine learning models for predicting binding affinities using popular data sets. ACS omega, 8(2):2389–2397, 2023. doi:10.1021/acsomega.2c06781
2023 doi
-
[35]
ReactionPredictor: prediction of complex chemical reactions at the mechanistic level using machine learning
Matthew A Kayala and Pierre Baldi. ReactionPredictor: prediction of complex chemical reactions at the mechanistic level using machine learning. Journal of chemical information and modeling, 52(10):2526–2540,
-
[36]
Pursuing a prospective perspective
Steven Kearnes. Pursuing a prospective perspective. Trends in chemistry , 3(2):77–79, 2021. doi:10.1016/j.trechm.2020.10.012
2021 doi
-
[37]
WILDS: A benchmark of in-the-wild distribution shifts
Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, Tony Lee, Etienne David, Ian Stavness, Wei Guo, Berton A Earnshaw, Imran S Haque, Sara Beery, Jure Leskovec, Ans...
2021
-
[38]
ChemBO: Bayesian optimization of small organic molecules with synthesizable recommendations
Ksenia Korovina, Sailun Xu, Kirthevasan Kandasamy, Willie Neiswanger, Barnabas Poczos, Jeff Schneider, and Eric P Xing. ChemBO: Bayesian optimization of small organic molecules with synthesizable recommendations. In Proceedings of the Twenty Third International Conference on A...
2020
-
[39]
Quantitative interpretation explains machine learning models for chemical reaction prediction and uncovers bias
Dávid Péter Kovács, William McCorkindale, and Alpha A Lee. Quantitative interpretation explains machine learning models for chemical reaction prediction and uncovers bias. Nature communications, 12(1):1695, 2021. doi:10.1038/s41467-021-21895-w
2021 doi
-
[40]
NameRxn: More than just a reaction classifier
Ingvar Lagerstedt, John Mayfield, and Roger Sayle. NameRxn: More than just a reaction classifier. ACS Fall 2021,
2021
-
[41]
Simple and scalable predictive uncertainty estimation using deep ensembles
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems 30, pages 6402–6413. Curran Associates, Inc., 2017
2017
-
[42]
SIMPD: An algorithm for generating simulated time splits for validating machine learning approaches
Gregory A Landrum, Maximilian Beckers, Jessica Lanini, Nadine Schneider, Nikolaus Stiefl, and Sereina Riniker. SIMPD: An algorithm for generating simulated time splits for validating machine learning approaches. Journal of cheminformatics, 15(1):119, 2023. doi:10.1186/s13321-0...
2023 doi
-
[43]
Handwritten digit recognition with a back-propagation network
Yann LeCun, Bernhard E Boser, John S Denker, Donnie Henderson, R E Howard, Wayne E Hubbard, and Lawrence D Jackel. Handwritten digit recognition with a back-propagation network. In Advances in Neural Information Processing Systems 2, pages 396–404. Morgan-Kaufmann, 1990. 13
1990
-
[44]
BART: Denoising sequence-to-sequence pre-training for natural language genera- tion, translation, and comprehension
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoy- anov, and Luke Zettlemoyer. BART: Denoising sequence-to-sequence pre-training for natural language genera- tion, translation, and comprehension. In Proceedings of the 58th Annu...
2020 doi
-
[45]
A system for massively parallel hyperparameter tuning
Liam Li, Kevin Jamieson, Afshin Rostamizadeh, Ekaterina Gonina, Jonathan Ben-tzur, Moritz Hardt, Benjamin Recht, and Ameet Talwalkar. A system for massively parallel hyperparameter tuning. In Proceedings of Machine Learning and Systems, volume 2, pages 230–246, 2020
2020
-
[46]
Decoupled weight decay regularization
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations 2019, 2019
2019
-
[47]
Extraction of chemical structures and reactions from the literature
Daniel Mark Lowe. Extraction of chemical structures and reactions from the literature. PhD thesis, University of Cambridge, 2012. URL http://dx.doi.org/10.17863/CAM.16293
2012 doi
-
[48]
Ideation and evaluation of novel multicomponent reactions via mechanistic network analysis and automation
Babak Mahjour, Juncheng Lu, Jenna Fromer, Nicholas Casetti, and Connor Coley. Ideation and evaluation of novel multicomponent reactions via mechanistic network analysis and automation. ChemRxiv, 2024. doi:10.26434/chemrxiv-2024-qfjh9-v3
2024 doi
-
[49]
Pistachio - search and faceting of large reaction databases
John Mayfield, Daniel Lowe, and Roger Sayle. Pistachio - search and faceting of large reaction databases. ACS Fall 2017, 2017. URL https://nextmovesoftware.com/talks/Mayfield_Pistachio_NIHReactions_202105. pdf
2017
-
[50]
Doubly stochastic graph-based non-autoregressive reaction prediction
Ziqiao Meng, Peilin Zhao, Yang Yu, and Irwin King. Doubly stochastic graph-based non-autoregressive reaction prediction. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence Main Track., pages 4064–4072, 2023. doi:10.24963/ijcai.2023/452
2023 doi
-
[51]
Ray: A distributed framework for emerging AI applications
Philipp Moritz, Robert Nishihara, Stephanie Wang, Alexey Tumanov, Richard Liaw, Eric Liang, Melih Elibol, Zongheng Yang, William Paul, Michael I Jordan, and Ion Stoica. Ray: A distributed framework for emerging AI applications. In 13th USENIX Symposium on Operating Systems Des...
2018
-
[52]
Pistachio
NextMove Software. Pistachio. https://www.nextmovesoftware.com/pistachio.html, 2021. Accessed: 2021-11-18
2021
-
[53]
NameRxn (expert system for named reaction identification and classification), 2022
NextMove Software. NameRxn (expert system for named reaction identification and classification), 2022. URL https://www.nextmovesoftware.com/namerxn.html
2022
-
[54]
Palladium-catalyzed formation of carbon-nitrogen bonds
Frederic Paul, Joe Patt, and John F Hartwig. Palladium-catalyzed formation of carbon-nitrogen bonds. reaction intermediates and catalyst improvements in the hetero cross-coupling of aryl halides and tin amides. Journal of the American Chemical Society, 116(13):5969–5970, 1994....
1994 doi
-
[55]
Transfer learning enables the molecular transformer to predict regio- and stereoselective reactions on carbohydrates.Nature communications, 11(1):4874, 2020
Giorgio Pesciullesi, Philippe Schwaller, Teodoro Laino, and Jean-Louis Reymond. Transfer learning enables the molecular transformer to predict regio- and stereoselective reactions on carbohydrates.Nature communications, 11(1):4874, 2020. doi:10.1038/s41467-020-18671-7
2020 doi
-
[56]
Elements of Causal Inference
Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of Causal Inference. The MIT Press, 2017
2017
-
[57]
MIT Press, 2008
Joaquin Quiñonero-Candela, Masashi Sugiyama, Anton Schwaighofer, and Neil D Lawrence, editors.Dataset Shift in Machine Learning. MIT Press, 2008
2008
-
[58]
RDKit: Open-source cheminformatics, 2021
RDKit Team. RDKit: Open-source cheminformatics, 2021. URL http://www.rdkit.org
2021
-
[59]
The medicinal chemist’ s toolbox: An analysis of reactions used in the pursuit of drug candidates
Stephen D Roughley and Allan M Jordan. The medicinal chemist’ s toolbox: An analysis of reactions used in the pursuit of drug candidates. Journal of medicinal chemistry, 54(10):3451–3479, 2011. doi:10.1021/jm200187y
2011 doi
-
[60]
RXNO: reaction ontologies, 2012
RSC. RXNO: reaction ontologies, 2012. URL https://github.com/rsc-ontologies/rxno. 14
2012
-
[61]
Molecule edit graph attention network: Modeling chemical reactions as sequences of graph edits
Mikołaj Sacha, Mikołaj Bła˙z, Piotr Byrski, Paweł D ˛ abrowski-Tuma ´ nski, Mikołaj Chromi ´ nski, Rafał Loska, Paweł Włodarczyk-Pruszy ´ nski, and Stanisław Jastrz˛ ebski. Molecule edit graph attention network: Modeling chemical reactions as sequences of graph edits. Journal ...
2021 doi
-
[62]
BREEDS: Benchmarks for subpopulation shift
Shibani Santurkar, Dimitris Tsipras, and Aleksander Madry. BREEDS: Benchmarks for subpopulation shift. In International Conference on Learning Representations 2021, 2021
2021
-
[63]
Big data from pharmaceutical patents: A computational analysis of medicinal chemists’ bread and butter.Journal of medicinal chemistry, 59(9):4385–4402, 2016
Nadine Schneider, Daniel M Lowe, Roger A Sayle, Michael A Tarselli, and Gregory A Landrum. Big data from pharmaceutical patents: A computational analysis of medicinal chemists’ bread and butter.Journal of medicinal chemistry, 59(9):4385–4402, 2016. doi:10.1021/acs.jmedchem.6b00153
2016 doi
-
[64]
Found in translation
Philippe Schwaller, Theophile Gaudin, David Lanyi, Costas Bekas, and Teodoro Laino. “Found in translation”: Predicting outcomes of complex organic chemistry reactions using neural sequence-to-sequence models. Chemical science, 9(28):6091–6098, 2018. doi:10.1039/C8SC02339E
2018 doi
-
[65]
Molecular transformer: A model for uncertainty-calibrated chemical reaction prediction
Philippe Schwaller, Teodoro Laino, Théophile Gaudin, Peter Bolgar, Christopher A Hunter, Costas Bekas, and Alpha A Lee. Molecular transformer: A model for uncertainty-calibrated chemical reaction prediction. ACS Central Science, 5(9):1572–1583, 2019. doi:10.1021/acscentsci.9b00576
2019 doi
-
[66]
Predicting retrosynthetic pathways using transformer- based models and a hyper-graph exploration strategy
Philippe Schwaller, Riccardo Petraglia, Valerio Zullo, Vishnu H Nair, Rico Andreas Haeuselmann, Riccardo Pisoni, Costas Bekas, Anna Iuliano, and Teodoro Laino. Predicting retrosynthetic pathways using transformer- based models and a hyper-graph exploration strategy. Chemical s...
2010 doi
-
[67]
On causal and anticausal learning
Bernhard Schölkopf, Dominik Janzing, Jonas Peters, Eleni Sgouritsa, Kun Zhang, and Joris Mooij. On causal and anticausal learning. In Proceedings of the 29th International Conference on Machine Learning, pages 1255–1262. Omnipress, 2012
2012
-
[68]
Modelling chemical reasoning to predict and invent reactions.Chemistry, 23(25):6118–6128, 2017
Marwin H S Segler and Mark P Waller. Modelling chemical reasoning to predict and invent reactions.Chemistry, 23(25):6118–6128, 2017. doi:10.1002/chem.201604556
2017 doi
-
[69]
Planning chemical syntheses with deep neural networks and symbolic AI
Marwin H S Segler, Mike Preuss, and Mark P Waller. Planning chemical syntheses with deep neural networks and symbolic AI. Nature, 555(7698):604–610, 2018. doi:10.1038/nature25978
2018 doi
-
[70]
Improving few- and zero-shot reaction template prediction using modern Hopfield networks
Philipp Seidl, Philipp Renz, Natalia Dyubankova, Paulo Neves, Jonas Verhoeven, Jörg K Wegner, Marwin Segler, Sepp Hochreiter, and Günter Klambauer. Improving few- and zero-shot reaction template prediction using modern Hopfield networks. Journal of chemical information and mod...
2022 doi
-
[71]
Time-split cross-validation as a method for estimating the goodness of prospective predic- tion
Robert P Sheridan. Time-split cross-validation as a method for estimating the goodness of prospective predic- tion. Journal of chemical information and modeling, 53(4):783–790, 2013. doi:10.1021/ci400084k
2013 doi
-
[72]
Improving predictive inference under covariate shift by weighting the log-likelihood function
Hidetoshi Shimodaira. Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of statistical planning and inference, 90(2):227–244, 2000. doi:10.1016/s0378-3758(00)00115-4
-
[73]
Beyond the imitation game: Quantifying and extrapolating the capabilities of language models
Aarohi Srivastava and others. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Transactions on Machine Learning Research, 2023
2023
-
[74]
Lo-Hi: Practical ML drug discovery benchmark
Simon Steshin. Lo-Hi: Practical ML drug discovery benchmark. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023
2023
-
[75]
Reproducing the invention of a named reaction: zero-shot prediction of unseen chemical reactions
An Su, Xinqiao Wang, Ling Wang, Chengyun Zhang, Yejian Wu, Xinyi Wu, Qingjie Zhao, and Hongliang Duan. Reproducing the invention of a named reaction: zero-shot prediction of unseen chemical reactions. Physical chemistry chemical physics: PCCP, 24(17):10280–10291, 2022. doi:10....
2022 doi
-
[76]
Machine Learning in Non-Stationary Environments: Introduction to Covariate Shift Adaptation
Masashi Sugiyama and Kawanabe Motoaki. Machine Learning in Non-Stationary Environments: Introduction to Covariate Shift Adaptation. MIT press, 2012. 15
2012
-
[77]
Generative AI for designing and validating easily synthesizable and structurally novel antibiotics
Kyle Swanson, Gary Liu, Denise B Catacutan, Autumn Arnold, James Zou, and Jonathan M Stokes. Generative AI for designing and validating easily synthesizable and structurally novel antibiotics. Nature machine intelligence, 6(3):338–353, 2024. doi:10.1038/s42256-024-00809-7
2024 doi
- [78]
-
[79]
State-of-the-art augmented NLP transformer models for direct and single-step retrosynthesis
Igor V Tetko, Pavel Karpov, Ruud Van Deursen, and Guillaume Godin. State-of-the-art augmented NLP transformer models for direct and single-step retrosynthesis. Nature communications, 11(1):5575, 2020. doi:10.1038/s41467-020-19266-y
2020 doi
-
[80]
Unas- sisted noise reduction of chemical reaction data sets
Alessandra Toniato, Philippe Schwaller, Antonio Cardinale, Joppe Geluykens, and Teodoro Laino. Unas- sisted noise reduction of chemical reaction data sets. Nature Machine Intelligence , 3(6):485–494, 2021. doi:10.1038/s42256-021-00319-w
2021 doi
-
[81]
Fast customization of chemical language models to out-of-distribution data sets
Alessandra Toniato, Alain C Vaucher, Marzena Maria Lehmann, Torsten Luksch, Philippe Schwaller, Marco Stenta, and Teodoro Laino. Fast customization of chemical language models to out-of-distribution data sets. Chemistry of materials: a publication of the American Chemical Soci...
2023 doi
-
[82]
Real-world molecular out-of-distribution: Specification and investigation
Prudencio Tossou, Cas Wognum, Michael Craig, Hadrien Mary, and Emmanuel Noutahi. Real-world molecular out-of-distribution: Specification and investigation. Journal of chemical information and modeling , 2024. doi:10.1021/acs.jcim.3c01774
2024 doi
-
[83]
Permutation Invariant Graph-to-Sequence Model for Template-Free Ret- rosynthesis and Reaction Prediction
Zhengkai Tu and Connor W Coley. Permutation Invariant Graph-to-Sequence Model for Template-Free Ret- rosynthesis and Reaction Prediction. Journal of chemical information and modeling, 62(15):3503–3513, 2022. doi:10.1021/acs.jcim.2c00321
2022 doi
-
[84]
Predictive chemistry: machine learning for reac- tion deployment, reaction development, and reaction discovery
Zhengkai Tu, Thijs Stuyver, and Connor W Coley. Predictive chemistry: machine learning for reac- tion deployment, reaction development, and reaction discovery. Chemical science , 14(2):226–244, 2023. doi:10.1039/d2sc05089g
2023 doi
-
[85]
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems 30, pages 5998–6008. Curran Associates, Inc., 2017
2017
-
[86]
SYNOPSIS: SYNthesize and OPtimize system in silico
H Maarten Vinkers, Marc R de Jonge, Frederik F D Daeyaert, Jan Heeres, Lucien M H Koymans, Joop H van Lenthe, Paul J Lewi, Henk Timmerman, Koen Van Aken, and Paul A J Janssen. SYNOPSIS: SYNthesize and OPtimize system in silico. Journal of medicinal chemistry, 46(13):2765–2773,...
2003 doi
-
[87]
ChemistGA: A chemical synthesizable accessible molecular generation algorithm for real-world drug discovery
Jike Wang, Xiaorui Wang, Huiyong Sun, Mingyang Wang, Yundian Zeng, Dejun Jiang, Zhenxing Wu, Zeyi Liu, Ben Liao, Xiaojun Yao, Chang-Yu Hsieh, Dongsheng Cao, Xi Chen, and Tingjun Hou. ChemistGA: A chemical synthesizable accessible molecular generation algorithm for real-world d...
2022 doi
-
[88]
Heck reaction prediction using a transformer model based on a transfer learning strategy
Ling Wang, Chengyun Zhang, Renren Bai, Jianjun Li, and Hongliang Duan. Heck reaction prediction using a transformer model based on a transfer learning strategy. Chemical communications (Cambridge, England), 56 (65):9368–9371, 2020. doi:10.1039/d0cc02657c
2020 doi
-
[89]
From theory to experiment: transformer-based generation enables rapid discovery of novel reactions
Xinqiao Wang, Chuansheng Yao, Yun Zhang, Jiahui Yu, Haoran Qiao, Chengyun Zhang, Yejian Wu, Renren Bai, and Hongliang Duan. From theory to experiment: transformer-based generation enables rapid discovery of novel reactions. Journal of cheminformatics, 14(1):60, 2022. doi:10.11...
2022 doi
-
[90]
A short review of chemical reaction database systems, computer-aided synthesis design, reaction prediction and synthetic feasibility
Wendy A Warr. A short review of chemical reaction database systems, computer-aided synthesis design, reaction prediction and synthetic feasibility. Molecular informatics, 33(6-7):469–476, 2014. doi:10.1002/minf.201400052. 16
2014 doi
-
[91]
Neural networks for the prediction of organic chemistry reactions
Jennifer N Wei, David Duvenaud, and Alán Aspuru-Guzik. Neural networks for the prediction of organic chemistry reactions. ACS Central Science, 2(10):725–732, 2016. doi:10.1021/acscentsci.6b00219
2016 doi
-
[92]
ORDerly: Data sets and benchmarks for chemical reaction data
Daniel S Wigh, Joe Arrowsmith, Alexander Pomberger, Kobi C Felton, and Alexei A Lapkin. ORDerly: Data sets and benchmarks for chemical reaction data. Journal of chemical information and modeling, 64(9):3790–3798,
-
[93]
HuggingFace’ s transformers: State-of-the-art natural language processing.arXiv [cs.CL],
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...
-
[94]
MoleculeNet: a benchmark for molecular machine learning
Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S Pappu, Karl Leswing, and Vijay Pande. MoleculeNet: a benchmark for molecular machine learning. Chemical science, 9(2): 513–530, 2018. doi:10.1039/c7sc02664a
2018 doi
-
[95]
OpenOOD: Benchmarking generalized out-of-distribution detection
Jingkang Yang, Pengyun Wang, Dejian Zou, Zitang Zhou, Kunyuan Ding, Wenxuan Peng, Haoqi Wang, Guangyao Chen, Bo Li, Yiyou Sun, Xuefeng Du, Kaiyang Zhou, Wayne Zhang, Dan Hendrycks, Yixuan Li, and Ziwei Liu. OpenOOD: Benchmarking generalized out-of-distribution detection. In Th...
2022
-
[96]
RetroOOD: Understanding out-of-distribution generalization in retrosynthesis prediction
Yemin Yu, Luotian Yuan, Ying Wei, Hanyu Gao, Xinhai Ye, Zhihua Wang, and Fei Wu. RetroOOD: Understanding out-of-distribution generalization in retrosynthesis prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 374–382, 2024. doi:10.16...
2024 doi
-
[97]
Retrosynthesis prediction using an end-to-end graph gen- erative architecture for molecular graph editing
Weihe Zhong, Ziduo Yang, and Calvin Yu-Chian Chen. Retrosynthesis prediction using an end-to-end graph gen- erative architecture for molecular graph editing. Nature communications, 14(1):3009, 2023. doi:10.1038/s41467- 023-38851-5. 17 Appendices S1 Methods In this section we d...
2023 doi
-
[103]
the reactants have fewer than 5 heavy atoms,
-
[104]
the reactants contain no carbon atoms,
-
[105]
none of the reactants have at least two bonds,
-
[106]
the reaction is easily identifiable as a (de)protonation (we neutralize commonly occurring charged atoms in the reactants and products using a SMARTS pattern and, having done so, see if the sets of reactants and products are then equal),
-
[107]
all products not already present in the reactant set contain fewer than 2 heavy atoms,
-
[108]
author-document-dense
the reaction is very long when tokenized 2 for our language model–based reaction predictor (e.g., over 800 tokens long). 1See https://docs.chemaxon.com/display/docs/formats_chemaxon-extended-smiles-and-smarts-cxsmiles-and-cxsmarts. md 2Details on the tokenizer we use can be fo...
-
[2012]
doi:10.1021/ci3003039
- [2019]
- [2020]
-
[2021]
URL https://www.nextmovesoftware.com/talks/ACS_2021_Fall_Lagerstedt_Namerxn.pdf
-
[2024]
doi:10.1021/acs.jcim.4c00292
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.