{"id":"bc306bef-548d-49b1-86a0-8c38884b80dd","arxiv_id":"2507.21762","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"TempRe shows that autoregressively generating reaction templates, and even whole synthesis routes as template sequences, beats template classification and SMILES generation on single-step and multi-step retrosynthesis benchmarks.","lead":"TempRe recasts retrosynthesis as generation of reaction templates (SMARTS strings) rather than classification or direct reactant generation, and extends this to generating entire multi-step routes as template sequences. A generalist should read it because it shows a lightweight alternative that combines chemical validity with scalability in computer-aided synthesis planning.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Top-k accuracy counts any RDChiral-applicable template as correct, so chemically implausible but precursor-recovering templates could inflate TempRe's headline advantage over P2R.","rationale":"The most load-bearing condition for the central claim is not only that templates are extracted and applied correctly, but that the metric used to demonstrate superiority actually measures chemical plausibility. The paper's own appendix concedes that generated templates can be chemically invalid, yet the evaluation pipeline has no filter for this; RDChiral application and precursor matching are sufficient for a correct count. This is a concrete, testable failure mode: an unrestricted sequence model under beam search could learn high-frequency, generic disconnection patterns that recover the exact precursors for many products without corresponding to real chemistry. Such behavior would inflate the non-strict P2T numbers without improving practical synthesis planning. The concern is load-bearing because the abstract's 'chemically plausible' promise and the claim that TempRe 'outperforms' P2R both rest on these accuracy numbers; if a substantial share of correct predictions are implausible, the headline should be reframed. The proposed audit is cheap and would either confirm the current interpretation or expose the conflation. I partially agree with the reader's weakest assumption: RDChiral fidelity matters, but the unmeasured plausibility of generated templates is an independent and more direct threat to the central claim. The final verdict remains CONDITIONAL: the framework may still be useful, but the 'chemically plausible' wording and the unrestricted-model comparisons require the additional evidence.","tokens_in":17535,"tokens_out":11919,"duration_ms":160737,"concrete_test":"Sample 200 Top-1-correct predictions of P2T (non-strict) on the PaRoutes single-step test set and 200 on the hard test set. Have two independent chemistry experts (or a validated automatic filter plus manual review) classify each RDChiral-applied template as chemically plausible or not. Compute the plausible fraction among correct predictions. If it is below 90%, rerun the headline Top-k comparison after discarding implausible templates; a material drop in P2T's accuracy relative to P2R would show that the 'chemically plausible' claim and part of the performance gap are artifacts of the precursor-only evaluation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim promises 'chemically plausible' retrosynthesis, but the evaluation in Section 3.6 measures only whether predicted templates, applied by RDChiral, reproduce the ground-truth precursor set; syntactically valid SMARTS that apply to the product are counted even if the transformation is chemically nonsensical. Appendix A.6.1 explicitly shows that P2T generates chemically implausible novel templates (Figure A.3c), yet the paper never quantifies how many of the correct Top-k predictions rely on such templates. If that fraction is substantial, the reported 96% Top-80 accuracy on PaRoutes, the hard-set gains, and the OOD robustness of P2T reflect template-conditioned precursor retrieval rather than chemically meaningful transformations. The concern is sharpest for the unrestricted P2T/P2T-Tok models, which are the basis of the flexibility and novel-template claims; the Strict variants sidestep the issue by construction, but they are not the models used to argue for generalization to rare or out-of-distribution reactions. The case study criticizes P2R for implausible 'shortcut' routes but does not perform an equivalent audit of TempRe's own top-ranked templates, so the symmetry of the critique is untested.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces TempRe, a generative framework for retrosynthesis that reformulates template-based approaches as sequence generation: a Transformer takes a product SMILES as input and autoregressively generates a reaction template (SMARTS), which is then applied with RDChiral to obtain precursors. The authors evaluate TempRe variants (P2T, P2T-Tok, and 'strict' versions that filter to training-set templates) on single-step top-k accuracy using the PaRoutes test sets, a newly constructed 'hard' test set enriched for rare templates, and a molecular-weight-based out-of-distribution split. They also integrate TempRe models as policies in an MCTS planner for multi-step retrosynthesis, and train a 'Direct TempRe' model that generates entire synthetic routes as sequences of templates. The central claims are that TempRe models outperform SMILES-based product-to-reactant (P2R) models across the board, that template generation is more robust to large out-of-distribution molecules, that a strict TempRe policy achieves strong route accuracy on PaRoutes, and that direct template-sequence route generation is a viable lightweight alternative to search-based planning.","tokens_in":17839,"tokens_out":6948,"duration_ms":85658,"significance":"If the empirical claims hold, the paper makes a useful contribution to computer-aided synthesis planning by showing that reaction templates can be treated as a generative modeling target rather than as a fixed classification library. The experimental design is careful in several respects: test reactions are removed from training, a hard set and an OOD split are constructed, a data-filtering ablation is reported, and the Direct TempRe-DMS comparison honestly acknowledges reaction-level leakage in the DMS split. The paper also provides a concrete case study illustrating qualitative differences between template-based and template-free route proposals. However, the headline superiority claims rest on single-seed runs without error bars, and the central notion of 'chemically plausible' retrosynthesis is not directly measured by the reported top-k accuracy, since any RDChiral-applicable template that recovers the ground-truth precursor set is counted as correct. These issues are load-bearing for the paper's main conclusions and require additional analysis before the claims can be fully accepted.","major_comments":[{"comment":"The 'chemically plausible' claim in the abstract and conclusion is not supported by the evaluation metric. Top-k accuracy is computed by applying each predicted template with RDChiral and checking whether the resulting precursor set matches the ground truth; a template that is syntactically valid and applicable but chemically nonsensical is counted as correct. Appendix A.6.1 (Figure A.3c) explicitly shows that P2T generates a chemically implausible template for a morpholine ring formation, yet the paper never quantifies how many of the correct top-k predictions rely on such templates, nor how many correct predictions come from novel versus training-set templates. This is especially important because the case study criticizes P2R for proposing chemically questionable 'shortcuts' but does not perform an equivalent audit of TempRe's own top-ranked templates. Please provide a breakdown of correct predictions by template provenance (known vs. novel) and an expert or rule-based plausibility audit on a sample of high-confidence correct predictions, or restrict the chemical-plausibility claim accordingly.","section":"Section 3.6 / Appendix A.6.1"},{"comment":"The main single-step and OOD results are reported without uncertainty estimates. All sequence-to-sequence models are trained once with a fixed random seed (42), and Figures 2a-2c present single runs without error bars or confidence intervals. The headline PaRoutes advantage (96% vs. 92% at Top-80) and the OOD MW-bin differences (57% vs. 42% at the highest MW bin) are therefore not assessable for statistical significance. Given that the paper's central contribution is empirical superiority, please add multiple seeds or bootstrap confidence intervals over the test set, and report the number of test reactions in each MW bin and template-frequency bucket.","section":"Section 5.1 / Appendix A.2.1"},{"comment":"The claim that the template-representation direct model 'consistently outperforms' DMS's Explorer rests on Direct TempRe-DMS, which is trained on the permissive DMS split that the paper itself identifies as containing reaction-level leakage (Appendix A.8). Although both Direct TempRe-DMS and Explorer use the same split, the comparison only demonstrates an advantage under a leaked training regime; the non-leaky Direct TempRe does not beat Explorer on the n1 solve rate (0.68 vs. 0.74). To support the broader claim that template-based route representation is superior to the nested-JSON representation, please either provide a non-leaky direct comparison or explicitly qualify the advantage as applying only under the DMS data-processing regime.","section":"Section 5.3 / Tables 2-3"}],"minor_comments":[{"comment":"The symbol T is used both for the generic template library in Eq. (2) and for the training-set template filter used in the strict variants; please use distinct notation or clarify that Eq. (2) applies to strict variants only, since the unrestricted models generate templates outside any fixed library.","section":"Section 3.1, Eq. (2)"},{"comment":"The text in Section 5.1 refers to Figure 2b for template-frequency performance and Figure 2c for OOD performance, but the caption lists subfigures a, b, and c with labels that do not exactly match the in-text ordering; please make the subfigure references consistent.","section":"Figure 2"},{"comment":"The row 'MCTS (AZF)(SB) PaRoutes 43K 22M' uses a different training set from all other rows; please separate it visually or add a footnote to make clear that it is an externally trained baseline and not directly comparable to the TempRe-trained models.","section":"Tables 2-3"},{"comment":"The statement that 'applying an equivalent filtering process to P2R outputs is less straightforward' would benefit from one or two sentences explaining why RXNMapper-based atom mapping followed by RDChiral template extraction is not a practical filter for P2R outputs in this setting.","section":"Appendix A.6.1"},{"comment":"The analysis of predicted route lengths reports only average step counts; because the discussion emphasizes that unrestricted models produce shorter routes, please consider also reporting medians or full distributions, which are more robust to the skew discussed in the text.","section":"Section 5.2, Figure 3c"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know: this is a solid empirical paper, not a revolution. What's genuinely new is the direct multi-step variant—generating a full route as a sequence of templates—and the careful evaluation of template generation inside MCTS search. Single-step template generation already exists in RetroComposer and the site-specific method from Shee et al.; those are reproduced and extended here. The OOD length-generalization analysis is also a real addition.\n\nThe data hygiene is the strongest part. The authors remove test reactions from training, build a hard test set enriched for rare templates, construct an OOD split by molecular weight, and ablate their data filters. They also explicitly acknowledge that the DMS-trained variant carries route-level leakage, which is more than most papers do. The comparison on PaRoutes is informative, and the finding that strict, library-constrained variants beat unrestricted ones on route accuracy is a nuanced result that cuts against any naive 'more generative is better' story. The case study is anecdotal but honestly presented.\n\nThe soft spots are real but not fatal. First, the headline single-step numbers (96% vs 92% at Top-80) come from a single seed with no error bars. That is minor and fixable. Second, and more substantive: top-k accuracy counts any template that RDChiral can apply and that recovers the ground-truth precursor set as correct, regardless of whether the transformation is chemically sensible. The paper itself shows a chemically invalid novel template generated by P2T (Figure A.3c), but never quantifies how many of the correct top-k predictions rely on such templates. That matters most for the unrestricted P2T/P2T-Tok models, which are exactly the ones carrying the flexibility and OOD robustness claims. The strict variants sidestep the issue, but they are not the models used to argue for generalization to rare reactions. So the abstract's 'chemically plausible' claim is stronger than the evidence supports. A plausibility audit or a metric that separates meaningful transformations from precursor retrieval would settle it.\n\nThe citation pattern is fine, and the acknowledged leakage in one variant is handled honestly. The paper would benefit from released code, but the setup is standard enough that the results are plausible.\n\nWho this is for: anyone working on retrosynthesis planning, especially on representations for direct route generation. It deserves a serious referee. My recommendation: engage, but ask for error bars, a quantification of invalid-template reliance, and ideally a small audit of top-ranked routes so the 'chemically plausible' claim matches the metric.","headline":"TempRe convincingly shows template generation as sequence modeling works for single-step and search-based multi-step planning, and adds a plausible direct multi-step variant; the 'chemically plausible' claim is softer than the evaluation supports.","tokens_in":18315,"tokens_out":1954,"would_cite":true,"duration_ms":24929,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Writing reaction rules beats writing products in retrosynthesis.","keywords":["template generation","retrosynthesis","reaction templates","SMARTS","sequence-to-sequence","multi-step synthesis planning","PaRoutes benchmark","out-of-distribution generalization"],"falsifier":"Take a random sample of top-10 predictions from P2T on PaRoutes test products and count how often the generated template fails to apply to the product with RDChiral or yields reactants that fail standard valence and stereochemistry checks; compare that failure rate with the invalid-SMILES rate of the product-to-reactant baseline. If the two rates are comparable, the claimed chemical-validity advantage of template generation evaporates.","tokens_in":17387,"feed_emoji":"🧪","tokens_out":7853,"duration_ms":83378,"temperature":0.7,"pith_summary":"TempRe reframes template-based retrosynthesis as a text-generation task: a Transformer reads a product SMILES and writes the reaction template as a SMARTS string, rather than choosing from a fixed template library or writing reactant SMILES directly. The paper's central claim is that this 'generate the rule, apply the rule' design keeps the chemical validity of template methods while removing their dependence on a fixed library, so the model can scale to hundreds of thousands of templates without growing its parameter count. On the PaRoutes benchmark, TempRe models reached 96% top-80 single-step accuracy, outperforming both a template classifier and a product-to-reactant Transformer, and on molecules far larger than those in training the template generator degraded much less than the reactant generator. In multi-step planning, a strict TempRe variant inside Monte Carlo Tree Search achieved high solve rates and top-k route accuracy, and a direct variant generated entire routes as a sequence of templates in a single pass. A sympathetic reader should care because this positions template generation as a scalable middle path between rigid rule libraries and chemically unreliable free-form generation.","feed_headline":"Writing reaction rules beats writing products in retrosynthesis","feed_subtitle":"TempRe generates templates as text, hitting 96% top-80 accuracy and staying accurate on large molecules.","key_machinery":"The central object is the reaction template as a SMARTS string—a substructural pattern describing which atoms react and how—treated as a token sequence for autoregressive generation. The framework factorizes the precursor distribution as $p(R|o)=\\sum_{t\\in\\mathcal{T}} q(R|o,t)\\,p_\\theta(t|o)$, where $p_\\theta$ is a product-to-template Transformer, $\\mathcal{T}$ is an external library of valid templates, and $q$ applies the generated template to the product through RDChiral. Two tokenization schemes are explored: plain BPE (P2T) and a frequency-sensitive scheme (P2T-Tok) that encodes popular templates as single tokens; 'strict' variants filter generated templates to those seen in training. For direct multi-step planning, the same generate-then-apply idea is extended by conditioning on the number of reaction steps and emitting a sequence of templates, which is then decoded into a route graph by iteratively applying each template to the current molecular state.","core_discovery":"The discovery the paper argues for is that retrosynthesis can be reformulated as generative modeling over reaction templates. Formally, the precursor distribution is written as $p(R|o)=\\sum_{t} q(R|o,t)\\,p_\\theta(t|o)$, where $p_\\theta(t|o)$ is an autoregressive Transformer that generates the template $t$ token by token from the product $o$, and $q(R|o,t)$ deterministically applies the template to the product to obtain reactants. Because the template library is external to the model, the network has 20--22 million parameters even when trained on 235K templates, whereas a classifier over the same library needs 122 million. The paper reports that template-generating models outperform a product-to-reactant Transformer on PaRoutes single-step reactions (96% vs 92% top-80 accuracy) and, crucially, maintain accuracy on out-of-distribution molecules with molecular weight above 500 g/mol, where the reactant generator falls from 70% to 42% top-5 accuracy. It also shows that a template generator restricted to known templates (P2T-Tok-Strict) is the best policy for search-based multi-step planning on PaRoutes, and that Direct TempRe can generate an entire route as a template sequence, outperforming a nested-JSON direct baseline when trained on the same data.","pith_inferences":["Editorial inference: because the output-template length is decoupled from input-molecule size, the out-of-distribution robustness should extend to even larger molecules and to other autoregressive architectures, though the paper only demonstrates it up to the tested molecular-weight bins.","Editorial inference: the strict-versus-unrestricted gap suggests a tunable 'creativity dial' for synthesis planning, where generated templates could be scored by similarity to known chemistry rather than simply accepted or discarded; this is a testable extension the paper does not pursue.","Editorial inference: the observation that the reactant-generating model's high solve rate is driven by chemically implausible shortcuts implies that solve-rate alone is a misleading progress metric for computer-aided synthesis planning; route-level chemical validity auditing should accompany route-accuracy reporting."],"forward_implications":["Single-step retrosynthesis models can be built as template generators with a compact Transformer, avoiding the parameter explosion of classifying over a 235K-template library (22M vs 122M parameters).","Template generation bypasses the length-generalization failure of reactant-SMILES generation, so it should be the preferred sequence-based formulation for large, drug-like target molecules.","Filtering generated templates to a known library (strict variants) trades a small loss of single-step diversity for substantially higher multi-step route accuracy, giving users a controllable exploration-fidelity dial.","Direct multi-step route generation is feasible as a lightweight single-pass alternative to search, with reasonable solve rates (0.68--0.75 on n1) even before any reward-guided search is added.","MCTS with a template-generating policy (P2T-Tok-Strict) sets a new reference point on PaRoutes with top-10 route accuracy of 0.62 on n1 and 0.54 on n5."],"supporting_citations":[{"why":"Supplies the PaRoutes n1/n5 benchmarks, stock sets, and route-evaluation protocol used for all single- and multi-step accuracy measurements.","marker":"[20]"},{"why":"RDChiral is the engine that extracts templates from the reaction corpus and applies generated templates to products, realizing the map q(R|o,t).","marker":"[32]"},{"why":"The Molecular Transformer defines the product-to-reactant sequence-to-sequence baseline and the SMILES tokenization used by all sequence models.","marker":"[19]"},{"why":"Syntheseus provides the MCTS implementation and search hyperparameters that embed single-step policies in multi-step planning.","marker":"[25]"},{"why":"DirectMultiStep supplies the direct-route-generation baseline and the permissive DMS data split used for the Direct TempRe-DMS comparison.","marker":"[29]"},{"why":"RXNMapper performs the atom-to-atom mapping that underpins template extraction and the novelty comparison against P2R.","marker":"[39]"},{"why":"The USPTO dataset is the source of all training and test reactions after the paper's filtering pipeline.","marker":"[30]"},{"why":"Prior template-generative approach for single-step retrosynthesis that TempRe extends to multi-step planning and search-based settings.","marker":"[27]"}],"fun_headline_variants":["Generate reaction rules, not products, for retrosynthesis","TempRe: template generation beats product generation in synthesis","Template-as-sequence: a new way to plan retrosynthesis","Direct multi-step routes via generated reaction templates","From product to template: 96% top-80 accuracy in retrosynthesis"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The results rest on the assumption that the template extraction and application pipeline—atom mapping, RDChiral template extraction, and the data filters—faithfully represents real chemical reactivity; if a class of reactions is poorly mapped or generated templates fail to apply cleanly, both the single-step accuracy and the multi-step route quality numbers would overstate the method's true performance.","fun_headline_variants_meta":{"raw":{"variants":["Generate reaction rules, not products, for retrosynthesis","TempRe: template generation beats product generation in synthesis","Template-as-sequence: a new way to plan retrosynthesis","Direct multi-step routes via generated reaction templates","From product to template: 96% top-80 accuracy in retrosynthesis"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000182,"raw_usage":{"total_tokens":1323,"prompt_tokens":969,"completion_tokens":354,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":585,"completion_tokens_details":{"reasoning_tokens":271}},"tokens_in":585,"tokens_out":354,"duration_ms":4973,"temperature":1.0,"reasoning_tokens":271,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T12:22:56.618954+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a random sample of top-10 predictions from P2T on PaRoutes test products and count how often the generated template fails to apply to the product with RDChiral or yields reactants that fail standard valence and stereochemistry checks; compare that failure rate with the invalid-SMILES rate of the product-to-reactant baseline. If the two rates are comparable, the claimed chemical-validity advantage of template generation evaporates.","supporting_citations":[{"cited_title":"Rdchiral: An rdkit wrapper for handling stereochemistry in retrosynthetic template extraction and application.Journal of chemical information and modeling, 59(6):2529–2537, 2019","cited_arxiv_id":null,"evidence_quote":"RDChiral is the engine that extracts templates from the reaction corpus and applies generated templates to products, realizing the map q(R|o,t)."},{"cited_title":"Molecular transformer: a model for uncertainty-calibrated chemical reaction prediction.ACS central science, 5(9):1572–1583, 2019","cited_arxiv_id":null,"evidence_quote":"The Molecular Transformer defines the product-to-reactant sequence-to-sequence baseline and the SMILES tokenization used by all sequence models."},{"cited_title":"Re-evaluating retrosynthesis algorithms with syntheseus","cited_arxiv_id":null,"evidence_quote":"Syntheseus provides the MCTS implementation and search hyperparameters that embed single-step policies in multi-step planning."},{"cited_title":"Directmultistep: Direct route generation for multistep retrosynthesis.Journal of Chemical Information and Modeling, 65(8): 3903–3914, 2025","cited_arxiv_id":null,"evidence_quote":"DirectMultiStep supplies the direct-route-generation baseline and the permissive DMS data split used for the Direct TempRe-DMS comparison."},{"cited_title":"PhD thesis, 2012","cited_arxiv_id":null,"evidence_quote":"The USPTO dataset is the source of all training and test reactions after the paper's filtering pipeline."},{"cited_title":"Site-specific template generative approach for retrosynthetic planning.Nature Communications, 15(1):7818, 2024","cited_arxiv_id":null,"evidence_quote":"Prior template-generative approach for single-step retrosynthesis that TempRe extends to multi-step planning and search-based settings."}],"review_version":1}