Pith. sign in

REVIEW 3 major objections 6 minor 26 references

Molecular LLMs fail basic structure tasks; SMILES–graph translation first restores reliable reasoning.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 05:29 UTC pith:NJEVQ5HV

load-bearing objection Solid empirical diagnosis and a useful structure benchmark; the transfer story is real but still entangled with GPT-5 CoT and multi-stage SFT volume. the 3 major comments →

arxiv 2607.03007 v1 pith:NJEVQ5HV submitted 2026-07-03 cs.LG cs.AIq-bio.BM

Back to Basics: Improving Molecular Understanding in LLMs via SMILES-Graph Translation

Wenda Wang , Jinjia Feng , Zhewei Wei This is my paper
classification cs.LG cs.AIq-bio.BM
keywords molecular LLMsSMILES-graph translationstructure comprehensionChain-of-Thoughtprogressive learningproperty predictionmolecular optimizationstructure determines function
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Current molecular language models can look strong on property prediction and molecule design, yet they often cannot count atoms or bonds or turn a SMILES string into a correct molecular graph. That gap breaks the chemistry principle that structure determines function. The paper argues the right order is structure first: teach models to convert both ways between SMILES and explicit graph adjacency lists, then use progressive training and standardized Chain-of-Thought to climb from structure to properties to optimization. On a multi-level structure benchmark, models that previously scored near zero on graph conversion reach high accuracy, and the same foundation improves property prediction and LogP-directed optimization. A sympathetic reader cares because high-level scores without structural grounding are brittle; fixing the foundation makes molecular reasoning more accurate, interpretable, and transferable.

Core claim

The paper shows that mainstream molecular LLMs almost universally fail basic structure perception—especially bidirectional SMILES–graph conversion—and that a structure-first pipeline, MolBasic, built on multi-level structure tasks with bidirectional conversion as the core, plus staircase progressive learning and CoT supervision, substantially raises structural accuracy and transfers to property prediction and objective molecular optimization.

What carries the argument

MolBasic: a staircase structure-first framework whose core is bidirectional SMILES–Graph translation on a multi-level structure comprehension benchmark, followed by two-stage answer-then-CoT training along the path structure → property → optimization.

Load-bearing premise

The claim depends on the idea that a fixed linear order—structure then property then optimization—plus CoT trajectories distilled from a strong external model is the right way to transfer structural skill; if that order or those trajectories are wrong, the transfer story weakens.

What would settle it

Train the same base model with matched data volume and CoT style but scramble or reverse the staircase order (or train property and optimization without prior SMILES–graph conversion) and check whether structure conversion stays near zero and downstream property and optimization metrics collapse relative to the full MolBasic order.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper argues that molecular LLMs achieve strong downstream scores without reliable structural grounding, conflicting with the chemistry principle that structure determines function. It introduces MolBasic, a structure-first post-training framework centered on bidirectional SMILES–Graph conversion within a multi-level structure comprehension (MSC) benchmark, then transfers that competence via a staircase curriculum (structure → structural property prediction → objective optimization) with two-stage LoRA training and standardized CoT distilled from GPT-5. Empirically, conversion accuracy rises from near zero to 94.65% (SMILES→Graph) and 85.66% (Graph→SMILES), with large gains on atom/bond counting and formula conversion (Table 1); property prediction becomes competitive with specialist models (Table 2); LogP-oriented optimization is best among LLM baselines (Table 3); and ablations (w/o Struct, Direct Opt, Mixed SFT, w/o CoT) plus external CleanMol/ChemCoTBench checks support the progressive structure-first design.

Significance. If the results hold, the work supplies a concrete, reproducible corrective to a real failure mode of molecular LLMs: near-total inability to recover 2D topology from canonical SMILES. The MSC benchmark, the explicit SMILES↔adjacency-list tasks, and the staged ablations are useful contributions beyond a single model checkpoint. Gains on held-out PubChem/TDC splits and external parsing benchmarks, together with honest Limitations on staircase weakening, make the paper a solid empirical argument that explicit structural alignment should precede high-level molecular reasoning. Strengths include thorough training/data documentation (Appendix B, Table 7), RDKit-verified conversion metrics, and public-dataset reproducibility.

major comments (3)
  1. §3.3–3.4, Eqs. (4)–(5), Table 4, Appendix B: The central causal claim is that SMILES–Graph translation is the foundation for downstream gains. Appendix B states that CoT is applied only to atom counting, total bond counting, and formula conversion, while conversion tasks remain answer-only—this is important and under-emphasized in the main text. Even so, every successful property/optimization path still receives GPT-5 Stage-2 CoT. Table 4 separates w/o Struct and w/o CoT but does not fully cross structure training with CoT source quality. Please state explicitly in the main paper that conversion supervision is not GPT-5 CoT, and discuss (even without a new run) how much of the transfer claim rests on high-quality external reasoning trajectories versus the learned SMILES–Graph equivalence.
  2. Table 11 vs Table 1 and §4.1: GPT-5 reaches 95% atom-counting accuracy and strong formula conversion, yet scores ~0–1.4% on bidirectional SMILES–Graph conversion, while MolBasic is the reverse pattern (weaker counting ACC, 85–95% conversion). The main narrative treats conversion as the missing structural foundation, but does not reconcile why elementary counting remains far below GPT-5 after structure-first training. This comparison should appear in the main results discussion, not only the appendix, because it qualifies what “structure comprehension” MolBasic actually acquires and how that differs from general LLM reasoning.
  3. §3.2 Eqs. (1)–(3) and Limitations: The linear path structure ≺ property ≺ optimization is presented as the natural chemist DAG, yet Limitations and Table 4(b) (Mixed SFT) already show that structural influence weakens on farther tasks and that joint training interferes. Frame the staircase more carefully as an effective curriculum choice rather than a validated dependency structure; otherwise the “structure determines function” rhetoric overclaims relative to the evidence that each stage plus CoT jointly drive gains.
minor comments (6)
  1. Figure 1 and §1: The pilot atom/bond counting failure is persuasive; state the pilot sample size and exact prompt format in the main text or caption so the figure is self-contained.
  2. Table 2: Chemprop remains stronger on several MAE columns (e.g., MW 1.63 vs 13.98). The text correctly says “comparable,” but a short note on when LLM structure reasoning still loses to graph specialists would avoid over-reading the LLM wins.
  3. Table 3 / §4.2.2: Optimization is single-objective LogP; multi-objective results are only in Appendix H.2. A one-sentence pointer in the main optimization section would help readers assess side-effect control.
  4. §3.1 / conversion evaluation: You verify exact RDKit molecular match and give an adjacency-list template to all models—good. Briefly note whether hydrogen handling, aromatic kekulization, or stereochemistry are normalized, since those choices affect “exact match.”
  5. Presentation: Expand acronyms on first use in figures (MSC, CoT); fix minor inconsistencies such as “MolBasic (BasicStructure Identification andComprehension)” spacing in §1; ensure Table 4 “Skipped” cells are defined in the caption.
  6. Related Work §2.1: Prior SMILES-parsing work (Jang et al., 2025; Hao et al., 2026) is cited; a clearer one-sentence contrast—local syntax/substructure vs full bidirectional graph equivalence—would sharpen novelty without lengthening the section.

Circularity Check

0 steps flagged

No circularity: empirical training framework evaluated on held-out and external benchmarks; no derivation reduces to its inputs by construction.

full rationale

MolBasic is an empirical post-training recipe (LoRA SFT on a multi-level MSC benchmark of SMILES–Graph conversion plus progressive property/optimization stages, with optional GPT-5-distilled CoT). All reported numbers—Table 1 conversion accuracies (0 % → 94.65 % / 85.66 %), Table 2 property MAE/Pearson, Table 3 ΔLogP/validity, ablations in Table 4, and external CleanMol/ChemCoTBench/few-shot results—are measured on held-out PubChem/TDC splits or independent benchmarks using RDKit exact-match or standard metrics. There is no mathematical derivation of a “prediction” from first principles, no parameter fitted to a subset and then re-presented as an independent forecast, no uniqueness theorem or ansatz imported via self-citation, and no renaming of a known empirical pattern. The staircase equations (1)–(3) and CoT objective (4)–(5) simply describe sequential supervised fine-tuning; they do not force the test-set numbers. Ablations and Limitations openly note that ordering and CoT source matter, confirming the claims remain experimental rather than definitional. Hence score 0 with empty steps.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 2 invented entities

The central empirical claim rests on standard chemistry and ML assumptions plus a handful of design choices (task weights, LoRA rank, GPT-5 CoT source, linear staircase). No new physical entities are postulated; the invented objects are the benchmark and the training curriculum themselves.

free parameters (3)
  • LoRA rank / alpha = 64 / 128
    Fixed at 64 / 128 for all stages; chosen by authors, not derived.
  • Stage-1 and Stage-2 task sampling weights = 2.0 / 3.0 / 1.5 / 0.5
    Conversion tasks weighted 2.0 then under-performing tasks re-weighted up to 3.0; hand-tuned to emphasize structure.
  • Learning rate and epoch schedule = 1e-4
    1e-4 cosine, 1–5 epochs per stage; standard but free choices that affect final numbers.
axioms (4)
  • domain assumption Structure determines function: reliable molecular reasoning requires first recovering the 2D graph from SMILES.
    Stated in Introduction and used to justify the entire staircase; standard chemistry principle but not proven for LLM training dynamics.
  • ad hoc to paper Chain-of-Thought trajectories distilled from GPT-5 plus manual standardization constitute valid intermediate supervision.
    Section 3.3; the quality of the distilled CoT is taken as given and never independently verified against human chemist reasoning.
  • ad hoc to paper Linear progressive order structure → property → optimization is the correct dependency DAG for molecular understanding.
    Section 3.2; the paper itself notes in Limitations that influence weakens with distance and that mixing interferes.
  • domain assumption Canonical SMILES plus textual adjacency lists are sufficient representations for teaching graph equivalence.
    Section 3.1; standard cheminformatics practice, but the particular adjacency-list format is paper-specific.
invented entities (2)
  • MSC multi-level structure comprehension benchmark no independent evidence
    purpose: Provide eight tasks (node/edge/graph) with bidirectional SMILES–graph conversion as core evaluation of structural grounding.
    Constructed by the authors from PubChemSTM; no prior public benchmark of this exact form is cited.
  • MolBasic staircase curriculum with two-stage (answer then CoT) LoRA training no independent evidence
    purpose: Enforce progressive acquisition of structure then property then optimization capabilities.
    Defined in Sections 3.2–3.4 and Algorithm 1; the specific ordering and CoT templates are new to this paper.

pith-pipeline@v1.1.0-grok45 · 25831 in / 2912 out tokens · 30136 ms · 2026-07-12T05:29:20.737901+00:00 · methodology

0 comments
read the original abstract

Recent advances in molecular large language models have led to strong performance on molecular understanding and generation tasks, yet these gains often come without reliable structural grounding. In particular, existing approaches conflict with the chemistry principle that structure determines function: despite their downstream success, current molecular LLMs perform poorly on basic structure recognition, suggesting that they fail to capture molecular graphs from canonical SMILES. To remedy this, we propose MolBasic, a structure-first framework that strengthens structural comprehension via SMILES-Graph translation. MolBasic is built around a multi-level structure perception benchmark, where bidirectional SMILES-Graph conversion serves as the core task to align sequential and topological representations. On top of this foundation, we employ a progressive learning scheme with a standardized Chain-of-Thought (CoT) to steer models from structure acquisition toward higher-level molecular reasoning. Experiments show that MolBasic substantially improves structural understanding and yields robust gains on downstream tasks, including property prediction and objective optimization, supporting our structure-first paradigm.

Figures

Figures reproduced from arXiv: 2607.03007 by Jinjia Feng, Wenda Wang, Zhewei Wei.

Figure 1
Figure 1. Figure 1: Preliminary evaluation of generalist and [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Comparison between the previous paradigm and our MolBasic framework for molecular understanding. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of the MolBasic framework. Left: The staircase learning path progresses through three stages: (1) structure comprehension at node, edge, and graph levels using the MSC benchmark; (2) Structural Property Prediction deriving molecular properties from structural features; (3) Objective Molecular Optimization modifying molecules toward desired properties. Right: The two-stage training strategy for eac… view at source ↗
Figure 4
Figure 4. Figure 4: Error distribution across four regression tasks [PITH_FULL_IMAGE:figures/full_fig_p015_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Distribution of sample-level absolute errors for seven molecular properties. Each subplot corresponds to [PITH_FULL_IMAGE:figures/full_fig_p017_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Property prediction errors with respect to molecular size. Each subplot shows the mean absolute error [PITH_FULL_IMAGE:figures/full_fig_p018_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Examples of standardized Chain-of-Thought [PITH_FULL_IMAGE:figures/full_fig_p019_7.png] view at source ↗
Figure 9
Figure 9. Figure 9: Examples of standardized Chain-of-Thought [PITH_FULL_IMAGE:figures/full_fig_p019_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Case study 1: Molecular optimization examples with CoT explanations. [PITH_FULL_IMAGE:figures/full_fig_p021_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Case study 2 and 3: Molecular optimization examples with CoT explanations. [PITH_FULL_IMAGE:figures/full_fig_p022_11.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

26 extracted references · 7 linked inside Pith

  1. [1]

    InProceedings of the 31st International Conference on Computational Linguistics, pages 354– 379

    Instructmol: Multi-modal integration for build- ing a versatile and reliable molecular assistant in drug discovery. InProceedings of the 31st International Conference on Computational Linguistics, pages 354– 379. Yongqiang Chen, Quanming Yao, Juzheng Zhang, James Cheng, and Yatao Bian. 2024a. Improving molecule-language alignment with hierarchical graph t...

    Pith/arXiv arXiv 2022

  2. [2]

    Esther Heid, Kevin P Greenman, Yunsie Chung, Shih- Cheng Li, David E Graff, Florence H Vermeire, Haoyang Wu, William H Green, and Charles J McGill

    Beyond chemical qa: Evaluating llm’s chemi- cal reasoning with modular chemical operations.Ad- vances in Neural Information Processing Systems, 38. Esther Heid, Kevin P Greenman, Yunsie Chung, Shih- Cheng Li, David E Graff, Florence H Vermeire, Haoyang Wu, William H Green, and Charles J McGill. 2023. Chemprop: a machine learning pack- age for chemical pro...

    Pith/arXiv arXiv 2023

  3. [3]

    Pubchem 2023 update.Nucleic acids research, 51(D1):D1373–D1380. TN Kipf. 2016. Semi-supervised classification with graph convolutional networks.arXiv preprint arXiv:1609.02907. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yu- taka Matsuo, and Yusuke Iwasawa. 2022. Large lan- guage models are zero-shot reasoners.Advances in neural information processing...

    Pith/arXiv arXiv 2023

  4. [4]

    InThe Twelfth Inter- national Conference on Learning Representations

    Let’s verify step by step. InThe Twelfth Inter- national Conference on Learning Representations. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023a. Visual instruction tuning. InNeurIPS. Shengchao Liu, Weili Nie, Chengpeng Wang, Jiarui Lu, Zhuoran Qiao, Ling Liu, Jian Tang, Chaowei Xiao, and Animashree Anandkumar. 2023b. Multi- modal molecule ...

    Pith/arXiv arXiv 2023

  5. [5]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others

    Graph attention networks.arXiv preprint arXiv:1710.10903. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elic- its reasoning in large language models.Advances in neural information processing systems, 35:24824– 24837. 10 David Weininger. 1988. Smiles, a chemica...

    Pith/arXiv arXiv 2022

  6. [6]

    Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie- Yan Liu

    Qwen3 technical report.arXiv preprint arXiv:2505.09388. Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie- Yan Liu. 2021. Do transformers really perform badly for graph representation?Advances in neural infor- mation processing systems, 34:28877–28888. Zheni Zeng, Yuan Yao, Zhiyuan Liu, and Maosong Sun

    Pith/arXiv arXiv 2021

  7. [7]

    Di Zhang, Wei Liu, Qian Tan, Jingdan Chen, Hang Yan, Yuliang Yan, Jiatong Li, Weiran Huang, Xiangyu Yue, Wanli Ouyang, and 1 others

    A deep-learning system bridging molecule structure and biomedical text with comprehension comparable to human professionals.Nature commu- nications, 13(1):862. Di Zhang, Wei Liu, Qian Tan, Jingdan Chen, Hang Yan, Yuliang Yan, Jiatong Li, Weiran Huang, Xiangyu Yue, Wanli Ouyang, and 1 others. 2024. Chemllm: A chemical large language model.arXiv preprint ar...

    Pith/arXiv arXiv 2024

  8. [8]

    AI chemist

    Atomas: Hierarchical adaptive alignment on molecule-text for unified molecule understanding and generation. InThe Thirteenth International Confer- ence on Learning Representations. Haiteng Zhao, Shengchao Liu, Ma Chang, Hannan Xu, Jie Fu, Zhihong Deng, Lingpeng Kong, and Qi Liu. 2023. Gimlet: A unified graph-text model for instruction-based molecule zero-...

    arXiv 2023

  9. [9]

    • Heavy atoms: 23

    Atoms and counts(Molecular formula: C18H16ClNO3): • C: 18, H: 16, N: 1, O: 3, Cl: 1. • Heavy atoms: 23

  10. [10]

    Functional groups identified: • Hydroxyl (–OH), ketone (C=O), amide (–C(=O)NH–), lactam (cyclic amide), and chlorine (–Cl)

  11. [11]

    Ring systems: • Total rings: 3, aromatic rings: 2, aliphatic rings: 1, heterocyclic rings: 1

  12. [12]

    • LogP: Moderately lipophilic due to aromatic rings, balanced by polar groups→3.37

    Property reasoning: • MW: Sum of atomic weights from C 18H16ClNO3 = 329.78 g/mol. • LogP: Moderately lipophilic due to aromatic rings, balanced by polar groups→3.37. • TPSA: Contributions from N and O atoms → 66.4 Å2. •HBD: 2, from NH/OH groups. •HBA: 3, from N/O atoms with lone pairs. • RB: 3, from non-ring single bonds excluding terminal bonds. •QED: Go...

  13. [13]

    Heavy atoms: 20

    Atoms and counts(Molecular formula: C15H16N3OS+): C: 15, H: 16, N: 3, O: 1, S: 1. Heavy atoms: 20

  14. [14]

    Functional groups: No common functional groups identified

  15. [15]

    Ring systems: Total rings: 3, aromatic rings: 3, hetero- cyclic rings: 3

  16. [16]

    •HBD: 0HBA: 4RB: 3QED: 0.69

    Property reasoning: • MW: 286.38 g/molLogP: 1.96, slightly lipophilic TPSA: 38.25 Å2. •HBD: 0HBA: 4RB: 3QED: 0.69. Answer:MW=286.38, LogP=1.96, TPSA=38.25, HBD=0, HBA=4, RB=3, QED=0.69 Figure 8: Examples of standardized Chain-of-Thought reasoning for structural property prediction. Task: Molecular Optimization (Example 1) Input: Cc1ccccc1/C=C1/Oc2c(ccc(O)...

  17. [17]

    Original molecule analysis: C 23H25NO+ 3 , 4 rings, 2 aromatic rings, LogP: 3.13, HBD: 2, HBA: 3, charge: +1

  18. [18]

    Polar groups that decrease LogP: quaternary ammo- nium [N+], hydroxyl –OH, ketone C=O, and ether –O–

  19. [19]

    Lipophilic groups that increase LogP: 2 phenyl rings and a 23-carbon skeleton

  20. [20]

    • Reduce HBD:2→1

    Optimization strategy: • Neutralize ionic charge, which is expected to provide a major LogP increase. • Reduce HBD:2→1. • Add a piperazine ring with a phenyl substituent

  21. [21]

    Output: Cc1ccccc1/C=C1\Oc2c(ccc(O)c2CN2CCN(c3 ccccc3)CC2)C1=O Task: Molecular Optimization (Example 2) Input: C[NH+](C)C(C)(C)CNC(=O)c1cnn(Cc2ccccc2 Cl)c1 Chain-of-Thought:

    Verification: LogP:3.13→4.64(∆ = +1.51)✓. Output: Cc1ccccc1/C=C1\Oc2c(ccc(O)c2CN2CCN(c3 ccccc3)CC2)C1=O Task: Molecular Optimization (Example 2) Input: C[NH+](C)C(C)(C)CNC(=O)c1cnn(Cc2ccccc2 Cl)c1 Chain-of-Thought:

  22. [22]

    Original molecule analysis: C 17H24ClN4O+, 2 aro- matic rings, LogP: 1.24, HBD: 2, HBA: 3, charge: +1

  23. [23]

    Polar groups that decrease LogP: quaternary ammo- nium [N+], secondary amine –NH–, and amide –CONH–

  24. [24]

    Lipophilic groups that increase LogP: phenyl ring, chlorine, and 17-carbon skeleton

  25. [25]

    • Reduce HBD:2→1

    Optimization strategy: • Remove charged quaternary N, which is expected to provide a major LogP increase. • Reduce HBD:2→1. • Add trifluoromethyl –CF3 and phenyl ring

  26. [26]

    Output: O=C(Nc1ccccc1C(F)(F)F)c1cnn(Cc2ccccc2 Cl)c1 Figure 9: Examples of standardized Chain-of-Thought reasoning for molecular optimization (LogP improve- ment)

    Verification: LogP:1.24→4.86(∆ = +3.62)✓. Output: O=C(Nc1ccccc1C(F)(F)F)c1cnn(Cc2ccccc2 Cl)c1 Figure 9: Examples of standardized Chain-of-Thought reasoning for molecular optimization (LogP improve- ment). K Use of AI Assistants We used AI assistants during the preparation of this work for limited auxiliary purposes, including code debugging and grammar-le...