{"id":"a891ede0-3ad2-4eb9-bff0-4d596d4c169d","arxiv_id":"2508.08657","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"M2LLM combines LLM embeddings prompted with structural and task questions with LLM-generated rule features, fused by trainable per-molecule weights, and reports state-of-the-art results on several MoleculeNet benchmarks.","lead":"A new system called M2LLM uses large language models to read a molecule's SMILES string from several angles: raw structure, the specific question being asked, and chemical rules the model generates itself. On standard drug-property benchmarks it reports top scores, though the gains over older LLM-based methods were not directly tested.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section 3.3 gives no way to compute the per-molecule fusion weights alpha_i for test molecules, so the reported test-set numbers cannot be reproduced from the method as written.","rationale":"The reader's contamination concern is legitimate: LLM pretraining corpora may contain MoleculeNet molecules or labels, and near-perfect ClinTox scores are suspicious without a contamination analysis. However, that concern is an external-validity risk that applies to many LLM-based benchmark papers and cannot be definitively settled from the manuscript alone. The more immediate problem is internal: Eq. (6) defines per-molecule fusion weights alpha_i as learnable parameters, yet no inference rule maps a test molecule to its alpha_i. The paper reports per-molecule component contributions in Figure 3, so alpha_i exists for every evaluated molecule, but without a formula or function the reported test metrics are not reproducible from the method as written. This is directly load-bearing because the large ESOL and FreeSolv gains are attributed to dynamic multi-view fusion. I would not reject the paper; the missing alpha mechanism could be a presentation omission that is fixed by releasing code and specifying the fusion function. The reader already assigned CONDITIONAL, and my concern reinforces that verdict rather than moving it, so the recommendation is UNCHANGED with the explicit requirement that the fusion rule and code be supplied.","tokens_in":16664,"tokens_out":6877,"duration_ms":82013,"concrete_test":"Obtain or reimplement the code and inspect the forward pass on the scaffold-split test set, identifying exactly how alpha_i is obtained for each test molecule. Then re-run the 10-seed evaluation for Table 2 with the fusion rule made explicit, e.g., alpha_i = softmax(W [z_struct_i; z_task_i; z_rule_i]) or whatever function the code actually uses, and compare ESOL and FreeSolv RMSE against the reported values. If the numbers reproduce within the reported standard deviations, the missing alpha specification is a presentation issue; if test-time alpha is undefined or the numbers shift materially, the central SOTA claim is not supported as written.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central SOTA claim depends on the dynamic fusion mechanism in Eq. (6), where z_fused_i = alpha_struct_i * z_struct_i + alpha_task_i * z_task_i + alpha_rule_i * z_rule_i, with alpha_struct_i + alpha_task_i + alpha_rule_i = 1 and all alpha >= 0. The paper states that these are 'learnable weights' that are 'specific to each molecule', and Figure 3 reports per-molecule component contributions. However, no formula, network, or inference procedure is given for producing alpha_struct_i, alpha_task_i, alpha_rule_i for a molecule in the scaffold-split test set. If alpha_i are free scalar parameters fit only on the training split, a test molecule has no alpha_i, so the test ROC-AUC and RMSE values in Tables 1 and 2 cannot be generated by the described method. If alpha_i are instead outputs of a learned fusion function h(z_i; theta), that function is the core of the multi-view claim and must be specified. This gap is load-bearing because the large reported gains, e.g., ESOL RMSE 1.55 -> 0.44 and FreeSolv 4.29 -> 2.01 in Table 2 and Figure 4, are attributed precisely to the dynamic multi-view fusion mechanism. The contamination concern raised by the reader is real, but it is an external-validity risk; the missing alpha computation is an internal gap that prevents the reported test-time forward pass from being defined at all.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes M2LLM, a multi-view molecular representation learning framework that combines three LLM-derived views: a molecular structure view (SMILES with structural questions), a molecular task view (SMILES with task-specific questions), and a molecular rules view (LLM-generated scientific and data-pattern rules converted to features). The views are combined through a per-molecule weighted fusion, Eq. (6), and an MLP performs the final classification or regression. The authors evaluate M2LLM on eight MoleculeNet datasets with scaffold splits using Galactica, LLaMa-3.1, and OpenAI embedding backbones, reporting state-of-the-art results on several classification and regression benchmarks, and they analyze per-view contributions and compare against a SMILES-only baseline.","tokens_in":16972,"tokens_out":6183,"duration_ms":63448,"significance":"If the method is fully specified and the empirical results hold, the paper offers a useful direction by showing that LLM semantic knowledge and rule-based reasoning can complement molecular structure encodings, and the multi-view fusion idea is potentially interesting for molecular property prediction. The paper has concrete strengths: it evaluates across multiple datasets and backbones, includes a SMILES-only ablation, and provides per-component contribution analysis. However, the central fusion mechanism is presently underspecified to the point that the test-time forward pass cannot be reproduced, and several performance claims are weaker than the text suggests when standard deviations are inspected. The significance of the contribution depends on resolving these issues.","major_comments":[{"comment":"The fusion mechanism is not computable as written. The per-molecule weights α_struct_i, α_task_i, and α_rule_i are described as 'learnable weights' that are 'specific to each molecule,' but no formula, network, or inference procedure is given for producing these weights for a molecule in the scaffold-split test set. If they are free scalar parameters fit on the training split, a test molecule would have no α_i. If they are the output of a learned function, that function is the core of the dynamic fusion claim and must be specified. In addition, the three view vectors have incompatible dimensionalities: z_struct_i is the concatenation of three LLM embeddings (Eq. (2)), z_task_i is a single LLM embedding (Eq. (3)), and z_rule_i is a rule-based feature vector whose length is the number of rules (Eq. (5)). No projection to a common space is described, so the weighted sum in Eq. (6) is not a well-defined vector operation as stated. Because the reported test-set numbers in Tables 1 and 2 are generated by this forward pass, the method is not reproducible from the manuscript.","section":"§3.3, Eq. (6)"},{"comment":"The claim of state-of-the-art performance is only partially supported by the reported means and standard deviations. On BACE, M2LLM (80.0) is below RF+ECFP4 (85.0), and the paper acknowledges this. On HIV, SIDER, and BBBP, the reported improvements over the best baseline are often within one standard deviation (e.g., HIV 79.5±0.7 vs Uni-Mol 78.3±1.3; SIDER 63.7±0.3 vs Uni-Mol 62.3±5.6; BBBP 75.5±1.3 vs MolCLR 73.1±1.6). No significance tests or paired comparisons are reported. The regression gains on FreeSolv for M2LLM(Galactica) (2.39±1.39) overlap substantially with the best baseline (2.38±0.40). The phrase 'surpassing existing baselines with significant improvements' should be qualified, and the state-of-the-art claim should be supported with appropriate statistical tests or restricted to the datasets where the improvement is clear.","section":"§4.2, Tables 1 and 2"},{"comment":"The text in §4.5 claims that M2LLM 'consistently improved the scores across all six datasets' and that on ClinTox 'our framework offering a marginal improvement.' This is contradicted by Appendix Table 5, where for most backbones the SMILES-only baseline achieves a higher ClinTox ROC-AUC than M2LLM (e.g., llama-3.1-8B: 99.68±0.09 vs 98.67±2.11; galactica-6.7b: 100.00±0.00 vs 99.93±0.05; text-embedding-3-large: 99.86±0.00 vs 99.38±0.09). The discrepancy between Figure 4, which shows ClinTox M2LLM at 1.000, and the appendix table needs to be resolved and the over-claim corrected.","section":"§4.5 and Appendix A.4, Table 5"},{"comment":"The evaluation does not address the possibility that the pretraining corpora of Galactica, LLaMa-3.1, and the OpenAI embedding models contain the MoleculeNet benchmark molecules and labels. Given the near-perfect ClinTox scores and the large ESOL improvement, the paper should at least discuss this contamination risk and, where feasible, provide an analysis such as checking SMILES overlap with public pretraining data or motivating why memorization is unlikely for held-out scaffold splits. This is an external-validity concern that directly affects the interpretation of the claimed state-of-the-art results.","section":"§4.1 and §3.2"}],"minor_comments":[{"comment":"There is a typo in the text: 'Sructure Insight 3' should be 'Structure Insight 3.'","section":"§3.1"},{"comment":"The table caption says 'with bold being the best result, and bold being the second best result,' but both are described as 'bold'; the two bold styles need to be distinguished or the caption clarified.","section":"§4.2, Table 1"},{"comment":"The dataset name is spelled inconsistently as 'Clintox' in the text and 'ClinTox' elsewhere; please standardize.","section":"§4.2"},{"comment":"In Figure 4, the classification subplot uses a scale starting near 0.5, which visually compresses the differences for ClinTox (near 1.0); consider using a separate scale or adding numerical labels above each bar.","section":"§4.5, Figure 4"},{"comment":"In the reference list, 'OpenAI et al., 2023' appears with a stray 'Josh' before 'OpenAI'; this should be corrected.","section":"References"},{"comment":"The Rule-to-Feature Code Translator is described as using LLMs to convert rules into executable code, but no details are given about the programming language, execution environment, or how the code functions are validated; this is important for reproducibility and should be specified, even in an appendix.","section":"§3.2"},{"comment":"The bar charts in Figure 5 are not very informative for the classification datasets because most values are close to 1.0; a table with numeric values or a zoomed plot would be clearer.","section":"Appendix A.2"}],"recommendation":"major_revision","confidential_remarks":"The core issue is that the dynamic fusion mechanism, which is the paper's main methodological contribution, is not specified in a way that allows the test-time forward pass to be reproduced. This is a load-bearing gap, not just a presentation issue. The discrepancy between Figure 4 and Appendix Table 5 on ClinTox is also serious and should be resolved before the paper can be considered reliable. The contamination risk with closed and large pretrained models is a known concern in this area and should be addressed head-on. If the authors can specify the fusion function, add appropriate projections for the view vectors, correct the performance claims, and add statistical support for the SOTA claims, the paper would be suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: the paper's central empirical claim — state-of-the-art on several MoleculeNet benchmarks via dynamic multi-view fusion — cannot be reproduced from the text. Eq. (6) defines the fused representation as a weighted sum with per-molecule weights alpha_i, but the paper never says how those weights are computed for a test molecule. If they are free parameters fit per training molecule, test molecules have none, so the test-set ROC-AUCs and RMSEs are undefined. If they come from a learned fusion function h(z_i), that function is the core of the multi-view mechanism and is not described. Figure 3 even plots per-molecule contributions, but the inference procedure is absent. That is the load-bearing gap, not a minor omission.\n\nWhat is actually new: the molecular rules view — using LLMs to generate scientific and data-pattern rules, then translating them into executable functions via an LLM code translator. That is a genuinely fresh idea, different from prior multi-view LLM work (Luo et al., Shirasuna et al., Rollins et al.), which fuse text and graph embeddings but don't turn reasoning outputs into features. The structure/task-view prompting is a known trick, but the rule-to-feature translation is the novel contribution.\n\nThe reported numbers are not all solid. ClinTox at 99.5% smells like contamination; the paper doesn't test whether the LLM pretraining corpora include the MoleculeNet test molecules. Galactica is explicitly trained on scientific literature, so this is a real risk. Also, BACE is below random forest, and several SIDER and FreeSolv gains are within reported standard deviations. The paper cites the closest prior multi-view LLM methods but never benchmarks against them, which weakens the state-of-the-art claim.\n\nAll that said, the framework is coherent and the idea deserves a serious referee rather than a desk reject. A referee should demand the fusion formula, a contamination check, and comparisons to the cited LLM baselines. If those are missing after revision, the result doesn't stand, but the core conceptual contribution could survive.","headline":"The rule-to-code feature idea is genuinely new, but the per-molecule fusion weights are unspecified, so the paper's test-set numbers cannot be produced from the method as written.","tokens_in":17515,"tokens_out":1829,"would_cite":false,"duration_ms":18934,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fusing three LLM views pushes ClinTox to 99.5% AUC.","keywords":["molecular property prediction","multi-view representation learning","large language models","molecular embedding","rule-based features","dynamic fusion","MoleculeNet benchmark","LLM reasoning"],"falsifier":"A concrete test: hold out a set of molecules synthesized and labeled after the training cutoff of the LLM backbones, rerun M2LLM and the baselines on these held-out molecules, and check whether the multi-view gain over the SMILES-only baseline persists; if the gain vanishes, the reported benchmark results are best explained by pretraining contamination rather than by the multi-view reasoning.","tokens_in":16465,"feed_emoji":"🧪","tokens_out":5912,"duration_ms":51384,"temperature":0.7,"pith_summary":"The paper tries to establish that a large language model can produce richer molecular representations than SMILES-only encoding or GNNs if it is prompted to view the molecule from three angles: its structure, the task being predicted, and the domain rules chemists use. To do this, it proposes M2LLM, which combines the LLM embeddings from the first two views with a feature vector built from LLM-generated rules, then fuses all three with per-molecule learned weights. The paper reports state-of-the-art results on several MoleculeNet benchmarks, including 99.5% ROC-AUC on ClinTox and RMSE 0.44 on ESOL, and shows the multi-view fusion consistently improves over a SMILES-only baseline. If true, this suggests that pretrained scientific knowledge in LLMs can substitute for much of the dataset-driven learning in molecular property prediction.","feed_headline":"Fusing three LLM views pushes ClinTox to 99.5% AUC.","feed_subtitle":"Structure, task, and rule views with per-molecule weights beat GNNs on MoleculeNet benchmarks.","key_machinery":"The central mechanism is the three-view dynamic fusion. The structure view concatenates the SMILES string with three fixed insight questions (3D shape, intermolecular forces, chemical equilibrium) and encodes the text through the LLM. The task view appends the specific prediction question, e.g., \"Will the chemical compound penetrate the blood-brain barrier?\" to the SMILES. The rules view first has the LLM generate 20 scientific rules for the task and 3 pattern rules from sampled training molecules, then a rule-to-feature code translator turns each rule into a binary or numeric function of the SMILES. The fused representation is $z^{\\text{fused}}_i = \\alpha^{\\text{struct}}_i z^{\\text{struct}}_i + \\alpha^{\\text{task}}_i z^{\\text{task}}_i + \\alpha^{\\text{rule}}_i z^{\\text{rule}}_i$, with non-negative per-molecule weights summing to 1, learned jointly with an MLP.","core_discovery":"M2LLM claims that the best molecular representations are not learned from graphs or fingerprints alone but assembled from three LLM-driven views: a structure view that encodes SMILES alongside generic chemistry questions, a task view that encodes SMILES alongside the specific prediction question, and a rules view in which the LLM generates chemical rules (from scientific knowledge and from observed training data) that are translated into executable feature functions. These three representations are combined by a weighted sum with molecule-specific, learnable weights that sum to one, and the resulting vector feeds an MLP for classification or regression. The paper reports that this fusion reaches higher ROC-AUC and lower RMSE than all tested GNN and transformer baselines on eight MoleculeNet datasets, highlighting near-perfect ClinTox scores and a 56.9% error reduction on ESOL relative to the best baseline.","pith_inferences":["The near-perfect ClinTox results (99.1-99.5% across backbones) are suspiciously high; if the backbones' pretraining corpora contained MoleculeNet molecules and labels, the comparison would partly measure memorization rather than the multi-view framework.","The rule-to-feature translator effectively performs code synthesis from natural language and could be applied outside property prediction, e.g., to generate physically interpretable descriptors for other scientific quantitative structure-activity problems.","The per-molecule weights alpha provide a cheap interpretability signal: examining which view dominates for a given molecule could reveal why the model is confident, turning the method into a hypothesis generator for chemists.","The approach is likely sensitive to the quality of the LLM's scientific knowledge; on molecules with novel chemistries the rules view might generate misleading rules, and the dynamic weights might not compensate."],"forward_implications":["If the framework is correct, molecular property prediction no longer needs hand-crafted fingerprints or graph-specific architectures; the same LLM backbone, prompted differently, supplies both semantic and structural information.","The rules view yields an interpretable feature set: each generated rule (e.g., \"molecular weight < 500 Da\") corresponds to a human-readable condition, so predictions can be traced back to chemical reasoning.","The component analysis suggests that different tasks recruit different views: classification relies more on the structure view, regression on rules and task views, implying the dynamic fusion weights capture task-relevant inductive biases automatically.","Since the structure view is modular and its questions can be replaced or added, the framework can be extended to new molecular properties without retraining the backbone, only reformulating the prompts."],"supporting_citations":[{"why":"Defines the MoleculeNet benchmark, the eight datasets, and the scaffold-split evaluation protocol used throughout.","marker":"[Wu et al., 2018]"},{"why":"Provides the Galactica backbone whose scientific pretraining powers the structure and task embeddings.","marker":"[Taylor et al., 2022]"},{"why":"Provides the LLaMa-3.1 backbone used to generate embeddings in one set of M2LLM variants.","marker":"[Dubey et al., 2024]"},{"why":"Supplies the closed-source text-embedding models used as the third backbone family.","marker":"[OpenAI, 2024]"},{"why":"Defines ECFP4 fingerprints, the input to the random-forest baseline that M2LLM must beat.","marker":"[Rogers and Hahn, 2010]"},{"why":"Establishes zero-shot reasoning in LLMs, the capability the rules view leverages for scientific rule generation.","marker":"[Kojima et al., 2022]"},{"why":"Situates LLM reasoning in molecular property prediction and motivates the task-view prompting strategy.","marker":"[Zheng et al., 2025]"},{"why":"Provides the MolCLR GNN baseline, the strongest prior method on ClinTox that M2LLM surpasses.","marker":"[Wang et al., 2022]"}],"fun_headline_variants":["Three LLM views beat GNNs on all eight MoleculeNet benchmarks","M2LLM fuses LLM views to cut ESOL error by 56.9%","Dynamic weight fusion of three LLM views hits 99.5% ClinTox AUC","M2LLM combines structure, task, and rule views for SOTA predictions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's empirical claims rest on the unstated assumption that the LLaMa, Galactica, and OpenAI embedding models were not trained on the MoleculeNet benchmark molecules and their labels; if those examples appear in the pretraining corpora, the reported gains could come from memorization rather than from the multi-view fusion.","fun_headline_variants_meta":{"raw":{"variants":["Three LLM views beat GNNs on all eight MoleculeNet benchmarks","M2LLM fuses LLM views to cut ESOL error by 56.9%","Dynamic weight fusion of three LLM views hits 99.5% ClinTox AUC","M2LLM combines structure, task, and rule views for SOTA predictions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00068,"raw_usage":{"total_tokens":3075,"prompt_tokens":920,"completion_tokens":2155,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":536,"completion_tokens_details":{"reasoning_tokens":2063}},"tokens_in":536,"tokens_out":2155,"duration_ms":17197,"temperature":1.0,"reasoning_tokens":2063,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:34:38.581355+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete test: hold out a set of molecules synthesized and labeled after the training cutoff of the LLM backbones, rerun M2LLM and the baselines on these held-out molecules, and check whether the multi-view gain over the SMILES-only baseline persists; if the gain vanishes, the reported benchmark results are best explained by pretraining contamination rather than by the multi-view reasoning.","supporting_citations":[{"cited_title":"New embedding models and api updates,","cited_arxiv_id":null,"evidence_quote":"Supplies the closed-source text-embedding models used as the third backbone family."},{"cited_title":"Extended-connectivity fingerprints","cited_arxiv_id":null,"evidence_quote":"Defines ECFP4 fingerprints, the input to the random-forest baseline that M2LLM must beat."},{"cited_title":"Large language models are zero-shot reason- ers","cited_arxiv_id":null,"evidence_quote":"Establishes zero-shot reasoning in LLMs, the capability the rules view leverages for scientific rule generation."},{"cited_title":"Large language models for scientific dis- covery in molecular property prediction","cited_arxiv_id":null,"evidence_quote":"Situates LLM reasoning in molecular property prediction and motivates the task-view prompting strategy."},{"cited_title":"Molecular contrastive learning of representations via graph neural networks","cited_arxiv_id":null,"evidence_quote":"Provides the MolCLR GNN baseline, the strongest prior method on ClinTox that M2LLM surpasses."}],"review_version":2}