Pith. sign in

REVIEW 4 major objections 5 minor 57 references

MolVision: Molecular Property Prediction with Vision Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MolVision shows that vision-language models predict molecular properties better when given a molecule's 2D structural image alongside its SMILES text, and that LoRA fine-tuning plus a Tanimoto-guided contrastive loss further improves…

desk verdict Useful benchmark hustle, but the headline claim that images help rests on a text-only baseline that is almost certainly broken; needs a major cleanup before you can trust the main comparison. read the letter →

arxiv 2507.03283 v1 pith:W2NXM5JI submitted 2025-07-04 cs.CV

Deepan Adak , Yogesh Singh Rawat , Shruti Vyas This is my paper · ORCID
classification cs.CV
keywords molecularpropertypredictionvision-languagemodelsmultimodallearningSMILESimagesLoRAfine-tuningTanimotosimilaritycontrastive
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MolVision introduces a benchmark and method that feed vision-language models both a rendered 2D skeletal image of a molecule and its SMILES string, testing nine VLMs across ten datasets spanning classification, regression, and description tasks. The central claim is that visual information alone is insufficient, but fusing the image with the textual representation consistently outperforms text-only prompting, especially after LoRA fine-tuning. This matters because existing LLM-based chemistry models rely only on linear string representations like SMILES, which omit spatial and stereochemical information that chemists read directly from bond-line diagrams. If the claim holds, adapting general-purpose VLMs with simple images becomes a practical and data-efficient route to better predictions of toxicity, solubility, bioactivity, and quantum properties.

What carries the argument

The central mechanism is pairing an RDKit-rendered 2D skeletal molecular image with a text prompt containing the SMILES string, then passing both through a vision-language model whose decoder is adapted with LoRA low-rank matrices. The secondary mechanism is a contrastive objective (NT-Xent loss) applied to the vision encoder, where positive pairs are molecules whose Tanimoto similarity exceeds 0.85, pulling their embeddings together and forming chemically meaningful clusters in visual feature space.

What would settle it

Run the same prompts on a held-out split while logging whether each text-only and image-plus-text response is parseable; if text-only parse failures account for most of the error gap on ESOL (Table 6), the visual-modality conclusion is not supported.

Watch

Extended reading notes

Core claim

The paper demonstrates that multimodal fusion of molecular structure images with SMILES text yields better property predictions than text alone across all ten benchmark datasets, while image-only inputs perform near chance (0.09-0.18 accuracy on several classification sets). After LoRA fine-tuning, BLIP-2 reaches an average classification accuracy of 0.92, surpassing the best graph-based LLM baseline (Molca at 0.83) and matching or beating traditional machine-learning models on most tasks. The paper also shows that adapting the vision encoder with a contrastive loss based on Tanimoto similarity (using molecules with similarity >0.85 as positive pairs) materially improves regression and description results, cutting ESOL RMSE by roughly 35% and LD50 MAE by roughly 51% relative to LoRA alone.

Load-bearing premise

The reported gains from adding images assume that the text-only baseline is a reliable predictor that produces parseable outputs; if the text-only condition mostly fails to emit usable numbers, the comparison overstates the value of the visual channel.

Editorial extensions

If this is right

  • Fused image-plus-text inputs raise average classification accuracy over text-only LLM baselines from roughly 0.67 to 0.92 after fine-tuning, establishing that visual structure helps the model generalize across diverse chemical properties.
  • LoRA fine-tuning of the language decoder alone brings open-source VLMs to parity with proprietary models on classification and regression, suggesting that parameter-efficient adaptation is sufficient for molecular tasks.
  • Tanimoto-guided contrastive adaptation of the vision encoder reduces regression error substantially (ESOL RMSE by about 35%, LD50 MAE by about 51%) and lifts classification F1 scores by 2-4 points over plain LoRA.
  • Visual information disproportionately benefits larger molecules, indicating that the image channel helps when long SMILES strings become hard for the model to parse.
  • SELFIES representations generally outperform SMILES in few-shot settings, so the choice of textual encoding still affects performance even after adding images.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported text-only ESOL RMSE of 9.89 for BLIP-2 versus 1.07 for image-plus-text is likely inflated by the text-only prompt failing to produce parseable numeric output; a parse-aware re-evaluation is needed to separate genuine chemical understanding from output-format stability.
  • Because the images are simple RDKit renderings, the same pipeline can be applied to any SMILES-based dataset, offering a lower-cost alternative to graph neural networks for property prediction.
  • The Tanimoto-guided contrastive adaptation should transfer to other VLMs with trainable vision encoders, and its benefit probably grows on datasets where stereochemistry and molecular geometry are decisive.
  • A direct test of whether the model truly reads structure from the image would be to scramble or rotate the molecular image while keeping text fixed; if performance stays the same, the visual channel is not providing chemical information.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. MolVision is a benchmark and method paper for molecular property prediction with vision-language models (VLMs). The authors render 2D skeletal structure images with RDKit and pair them with SMILES/SELFIES text, building ten datasets covering classification, regression, and description tasks. They evaluate nine VLMs in zero-shot, few-shot (ICL and CoT), and LoRA fine-tuning settings, and propose a contrastive learning objective that uses Tanimoto similarity to form positive pairs for adapting the vision encoder. The central claim is that combining molecular images with text yields better property predictions than text alone, while images alone are insufficient. The paper also reports that LoRA fine-tuning and Tanimoto-guided contrastive adaptation further improve performance.

Significance. If the central claim is correct, MolVision would provide a useful, easily reproducible benchmark for multimodal chemical property prediction and a simple domain-adaptation recipe for vision encoders. The paper's strengths are its breadth (nine models, ten datasets), the concrete contrastive adaptation with t-SNE analysis, and the stated intention to release code and data. However, the main evidence for the visual-information claim rests on Table 6, which mixes experimental settings and contains a text-only baseline that appears to have failed numerically; without fixing this comparison and adding statistical rigor, the significance of the benchmark is conditional.

major comments (4)
  1. [§5.3, Table 6] The central comparison is not a controlled experiment. The caption and text do not state whether the rows correspond to zero-shot, ICL, CoT, or fine-tuning. The Image+Text column for BLIP-2 exactly reproduces the LoRA fine-tuning accuracies from Table 4 for BACE-V (0.86), BBBP-V (0.93), HIV-V (0.92), ClinTox-V (0.89), and Tox21-V (0.99), while the ESOL RMSE (1.07) and LD50 MAE (0.49) match the CoT values in Table 3, and the QM9 MAE (4.92) matches the LoRA values in Tables 8 and 29. The PCQM4Mv2 MAE of 1.99 appears in no other table (LoRA reports 0.24, ICL 1.30, CoT 1.25). Because the rows mix settings, Table 6 cannot support the claim that adding the image causes the observed gains.
  2. [§5.3, Table 6, Text Only row] The text-only baseline is not credible. BLIP-2's text-only ESOL RMSE of 9.89 is worse than a constant predictor on a log-solubility scale whose training-set RMSE would be about 1.7-2.5, implying the model produced non-numeric or out-of-range outputs that were treated as valid numbers. The paper describes no parsing or validation protocol for numerical answers (§5.1 only states temperature 0). Without a documented extraction scheme, the 9x gap between Text Only and Image+Text cannot be attributed to chemical understanding; it likely reflects a broken baseline. JanusPro's text-only ESOL (1.23) is more plausible, so a weaker version of the claim may survive for that model, but the broad claim in the abstract is not established.
  3. [§4, Table 1 and §5.1] The benchmark does not report the data-split protocol (random, scaffold, or stratified) or any measure of variance such as seeds or error bars. Since the benchmark is a contribution and the fine-tuning improvements in Table 8 are small (2-4% accuracy, 35-51% error reductions), the absence of repeated runs makes it impossible to assess whether the reported differences are significant.
  4. [§5.4, Eq. (1)-(2) and Table 8] The hyperparameters of the proposed contrastive adaptation (τ=0.5, λ, Tanimoto threshold >0.85, three positive pairs) appear to be chosen using the same target datasets on which the final results are reported; no validation split is described. This opens the reported gains to selection bias. The paper should either tune on a validation split or clearly state that the reported numbers are fixed-hyperparameter test results.
minor comments (5)
  1. [Table 6 caption] The caption uses 'PV' for PCQM4Mv2-V without defining it; all column abbreviations should be expanded in captions.
  2. [Throughout] The notation SELFIES/SELFIE is inconsistent; please choose one spelling and use it uniformly.
  3. [Appendix Tables 32-34] Several tables in the appendix have numbers that run together without clear separators, making them very hard to read; please reformat them with proper column alignment.
  4. [Figure 5] The heatmaps do not define their color scale or value range; a colorbar and numeric range should be added.
  5. [Abstract / Resources] The abstract states 'The code and data is available' with only a project page link; please provide a stable repository or DOI, and note the sentence should read 'are available'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MolVision is an empirical benchmark and its claims rest on measured VLM comparisons, not on a derivation that reduces to its own inputs.

full rationale

The paper's central claim (visual information improves molecular property prediction when combined with text) is supported by cross-model benchmark tables, not by a closed-form derivation or a fitted constant renamed as a prediction. The only tuned quantities are ordinary training/contrastive hyperparameters (tau = 0.5, lambda, Tanimoto threshold > 0.85) selected on training data and evaluated on held-out test splits, which is standard supervised benchmarking rather than a definitional identity. The proposed T-Aug contrastive loss uses positive pairs defined by Tanimoto fingerprint similarity, computed from molecular structure independently of the property labels being predicted, so the reported gains are not forced by construction. The paper does not invoke a uniqueness theorem from prior work by the same authors, and I found no self-citations by the author team that carry a load-bearing argument. The concern raised about the BLIP-2 text-only ESOL baseline (RMSE 9.89) is an experimental validity issue about output parsing and controlled comparison in Table 6, not a circularity: even if that baseline is broken, the image-plus-text result does not equal the text-only input by definition. Appendix A limitations concern adaptation of closed-source models and multi-image input, and do not disclose any circular step. Accordingly, no step in the paper's reasoning reduces to its own input, and the non-finding of circularity is the honest verdict.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The benchmark's claims rest on the chosen image rendering, the assumed standard data splits (not stated), and the accuracy of output parsing, together with a set of contrastive hyperparameters that are not all reported.

free parameters (5)
  • tau (NT-Xent temperature) = 0.5
    Chosen for the contrastive loss in Eq. 1; no ablation on validation is reported (Appendix G.3).
  • lambda (contrastive loss weight) = not reported
    Weights L_contrastive in Eq. 2; no value or tuning curve is given (Appendix G.3).
  • Tanimoto similarity threshold for positive pairs = 0.85
    Threshold used to define structurally similar positive pairs in T-Aug (Appendix G.2).
  • number of Tanimoto positive pairs = 3
    Each molecule is paired with three similar molecules (Appendix G.2).
  • LoRA hyperparameters (rank, alpha, LR, epochs) = not reported
    LoRA fine-tuning is used for all finetuning tables but no hyperparameters or seeds are given (Section 3.1).
assumptions (3)
  • domain assumption RDKit-rendered 2D skeletal images are a valid visual representation of molecular structure for VLM processing.
    The whole benchmark construction assumes these images encode chemically meaningful information (Section 4, Figure 4).
  • domain assumption Evaluation splits match the standard splits of the original datasets (random or scaffold) and no test labels leak into LoRA or contrastive training.
    The paper does not state the splitting protocol for the image-augmented benchmark (Section 4).
  • ad hoc to paper VLM output parsing (extracting Yes/No or a number from free-form generation) is accurate and consistent across conditions.
    The text-only versus image-plus-text comparison in Table 6 shows a 9x RMSE gap on ESOL, suggesting parsing failures in the text-only condition rather than chemical understanding differences.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MolVision: Molecular Property Prediction with Vision Language Models." pith.science (2026). https://pith.science/paper/W2NXM5JI

@misc{pith2026250703283,
  author       = {Pith},
  title        = {Pith review of: MolVision: Molecular Property Prediction with Vision Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W2NXM5JI}},
  note         = {Machine review of arXiv:2507.03283}
}
abstract

Molecular property prediction is a fundamental task in computational chemistry with critical applications in drug discovery and materials science. While recent works have explored Large Language Models (LLMs) for this task, they primarily rely on textual molecular representations such as SMILES/SELFIES, which can be ambiguous and structurally less informative. In this work, we introduce MolVision, a novel approach that leverages Vision-Language Models (VLMs) by integrating both molecular structure as images and textual descriptions to enhance property prediction. We construct a benchmark spanning ten diverse datasets, covering classification, regression and description tasks. Evaluating nine different VLMs in zero-shot, few-shot, and fine-tuned settings, we find that visual information improves prediction performance, particularly when combined with efficient fine-tuning strategies such as LoRA. Our results reveal that while visual information alone is insufficient, multimodal fusion significantly enhances generalization across molecular properties. Adaptation of vision encoder for molecular images in conjunction with LoRA further improves the performance. The code and data is available at : $\href{https://molvision.github.io/MolVision/}{https://molvision.github.io/MolVision/}$.

Figures

Figures reproduced from arXiv: 2507.03283 by the authors.

Figure 1
Figure 1. MolVision overview: Average performance comparison of models in zero-shot (ZS), in￾context (ICL), chain-of-thoughts (CoT), and finetuning (FT) for classification (Left ↑) and regression tasks (Center ↓). (Right:) Impact of using visual information on model performance (↑) (JanusPro). Abstract Molecular property prediction is a fundamental task in computational chemistry with critical applications in drug discovery a… view at source ↗
Figure 2
Figure 2. MolVision comparison: Comparison of relevant molecular property prediction approaches. structures are inherently visual, leveraging vision in molecular analysis presents an exciting, yet underexplored, research direction. Recent works such as ChemLLM [9] and ChemLLM-Bench [10] have begun to explore LLMs for molecular property prediction. These methods primarily rely on textual molecular representations, such as SMIL… view at source ↗
Figure 3
Figure 3. Overview of visual-textual approach for property prediction: The image representation along with textual description are used as input by the VLM where the image is encoded by a vision encoder and textual description is encoded by a text encoder. These multimodal features are used to generate the output with the help of a decoder. We show template prompt used for property prediction, including general outline, task … view at source ↗
Figures from the paper (18 more)
Figure 4
Figure 4. Figure 4: Sample visual and textual representation pairs: The images in top row shows skeletal structure of molecules and bottom row shows their corresponding SMILES representations. Text prompt: The prompt consists of three components passed to the text encoder: 1) General outl…
Figure 5
Figure 5. Figure 5: Zero-shot generalization: Visualization of accuracy (left) and F1-score (right) for zero-shot cross-dataset perfor￾mance using BLIP-2. Each heatmap illustrates results from fine-tuning on one dataset (y-axis) and evaluating on others. Models Zero ICL CoT LoRA GPT-4v 2/…
Figure 6
Figure 6. Figure 6: Analysis on molecular-size, model-size and effect of in-context examples: The first plot shows the impact of molecular size on regression error in LD50 with JanusPro, highlighting how visual data improves performance. The middle figure shows comparison of VLMs across d…
Figure 7
Figure 7. Figure 7: Analysis on finetuning, temperature and SELFIES vs. SMILES: The first plot shows the impact of percentage of finetuning data. The middle figure shows performance variation with temperature across datasets for BLIP2. The last figure shows analysis of SMILE vs SELFIES st…
Figure 8
Figure 8. Figure 8: Analyzing visual features: The left two plots show t-SNE visualizations of visual encodings of BLIP-2 before and after cross-modal fusion respectively. The right two plots show corresponding t-SNE plots with the proposed contrastive loss using Tanimoto augmentation (T-…
Figure 9
Figure 9. Figure 9: Performance comparison of Vision-Language Models (VLMs) across BACE, BBBP, HIV, [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: shows categorization of these datasets by tasks, Classification, Regression and Molecular Description. The default representation that is included with these datasets is SMILES and we generated the corresponding SELFIES representation and performed additional evaluati…
Figure 11
Figure 11. Figure 11: Radar plot comparing regression performance across various models (GPT-4v, GPT-4o, [PITH_FULL_IMAGE:figures/full_fig_p024_11.png]
Figure 12
Figure 12. Figure 12: Analyzing visual features: The two plots show t-SNE visualizations of visual encodings of BLIP-2 before and after cross-modal fusion respectively using augmentative technique. (positive pairs) while pushing apart embeddings of dissimilar molecules (negative pairs) in …
Figure 13
Figure 13. Figure 13: Sample prompt for BACE-V: A general outline is provided at first followed by set of instructions to be more specific about the task. The task is explained briefly and expected output is stated. In our case it should be Yes/No. This includes ICL examples with k=2 (No o…
Figure 14
Figure 14. Figure 14: Example prompt: The figure presents a task designed for predicting molecular properties, specifically penetration through the blood-brain barrier (BBBP-V dataset), using the SMILES string representation. The general outline and specific instructions detail the expecte…
Figure 15
Figure 15. Figure 15: Example prompt: The figure outlines a task for predicting the ability of molecules to inhibit HIV replication (HIV-V dataset), based on their SMILES string representation. The general outline and specific instructions require a binary output (Yes/No). Example molecule…
Figure 16
Figure 16. Figure 16: Example prompt: The figure outlines a task for predicting whether molecules are clinically trial-toxic (ClinTox-V dataset), using their SMILES string representation. The general outline and specific instructions require a binary output (Yes/No) to indicate if the mole…
Figure 17
Figure 17. Figure 17: Example prompt: The figure outlines a task for predicting the toxicity of molecules based on their SMILES string representation, specifically in the context of the Tox21 dataset. The general outline and specific instructions require a binary output (Yes/No) to indicat…
Figure 18
Figure 18. Figure 18: Example prompt: The figure outlines a task for predicting the ability of molecules for BACE-Inhibit (BACE-V dataset), using their SELFIES string representation. The general outline and specific instructions require a binary output (Yes/No). Example molecules are provi…
Figure 19
Figure 19. Figure 19: Example prompt: The figure outlines a task for predicting the log solubility of molecules based on their SMILES string representation, using the ESOL dataset. The general outline and specific instructions require a numerical output for log solubility in mols per litre…
Figure 20
Figure 20. Figure 20: Example Molecular Description prompt: The figure outlines a task for predicting the Molecular Description based on their SMILES string representation, using the Chebi dataset. The general outline and specific instructions requires a captioning output. 38 [PITH_FULL_I…
Figure 21
Figure 21. Figure 21: Example CoT prompt: The figure outlines a task for predicting the Brain Penetration of molecules based on their SMILES string representation, using the BBBP dataset. The general outline and specific instructions requires a binary output (Yes/No). 39 [PITH_FULL_IMAGE:…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 45 canonical work pages

  1. [1]

    Gpt-4 technical report,

    OpenAI, J. Achiam, and et. al., “Gpt-4 technical report,” 2024

    work page 2024

  2. [2]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. , “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  3. [3]

    Gpt-3: Its nature, scope, limits, and consequences,

    L. Floridi and M. Chiriatti, “Gpt-3: Its nature, scope, limits, and consequences,” Minds and Machines, vol. 30, pp. 681–694, 2020

    work page 2020

  4. [4]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V . Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan, M. Kardas, V . Kerkez, M. Khabsa, I. Kloumann, A. Koren...

    2023

  5. [5]

    Mistral 7b,

    A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. de las Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, L. R. Lavaud, M.-A. Lachaux, P. Stock, T. L. Scao, T. Lavril, T. Wang, T. Lacroix, and W. E. Sayed, “Mistral 7b,” 2023

    2023

  6. [6]

    Visual instruction tuning,

    H. Liu, C. Li, Q. Wu, and Y . J. Lee, “Visual instruction tuning,” 2023

    2023

  7. [7]

    Llama-adapter v2: Parameter-efficient visual instruction model,

    P. Gao, J. Han, R. Zhang, Z. Lin, S. Geng, A. Zhou, W. Zhang, P. Lu, C. He, X. Yue, H. Li, and Y . Qiao, “Llama-adapter v2: Parameter-efficient visual instruction model,” 2023

    work page 2023

  8. [8]

    Gemini: A family of highly capable multimodal models,

    G. Team and et. al., “Gemini: A family of highly capable multimodal models,” 2024

    work page 2024

Show all 57 references
  1. [9]

    Chemllm: A chemical large language model,

    D. Zhang, W. Liu, Q. Tan, J. Chen, H. Yan, Y . Yan, J. Li, W. Huang, X. Yue, W. Ouyang, D. Zhou, S. Zhang, M. Su, H.-S. Zhong, and Y . Li, “Chemllm: A chemical large language model,” 2024

    2024

  2. [10]

    What can large language models do in chemistry? a comprehensive benchmark on eight tasks,

    T. Guo, K. Guo, B. Nan, Z. Liang, Z. Guo, N. V . Chawla, O. Wiest, and X. Zhang, “What can large language models do in chemistry? a comprehensive benchmark on eight tasks,” 2023

    2023

  3. [11]

    Highly accurate quantum chemical property prediction with uni-mol+,

    S. Lu, Z. Gao, D. He, L. Zhang, and G. Ke, “Highly accurate quantum chemical property prediction with uni-mol+,” arXiv preprint arXiv:2303.16982, 2023

    2023 arXiv

  4. [12]

    Artificial intelligence for drug discovery: Are we there yet?,

    C. Hasselgren and T. I. Oprea, “Artificial intelligence for drug discovery: Are we there yet?,” Annual Review of Pharmacology and Toxicology, vol. 64, p. 527–550, Jan. 2024

    2024

  5. [13]

    Molecular fingerprints for robust and efficient ml-driven molecular generation,

    R. N. Tazhigulov, J. Schiller, J. Oppenheim, and M. Winston, “Molecular fingerprints for robust and efficient ml-driven molecular generation,” 2022

    2022

  6. [14]

    Coarse-grained configurational polymer fingerprints for property prediction using machine learning,

    I. Kumar and P. K. Jha, “Coarse-grained configurational polymer fingerprints for property prediction using machine learning,” 2023

    2023

  7. [15]

    Todeschini and V

    R. Todeschini and V . Consonni,Handbook of molecular descriptors . John Wiley & Sons, 2008

    2008

  8. [16]

    Chemnlp: A natural language-processing-based library for materials chemistry text data,

    K. Choudhary and M. L. Kelley, “Chemnlp: A natural language-processing-based library for materials chemistry text data,” The Journal of Physical Chemistry C , vol. 127, p. 17545–17555, Aug. 2023. 10

    2023

  9. [17]

    Exploring chemical space using natural language processing methodologies for drug discovery,

    H. Öztürk, A. Özgür, P. Schwaller, T. Laino, and E. Ozkirimli, “Exploring chemical space using natural language processing methodologies for drug discovery,” Drug Discovery Today, vol. 25, p. 689–705, Apr. 2020

    2020

  10. [18]

    Natural language processing models that automate programming will transform chemistry research and teaching,

    G. M. Hocky and A. D. White, “Natural language processing models that automate programming will transform chemistry research and teaching,” Digital Discovery, vol. 1, no. 2, p. 79–83, 2022

    2022

  11. [19]

    Neural message passing for quantum chemistry,

    J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, “Neural message passing for quantum chemistry,” in International conference on machine learning , pp. 1263–1272, PMLR, 2017

    2017

  12. [20]

    Moleculenet: a benchmark for molecular machine learning,

    Z. Wu, B. Ramsundar, E. N. Feinberg, J. Gomes, C. Geniesse, A. S. Pappu, K. Leswing, and V . Pande, “Moleculenet: a benchmark for molecular machine learning,” Chemical science, vol. 9, no. 2, pp. 513–530, 2018

    2018

  13. [21]

    Smiles, a chemical language and information system. 1. introduction to method- ology and encoding rules,

    D. Weininger, “Smiles, a chemical language and information system. 1. introduction to method- ology and encoding rules,” Journal of Chemical Information and Computer Sciences , vol. 28, no. 1, pp. 31–36, 1988

    1988

  14. [22]

    A self-attention based message passing neural network for predicting molecular lipophilicity and aqueous solubility,

    B. Tang, S. T. Kramer, M. Fang, Y . Qiu, Z. Wu, and D. Xu, “A self-attention based message passing neural network for predicting molecular lipophilicity and aqueous solubility,”Journal of cheminformatics, vol. 12, pp. 1–9, 2020

    2020

  15. [23]

    Deep learning for computational chemistry,

    G. B. Goh, N. O. Hodas, and A. Vishnu, “Deep learning for computational chemistry,” Journal of computational chemistry, vol. 38, no. 16, pp. 1291–1307, 2017

    2017

  16. [24]

    Deep learning for molecular design—a review of the state of the art,

    D. C. Elton, Z. Boukouvalas, M. D. Fuge, and P. W. Chung, “Deep learning for molecular design—a review of the state of the art,”Molecular Systems Design & Engineering , vol. 4, no. 4, pp. 828–849, 2019

    2019

  17. [25]

    Language models are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. , “Language models are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020

    1901

  18. [26]

    Eva: Exploring the limits of masked visual representation learning at scale,

    Y . Fang, W. Wang, B. Xie, Q. Sun, L. Wu, X. Wang, T. Huang, X. Wang, and Y . Cao, “Eva: Exploring the limits of masked visual representation learning at scale,” 2022

    2022

  19. [27]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning transferable visual models from natural language supervision,” 2021

    2021

  20. [28]

    Judging llm-as-a-judge with mt-bench and chatbot arena,

    L. Zheng, W.-L. Chiang, Y . Sheng, S. Zhuang, Z. Wu, Y . Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica, “Judging llm-as-a-judge with mt-bench and chatbot arena,” 2023

    2023

  21. [29]

    Flamingo: a visual language model for few-shot learning,

    J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y . Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds, R. Ring, E. Rutherford, S. Cabi, T. Han, Z. Gong, S. Samangooei, M. Monteiro, J. Menick, S. Borgeaud, A. Brock, A. Nematzadeh, S. Sharifzadeh, M. Binkowski, R. Barre...

    2022

  22. [30]

    Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts,

    S. Changpinyo, P. Sharma, N. Ding, and R. Soricut, “Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts,” 2021

    2021

  23. [31]

    Microsoft coco: Common objects in context,

    T.-Y . Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, C. L. Zitnick, and P. Dollár, “Microsoft coco: Common objects in context,” 2015

    2015

  24. [32]

    Vqa: Visual question answering,

    A. Agrawal, J. Lu, S. Antol, M. Mitchell, C. L. Zitnick, D. Batra, and D. Parikh, “Vqa: Visual question answering,” 2016

    2016

  25. [33]

    Improved baselines with visual instruction tuning,

    H. Liu, C. Li, Y . Li, and Y . J. Lee, “Improved baselines with visual instruction tuning,” 2024

    2024

  26. [34]

    Why is tanimoto index an appropriate choice for fingerprint-based similarity calculations?,

    H. K. Bajusz D, Rácz A, “Why is tanimoto index an appropriate choice for fingerprint-based similarity calculations?,” 2015. 11

    2015

  27. [35]

    nach0: Multimodal natural and chemical languages foundation model,

    M. Livne, Z. Miftahutdinov, E. Tutubalina, M. Kuznetsov, D. Polykovskiy, A. Brundyn, A. Jhun- jhunwala, A. Costa, A. Aliper, A. Aspuru-Guzik, and A. Zhavoronkov, “nach0: Multimodal natural and chemical languages foundation model,” 2024

    2024

  28. [36]

    Molca: Molecular graph-language modeling with cross-modal projector and uni-modal adapter,

    Z. Liu, S. Li, Y . Luo, H. Fei, Y . Cao, K. Kawaguchi, X. Wang, and T.-S. Chua, “Molca: Molecular graph-language modeling with cross-modal projector and uni-modal adapter,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pp. 15623–15...

    2023

  29. [37]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,

    J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,” 2023

    2023

  30. [38]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  31. [39]

    Janus-pro: Uni- fied multimodal understanding and generation with data and model scaling,

    X. Chen, Z. Wu, X. Liu, Z. Pan, W. Liu, Z. Xie, X. Yu, and C. Ruan, “Janus-pro: Uni- fied multimodal understanding and generation with data and model scaling,” arXiv preprint arXiv:2501.17811, 2025

    2025 arXiv

  32. [40]

    Cogvlm: Visual expert for pretrained language models,

    W. Wang, Q. Lv, W. Yu, W. Hong, J. Qi, Y . Wang, J. Ji, Z. Yang, L. Zhao, X. Song, J. Xu, B. Xu, J. Li, Y . Dong, M. Ding, and J. Tang, “Cogvlm: Visual expert for pretrained language models,” 2024

    2024

  33. [41]

    mplug-owl: Modularization empowers large language models with multimodality,

    Q. Ye, H. Xu, G. Xu, J. Ye, M. Yan, Y . Zhou, J. Wang, A. Hu, P. Shi, Y . Shi, C. Li, Y . Xu, H. Chen, J. Tian, Q. Qian, J. Zhang, F. Huang, and J. Zhou, “mplug-owl: Modularization empowers large language models with multimodality,” 2024

    2024

  34. [42]

    Rdkit documentation,

    G. Landrum, “Rdkit documentation,” Release, vol. 1, no. 1-79, p. 4, 2013

    2013

  35. [43]

    Computational modeling ofβ- secretase 1 (bace-1) inhibitors using ligand based approaches,

    G. Subramanian, B. Ramsundar, V . Pande, and R. A. Denny, “Computational modeling ofβ- secretase 1 (bace-1) inhibitors using ligand based approaches,”Journal of Chemical Information and Modeling, vol. 56, no. 10, pp. 1936–1949, 2016. PMID: 27689393

    1936

  36. [44]

    Prediction of blood-brain barrier penetration (bbbp) based on molecular descriptors of the free-form and in-blood-form datasets. molecules.,

    O. T. Sakiyama H, Fukuda M, “Prediction of blood-brain barrier penetration (bbbp) based on molecular descriptors of the free-form and in-blood-form datasets. molecules.,” 2021

    2021

  37. [45]

    The nci’s aids antiviral drug screening program,

    Y . SS, “The nci’s aids antiviral drug screening program,” 1995

    1995

  38. [46]

    Tox21challenge to build predictive models of nuclear receptor and stress response pathways as mediated by exposure to environmental chemicals and drugs,

    R. Huang, M. Xia, D.-T. Nguyen, T. Zhao, S. Sakamuru, J. Zhao, S. A. Shahane, A. Rossoshek, and A. Simeonov, “Tox21challenge to build predictive models of nuclear receptor and stress response pathways as mediated by exposure to environmental chemicals and drugs,” Frontiers in ...

    2016

  39. [47]

    Esol: estimating aqueous solubility directly from molecular structure,

    J. S. Delaney, “Esol: estimating aqueous solubility directly from molecular structure,” Journal of chemical information and computer sciences , vol. 44, no. 3, pp. 1000–1005, 2004

    2004

  40. [48]

    Variability of ld50 values from rat oral acute toxicity studies: implications for alternative model development,

    A. Karmaus, J. Fitzpatrick, D. Allen, G. Patlewicz, N. Kleinstreuer, and W. Casey, “Variability of ld50 values from rat oral acute toxicity studies: implications for alternative model development,” Society of Toxicology, San Antonio, TX, vol. 3, pp. 11–15, 2018

    2018

  41. [49]

    Commentary: The materials project: A materials genome approach to accelerating materials innovation,

    A. Jain, S. P. Ong, G. Hautier, W. Chen, W. D. Richards, S. Dacek, S. Cholia, D. Gunter, D. Skinner, G. Ceder,et al., “Commentary: The materials project: A materials genome approach to accelerating materials innovation,” APL materials, vol. 1, no. 1, 2013

    2013

  42. [50]

    Ogb-lsc: A large-scale challenge for machine learning on graphs,

    W. Hu, M. Fey, H. Ren, M. Nakata, Y . Dong, and J. Leskovec, “Ogb-lsc: A large-scale challenge for machine learning on graphs,” arXiv preprint arXiv:2103.09430, 2021

    2021 arXiv

  43. [51]

    Text2Mol: Cross-modal molecule retrieval with natural language queries,

    C. Edwards, C. Zhai, and H. Ji, “Text2Mol: Cross-modal molecule retrieval with natural language queries,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (M.-F. Moens, X. Huang, L. Specia, and S. W.-t. Yih, eds.), (Online and Punta Can...

    2021

  44. [52]

    What can large language models do in chemistry? a comprehensive benchmark on eight tasks,

    T. Guo, B. Nan, Z. Liang, Z. Guo, N. Chawla, O. Wiest, X. Zhang, et al., “What can large language models do in chemistry? a comprehensive benchmark on eight tasks,” Advances in Neural Information Processing Systems, vol. 36, pp. 59662–59688, 2023

    2023

  45. [53]

    Galactica: A large language model for science,

    R. Taylor, M. Kardas, G. Cucurull, T. Scialom, A. Hartshorn, E. Saravia, A. Poulton, V . Kerkez, and R. Stojnic, “Galactica: A large language model for science,” arXiv preprint arXiv:2211.09085, 2022

    2022 arXiv

  46. [54]

    Transitioning the generalised read-across approach (genra) to quantitative predictions: a case study using acute oral toxicity data,

    G. Helman, I. Shah, and G. Patlewicz, “Transitioning the generalised read-across approach (genra) to quantitative predictions: a case study using acute oral toxicity data,” Computational Toxicology, vol. 12, p. 100097, 2019

    2019

  47. [55]

    Translation between molecules and natural language,

    C. Edwards, T. Lai, K. Ros, G. Honke, K. Cho, and H. Ji, “Translation between molecules and natural language,” 2022

    2022

  48. [56]

    Selfies and the future of molecular string representations,

    M. Krenn, Q. Ai, S. Barthel, N. Carson, A. Frei, N. C. Frey, P. Friederich, T. Gaudin, A. A. Gayle, K. M. Jablonka, et al. , “Selfies and the future of molecular string representations,” Patterns, vol. 3, no. 10, 2022

    2022

  49. [57]

    A simple framework for contrastive learning of visual representations,

    T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in International conference on machine learning , pp. 1597–1607, PmLR, 2020. 13 Table of Contents A Limitations 15 B Ethical considerations 15 C Datasets:...

    2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.