MolE: a molecular foundation model for drug discovery
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:J6DAO7UGrecord.jsonopen to challenge →
read the original abstract
Models that accurately predict properties based on chemical structure are valuable tools in drug discovery. However, for many properties, public and private training sets are typically small, and it is difficult for the models to generalize well outside of the training data. Recently, large language models have addressed this problem by using self-supervised pretraining on large unlabeled datasets, followed by fine-tuning on smaller, labeled datasets. In this paper, we report MolE, a molecular foundation model that adapts the DeBERTa architecture to be used on molecular graphs together with a two-step pretraining strategy. The first step of pretraining is a self-supervised approach focused on learning chemical structures, and the second step is a massive multi-task approach to learn biological information. We show that fine-tuning pretrained MolE achieves state-of-the-art results on 9 of the 22 ADMET tasks included in the Therapeutic Data Commons.
This paper has not been read by Pith yet.
Forward citations
Cited by 3 Pith papers
-
Interpretable Meta-Learning for Multi-Objective Chemical Search
Linear meta-learning surrogates trained across chemical objectives and auxiliary properties adapt rapidly to new multi-objective molecular searches and outperform baselines by 78% in Pareto performance on spin-crossov...
-
The Metric Picks the Winner: Evaluation Choice Flips Model Rankings for Drug-Response Prediction in Unseen Chemistry
Under Bemis-Murcko scaffold split on THP-1 DRUG-seq data, inverse-variance proxy ranks linear Morgan fingerprint regression highest while contest wMSE ranks deep fusion models highest, with fusion beating linear by -0...
-
MSAlign: Aligning Molecule and Mass Spectra Foundation Models for Metabolite Identification
MSAlign aligns frozen DreaMS and ChemBERTa models with MLPs and candidate-based contrastive learning to outperform prior methods on molecule retrieval from MS/MS spectra while quantifying distribution shift in data splits.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.