pith. sign in

arxiv: 2206.00133 · v2 · pith:7PNKTCILnew · submitted 2022-05-31 · 💻 cs.LG · q-bio.BM· stat.ML

Pre-training via Denoising for Molecular Property Prediction

Sheheryar Zaidi , Michael Schaarschmidt , James Martens , Hyunjik Kim , Yee Whye Teh , Alvaro Sanchez-Gonzalez , Peter Battaglia , Razvan Pascanu
show 1 more author
Jonathan Godwin
This is my paper
classification 💻 cs.LG q-bio.BMstat.ML
keywords moleculardenoisingpre-trainingpredictionpropertystructuresdatasetdatasets
0
0 comments X
read the original abstract

Many important problems involving molecular property prediction from 3D structures have limited data, posing a generalization challenge for neural networks. In this paper, we describe a pre-training technique based on denoising that achieves a new state-of-the-art in molecular property prediction by utilizing large datasets of 3D molecular structures at equilibrium to learn meaningful representations for downstream tasks. Relying on the well-known link between denoising autoencoders and score-matching, we show that the denoising objective corresponds to learning a molecular force field -- arising from approximating the Boltzmann distribution with a mixture of Gaussians -- directly from equilibrium structures. Our experiments demonstrate that using this pre-training objective significantly improves performance on multiple benchmarks, achieving a new state-of-the-art on the majority of targets in the widely used QM9 dataset. Our analysis then provides practical insights into the effects of different factors -- dataset sizes, model size and architecture, and the choice of upstream and downstream datasets -- on pre-training.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. h-MINT: Modeling Pocket-Ligand Binding with Hierarchical Molecular Interaction Network

    cs.LG 2026-04 unverdicted novelty 7.0

    h-MINT improves ligand-protein binding affinity prediction by 2-4% and virtual screening metrics by 1-3% via overlapping fragment tokenization and hierarchical modeling.

  2. DiffusionBench: On Holistic Evaluation of Diffusion Transformers

    cs.CV 2026-06 conditional novelty 6.0

    NanoGen unifies DiT training on ImageNet and T2I, reveals negative Pearson correlations (-0.377 to -0.580) in method rankings across metrics from 21 models, and motivates DiffusionBench for holistic evaluation.

  3. Toward Better Geometric Representations for Molecule Generative Models

    cs.LG 2026-05 unverdicted novelty 6.0

    LENSEs improves representation-conditioned molecule generation by jointly training a multi-level representation head, perceptual loss, and REPA alignment on pretrained encoders, yielding 97.28% validity and 98.51% sta...

  4. A probabilistic framework for crystal structure denoising, phase classification, and order parameters

    cond-mat.mtrl-sci 2025-12 unverdicted novelty 6.0

    A unified probabilistic model uses per-atom logits over crystal prototypes to denoise atomic configurations, classify phases, and derive order parameters from a single differentiable scalar field.

  5. TriForces: Augmenting Atomistic GNNs for Transferable Representations

    cs.LG 2026-05 unverdicted novelty 5.0

    TriForces adds a model-agnostic three-stream architecture plus self-supervised objectives to atomistic GNNs, improving transfer performance on MatBench, QM9, and limited-data OMat24 without DFT labels.

  6. On Improving Graph Neural Networks for QSAR by Pre-training on Extended-Connectivity Fingerprints

    cs.LG 2026-05 unverdicted novelty 4.0

    Pre-training GNNs on ECFP prediction produces statistically significant QSAR gains on five of six Biogen benchmarks with OOD splits, but underperforms on heterogeneous datasets and complex endpoints like binding affinity.