Pith. sign in

REVIEW 5 cited by

Molecule Attention Transformer

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2002.08264 v1 pith:CYNLGDAD submitted 2020-02-19 cs.LG physics.comp-phstat.ML

{L}ukasz Maziarka , Tomasz Danel , S{l}awomir Mucha , Krzysztof Rataj , Jacek Tabor , Stanis{l}aw Jastrzębski This is my paper · ORCID
classification cs.LGphysics.comp-phstat.ML
keywords attentionmoleculetaskstransformercompetitivelymolecularperformsprediction
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Designing a single neural network architecture that performs competitively across a range of molecule property prediction tasks remains largely an open challenge, and its solution may unlock a widespread use of deep learning in the drug discovery industry. To move towards this goal, we propose Molecule Attention Transformer (MAT). Our key innovation is to augment the attention mechanism in Transformer using inter-atomic distances and the molecular graph structure. Experiments show that MAT performs competitively on a diverse set of molecular prediction tasks. Most importantly, with a simple self-supervised pretraining, MAT requires tuning of only a few hyperparameter values to achieve state-of-the-art performance on downstream tasks. Finally, we show that attention weights learned by MAT are interpretable from the chemical point of view.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Predicting Therapeutic Outcome via Aligning Patient-Specific Knowledge Graph and Gene-Level Perturbation Representations

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Aligning patient-specific gene-regulatory graphs with LINCS-pretrained perturbation embeddings via CLIP-style contrastive learning improves clinical drug-response prediction on TCGA and zero-shot I-SPY2.

  2. From Molecules to Mixtures: Learning Representations of Olfactory Mixture Similarity using Inductive Biases

    cs.LG 2025-01 conditional novelty 6.0 of 10

    POMMix, a graph-based model with attention and cosine similarity heads, extends the Principal Odor Map to predict human perceptual similarity of odor mixtures, reporting a test correlation of 0.78 on a compiled datase...

  3. Multi-Alignment Contrastive Learning for Enzyme--Reaction Retrieval

    q-bio.BM 2025-12 conditional novelty 5.0 of 10

    FGW-CLIP, a contrastive method that aligns enzymes and reactions while also aligning within-domain EC structure with a Gromov-Wasserstein regularizer, reports state-of-the-art retrieval on EnzymeMap and ReactZyme.

  4. KEPLA: A Knowledge-Enhanced Deep Learning Framework for Accurate Protein-Ligand Binding Affinity Prediction

    cs.LG 2025-06 conditional novelty 5.0 of 10

    KEPLA jointly optimizes knowledge graph embeddings and cross attention to improve protein-ligand affinity prediction, reporting RMSE 1.202 on PDBbind core and 1.459 on CSAR-HiQ, beating all tested baselines.

  5. Multi-Level Fusion Graph Neural Network for Molecule Property Prediction

    cs.LG 2025-07 conditional novelty 4.0 of 10

    MLFGNN, a GAT combined with a DyT-augmented Graph Transformer and fingerprint cross-attention, reports best regression scores on five MoleculeNet benchmarks and best classification scores on two of five.

Pith tools