Pith. sign in

REVIEW 3 major objections 7 minor 72 references

Learning the Neighborhood: Contrast-Free Multimodal Self-Supervised Molecular Graph Pretraining

T0 review · 3 major / 7 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read C-FREE learns molecular representations by predicting a subgraph's latent embedding from its complementary neighborhood, and this contrast-free objective outperforms contrastive and generative baselines on MoleculeNet.

desk verdict A simple and promising JEPA-style multimodal pretraining method for molecules—new enough to cite, strong enough to referee, with an evaluation-comparability caveat that needs a close look. read the letter →

arxiv 2509.22468 v2 pith:ORDPFH6U submitted 2025-09-26 cs.LG cs.AI

Boshra Ariguib , Mathias Niepert , Andrei Manolache This is my paper
classification cs.LGcs.AI
keywords molecularrepresentationlearningself-supervisedpretraining3Dconformersego-netscontrast-freegraphneuralnetworksMoleculeNetmultimodal
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

C-FREE is a self-supervised pretraining method for molecular property prediction. It learns by predicting the embedding of a k-hop neighborhood subgraph from the embedding of its complementary neighborhood, using both 2D topology and ensembles of 3D conformers. The authors claim this contrast-free, non-generative objective—no negative samples, no hand-crafted augmentations, no positional encodings, no input-space reconstruction—produces state-of-the-art representations: average ROC-AUC of 71.07 with frozen linear probes and 77.50 with full fine-tuning on MoleculeNet, above the best multimodal baseline (MoleBlend, 76.16). They also claim the method transfers to regression and low-label settings, and that with a DeepSets head it is strictly more expressive than the 1-WL test while remaining computationally cheap. The paper matters because molecular labels are scarce and 3D structure is usually underused; if these claims hold, a simple predictive ego-net objective can replace contrastive and generative pipelines while exploiting 3D conformers.

What carries the argument

The key object is the fixed-radius k-EgoNet: the k-hop neighborhood of a sampled atom, paired with its complement to make a context–target pair. Each subgraph is encoded twice—2D structure by GINE and 3D conformers by SchNet—and the token sequences are concatenated with modality markers and processed by a transformer. The context embedding is fed through a predictor transformer to match the target embedding under a mean-squared L2 loss, and the target encoder is an exponential moving average of the context encoder. This ego-net decomposition is what makes the method contrast-free: instead of pulling positive views together and pushing negatives apart, the model reconstructs a neighborhood's

What would settle it

Run the strongest baselines (e.g., MoleBlend, GraphMVP, GraphMAE) under C-FREE's exact scaffold splits, linear-probe selection rule, and fine-tuning protocol with the same RDKit-generated conformers. If their average ROC-AUC rises to within noise of C-FREE's 77.50, the claimed state-of-the-art margin collapses. Independently, compute the scaffold overlap between GEOM pretraining molecules and the MoleculeNet test sets; high overlap would indicate leakage that could explain the gains.

Watch

Extended reading notes

Core claim

The central claim is that the right self-supervised task for molecules is not matching augmented views or reconstructing graphs, but aligning a subgraph with its complement in latent space. C-FREE samples atoms, extracts fixed-radius k-ego-nets (k=3 or 4) as targets and the remaining graph as context, encodes both views with a multimodal backbone (GINE for 2D, SchNet for 3D, fused by a transformer), and minimizes the L2 distance between a predicted context embedding and the target embedding, with an EMA-updated target encoder preventing collapse. The paper reports that this objective, pretrained on 0.33M GEOM molecules, sets a new average ROC-AUC on MoleculeNet: 71.07 for frozen linear probe

Load-bearing premise

The load-bearing premise is that the published baselines are directly comparable under the same scaffold splits, probe-selection rules, and 3D-conformer protocol, and that RDKit-generated conformers at fine-tuning transfer the GEOM pretraining signal; if the harnesses differ or GEOM overlaps the MoleculeNet scaffolds, the margins could shrink or vanish.

Editorial extensions

If this is right

  • A frozen GEOM-pretrained backbone works as a general feature extractor: linear probes on C-FREE embeddings beat all compared pretraining baselines on MoleculeNet's eight classification tasks on average.
  • The same backbone transfers to molecular regression: pretrained models start with lower error and converge faster than random initialization on the Kraken Sterimol tasks.
  • The method exploits 3D conformers when present but does not require them: the 2D-only variant still outperforms the published baselines on average, and the modality ablation shows 3D and 2D contribute complementary signal.
  • In label-scarce regimes, pretraining is the main driver of performance: at 1% and 10% of the Drugs-75K labels, fine-tuned pretrained models beat models trained from scratch.
  • The architecture with a DeepSets head simulates ESAN and is provably more expressive than 1-WL; the EXP benchmark results place 2- and 3-ego-net variants at about 99% accuracy where 1-WL GNNs are stuck at chance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A head-to-head re-run of the published baselines under C-FREE's exact scaffold splits, checkpoint-selection rule, and RDKit conformer pipeline would be the cleanest test of the claimed margin; the paper does not report such a re-run, nor an overlap analysis between GEOM pretraining molecules and MoleculeNet scaffolds.
  • Because the ego-net objective is a generic structural prior, the same contrast-free recipe could transfer to proteins, materials, or reaction graphs where 3D conformer ensembles are available—domains the paper does not evaluate.
  • The paper's own caveat that 1-WL already distinguishes nearly all molecular graphs suggests the expressiveness result may matter more for non-molecular graphs; one could test this by running C-FREE on graph datasets designed to require higher-order substructure awareness beyond molecules.
  • A direct ablation separating the ego-net objective from the multimodal backbone—for example, pretraining the same encoder with whole-graph prediction instead of subgraph-complement prediction—would isolate how much of the gain actually comes from neighborhood prediction, a comparison the paper does not make.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper presents C-FREE, a self-supervised framework that pretrains a GINE/SchNet/Transformer encoder by predicting the embedding of a target k-ego-net subgraph from its complementary context subgraph, using an EMA target encoder and an L2 loss. Pretraining is on GEOM (304k molecules); downstream evaluation includes MoleculeNet classification under frozen linear probe and full fine-tuning, regression on Kraken, label-efficiency on Drugs-75K, and expressiveness on EXP. The central claims are state-of-the-art MoleculeNet results (average ROC-AUC 77.50 full fine-tuning, 71.07 linear probe), strong 2D-only transfer, and >1-WL expressiveness with a DeepSets head.

Significance. If the empirical claims survive scrutiny, the contribution is useful: a contrast-free, non-generative, multimodal pretraining task with no negatives or clustering, public code/checkpoints, clear ablations, and an honest limitation statement on expressiveness. The Kraken and Drugs-75K experiments provide additional transfer evidence. However, the SOTA claim rests on comparison with published baseline numbers and on GEOM/MoleculeNet overlap assumptions that are not documented.

major comments (3)
  1. [§4.1, Table 1] Table 1 is the sole evidence for the frozen-linear-probe SOTA claim, but it omits all multimodal baselines (GraphMVP, 3D InfoMax, MoleculeSDE, MoleBlend) and several recent 2D methods (GraphMAE, MGSSL, Mole-BERT). The text in §4.1 only claims comparison to 'contrastive and non-contrastive' baselines, yet the abstract and contribution list claim to surpass 'other multimodal self-supervised methods.' To support the SOTA claim, either add these baselines under the same frozen protocol or restrict the claim to the methods actually tabulated.
  2. [§4.1/§4.2.1, Tables 1–2] The reported margins over published baselines are only meaningful if the evaluation harnesses are identical. The paper states that it follows Wang et al. (2023a) and uses the same linear head as Yu et al. (2023), but it does not demonstrate that scaffold splits, checkpoint selection, probe selection, and 3D conformer generation match those used for the tabulated baselines. C-FREE uses three RDKit-generated conformers at fine-tuning; the baselines may use different geometry pipelines. Additionally, no overlap analysis is reported between the GEOM pretraining set and the MoleculeNet test scaffolds, so test molecules may have been seen during pretraining, potentially inflating C-FREE's gains. Please provide split definitions, report overlap statistics, or pretrain on a disjoint set; the Kraken 'disjoint' claim should also be substantiated.
  3. [§4.2.2, Table 4] The Drugs-75K label-efficiency experiments and the Kraken full-fine-tuning results compare C-FREE only against random initialization. They therefore support transfer and label efficiency, but not the paper's broader 'state-of-the-art' or 'surpassing other self-supervised methods' claims in the abstract and introduction. If those claims are intended to cover regression benchmarks, baseline SSL methods need to be evaluated under the same protocol.
minor comments (7)
  1. [§4.2.2] 'Table 10' should be 'Table 4' for the Drugs-75K results; Table 10 is the predictor ablation.
  2. [§4.3] The text lists 'none, a linear predictor, and a transformer' as the three predictor variants, but Table 10 and Fig. 4 use a 2-layer MLP, not a linear predictor. Make the terminology consistent.
  3. [§3.1 / Appendix A.1] The main text calls the expressiveness statement 'Informal Lemma 1', while Appendix A.1 labels the invariance statement Lemma 1 and the expressiveness statement Lemma 2. Renumber to avoid confusion.
  4. [Appendix A.5, Fig. 5] The caption says '(Sterimol BurL and Sterimol L)' but the figure and text describe B5 and BurB5; correct the caption.
  5. [Table 2 caption] 'outperforming both multi-modal baselines' should read 'the multi-modal baselines' since four are listed.
  6. [Eq. (1), §3] The loss uses k both for the ego-net radius and for the number of sampled views; introduce a separate symbol (e.g., K views, r radius).
  7. [§4.1] The sentence about 'three additional conformers generated using RDKit at fine-tuning' is ambiguous—does the multimodal variant receive GEOM conformers, RDKit conformers, or both during fine-tuning? Clarify and state whether baseline multimodal methods use the same geometry inputs.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the SSL objective and expressiveness claims are independent of downstream labels; one minor non-load-bearing self-citation for invariance.

full rationale

The paper's central derivation chain is self-contained. The pretraining objective (Section 3, Eq. 1) is a standard non-contrastive latent alignment: it minimizes the L2 distance between the predictor output on the context subgraph and the target encoder output on the target subgraph, with no downstream labels or benchmark constants entering the loss. Downstream numbers are measured under scaffold splits, not refit. The expressiveness claim (Lemma 2) is explicitly a corollary of an external theorem: the paper states it holds 'under the assumptions from Theorem 2 of (Bevilacqua et al., 2022)', and the appendix proof says 'we can use the same proof argument as in Theorem 2'. Since k-EgoNets with their complements form an edge-covering EGO policy and a DeepSets head gives permutation-invariant aggregation, this reduction is legitimate, not circular. The only self-citation worth flagging is the invariance lemma: Appendix A.1 're-state[s] the result of Manolache et al. (2024)' and delegates the proof to that prior paper, which shares two authors. This is a minor self-citation, but it is not load-bearing: the SOTA results and the expressiveness lemma do not depend on it. The comparability of reprinted baselines is a benchmarking/correctness risk, not a circularity, and the paper's own pretrained-vs-random-init comparisons on Kraken and Drugs-75K are unaffected. No fitted parameter is renamed as a prediction, and no benchmark value is used to design the loss. Overall: no significant circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

C-FREE introduces no new physical entity, force, or conserved quantity. The main assumptions are about the reliability of the encoders, the quality of the conformers, and the comparability of baseline numbers. The free parameters are architectural and training hyperparameters, none of which are fitted to downstream benchmark targets.

free parameters (5)
  • ego-net radius k = 3, 4
    Chosen by hand as fixed-size patches, analogous to vision JEPA. The value affects the size and composition of the prediction task.
  • number of conformers per molecule = 3 (pretraining), 3 RDKit (fine-tuning)
    Chosen by hand; the paper uses 3 GEOM conformers at pretraining and 3 RDKit-generated conformers at fine-tuning for MoleculeNet.
  • EMA decay schedule = tau from 0.995 to 1.0
    Standard in BYOL/I-JEPA; the schedule is a design choice affecting stability.
  • learning rate schedule = 2e-6 warmup to 5e-5 over 30 epochs
    Chosen to stabilize training; not tied to any benchmark metric.
  • architecture dimensions = GINE 128, SchNet 128, Transformer 512/387, 6 layers, 8 heads
    Hyperparameters chosen by hand; reported in Appendix A.4.
assumptions (5)
  • domain assumption SchNet is E(3)-invariant and produces geometrically meaningful atom embeddings
    The 3D encoder is SchNet; the invariance lemma is restated from Manolache et al. (2024) and assumed to hold for the fused 2D+3D sequence.
  • domain assumption A Transformer without positional encodings can approximate the identity map, preserving MPNN expressive power
    Lemma 2's proof in Appendix A.1 relies on the existence of an identity parametrization of the Transformer to conclude that C-FREE DS is as expressive as ESAN.
  • domain assumption GEOM conformers and RDKit-generated conformers are accurate enough for the 3D signal to transfer
    Section 4.1 uses RDKit conformers for MoleculeNet fine-tuning; no validation of conformer quality or consistency with GEOM is reported.
  • domain assumption Published baseline numbers are directly comparable to C-FREE's runs
    Tables 1 and 2 compare against numbers reprinted from Wang et al. (2023a) and Yu et al. (2023) without a shared code harness, identical split seeds, or protocol verification.
  • domain assumption Scaffold splits prevent leakage between GEOM pretraining and MoleculeNet test sets
    The paper pretrains on 304,466 GEOM molecules and fine-tunes on MoleculeNet, but does not report an overlap analysis between GEOM and the MoleculeNet test scaffolds.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning the Neighborhood: Contrast-Free Multimodal Self-Supervised Molecular Graph Pretraining." pith.science (2026). https://pith.science/paper/ORDPFH6U

@misc{pith2026250922468,
  author       = {Pith},
  title        = {Pith review of: Learning the Neighborhood: Contrast-Free Multimodal Self-Supervised Molecular Graph Pretraining},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ORDPFH6U}},
  note         = {Machine review of arXiv:2509.22468}
}
read the original abstract

High-quality molecular representations are essential for property prediction and molecular design, yet large labeled datasets remain scarce. While self-supervised pretraining on molecular graphs has shown promise, many existing approaches either depend on hand-crafted augmentations or complex generative objectives, and often rely solely on 2D topology, leaving valuable 3D structural information underutilized. To address this gap, we introduce C-FREE (Contrast-Free Representation learning on Ego-nets), a simple framework that integrates 2D graphs with ensembles of 3D conformers. C-FREE learns molecular representations by predicting subgraph embeddings from their complementary neighborhoods in the latent space, using fixed-radius ego-nets as modeling units across different conformers. This design allows us to integrate both geometric and topological information within a hybrid Graph Neural Network (GNN)-Transformer backbone, without negatives, positional encodings, or expensive pre-processing. Pretraining on the GEOM dataset, which provides rich 3D conformational diversity, C-FREE achieves state-of-the-art results on MoleculeNet, surpassing contrastive, generative, and other multimodal self-supervised methods. Fine-tuning across datasets with diverse sizes and molecule types further demonstrates that pretraining transfers effectively to new chemical domains, highlighting the importance of 3D-informed molecular representations.

Figures

Figures reproduced from arXiv: 2509.22468 by the authors.

Figure 1
Figure 1. From each molecular graph, we sample a random node and extract its k-EgoNet (Bevilac [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. To generate subgraphs, we sample a random node from the origi￾nal graph (here, the oxygen atom) and extract its 2-EgoNet as the context sub￾graph (outlined by red square). The re￾maining components (outlined by blue squares) constitute the target subgraph. To further diversify training, we sample multiple nodes v1, v2, . . . , vn per molecule and construct their corre￾sponding k-EgoNets E(v1), E(v2), . . . , E(vn), … view at source ↗
Figure 3
Figure 3. Test MAE on the Kraken regression tasks (Sterimol BurL and Sterimol L) with frozen [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Predictor ablation. Left: SSL validation loss on GEOM during pretraining. Right: Aver￾age linear-probe MAE (↓) on Kraken with frozen backbones. Without a predictor, training collapses (loss ∼0) and probes perform worst; an MLP predictor helps but underperforms, while a…
Figure 5
Figure 5. Figure 5: Test MAE on the Kraken regression tasks (Sterimol BurL and Sterimol L) with frozen [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: Test MAE on Kraken regression tasks (Sterimol L,B5, BurB5 and BurL) comparing ran [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

72 extracted references · 1 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    The surprising power of graph neural networks with random node initialization, 2021

    Ralph Abboud, İsmail İlkan Ceylan, Martin Grohe, and Thomas Lukasiewicz. The surprising power of graph neural networks with random node initialization, 2021. URL https://arxiv.org/abs/2010.01179

    arXiv 2021

  3. [3]

    Self- Supervised Learning From Images With a Joint-Embedding Predictive Architecture

    Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self- Supervised Learning From Images With a Joint-Embedding Predictive Architecture . In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition , pp.\ 15619--15629, 2023

    2023

  4. [4]

    Geom, energy-annotated molecular conformations for property prediction and molecular generation

    Simon Axelrod and Rafael Gomez-Bombarelli. Geom, energy-annotated molecular conformations for property prediction and molecular generation. Scientific Data, 9 0 (1): 0 185, 2022

    2022

  5. [5]

    Geometry-aware line graph transformer pre-training for molecular property prediction, 2023

    Peizhen Bai, Xianyuan Liu, and Haiping Lu. Geometry-aware line graph transformer pre-training for molecular property prediction, 2023. URL https://arxiv.org/abs/2309.00483

    arXiv 2023

  6. [6]

    Towards foundational models for molecular learning on large-scale multi-task datasets

    Dominique Beaini, Shenyang Huang, Joao Alex Cunha, Zhiyi Li, Gabriela Moisescu-Pareja, Oleksandr Dymov, Samuel Maddrell-Mander, Callum McLean, Frederik Wenkel, Luis M \"u ller, Jama Hussein Mohamud, Ali Parviz, Michael Craig, Micha Koziarski, Jiarui Lu, Zhaocheng Zhu, Cristian Gabellini, Kerstin Klaser, Josef Dean, Cas Wognum, Maciej Sypetkowski, Guillaum...

    2024

  7. [7]

    Bronstein, and Haggai Maron

    Beatrice Bevilacqua, Fabrizio Frasca, Derek Lim, Balasubramaniam Srinivasan, Chen Cai, Gopinath Balamurugan, Michael M. Bronstein, and Haggai Maron. Equivariant Subgraph Aggregation Networks , March 2022

    2022

  8. [8]

    Design of protein-binding proteins from the target structure alone

    Longxing Cao, Brian Coventry, Inna Goreshnik, Buwei Huang, William Sheffler, Joon Sung Park, Kevin M Jude, Iva Markovi \'c , Rameshwar U Kadam, Koen HG Verschueren, et al. Design of protein-binding proteins from the target structure alone. Nature, 605 0 (7910), 2022

    2022

Show all 72 references
  1. [9]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jegou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp.\ 9630--9640, 2021. doi:10.1109/ICCV4...

    2021

  2. [10]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations . In Guide Proceedings , volume 119, pp.\ 1597--1607. JMLR.org, July 2020. doi:10.5555/3524938.3525087

    2020

  3. [11]

    BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding . In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of...

    2019

  4. [12]

    Elton, Zois Boukouvalas, Mark D

    Daniel C. Elton, Zois Boukouvalas, Mark D. Fuge, and Peter W. Chung. Deep learning for molecular design --- a review of the state of the art . Mol. Syst. Des. Eng., 4 0 (4): 0 828--849, August 2019. ISSN 2058-9689. doi:10.1039/C9ME00039A

    2019 doi

  5. [13]

    UniCorn : A Unified Contrastive Learning Approach for Multi-view Molecular Representation Learning

    Shikun Feng, Yuyan Ni, Minghao Li, Yanwen Huang, Zhi-Ming Ma, Wei-Ying Ma, and Yanyan Lan. UniCorn : A Unified Contrastive Learning Approach for Multi-view Molecular Representation Learning . In Proceedings of the 41st International Conference on Machine Learning , pp.\ 13256-...

    2024

  6. [14]

    Fast graph representation learning with pytorch geometric

    Matthias Fey and Jan Eric Lenssen. Fast graph representation learning with pytorch geometric. arXiv preprint arXiv:1903.02428, 2019

    1903 arXiv

  7. [15]

    Gemnet: Universal directional graph neural networks for molecules

    Johannes Gasteiger, Florian Becker, and Stephan G \"u nnemann. Gemnet: Universal directional graph neural networks for molecules. Advances in Neural Information Processing Systems, 34: 0 6790--6802, 2021

    2021

  8. [16]

    A comprehensive discovery platform for organophosphorus ligands for catalysis

    Tobias Gensch, Gabriel dos Passos Gomes, Pascal Friederich, Ellyn Peters, Th \'e ophile Gaudin, Robert Pollice, Kjell Jorner, AkshatKumar Nigam, Michael Lindner-D’Addario, Matthew S Sigman, et al. A comprehensive discovery platform for organophosphorus ligands for catalysis. J...

    2022

  9. [17]

    Schoenholz, Patrick F

    Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural Message Passing for Quantum Chemistry . In Proceedings of the 34th International Conference on Machine Learning , pp.\ 1263--1272. PMLR, July 2017

    2017

  10. [18]

    Bootstrap your own latent - a new approach to self-supervised learning

    Jean-Bastien Grill, Florian Strub, Florent Altch\' e , Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, Bilal Piot, koray kavukcuoglu, Remi Munos, and Michal Valko. Bootstrap your own latent - a ne...

    2020

  11. [19]

    Bootstrap Your Own Latent - A New Approach to Self-Supervised Learning

    Jean-Bastien Grill, Florian Strub, Florent Altch \'e , Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, Bilal Piot, koray kavukcuoglu , Remi Munos, and Michal Valko. Bootstrap Your Own Latent - A N...

    2020

  12. [20]

    Inductive Representation Learning on Large Graphs

    Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive Representation Learning on Large Graphs . In Advances in Neural Information Processing Systems , volume 30. Curran Associates, Inc., 2017

    2017

  13. [21]

    Masked Autoencoders Are Scalable Vision Learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll \'a r, and Ross Girshick. Masked Autoencoders Are Scalable Vision Learners . In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition , pp.\ 16000--16009, 2022

    2022

  14. [22]

    Convolutional neural network based on smiles representation of compounds for detecting chemical motif

    Maya Hirohara, Yutaka Saito, Yuki Koda, Kengo Sato, and Yasubumi Sakakibara. Convolutional neural network based on smiles representation of compounds for detecting chemical motif. BMC bioinformatics, 19 0 (Suppl 19): 0 526, 2018

    2018

  15. [23]

    GraphMAE : Self-Supervised Masked Graph Autoencoders

    Zhenyu Hou, Xiao Liu, Yukuo Cen, Yuxiao Dong, Hongxia Yang, Chunjie Wang, and Jie Tang. GraphMAE : Self-Supervised Masked Graph Autoencoders . In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , KDD '22, pp.\ 594--604, New York, NY, USA, A...

    2022

  16. [24]

    Strategies for Pre-training Graph Neural Networks , February 2020 a

    Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. Strategies for Pre-training Graph Neural Networks , February 2020 a

    2020

  17. [25]

    Gpt-gnn: Generative pre-training of graph neural networks

    Ziniu Hu, Yuxiao Dong, Kuansan Wang, Kai-Wei Chang, and Yizhou Sun. Gpt-gnn: Generative pre-training of graph neural networks. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp.\ 1857--1867, 2020 b

    2020

  18. [26]

    A fast and high quality multilevel scheme for partitioning irregular graphs

    George Karypis and Vipin Kumar. A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM Journal on scientific Computing, 20 0 (1): 0 359--392, 1998

    1998

  19. [27]

    Kipf and Max Welling

    Thomas N. Kipf and Max Welling. Semi- Supervised Classification with Graph Convolutional Networks , February 2017

    2017

  20. [28]

    3D-Mol : A Novel Contrastive Learning Framework for Molecular Property Prediction with 3D Information , June 2024

    Taojie Kuang, Yiming Ren, and Zhixiang Ren. 3D-Mol : A Novel Contrastive Learning Framework for Molecular Property Prediction with 3D Information , June 2024

    2024

  21. [29]

    Rdkit: Open-source cheminformatics http://www.rdkit.org

    G Landrum. Rdkit: Open-source cheminformatics http://www.rdkit.org. 2016

    2016

  22. [30]

    A path towards autonomous machine intelligence

    Yann LeCun and Courant. A path towards autonomous machine intelligence. 2022. URL https://api.semanticscholar.org/CorpusID:251881108

    2022

  23. [31]

    Augmentation- Free Self-Supervised Learning on Graphs

    Namkyeong Lee, Junseok Lee, and Chanyoung Park. Augmentation- Free Self-Supervised Learning on Graphs . Proceedings of the AAAI Conference on Artificial Intelligence, 36 0 (7): 0 7372--7380, June 2022. ISSN 2374-3468. doi:10.1609/aaai.v36i7.20700

    2022 doi

  24. [32]

    Pre-training Molecular Graph Representation with 3D Geometry , May 2022 a

    Shengchao Liu, Hanchen Wang, Weiyang Liu, Joan Lasenby, Hongyu Guo, and Jian Tang. Pre-training Molecular Graph Representation with 3D Geometry , May 2022 a

    2022

  25. [33]

    A group symmetric stochastic differential equation model for molecule multi-modal pretraining

    Shengchao Liu, Weitao Du, Zhi-Ming Ma, Hongyu Guo, and Jian Tang. A group symmetric stochastic differential equation model for molecule multi-modal pretraining. In International Conference on Machine Learning, pp.\ 21497--21526. PMLR, 2023 a

    2023

  26. [34]

    Auto3d: Automatic generation of the low-energy 3d structures with ani neural network potentials

    Zhen Liu, Tetiana Zubatiuk, Adrian Roitberg, and Olexandr Isayev. Auto3d: Automatic generation of the low-energy 3d structures with ani neural network potentials. Journal of Chemical Information and Modeling, 62 0 (22), 2022 b . PMID: 36112860

    2022

  27. [35]

    The challenge of balancing model sensitivity and robustness in predicting yields: a benchmarking study of amide coupling reactions

    Zhen Liu, Yurii S Moroz, and Olexandr Isayev. The challenge of balancing model sensitivity and robustness in predicting yields: a benchmarking study of amide coupling reactions. Chemical Science, 14 0 (39): 0 10835--10846, 2023 b

    2023

  28. [36]

    Kha-Dinh Luong and Ambuj K. Singh. Fragment-based Pretraining and Finetuning on Molecular Graphs . Advances in Neural Information Processing Systems, 36: 0 17584--17601, December 2023

    2023

  29. [37]

    MolMix : A Simple Yet Effective Baseline for Multimodal Molecular Representation Learning

    Andrei Manolache, Dragos Tantaru, and Mathias Niepert. MolMix : A Simple Yet Effective Baseline for Multimodal Molecular Representation Learning . In Advances in Neural Information Processing Systems (NeurIPS), Machine Learning for Structural Biology Workshop, 2024

    2024

  30. [38]

    Duy M. H. Nguyen, Nina Lukashina, Tai Nguyen, An T. Le, TrungTin Nguyen, Nhat Ho, Jan Peters, Daniel Sonntag, Viktor Zaverkin, and Mathias Niepert. Structure-aware e(3)-invariant molecular conformer aggregation networks. In Proceedings of the 41st International Conference on M...

    2024

  31. [39]

    Graph neural networks can (often) count substructures

    Paolo Pellizzoni, Till Hendrik Schulz, and Karsten Borgwardt. Graph neural networks can (often) count substructures. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=sZQRUrvLn4

    2025

  32. [40]

    Richemond, Jean-Bastien Grill, Florent Altché, Corentin Tallec, Florian Strub, Andrew Brock, Samuel Smith, Soham De, Razvan Pascanu, Bilal Piot, and Michal Valko

    Pierre H. Richemond, Jean-Bastien Grill, Florent Altché, Corentin Tallec, Florian Strub, Andrew Brock, Samuel Smith, Soham De, Razvan Pascanu, Bilal Piot, and Michal Valko. Byol works even without batch statistics, 2020. URL https://arxiv.org/abs/2010.10241

    2020 arXiv

  33. [41]

    Self- Supervised Graph Transformer on Large-Scale Molecular Data

    Yu Rong, Yatao Bian, Tingyang Xu, Weiyang Xie, Ying WEI, Wenbing Huang, and Junzhou Huang. Self- Supervised Graph Transformer on Large-Scale Molecular Data . In Advances in Neural Information Processing Systems , volume 33, pp.\ 12559--12571. Curran Associates, Inc., 2020

    2020

  34. [42]

    u tt, Pieter-Jan Kindermans, Huziel E. Sauceda, Stefan Chmiela, Alexandre Tkatchenko, and Klaus-Robert M \

    Kristof T. Sch \"u tt, Pieter-Jan Kindermans, Huziel E. Sauceda, Stefan Chmiela, Alexandre Tkatchenko, and Klaus-Robert M \"u ller. SchNet : A continuous-filter convolutional neural network for modeling quantum interactions, December 2017

    2017

  35. [43]

    Graph-level Representation Learning with Joint-Embedding Predictive Architectures , January 2025

    Geri Skenderi, Hang Li, Jiliang Tang, and Marco Cristani. Graph-level Representation Learning with Joint-Embedding Predictive Architectures , January 2025

    2025

  36. [44]

    a rk, Dominique Beaini, Gabriele Corso, Prudencio Tossou, Christian Dallago, Stephan G \

    Hannes St \"a rk, Dominique Beaini, Gabriele Corso, Prudencio Tossou, Christian Dallago, Stephan G \"u nnemann, and Pietro Li \'o . 3D Infomax improves GNNs for Molecular Property Prediction . In Proceedings of the 39th International Conference on Machine Learning , pp.\ 20479...

    2022

  37. [45]

    InfoGraph : Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization , January 2020

    Fan-Yun Sun, Jordan Hoffmann, Vikas Verma, and Jian Tang. InfoGraph : Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization , January 2020

    2020

  38. [46]

    Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results

    Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural...

    2017

  39. [47]

    Dyer, R \'e mi Munos, Petar Veli c kovi \'c , and Michal Valko

    Shantanu Thakoor, Corentin Tallec, Mohammad Gheshlaghi Azar, Mehdi Azabou, Eva L. Dyer, R \'e mi Munos, Petar Veli c kovi \'c , and Michal Valko. Large- Scale Representation Learning on Graphs via Bootstrapping , February 2023

    2023

  40. [48]

    Attention is All you Need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is All you Need . In Advances in Neural Information Processing Systems , volume 30. Curran Associates, Inc., 2017

    2017

  41. [49]

    Evaluating self-supervised learning for molecular graph embeddings

    Hanchen Wang, Jean Kaddour, Shengchao Liu, Jian Tang, Joan Lasenby, and Qi Liu. Evaluating self-supervised learning for molecular graph embeddings. Advances in Neural Information Processing Systems, 36: 0 68028--68060, 2023 a

    2023

  42. [50]

    Smiles-bert: large scale unsupervised pre-training for molecular property prediction

    Sheng Wang, Yuzhi Guo, Yuhong Wang, Hongmao Sun, and Junzhou Huang. Smiles-bert: large scale unsupervised pre-training for molecular property prediction. In Proceedings of the 10th ACM international conference on bioinformatics, computational biology and health informatics, pp...

    2019

  43. [51]

    Automated 3d pre-training for molecular property prediction

    Xu Wang, Huan Zhao, Wei-wei Tu, and Quanming Yao. Automated 3d pre-training for molecular property prediction. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.\ 2419--2430, 2023 b

    2023

  44. [52]

    Molecular contrastive learning of representations via graph neural networks

    Yuyang Wang, Jianren Wang, Zhonglin Cao, and Amir Barati Farimani. Molecular contrastive learning of representations via graph neural networks. Nature Machine Intelligence, 4 0 (3): 0 279--287, 2022

    2022

  45. [53]

    The reduction of a graph to canonical form and the algebra which appears therein

    Boris Weisfeiler and Andrei Leman. The reduction of a graph to canonical form and the algebra which appears therein. nti, Series, 2 0 (9): 0 12--16, 1968

    1968

  46. [54]

    The mechanism of prediction head in non-contrastive self-supervised learning

    Zixin Wen and Yuanzhi Li. The mechanism of prediction head in non-contrastive self-supervised learning. Advances in Neural Information Processing Systems, 35: 0 24794--24809, 2022

    2022

  47. [55]

    Wigh, Jonathan M

    Daniel S. Wigh, Jonathan M. Goodman, and Alexei A. Lapkin. A review of molecular representation in the age of machine learning . WIREs Comput. Mol. Sci., 12 0 (5): 0 e1603, September 2022. ISSN 1759-0876. doi:10.1002/wcms.1603

    2022 doi

  48. [56]

    a ger, Niklas Kemper, Leon Hetzel, Johanna Sommer, and Stephan G \

    Tom Wollschl \"a ger, Niklas Kemper, Leon Hetzel, Johanna Sommer, and Stephan G \"u nnemann. Expressivity and generalization: Fragment-biases for molecular gnns. In International Conference on Machine Learning, 2024

    2024

  49. [57]

    Moleculenet: a benchmark for molecular machine learning

    Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S Pappu, Karl Leswing, and Vijay Pande. Moleculenet: a benchmark for molecular machine learning. Chemical science, 9 0 (2): 0 513--530, 2018

    2018

  50. [58]

    Jun Xia, Chengshuai Zhao, Bozhen Hu, Zhangyang Gao, Cheng Tan, Yue Liu, Siyuan Li, and Stan Z. Li. Mole- BERT : Rethinking Pre-training Graph Neural Networks for Molecules , April 2023

    2023

  51. [59]

    Self- Supervised Representation Learning via Latent Graph Prediction

    Yaochen Xie, Zhao Xu, and Shuiwang Ji. Self- Supervised Representation Learning via Latent Graph Prediction . In Proceedings of the 39th International Conference on Machine Learning , pp.\ 24460--24477. PMLR, June 2022

    2022

  52. [60]

    How Powerful are Graph Neural Networks ?, February 2019

    Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How Powerful are Graph Neural Networks ?, February 2019

    2019

  53. [61]

    Self-supervised graph-level representation learning with local and global structure

    Minghao Xu, Hang Wang, Bingbing Ni, Hongyu Guo, and Jian Tang. Self-supervised graph-level representation learning with local and global structure. In International conference on machine learning, pp.\ 11548--11558. PMLR, 2021

    2021

  54. [62]

    Graph contrastive learning automated

    Yuning You, Tianlong Chen, Yang Shen, and Zhangyang Wang. Graph contrastive learning automated. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp.\ 12121--1213...

    2021

  55. [63]

    Graph Contrastive Learning with Augmentations , April 2021 b

    Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph Contrastive Learning with Augmentations , April 2021 b

    2021

  56. [64]

    Multimodal molecular pretraining via modality blending

    Qiying Yu, Yudi Zhang, Yuyan Ni, Shikun Feng, Yanyan Lan, Hao Zhou, and Jingjing Liu. Multimodal molecular pretraining via modality blending. arXiv preprint arXiv:2307.06235, 2023

    2023 arXiv

  57. [65]

    Deep sets

    Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander J Smola. Deep sets. Advances in neural information processing systems, 30, 2017

    2017

  58. [66]

    From Canonical Correlation Analysis to Self-supervised Graph Neural Networks

    Hengrui Zhang, Qitian Wu, Junchi Yan, David Wipf, and Philip S Yu. From Canonical Correlation Analysis to Self-supervised Graph Neural Networks . In Advances in Neural Information Processing Systems , volume 34, pp.\ 76--89. Curran Associates, Inc., 2021

    2021

  59. [67]

    Motif-based Graph Self-Supervised Learning for Molecular Property Prediction

    ZAIXI ZHANG, Qi Liu, Hao Wang, Chengqiang Lu, and Chee-Kong Lee. Motif-based Graph Self-Supervised Learning for Molecular Property Prediction . In Advances in Neural Information Processing Systems , volume 34, pp.\ 15870--15882. Curran Associates, Inc., 2021

    2021

  60. [68]

    Unified 2d and 3d pre-training of molecular representations

    Jinhua Zhu, Yingce Xia, Lijun Wu, Shufang Xie, Tao Qin, Wengang Zhou, Houqiang Li, and Tie-Yan Liu. Unified 2d and 3d pre-training of molecular representations. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, pp.\ 2626--2636, 2022

    2022

  61. [69]

    Coley, Yizhou Sun, and Wei Wang

    Yanqiao Zhu, Jeehyun Hwang, Keir Adams, Zhen Liu, Bozhao Nan, Brock Stenfors, Yuanqi Du, Jatin Chauhan, Olaf Wiest, Olexandr Isayev, Connor W. Coley, Yizhou Sun, and Wei Wang. Learning Over Molecular Conformer Ensembles : Datasets and Benchmarks , July 2024

    2024

  62. [70]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  63. [71]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  64. [72]

    a rk, Hannes and Beaini, Dominique and Corso, Gabriele and Tossou, Prudencio and Dallago, Christian and G \

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

    2024 doi arXiv

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.