REVIEW 6 cited by
Transformer for Graphs: An Overview from Architecture Perspective
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recently, Transformer model, which has achieved great success in many artificial intelligence fields, has demonstrated its great potential in modeling graph-structured data. Till now, a great variety of Transformers has been proposed to adapt to the graph-structured data. However, a comprehensive literature review and systematical evaluation of these Transformer variants for graphs are still unavailable. It's imperative to sort out the existing Transformer models for graphs and systematically investigate their effectiveness on various graph tasks. In this survey, we provide a comprehensive review of various Graph Transformer models from the architectural design perspective. We first disassemble the existing models and conclude three typical ways to incorporate the graph information into the vanilla Transformer: 1) GNNs as Auxiliary Modules, 2) Improved Positional Embedding from Graphs, and 3) Improved Attention Matrix from Graphs. Furthermore, we implement the representative components in three groups and conduct a comprehensive comparison on various kinds of famous graph data benchmarks to investigate the real performance gain of each component. Our experiments confirm the benefits of current graph-specific modules on Transformer and reveal their advantages on different kinds of graph tasks.
Forward citations
Cited by 6 Pith papers
-
DiffNMR: Diffusion Models for Nuclear Magnetic Resonance Spectra Elucidation
DiffNMR uses a discrete graph diffusion model conditioned on NMR spectra to predict molecular structures, achieving 68.26% top-1 accuracy with formula on molecules up to 15 heavy atoms.
-
Learnable Spatial-Temporal Positional Encoding for Link Prediction
L-STEP learns time-evolving positional encodings for graph nodes via a learnable spectral filter and predicts links with MLPs only, matching or beating attention-based baselines on 13 temporal datasets.
-
Spectro-Riemannian Graph Neural Networks
CUSP is a graph neural network that combines Ollivier-Ricci curvature with spectral filters on a product of hyperbolic, spherical, and Euclidean spaces, claiming SOTA results on eight node and link prediction benchmarks.
-
A Blueprint for Equilibrium-Based Differentiable Continuous-Variable Thermodynamic Computing
Tunable energy landscapes whose thermal averages equal sigmoid, softmax, and matrix-vector products can, in principle, form the basis of a low-energy analog computer, with a superconducting double-well device as a fir...
-
Large Scalable Cross-Domain Graph Neural Networks for Personalized Notification at LinkedIn
At LinkedIn, a cross-domain GNN trained on a unified 8.6 billion-node graph with temporal modeling and multi-task learning reports a 0.62% CTR lift and a 0.10% WAU lift online.
-
Graph Data Management and Graph Machine Learning: Synergies and Opportunities
This survey reviews the two-way synergy between graph data management and graph machine learning, organized as a pipeline of cleaning, embedding, training, indexing, explanation, and query answering.
Discussion (0). Continue with ORCID to comment.