REVIEW 23 cited by
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
On the Bottleneck of Graph Neural Networks and its Practical Implications
read the original abstract
Since the proposal of the graph neural network (GNN) by Gori et al. (2005) and Scarselli et al. (2008), one of the major problems in training GNNs was their struggle to propagate information between distant nodes in the graph. We propose a new explanation for this problem: GNNs are susceptible to a bottleneck when aggregating messages across a long path. This bottleneck causes the over-squashing of exponentially growing information into fixed-size vectors. As a result, GNNs fail to propagate messages originating from distant nodes and perform poorly when the prediction task depends on long-range interaction. In this paper, we highlight the inherent problem of over-squashing in GNNs: we demonstrate that the bottleneck hinders popular GNNs from fitting long-range signals in the training data; we further show that GNNs that absorb incoming edges equally, such as GCN and GIN, are more susceptible to over-squashing than GAT and GGNN; finally, we show that prior work, which extensively tuned GNN models of long-range problems, suffers from over-squashing, and that breaking the bottleneck improves their state-of-the-art results without any tuning or additional weights. Our code is available at https://github.com/tech-srl/bottleneck/ .
Forward citations
Cited by 23 Pith papers
-
Quantum machine learning models for graphs
Characterizes constituents of n-qubit graph quantum ML models and supplies a toolbox enabling integration with classical models, generalization of prior GQML approaches, and classical pre-training.
-
AGDN: Learning to Solve Traveling Salesman Problem with Anisotropic Graph Diffusion Network
AGDN is a new GNN framework using a MixScore matrix and anisotropic graph diffusion to outperform prior methods on TSP instances across sizes and distributions.
-
Learning Laplacian Eigenspace with Mass-Aware Neural Operators on Point Clouds
NEO is a mass-aware neural operator that learns the invariant low-frequency eigenspace of the LBO on point clouds for fast spectral geometry.
-
Gaussian Sheaf Neural Networks
Gaussian Sheaf Neural Networks derive a sheaf Laplacian for Gaussian node features on graphs to preserve their geometric structure during message passing.
-
Topology-Preserving Neural Operator Learning via Hodge Decomposition
Hodge Spectral Duality provides a topology-preserving neural operator by isolating unlearnable topological components via Hodge orthogonality and operator splitting.
-
Evaluating LLMs on Large-Scale Graph Property Estimation via Random Walks
EstGraph benchmark evaluates LLMs on estimating properties of very large graphs from random-walk samples that fit in context limits.
-
RopeDreamer: A Kinematic Recurrent State Space Model for Dynamics of Flexible Deformable Linear Objects
RopeDreamer uses quaternionic kinematic chains in a recurrent state space model with a dual decoder to cut open-loop prediction error by 40.52% over 50 steps on simulated DLO trajectories while preserving physical con...
-
A Mechanistic Analysis of Looped Reasoning Language Models
Looped LLMs converge to distinct cyclic fixed points per layer, repeating feedforward-style inference stages across recurrences.
-
Enhancing LLMs for Graph Tasks via Graph-aware LoRA Generation
GaRA generates task-specific LoRA weight updates conditioned on graph structures to enable better whole-graph encoding in LLMs for zero-shot graph learning.
-
MMGNN: Multi-level, multi-color graph neural networks for molecular property prediction
MMGNN decomposes molecular graphs into multi-color subgraphs by atom-type pairs and applies shared message-passing per subgraph, achieving top macro AUC-ROC of 0.838 on classification and best RMSE on ESOL and FreeSol...
-
EqGINO: Equivariant Geometry-Informed Fourier Neural Operators for 3D PDEs
EqGINO adds a spectral isotropy prior to FNOs to guarantee discrete equivariance and enable generalization to continuous SE(3) transformations on 3D PDEs with limited training data.
-
Learning from Historical Activations in Graph Neural Networks
HISTOGRAPH applies unified layer-wise attention followed by node-wise attention over historical GNN activations to improve graph classification, especially in deep models.
-
How Wide and How Deep? Mitigating Over-Squashing of GNNs via Channel Capacity Constrained Estimation
C3E estimates hidden dimensions and depths for GNNs by treating them as communication channels to reduce over-squashing and improve representation learning.
-
Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
Geometric deep learning provides a unified mathematical framework based on grids, groups, graphs, geodesics, and gauges to explain and extend neural network architectures by incorporating physical regularities.
-
GeoFlow: Geo-Aware Modeling of Inter-Area Relationships in Origin-Destination Flow Prediction and Generation
GeoFlow improves OD flow prediction and generation by augmenting area representations with geospatial attributes and using a geometric-intrinsic fusion encoder with axial-global attention decoder.
-
Graph Grounded Cross Attention Transformer Neural Network for Structurally Constrained Full Event Sequence Generation in Predictive Process Monitoring
GGATN combines graph grounding with transformer self- and cross-attention to generate full event sequences, timestamps, length, and attributes in a single pass followed by Viterbi-style constrained decoding, outperfor...
-
Topology-Preserving Neural Operator Learning via Hodge Decomposition
Introduces Hodge Spectral Duality, a hybrid neural architecture that applies Hodge orthogonality and operator splitting to isolate unlearnable topological degrees of freedom from learnable geometric dynamics in soluti...
-
From raw data to neutrino candidates: a neural-network pipeline for Baikal-GVD
A transformer-based three-stage neural network pipeline filters Baikal-GVD data to suppress air showers and noise while selecting neutrino candidates faster and more accurately than standard methods, using domain adap...
-
Attention-based graph neural networks: a survey
The survey groups attention-based GNNs into three stages—graph recurrent attention networks, graph attention networks, and graph transformers—while reviewing architectures and future directions.
-
SR-CGCNN: Shared Recurrent Convolution in Crystal Graph Neural Networks for Materials Property Prediction
SR-CGCNN applies shared weights across recurrent steps in crystal graph convolutions, matching three-layer CGCNN accuracy on Materials Project data with 34.5% of the parameters.
-
SR-CGCNN: Shared Recurrent Convolution in Crystal Graph Neural Networks for Materials Property Prediction
SR-CGCNN shares convolutional weights recurrently to approximate deeper CGCNN performance with 34.5% of the parameters, raising formation-energy MAE from 0.0945 to 0.0986 eV/atom and band-gap MAE from 0.4346 to 0.4503...
-
Information Bottleneck-Guided Heterogeneous Graph Learning for Interpretable Neurodevelopmental Disorder Diagnosis
Proposes the I2B-HGNN framework using information bottleneck-guided graph transformers and heterogeneous graph attention for interpretable multimodal NDD diagnosis.
-
Six Open Questions in Machine-Learned Interatomic Potential Foundation Models
This perspective article develops a definition of foundational MLIPs and poses six open questions that the authors believe will define future research in machine-learned interatomic potentials.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.