Pith. sign in

REVIEW 9 cited by

Gophormer: Ego-Graph Transformer for Node Classification

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2110.13094 v1 pith:ESN6PB4R submitted 2021-10-25 cs.LG

Jianan Zhao , Chaozhuo Li , Qianlong Wen , Yiqi Wang , Yuming Liu , Hao Sun , Xing Xie , Yanfang Ye This is my paper · ORCID
classification cs.LG
keywords transformersgraphattentiongophormerperformanceproposeachievedclassification
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Transformers have achieved remarkable performance in a myriad of fields including natural language processing and computer vision. However, when it comes to the graph mining area, where graph neural network (GNN) has been the dominant paradigm, transformers haven't achieved competitive performance, especially on the node classification task. Existing graph transformer models typically adopt fully-connected attention mechanism on the whole input graph and thus suffer from severe scalability issues and are intractable to train in data insufficient cases. To alleviate these issues, we propose a novel Gophormer model which applies transformers on ego-graphs instead of full-graphs. Specifically, Node2Seq module is proposed to sample ego-graphs as the input of transformers, which alleviates the challenge of scalability and serves as an effective data augmentation technique to boost model performance. Moreover, different from the feature-based attention strategy in vanilla transformers, we propose a proximity-enhanced attention mechanism to capture the fine-grained structural bias. In order to handle the uncertainty introduced by the ego-graph sampling, we further propose a consistency regularization and a multi-sample inference strategy for stabilized training and testing, respectively. Extensive experiments on six benchmark datasets are conducted to demonstrate the superiority of Gophormer over existing graph transformers and popular GNNs, revealing the promising future of graph transformers.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Toward a Graph Foundation Model: Pre-Training Transformers With Random Walks

    cs.LG 2025-06 conditional novelty 6.0 of 10

    A random-walk-based Transformer pre-trained on multiple graph datasets transfers to unseen node, link, and graph tasks, though its expressivity proofs are incomplete.

  2. DAM-GT: Dual Positional Encoding-Based Attention Masking Graph Transformer for Node Classification

    cs.LG 2025-05 conditional novelty 6.0 of 10

    A graph Transformer with dual positional encoding and a star-shaped attention mask yields consistent accuracy improvements on node classification benchmarks.

  3. Adaptive Tokenization: On the Hop-Overpriority Problem in Tokenized Graph Learning Models

    cs.LG 2025-05 conditional novelty 6.0 of 10

    A learnable graph token list that adaptively reweights hops and selects informative within-hop nodes alleviates the hop-overpriority problem in tokenized graph learning models, especially on heterophilic graphs.

  4. Tokenphormer: Structure-aware Multi-token Graph Transformer for Node Classification

    cs.LG 2024-12 conditional novelty 6.0 of 10

    Tokenphormer combines walk-tokens, SGPM-tokens, and hop-tokens in a graph transformer and reports improved node classification accuracy on several benchmark graphs.

  5. ReHub: Linear Complexity Graph Transformers with Adaptive Hub-Spoke Reassignment

    cs.LG 2024-12 conditional novelty 6.0 of 10

    ReHub keeps spoke-to-hub attention sparse by connecting each node to only k hubs per layer and reassigning those hubs via hub-hub similarity, reaching O(N) complexity with accuracy on par with its dense variant.

  6. Even Sparser Graph Transformers

    cs.LG 2024-11 conditional novelty 6.0 of 10

    Spexphormer trains a narrow graph transformer to identify important attention edges, then trains a wider model on the sparsified graph, achieving competitive accuracy with far less memory.

  7. Rethinking Tokenized Graph Transformers for Node Classification

    cs.LG 2025-02 conditional novelty 5.0 of 10

    SwapGT augments tokenized graph Transformers with a random token swapping operation and a center alignment loss, reporting state-of-the-art accuracy on eight node classification datasets.

  8. Position-aware Graph Transformer for Recommendation

    cs.IR 2024-12 conditional novelty 4.0 of 10

    PGTR combines a GCN recommender with a kernelized graph transformer and four positional encodings, reporting improved Recall and NDCG on four recommendation datasets.

  9. Relational Deep Learning: Challenges, Foundations and Next-Generation Architectures

    cs.LG 2025-06 conditional novelty 3.0 of 10

    A review of Relational Deep Learning that formalizes relational entity graphs, catalogs datasets and methods, and advocates for unified GNN and foundation model research directions.

Pith tools