REVIEW 10 cited by
Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Representation learning on text-attributed graphs (TAGs) has become a critical research problem in recent years. A typical example of a TAG is a paper citation graph, where the text of each paper serves as node attributes. Initial graph neural network (GNN) pipelines handled these text attributes by transforming them into shallow or hand-crafted features, such as skip-gram or bag-of-words features. Recent efforts have focused on enhancing these pipelines with language models (LMs), which typically demand intricate designs and substantial computational resources. With the advent of powerful large language models (LLMs) such as GPT or Llama2, which demonstrate an ability to reason and to utilize general knowledge, there is a growing need for techniques which combine the textual modelling abilities of LLMs with the structural learning capabilities of GNNs. Hence, in this work, we focus on leveraging LLMs to capture textual information as features, which can be used to boost GNN performance on downstream tasks. A key innovation is our use of explanations as features: we prompt an LLM to perform zero-shot classification, request textual explanations for its decision-making process, and design an LLM-to-LM interpreter to translate these explanations into informative features for downstream GNNs. Our experiments demonstrate that our method achieves state-of-the-art results on well-established TAG datasets, including Cora, PubMed, ogbn-arxiv, as well as our newly introduced dataset, tape-arxiv23. Furthermore, our method significantly speeds up training, achieving a 2.88 times improvement over the closest baseline on ogbn-arxiv. Lastly, we believe the versatility of the proposed method extends beyond TAGs and holds the potential to enhance other tasks involving graph-text data. Our codes and datasets are available at: https://github.com/XiaoxinHe/TAPE.
Forward citations
Cited by 10 Pith papers
-
Agentic Graph Token Reasoning
Agentic Graph Token (AGT) reasoning lets an LLM iteratively select graph views, encode them as tokens, and reason step-by-step, beating static graph-token and text-agent baselines on seven graph domains.
-
Delving into Instance-Dependent Label Noise in Graph Data: A Comprehensive Study and Benchmark
BeGIN provides 10 graph datasets with six label-noise types and a broad evaluation, showing that LLM-simulated noise hurts GNNs more than uniform or pairwise noise.
-
One Model, Many Graphs: Learning over Attributed Graphs across Heterogeneous Modalities with Vision-Language Models
OMG-VLM is a single VLM-based model that handles text-, image-, and multi-attributed graphs through structure-aware adapters, reporting gains on several node/link prediction benchmarks.
-
GraphLAMA: Enabling Efficient Adaptation of Graph Language Models with Limited Annotations
A frozen-LLM graph model with a small tuned adapter (about 726k parameters, 3MB) outperforms in-context learning and instruction-tuned graph LLMs in few/zero-shot node classification and summary generation.
-
Efficient Text-Attributed Graph Learning through Selective Annotation and Graph Alignment
GAGA matches or exceeds state-of-the-art accuracy on several text-attributed graph benchmarks while requiring large language model annotations for only 1% of nodes or edges.
-
UNIT: Unleash Large Language Models Potential for Graph Continual Learning
UNIT reaches SOTA graph continual learning by one-shot LLM instruction tuning plus uncertainty-aware semantic and structural class anchors that jointly set classifier weights.
-
OpenRTAG: A Comprehensive Benchmark for Robust Text-Attributed Graph Learning under Data Quality Degradation
OpenRTAG is a benchmark that organizes text-attributed-graph data-quality issues into a 3x3 taxonomy (text/structure/label by sparsity/noise/imbalance) and evaluates model robustness across nine datasets and three tasks.
-
GSTBench: A Benchmark Study on the Transferability of Graph Self-Supervised Learning
GSTBench finds that masked feature reconstruction (GraphMAE) is the only one of five graph self-supervised pretraining objectives that consistently transfers across eight datasets, while contrastive methods often perf...
-
CS-Agent: LLM-based Community Search via Dual-agent Collaboration
CS-Agent, a Solver-Validator two-agent dialogue with a Decider selector, improves LLM community search on synthetic graphs, and GraphCS is a new benchmark for measuring it.
-
NOCL: Node-Oriented Conceptualization LLM for Graph Tasks without Message Passing
NOCL lets an LLM handle node, edge, and graph tasks on text and non-text graphs by compressing each node's description into one semantic embedding and turning the graph into a text prompt.
Discussion (0). Sign in to comment.