When Do LLMs Help With Node Classification? A Comprehensive Analysis

Caihua Shan; Fangzhou Ge; Hong Cheng; Xiangguo Sun; Xixi Wu; Yifei Shen; Yizhu Jiao

REVIEW 3 major objections 4 minor 1 cited by

When Do LLMs Help With Node Classification? A Comprehensive Analysis

T0 review · 3 major / 4 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that LLM-based node classification methods give large gains in semi-supervised settings but only marginal gains once supervision is abundant, and that stronger closed-source LLMs like GPT-4o still beat graph foundation…

desk verdict A solid, useful benchmark with one takeaway built on four tiny graphs and an abstract that outruns its own Table 13. read the letter →

arxiv 2502.00829 v2 pith:ZPNLXJEB submitted 2025-02-02 cs.LG cs.SI

Xixi Wu , Yifei Shen , Fangzhou Ge , Caihua Shan , Yizhu Jiao , Xiangguo Sun , Hong Cheng This is my paper · ORCID

classification cs.LGcs.SI

keywords nodeclassificationtext-attributedgraphslargelanguagemodelsgraphfoundationsemi-supervisedlearningzero-shothomophilybenchmark

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to answer a practical question: when does paying for a large language model actually improve node classification on text-attributed graphs? To answer it, the authors built LLMNodeBed, a standardized testbed with 14 datasets, 8 LLM-based algorithms, 8 classic baselines, and 3 learning paradigms, and trained or evaluated more than 2,700 models under consistent backbones and splits. The central finding is that LLM-based methods help most when labels are scarce: in semi-supervised settings they clearly beat classic GNNs and language models, while in fully supervised settings the advantage shrinks to a few points. The paper also finds that in zero-shot settings, direct prompting of a strong LLM such as GPT-4o with neighbor-summary context beats graph foundation models. The practical upshot is a set of design guidelines: use LLM-as-Encoder for balanced cost and accuracy, LLM-as-Explainer when labels depend on text, and reserve LLM-as-Predictor for abundant supervision.

What carries the argument

The load-bearing object is LLMNodeBed, a PyG-based codebase that standardizes dataloaders, backbones, and evaluation across algorithms, plus the mutual-information decomposition $I(G; Y_l) = I(E; Y_l) + I(X; Y_l|E)$ used to explain when LLM text features matter. The testbed organizes LLM methods into four roles — Encoder, Explainer, Predictor, and Direct Inference/GFM — and fixes GCN, RoBERTa-355M, and Mistral-7B as default components so performance differences are attributable to method design rather than architecture. The decomposition does the explanatory work: structure contributes more to labels under abundant supervision, narrowing the room for LLM text features.

What would settle it

Run the same encoder comparison (GCNLLMEmb with Mistral-7B versus RoBERTa-355M) on a heterophilic text-attributed graph with thousands of nodes; if the LLM encoder fails to beat the LM encoder by a large margin, the paper's claim that LLMs matter most in less informative graphs is falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that the value of LLMs in node classification is governed by how much information the graph structure and the supervision already carry. Using the decomposition $I(G; Y_l) = I(E; Y_l) + I(X; Y_l|E)$, the authors argue that when labels are scarce, structure carries less label information and the text-encoding power of LLMs contributes more; when supervision is abundant, structure dominates and LLM gains become marginal. Empirically, this yields the paper's eight takeaways, including the headline results that LLM-based methods significantly outperform traditional methods in semi-supervised settings but only marginally in supervised settings, and that graph foundation models beat open-source LLMs but fall short of GPT-4o in zero-shot classification. The paper frames these findings as design guidance for practitioners deciding whether to adopt LLM-based node classifiers.

Load-bearing premise

The load-bearing premise is that the 14 datasets, especially the four heterophilic graphs with only 191-265 nodes each, faithfully represent real-world text-attributed node classification; if those graphs are not representative, the guidance about less informative graphs does not transfer.

Editorial extensions

If this is right

Practitioners with 10% or fewer labeled nodes should expect LLM-based encoders or explainers to outperform classic GNNs, and should budget for their extra compute.
With 60% labeled nodes, classic methods like GCN with shallow embeddings or RoBERTa are competitive, so the extra cost of LLMs may not be justified.
LLM-as-Predictor methods should be used mainly when labels are abundant; under scarce labels they produce biased and hallucinated outputs.
In zero-shot settings, prompt a strong LLM with a summary of neighbor texts rather than relying on complex reasoning prompts or graph foundation models.
On graphs with low homophily, replacing a language-model encoder with an LLM encoder gives larger gains than on homophilic graphs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The mutual-information explanation predicts that the advantage of LLM encoders should shrink continuously as label fraction increases; a direct sweep of label percentages on a fixed graph would test this more precisely than the two-point comparison in the paper.
Because the heterophilic evidence comes from four graphs with 191-265 nodes, the 'less informative graphs' conclusion is the least robust part of the guidance; re-running the encoder comparison on larger heterophilic text-attributed graphs would show whether the reported 10-point gap persists.
The ranking of open-source backbones (Mistral-7B over Qwen-7B and LLaMA-8B) is tied to current model generations and could shift as newer models appear, although the qualitative role-level findings are likely more stable.
The neighbor-summary prompt finding suggests a testable recipe the paper does not directly evaluate: combine a graph foundation model's embeddings with a strong LLM's summarization in zero-shot settings.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This is a genuinely useful benchmark paper. The new thing is LLMNodeBed—a single codebase that standardizes backbones and implementations across eight LLM-based node classification methods, evaluated on fourteen datasets and three learning paradigms. That standardization matters: prior comparisons like GLBench let each method bring its own code, which confounds method with implementation. The authors also released code, data, and cost measurements, and ran over 2,700 models with error bars. That is real work and the community will use it.

The headline finding holds up: LLM-based methods give large gains over classic methods in semi-supervised settings and only small gains once you have 60% labels. The evidence on the nine larger homophilic datasets is consistent and convincing. The paper also does a nice job showing that CoT/ToT/ReAct prompts add little for direct inference, and that enriched prompts with neighbor summaries help more.

The soft spots are at the edges. Takeaway 8—that LLM-as-Encoder beats LM-as-Encoder more on 'less informative' heterophilic graphs—rests on four tiny web-page networks (Cornell, Texas, Wisconsin, Washington) with 191–265 nodes and 1:1:8 splits, so about 4–5 training nodes per class. Standard deviations are large (e.g., 59.87±6.72) and there are no significance tests. The paper itself concedes the scarcity of text-rich heterophilic datasets, but the main text still presents this as a generalized takeaway. It should be labeled as suggestive, not a firm guideline.

The abstract's 'Graph Foundation Models can beat open-source LLMs' also overstates the evidence. In Table 13, ZeroG (55.56 avg acc) loses to Mistral-7B with summary (59.15) and DeepSeek-V3 with summary (65.02); it only beats LLaMA-8B (49.56). The body is more careful—Takeaway 6 names LLaMA-8B—but the abstract should match the data.

The mutual-information identity in Eq. (1) is a standard decomposition, and the paper uses it as intuition about where gains come from. That is fine, as long as readers don't treat it as a mechanistic proof. Minor issue.

Who is this for? Practitioners choosing between LLM-based and classic methods, and researchers building on LLM-as-Encoder or predictor designs. It deserves a serious referee. Send it to review, with a request to soften Takeaway 8 and fix the abstract's overgeneralization. The code release alone justifies the referee time.

Referee Report

3 major / 4 minor

Summary. The paper introduces LLMNodeBed, a PyG-based testbed for node classification on text-attributed graphs, and reports a comparative study of 8 LLM-based methods, 8 classic baselines, 14 datasets, and three learning paradigms (semi-supervised, supervised, zero-shot), totaling over 2,700 trained or evaluated models. The central empirical findings are that LLM-based methods, especially LLM-as-Encoder and LLM-as-Explainer, give larger gains over classic methods in semi-supervised settings than in supervised settings; that zero-shot direct inference with structurally enriched prompts (w. Summary) performs best; that GFMs such as ZeroG beat LLaMA-8B but not GPT-4o; and that LLM encoders surpass LM encoders mainly on less informative graphs, with heterophilic graphs as the key evidence. The authors release code and processed datasets at the LLMNodeBed website.

Significance. The paper's main strength is as a benchmark artifact: it standardizes backbones, data loaders, and hyperparameter search, reports means and standard deviations across runs, and releases code and data, which should make future LLM-for-graph comparisons cheaper and more reproducible. The headline semi-supervised-versus-supervised finding is supported by nine larger homophilic datasets and is a practically useful guideline. The zero-shot prompt comparison is also valuable. However, the most generalizable-sounding claims, especially Takeaway 8 about less informative graphs and the abstract's statement about GFMs beating open-source LLMs, are not matched by the strength of the evidence. The paper is likely to be influential as a benchmark and should be published after the evidentiary claims are brought in line with the experiments.

major comments (3)

[5.1 / Table 5 / Appendix C.1] Takeaway 8, which answers the title question for 'less informative graphs', is supported only by four tiny web-page graphs (Cornell 191, Texas 187, Wisconsin 265, Washington 229 nodes) from the same source and with the same five-class label schema. With 1:1:8 splits this provides only 19-26 labeled training nodes and test sets of roughly 150-210 nodes, and only four runs are reported, without significance tests, confidence intervals, or multiple-comparison corrections. Several entries in Table 5 are inconsistent with a clean LLM-over-LM conclusion, for example semi-supervised MLP on Cornell has Mistral-7B at 59.87±6.72 versus RoBERTa at 59.08±2.57, and H2GCN on Wisconsin has Qwen-3B at 84.53±0.70 versus Mistral-7B at 81.89±1.51. The paper itself concedes the scarcity of text-rich heterophilic datasets in the concluding section, but this caveat is not carried into the strength of Takeaway 8. I recommend either softening Takeaway 8 to a case-study finding or providing external validation, such as more heterophilic text-attributed graphs, paired bootstrap tests on the existing splits, or a homophily-perturbation experiment on larger graphs.
[Abstract / §4.2 / Table 13] The claim that 'Graph Foundation Models can beat open-source LLMs' is contradicted by the paper's own zero-shot results. ZeroG reaches an average accuracy of 55.56, whereas DeepSeek-V3 with w. Summary reaches 65.02 and Mistral-7B with w. Summary reaches 59.15 (Table 13). The body text correctly limits the comparison to LLaMA-8B, but the abstract and Takeaway 6 overgeneralize it to open-source LLMs as a class. Please revise the wording to 'some open-source LLMs' or specify LLaMA-8B explicitly.
[§4.1 / Tables 2 and 12] The phrase 'significantly outperform' in Takeaway 1 and Takeaway 2 is not backed by statistical inference. All comparisons are reported as mean ± standard deviation over four runs, with no paired significance tests, effect sizes, or confidence intervals. Some individual gaps are within the reported variation: for example, in semi-supervised Pubmed, GCNLLMEmb (78.71) does not beat GCNShallowEmb (78.94). If 'significantly' is meant statistically, the authors should report paired comparisons across the same data splits; if it is meant practically, they should use weaker wording such as 'on average'.

minor comments (4)

[§4.1, Eq. (1)] The mutual-information explanation is presented as a formal argument, but the terms I(E; Y_l) and I(X; Y_l | E) are never estimated or shown to behave as claimed. I recommend labeling this passage as intuition and, if the formal framing is retained, defining the empirical estimators used.
[Table 2, arXiv row] LLMIT on arXiv is a single run, and the table gives no uncertainty for the 76.08 value. The caption mentions this, but the main-text analysis of Takeaway 5 should also state that this particular result has no error bar.
[Table 14 and Table 1] The homophily ratio for Reddit is reported as 55.52 in Table 1 and 55.22 in Table 14; please make the statistics consistent.
[Appendix C.1] The statement that the four heterophilic datasets are the only available text-attributed options is an important limitation and should appear in Section 5.1 where Takeaway 8 is stated, rather than only in the concluding paragraph.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is an empirical benchmark whose insights summarize direct measurements; the mutual-information identity is used as a post-hoc explanation, not as a fitted input or derivation.

full rationale

The paper's central claims are empirical summaries of a systematic benchmark: over 2,700 trained and evaluated models on 14 datasets (Section 3 and 4). No claim is derived by fitting a parameter and then renaming that fit a 'prediction'. The mutual-information decomposition in Equation (1) is a standard identity, I(G;Y_l)=I(E;Y_l)+I(X;Y_l|E), and the paper uses it only to interpret observed performance gaps, not to compute or force those gaps; the empirical tables are independent of the identity. Takeaway 8, about LLM-as-Encoder advantages in less informative graphs, is grounded in measured accuracy differences on heterophilic datasets (Table 5), with the 'less informative' characterization based on the datasets' homophily ratios, which are external dataset statistics rather than outputs of the tested models. Self-citations appear (e.g., Li et al. 2023b for a survey, Wu et al. 2024 in a related-work discussion), but none is load-bearing for the empirical conclusions, and no uniqueness theorem or ansatz is imported from prior work to make a choice forced. The absence of significance tests and the small size of the four heterophilic graphs are validity limitations, not circularity, and are explicitly flagged in the paper's conclusion as 'the scarcity of such text-rich datasets.' The derivation chain therefore does not reduce to its own inputs.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no free parameters or invented entities. Its conclusions rest on empirical observations over the selected benchmark; the main unstated premises are the representativeness of the datasets and the adequacy of the mutual-information decomposition as an explanation of the observed trends.

assumptions (3)

domain assumption The mutual information identity I(G; Yl) = I(E; Yl) + I(X; Yl|E) separates the contributions of graph structure and node features to label information.
Invoked in Section 4.1 after Takeaway 2 to explain why LLM-based methods gain more in semi-supervised settings; assumes classic GNNs primarily exploit the first term and LLMs the second, which is an oversimplification because LLM-based methods also use structure via GNNs.
domain assumption The 14 datasets, including the 4 heterophilic graphs with 191-265 nodes, are representative of text-attributed node classification tasks.
The paper generalizes its 8 takeaways to guidance for practitioners (Sections 1 and 6); if the small heterophilic datasets are not representative, Takeaway 8 and the 'less informative graph' guidance may not transfer.
domain assumption Standardizing backbones (GCN, RoBERTa-355M, Mistral-7B) across methods yields a fair comparison.
The paper deliberately fixes backbones to isolate the effect of LLM integration (Section 4.1), but this choice can favor or disfavor specific methods if the fixed backbone is not equally suitable for all roles.

how reviews work

0 comments

Cite this review

Pith. "Pith review of When Do LLMs Help With Node Classification? A Comprehensive Analysis." pith.science (2026). https://pith.science/paper/ZPNLXJEB

@misc{pith2026250200829,
  author       = {Pith},
  title        = {Pith review of: When Do LLMs Help With Node Classification? A Comprehensive Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZPNLXJEB}},
  note         = {Machine review of arXiv:2502.00829}
}

read the original abstract

Node classification is a fundamental task in graph analysis, with broad applications across various fields. Recent breakthroughs in Large Language Models (LLMs) have enabled LLM-based approaches for this task. Although many studies demonstrate the impressive performance of LLM-based methods, the lack of clear design guidelines may hinder their practical application. In this work, we aim to establish such guidelines through a fair and systematic comparison of these algorithms. As a first step, we developed LLMNodeBed, a comprehensive codebase and testbed for node classification using LLMs. It includes 10 homophilic datasets, 4 heterophilic datasets, 8 LLM-based algorithms, 8 classic baselines, and 3 learning paradigms. Subsequently, we conducted extensive experiments, training and evaluating over 2,700 models, to determine the key settings (e.g., learning paradigms and homophily) and components (e.g., model size and prompt) that affect performance. Our findings uncover 8 insights, e.g., (1) LLM-based methods can significantly outperform traditional methods in a semi-supervised setting, while the advantage is marginal in a supervised setting; (2) Graph Foundation Models can beat open-source LLMs but still fall short of strong LLMs like GPT-4o in a zero-shot setting. We hope that the release of LLMNodeBed, along with our insights, will facilitate reproducible research and inspire future studies in this field. Codes and datasets are released at \href{https://llmnodebed.github.io/}{\texttt{https://llmnodebed.github.io/}}.

Figures

Figures reproduced from arXiv: 2502.00829 by the authors.

**Figure 1.** Overview of LLMNodeBed. best and the key components for achieving this performance. Our work provides intuitive explanations, practical tips, and insights about the strengths and limitations of each algorithm category. 2. Preliminaries on Node Classification To leverage the language abilities of LLMs, we study the node classification task within the context of text-attributed graphs (TAGs) (Ma & Tang, 2021). TAGs … view at source ↗

**Figure 2.** Illustrations of LLM-based node classification algorithms under supervised and zero-shot settings. LLMNodeBed considers several key factors: (1) Multidomain Diversity to reflect different contexts, (2) Varying Scales to examine algorithm scalability and the associated costs of leveraging LLMs, and (3) Diverse Levels of Homophily to understand its impact on performance. Therefore, LLMNodeBed comprises 14 datasets … view at source ↗

**Figure 3.** Performance trends within Qwen-series in different scales using LLaGA framework in semi-supervised settings. 3B 7B 14B 32B Qwen Size (Billion) 75 80 85 90 Performance (%) Accuracy Macro-F1 (a) Cora 3B 7B 14B 32B Qwen Size (Billion) 75 80 85 Performance (%) Accuracy Macro-F1 (b) WikiCS 3B 7B 14B 32B Qwen Size (Billion) 45 55 65 Performance (%) Accuracy Macro-F1 (c) Instagram [PITH_FULL_IMAGE:figures/full_fig_p024_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Performance trends within Qwen-series in different scales using LLaGA framework in supervised settings [PITH_FULL_IMAGE:figures/full_fig_p024_4.png]

**Figure 5.** Figure 5: Biased predictions by LLM-as-Predictor methods on the Instagram dataset: Comparison of ground-truth label distributions with predictor-generated label distributions. Valid Type 1 Valid Type 2 Valid Type 3 Hallucination 0 10 20 30 40 50 Distribution (%) 0.02 9.82 14.41 …

**Figure 6.** Figure 6: Biased predictions by LLM-as-Predictor methods on the Pubmed dataset. F.3. LLM-as-Predictor: Biased and Hallucinated Predictions During our experiments, we found that LLM-as-Predictor methods are vulnerable to limited supervision. In addition to standard metrics such a…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

UNIT: Unleash Large Language Models Potential for Graph Continual Learning
cs.AI 2026-07 conditional novelty 5.5 of 10

UNIT reaches SOTA graph continual learning by one-shot LLM instruction tuning plus uncertainty-aware semantic and structural class anchors that jointly set classifier weights.

Reference graph

Works this paper leans on

27 extracted references · 16 canonical work pages · cited by 1 Pith paper

[4]

The number of training epochs is set to 10 for semi-supervised settings and 4 for supervised settings. For Qwen2.5-series, we encounter over-fitting issues in the Photo, Computer, and Books datasets, leading us to adjust the learning rate to 5e − 5 and reduce the number of epochs to 2 under supervised settings. • For GraphGPT (Tang et al., 2023), it inclu...

work page 2023
[5]

Nt-llm: A novel node tokenizer for integrat- ing graph structure into large language models

Ji, Y ., Liu, C., Chen, X., Ding, Y ., Luo, D., Li, M., Lin, W., and Lu, H. Nt-llm: A novel node tokenizer for integrat- ing graph structure into large language models. ArXiv, abs/2410.10743,

arXiv
[6]

Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D

Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., de Las Casas, D., Bressand, F., Lengyel, 10 When Do LLMs Help With Node Classification? A Comprehensive Analysis G., Lample, G., Saulnier, L., Lavaud, L. R., Lachaux, M.- A., Stock, P., Scao, T. L., Lavril, T., Wang, T., Lacroix, T., and Sayed, W. E. Mistral 7b. ArXiv, abs/2310.06825,

arXiv
[7]

Large language models on graphs: A comprehensive survey

Jin, B., Liu, G., Han, C., Jiang, M., Ji, H., and Han, J. Large language models on graphs: A comprehensive survey. arXiv preprint arXiv:2312.02783, 2023a. Jin, B., Zhang, W., Zhang, Y ., Meng, Y ., Zhang, X., Zhu, Q., and Han, J. Patton: Language model pretraining on text-rich networks. In 61st Annual Meeting of the Association for Computational Linguisti...

arXiv 2023
[9]

Roberta: A robustly optimized bert pretraining approach

Liu, Y ., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V . Roberta: A robustly optimized bert pretraining approach. ArXiv, abs/1907.11692,

arXiv 1907
[10]

and Cangea, C

Mernyei, P. and Cangea, C. Wiki-cs: A wikipedia- based benchmark for graph neural networks. ArXiv, abs/2007.02901,

arXiv 2007
[11]

and Gurevych, I

Reimers, N. and Gurevych, I. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, 11

work page 2019
[13]

Text embeddings by weakly-supervised contrastive pre-training

Wang, L., Yang, N., Huang, X., Jiao, B., Yang, L., Jiang, D., Majumder, R., and Wei, F. Text embeddings by weakly-supervised contrastive pre-training. ArXiv, abs/2212.03533,

arXiv

Show all 27 references

[14]

Learning on large-scale text-attributed graphs via variational inference

Zhao, J., Qu, M., Li, C., Yan, H., Liu, Q., Li, R., Xie, X., and Tang, J. Learning on large-scale text-attributed graphs via variational inference. In International Conference on Learning Representations, 2023a. Zhao, J., Zhuo, L., Shen, Y ., Qu, M., Liu, K., Bronstein, M. M.,...

arXiv
[15]

Graphclip: Enhancing trans- ferability in graph foundation models for text-attributed graphs

Zhu, Y ., Shi, H., Wang, X., Liu, Y ., Wang, Y ., Peng, B., Hong, C., and Tang, S. Graphclip: Enhancing trans- ferability in graph foundation models for text-attributed graphs. Proceedings of the ACM on Web Conference 2025,

2025
[16]

Related Works and Discussion In this section, we present a comprehensive taxonomy of node classification methods, ranging from classic approaches to those leveraging LLMs

12 When Do LLMs Help With Node Classification? A Comprehensive Analysis A. Related Works and Discussion In this section, we present a comprehensive taxonomy of node classification methods, ranging from classic approaches to those leveraging LLMs. A.1. Classic Methods Early app...

2006
[18]

share similar frameworks with LLaGA but vary in integration approaches, training objectives, and tackled tasks. A.3. Zero-shot Learning with LLMs Supervised learning approaches, which rely on labeled data, often struggle to keep pace with the rapid evolution of real-world grap...

2022
[19]

deals with more complex graph computational problems, with the complexity of tasks ranging from polynomial to NP-Complete like the Traveling Salesman Problem. ProGraph (Li et al., 2024a) evaluates the scalability of LLMs by handling large graphs with up to 106 nodes, necessita...

2025
[20]

w. Neighbor

and LLaGA (Chen et al., 2024a), we utilize the prompt templates provided in their original papers for several datasets, including Cora and arXiv. For datasets not originally addressed, such as Photo, we adapt their prompt designs to create similarly formatted prompts. For LLM ...

2022
[21]

Photo”) and Ele-Computer (abbreviated as “Computer

datasets consist of nodes representing papers, with edges indicating citation relationships. The associated text attributes include each paper’s title and abstract, which we use the collected version as follows: Cora and Pubmed from He et al. (2024), Citeseer from Chen et al. ...

2024
[25]

In experiments involving GFMs, the intra-domain training mode utilizes the following source-target pairs: arXiv → Cora, arXiv → WikiCS, Reddit → Instagram, and Computer → Photo. C.3. Distinct Features A fair comparison necessitates a benchmark that evaluates all methods using ...

2017
[26]

These GNNs can be customized with various layers and embedding dimensions

and H2GCN (Zhu et al., 2020). These GNNs can be customized with various layers and embedding dimensions. – LMs and LLMs: Open-source models can be easily loaded via the Transformers library3. In our experiments, we primarily utilize SenBERT-66M (Reimers & Gurevych, 2019), RoBE...

2020
[27]

Additionally, LLMNodeBed allows users to specify and invoke any LM or LLM of their choice, providing flexibility for diverse research needs

and GPT-4o (OpenAI, 2024). Additionally, LLMNodeBed allows users to specify and invoke any LM or LLM of their choice, providing flexibility for diverse research needs. • Robust Evaluation Protocols: LLMNodeBed incorporates comprehensive hyperparameter tuning and design space e...

2024
[32]

Random Walk

Except for the Pubmed, arXiv, and Computer datasets, where the LM is trained first during the EM iteration, all remaining datasets train the GNN first. This choice is based on empirical findings that suggest better performance. • For ENGINE (Zhu et al., 2024), we refer to the ...

2024
[2006]

GraphLLM: Boosting graph reasoning ability of large language model

Chai, Z., Zhang, T., Wu, L., Han, K., Hu, X., Huang, X., and Yang, Y . GraphLLM: Boosting graph reasoning ability of large language model. arXiv preprint arXiv:2310.05845,

arXiv
[2016]

techniques, generating a fixed 300- dimensional embedding for each node based on a walk length of 30 and a total of 10 walks. For the heterophilic GNN, H2GCN (Zhu et al., 2020), which we employed on heterophilic graphs in the fine-grained analysis of LLM-as-Encoder, we perform...

2020
[2020]

Can llms ef- fectively leverage graph structural information through prompts, and why? Trans

Huang, J., Zhang, X., Mei, Q., and Ma, J. Can llms ef- fectively leverage graph structural information through prompts, and why? Trans. Mach. Learn. Res. , 2024,

2024
[2021]

GraphGPT: Graph instruc- tion tuning for large language models

Tang, J., Yang, Y ., Wei, W., Shi, L., Su, L., Cheng, S., Yin, D., and Huang, C. GraphGPT: Graph instruc- tion tuning for large language models. arXiv preprint arXiv:2310.13023,

arXiv
[2022]

allows these models to map textual information directly to node labels, leveraging their strong language understanding capabilities to predict labels. To harness the complementary strengths of GNNs and LMs, i.e., structural and textual information, hybrid LM+GNN architectures ...

2024
[2023]

Can gnn be good adapter for llms? Proceedings of the ACM on Web Conference 2024,

Huang, X., Han, K., Yang, Y ., Bao, D., Tao, Q., Chai, Z., and Zhu, Q. Can gnn be good adapter for llms? Proceedings of the ACM on Web Conference 2024,

2024
[2024]

Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., and et al, A. F. The llama 3 herd of models. ArXiv, abs/2407.21783,

arXiv
[2025]

Can large language models analyze graphs like professionals? a benchmark, datasets and models, 2024a

Li, X., Chen, W., Chu, Q., Li, H., Sun, Z., Li, R., Qian, C., Wei, Y ., Liu, Z., Shi, C., Sun, M., and Yang, C. Can large language models analyze graphs like professionals? a benchmark, datasets and models, 2024a. Li, Y ., Ding, K., and Lee, K. Grenade: Graph-centric lan- guag...

arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.

[1] [4]

The number of training epochs is set to 10 for semi-supervised settings and 4 for supervised settings. For Qwen2.5-series, we encounter over-fitting issues in the Photo, Computer, and Books datasets, leading us to adjust the learning rate to 5e − 5 and reduce the number of epochs to 2 under supervised settings. • For GraphGPT (Tang et al., 2023), it inclu...

work page 2023

[2] [5]

Nt-llm: A novel node tokenizer for integrat- ing graph structure into large language models

Ji, Y ., Liu, C., Chen, X., Ding, Y ., Luo, D., Li, M., Lin, W., and Lu, H. Nt-llm: A novel node tokenizer for integrat- ing graph structure into large language models. ArXiv, abs/2410.10743,

arXiv

[3] [6]

Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D

Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., de Las Casas, D., Bressand, F., Lengyel, 10 When Do LLMs Help With Node Classification? A Comprehensive Analysis G., Lample, G., Saulnier, L., Lavaud, L. R., Lachaux, M.- A., Stock, P., Scao, T. L., Lavril, T., Wang, T., Lacroix, T., and Sayed, W. E. Mistral 7b. ArXiv, abs/2310.06825,

arXiv

[4] [7]

Large language models on graphs: A comprehensive survey

Jin, B., Liu, G., Han, C., Jiang, M., Ji, H., and Han, J. Large language models on graphs: A comprehensive survey. arXiv preprint arXiv:2312.02783, 2023a. Jin, B., Zhang, W., Zhang, Y ., Meng, Y ., Zhang, X., Zhu, Q., and Han, J. Patton: Language model pretraining on text-rich networks. In 61st Annual Meeting of the Association for Computational Linguisti...

arXiv 2023

[5] [9]

Roberta: A robustly optimized bert pretraining approach

Liu, Y ., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V . Roberta: A robustly optimized bert pretraining approach. ArXiv, abs/1907.11692,

arXiv 1907

[6] [10]

and Cangea, C

Mernyei, P. and Cangea, C. Wiki-cs: A wikipedia- based benchmark for graph neural networks. ArXiv, abs/2007.02901,

arXiv 2007

[7] [11]

and Gurevych, I

Reimers, N. and Gurevych, I. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, 11

work page 2019

[8] [13]

Text embeddings by weakly-supervised contrastive pre-training

Wang, L., Yang, N., Huang, X., Jiao, B., Yang, L., Jiang, D., Majumder, R., and Wei, F. Text embeddings by weakly-supervised contrastive pre-training. ArXiv, abs/2212.03533,

arXiv