REVIEW 2 major objections 5 minor 43 references
Node-as-Agent: Graph Agentic Network
T0 review · 2 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read ReaGAN treats each graph node as an autonomous agent that plans its own message passing with a frozen LLM, achieving 84.95% test accuracy on Cora without fine-tuning.
desk verdict Genuinely new agentic-LLM framework for node classification, but the reported accuracies are not trustworthy until the authors clarify which labels populate the retrieval database and how the per-dataset prompt strategy was chosen. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing machinery is the per-node agentic loop: memory initialization with the node's text; a planning prompt to a frozen LLM; a discrete action space consisting of NoOp, Local Aggregation, Global Aggregation, and Local+Global Aggregation; and a RAG tool that searches a structure-free database of all node texts (and labels when available) by embedding similarity. Local and global aggregation each have two effects: they produce an aggregated text feature via natural-language summarization, and they collect a small set of (text, label) examples into memory. At the final layer, a prediction prompt injects the memory's anonymized labeled examples to force the LLM to reason from examples rather than from label semantics. The TextAgg function, whether concatenation or summarization, is what converts node and neighbor texts into a context-enriched representation that the LLM consumes.
What would settle it
Re-run ReaGAN on Cora with the retrieval database built strictly from the 60% training split, excluding all validation and test labels, and report test accuracy; if the accuracy drops materially below 84.95% or below GCN's 84.71%, the claimed parity is not reproduced.
Extended reading notes
Core claim
ReaGAN's central claim is that node-level autonomy plus local-global retrieval can substitute for supervised training in node classification. Each node maintains a memory of its original text, aggregated summaries, and labeled neighbor examples; a planning prompt asks the frozen LLM to pick actions per layer, and a prediction prompt asks it to output a label based on the accumulated memory. The few-shot examples are anonymized (Label_1, Label_2, ...) because the paper finds that revealing semantic label names hurts accuracy by letting the LLM shortcut on keyword matching. The strongest evidence is the Cora result, where ReaGAN's 84.95% test accuracy slightly beats fully trained GCN (84.71%) and GraphSAGE (84.35%); ablations show that removing either the planning step or global retrieval degrades performance. The paper frames this as a new paradigm where a frozen LLM's reasoning, not learned parameters, carries the graph-learning signal.
Load-bearing premise
The reported accuracies assume that the only labels available for retrieval are from the training split; if validation or test labels can enter the retrieval database, the few-shot examples could leak the answer and inflate accuracy.
Editorial extensions
If this is right
- Text-attributed node classification can be performed in a few-shot, zero-parameter fashion with a frozen LLM, avoiding GNN training on the target graph.
- Nodes in sparse or disconnected regions can benefit from global semantic retrieval that conventional local message passing cannot provide.
- Label anonymization during few-shot prompting is not just a presentational choice but a substantive accuracy driver, so other LLM-based graph reasoning systems should adopt it.
- The ablation results imply that both node-level planning and global retrieval are necessary; removing either component drops accuracy consistently.
- ReaGAN's performance makes agentic, retrieval-augmented prompting a viable alternative to supervised GNNs in settings where task-specific training is expensive or labels are scarce.
Reading between the lines
- The paper never states which data split populates the retrieval database; if validation or test labels are included, the reported few-shot accuracies may reflect label leakage rather than genuine reasoning.
- The same agentic loop could extend to link prediction, graph classification, or dynamic graph tasks, but the paper only demonstrates static node classification.
- Per-node LLM calls across layers make ReaGAN expensive relative to a single GNN forward pass; practical adoption would require caching, parallel orchestration, or smaller backbone models.
- A direct test of the approach's generality would replace the frozen LLM with a different-sized model; if accuracy scales with model size, the paradigm's value may come more from the LLM than from the agentic structure.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes ReaGAN, a node-classification framework that treats each node as an LLM-powered agent. Each agent maintains a memory, plans actions (local aggregation, global aggregation via RAG, no-op) through a frozen LLM, and predicts a label at the final layer. The method is evaluated on Cora, Citeseer, and Chameleon under a 60/20/20 split and reported to achieve 84.95% on Cora, 60.25% on Citeseer, and 43.80% on Chameleon without fine-tuning, comparing favorably with trained GNN baselines.
Significance. If the reported results are valid, ReaGAN would be a noteworthy contribution: it suggests that a frozen LLM with agentic planning and retrieval can match supervised GNNs on text-attributed graphs, avoiding gradient-based training. The paper includes a clean agentic formulation, ablations (No Prompt Planning, Local Only, Global Only), and a useful RQ4 analysis of label semantics. However, the evaluation as written contains two unresolved methodological issues—an unspecified label-split rule for the retrieval database and per-dataset prompt-strategy selection—that directly affect the validity of the headline numbers. These issues must be resolved before the central claim can be assessed.
major comments (2)
- [Section 2.6 and Algorithm 1] The paper never states which split's labels are used to construct the retrieval database D or to populate the few-shot example sets E_v^(l). Section 2.6 says D consists of 'all nodes' text features and, when available, their labels,' and Algorithm 1 lines 9 and 14 collect (text, label) pairs from any neighbor u with y_u in Y. Since the raw Cora, Citeseer, and Chameleon datasets contain labels for all nodes, the phrase 'when available' is ambiguous. If validation or test labels are included, then a test node's retrieved global neighbors (or structural neighbors) can supply the true labels of other test nodes as in-context examples, directly leaking the answer and inflating the accuracies in Table 1. Please state explicitly that only training-split labels are ever used in D and in example collection, and if that is not the case, the reported results must be re-evaluated.
- [Section 3.2, Table 3] The final ReaGAN results in Table 1 appear to be the per-dataset best of two prompt strategies: Strategy A is reported for Cora (84.95) and Chameleon (43.80), while Strategy B is reported for Citeseer (60.25). The manuscript does not describe any mechanism by which the method selects Strategy A or B without access to test labels, nor does it present a fixed strategy applied uniformly. Choosing the better of two configurations after seeing test accuracy constitutes test-set selection and overstates the performance of any single method. Please clarify how the strategy is selected per dataset (e.g., via validation accuracy) or report results for a single, a priori chosen strategy across all datasets.
minor comments (5)
- [Appendix A.1.1] The planning prompt example shows 'Respond strictly in JSON: [{"action_type": "local aggregate", "global aggregate" or "no_op"}, ...]' which is not valid JSON; the example should contain actual string values for action_type. This makes the prompt specification ambiguous.
- [Section 2.4.2] The similarity metric is referred to as 'cosine distance' in the text and formulas, but cosine distance is 1 minus cosine similarity, and the indexing direction matters; please clarify whether higher similarity corresponds to smaller distance.
- [Section 3.2, Table 4] The RQ4 results are only shown for Cora and Citeseer, while the text says 'consistently degrades classification accuracy across all datasets'; a Chameleon row is missing.
- [Algorithm 1] In Algorithm 1, the final prediction is described as 'if generated,' but Section 2.3 states the label is queried at the final layer; please clarify under what conditions prediction is skipped.
- [Section 6] The Limitations section is very brief and does not mention the label-split ambiguity or the prompt-strategy selection issue; it should be expanded to address these threats to validity.
Circularity Check
ReaGAN's reported accuracy is partially self-referential: the retrieval database is defined over all nodes' labels with no split restriction, so test labels can be injected as few-shot examples, and the per-dataset prompt strategy is chosen from test accuracy.
-
self definitional
[Section 2.6 (Tools) and Algorithm 1 (lines 12-21)]
"A structure-free database is constructed, consisting of all nodes' text features and, when available, their labels. ... N_global(v)=RAG(t_v^{l-1}, top=K) ... E_v^{(l)} = {(t_u^{l-1}, y_u) | u in N_global(v), y_u in Y} ... return Predicted label y_hat_v (if generated)."
The prediction prompt is built from (text, label) pairs stored in memory. These pairs come from the retrieval database D, which Section 2.6 defines over 'all nodes' text features and, when available, their labels' without restricting labels to the training split. In the Cora, Citeseer, and Chameleon datasets, every node ships with a label, so a test node's own label and other test nodes' labels are 'available' and can be retrieved as top-K neighbors and written into memory as few-shot examples. Algorithm 1 never excludes the query node or test-split nodes. Thus, as written, the method's 'prediction' can consist of copying the ground-truth label from the retrieved context; the reported 84.95% on Cora is not independent of the answer it is supposed to predict.
-
fitted input called prediction
[Section 3.2, Table 3, versus Section 3.1, Table 1]
"Strategy B includes global memory only when fewer than two local entries are available. ... Citeseer A 50.14 B 60.25 ... ReaGAN still maintains a strong competitive standing. For instance, its performance on Citeseer (60.25%) is comparable to several established GNNs."
The final ReaGAN accuracy on Citeseer (60.25%) is exactly the better of the two prompt strategies in Table 3, with Strategy B selected after observing that it outperforms Strategy A on the test split (60.25 vs 50.14). No a priori rule for choosing A or B per dataset is specified before the ablation; the rule is inferred from the test results. The reported 'prediction' therefore includes a hyperparameter (the prompt-construction strategy) that was fit to the test outcome, so part of the claimed performance is the result of test-set selection rather than an out-of-sample prediction.
full rationale
This is an empirical systems paper with no formal derivation chain, so circularity must be assessed in the evaluation protocol. The central claim—that a frozen LLM with few-shot in-context learning matches trained GNNs—depends on two self-referential choices. First, Section 2.6 builds the retrieval database from 'all nodes' text features and, when available, their labels' and never restricts 'available' to the training split; since the raw benchmark datasets contain labels for all nodes, a test node can retrieve itself or other test nodes with true labels, and those labels are injected into the prediction prompt as few-shot examples. Under that reading, the reported accuracy is partly an answer-retrieval score, not a label prediction. Second, the prompt strategy (A vs B) is chosen per dataset using test accuracy, and the winning value is then presented as ReaGAN's fixed performance, which is a fitted-input-called-prediction pattern. There is no load-bearing self-citation: the paper cites several works by the same group, but none is used to justify the central empirical claim, and the GNN baselines are standard external methods. The limitations section does not acknowledge either issue. Because the label-leakage ambiguity is stated in the method itself and never resolved, and because the headline numbers are partly selected from the test set, the paper's strongest claim partially reduces to its own evaluation choices; score 6.
Assumptions & free parameters
free parameters (5)
- per-dataset prompt strategy (A or B) =
A on Cora and Chameleon, B on Citeseer
- RAG top-K =
5
- max reasoning layers L =
3
- few-shot examples per node =
5 local + 5 global
- label anonymization =
anonymized labels (e.g., Label_1)
assumptions (4)
- domain assumption A frozen, instruction-tuned LLM (Qwen2.5-14B-Instruct) can reliably produce well-formed plans and labels from natural-language prompts without task-specific training.
- domain assumption Node text features (title+abstract or full Wikipedia text) faithfully represent the classification signal, and the embedding model (all-MiniLM-L6-v2) ranks nodes by semantic similarity that correlates with label similarity.
- domain assumption The retrieval database contains labels only for nodes whose labels are legitimately observable, i.e., no validation or test labels leak into the few-shot examples.
- domain assumption Per-node planning decisions (local, global, no-op) improve over fixed message-passing schedules.
Cite this review
Pith. "Pith review of Node-as-Agent: Graph Agentic Network." pith.science (2026). https://pith.science/paper/QWNO52O3
@misc{pith2026250800429,
author = {Pith},
title = {Pith review of: Node-as-Agent: Graph Agentic Network},
year = {2026},
howpublished = {\url{https://pith.science/paper/QWNO52O3}},
note = {Machine review of arXiv:2508.00429}
}
read the original abstract
Graph Neural Networks (GNNs) have achieved remarkable success in graph-based learning by propagating information among neighbor nodes via predefined aggregation mechanisms. However, such fixed schemes often suffer from two key limitations. First, they cannot handle the imbalance in node informativeness -- some nodes are rich in information, while others remain sparse. Second, predefined message passing primarily leverages local structural similarity while ignoring global semantic relationships across the graph, limiting the model's ability to capture distant but relevant information. We propose Retrieval-augmented Graph Agentic Network (ReaGAN), an agent-based framework that empowers each node with autonomous, node-level decision-making. Each node acts as an agent that independently plans its next action based on its internal memory, enabling node-level planning and adaptive message propagation. Additionally, retrieval-augmented generation (RAG) allows nodes to access semantically relevant content and build global relationships in the graph. ReaGAN achieves competitive performance under few-shot in-context settings using a frozen LLM backbone without fine-tuning, showcasing the potential of agentic planning and local-global retrieval in graph learning.
Figures
Reference graph
Works this paper leans on
-
[1]
Sami Abu-El-Haija, Bryan Perozzi, Rami Al-Rfou, and Alexander Alemi. 2019. MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neigh- borhood Mixing. InInternational Conference on Machine Learning (ICML). PMLR, 21–29
work page 2019
-
[2]
Yurun Chen, Xavier Hu, Yuhan Liu, Ziqi Wang, Zeyi Liao, Lin Chen, Feng Wei, Yuxi Qian, Bo Zheng, Keting Yin, and Shengyu Zhang. 2025. Graph2Eval: Automatic Multimodal Task Generation for Agents via Knowledge Graphs. arXiv:2510.00507 [cs.CL] https://arxiv.org/abs/2510.00507 ReaGAN: Node-as-Agent-Reasoning Graph Agentic Network
arXiv 2025
-
[3]
Eli Chien, Jianhao Peng, Pan Li, and Olgica Milenkovic. 2020. Adaptive universal generalized pagerank graph neural network.arXiv preprint arXiv:2006.07988 (2020)
arXiv 2020
-
[4]
Ben Finkelshtein, Xingyue Huang, Michael Bronstein, and Ismail Ilkan Ceylan
-
[5]
Hang Gao and Yongfeng Zhang. 2024. Memory Sharing for Large Language Model based Agents. arXiv:2404.09982 [cs.CL] https://arxiv.org/abs/2404.09982
arXiv 2024
-
[6]
Talia Gershon, Seetharami Seelam, Brian Belgodere, Milton Bonilla, Lan Hoang, Danny Barnett, I-Hsin Chung, Apoorve Mohan, Ming-Hung Chen, Lixiang Luo, Robert Walkup, Constantinos Evangelinos, Shweta Salaria, Marc Dombrowa, Yoonho Park, Apo Kayi, Liran Schour, Alim Alim, Ali Sydney, Pavlos Manio- tis, Laurent Schares, Bernard Metzler, Bengi Karacali-Akyama...
arXiv 2025
-
[7]
Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs.Advances in neural information processing systems30 (2017)
2017
-
[8]
Junfeng Hu, Xu Liu, Zhencheng Fan, Yifang Yin, Shili Xiang, Savitha Ra- masamy, and Roger Zimmermann. 2024. Prompt-Based Spatio-Temporal Graph Transfer Learning. InProceedings of the 33rd ACM International Conference on Information and Knowledge Management(Boise, ID, USA)(CIKM ’24). As- sociation for Computing Machinery, New York, NY, USA, 890–899. https:...
arXiv 2024
Show all 43 references
-
[9]
Jingyuan Huang, Xi Zhu, Minghao Guo, and Yongfeng Zhang. 2025. DeSocial: Blockchain-based Decentralized Social Networks. arXiv:2505.21388 [cs.SI] https://arxiv.org/abs/2505.21388
2025
-
[10]
Thomas Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. InInternational Conference on Learning Representations (ICLR)
2017
-
[11]
Johannes Klicpera, Aleksandar Bojchevski, and Stephan Günnemann. 2019. Pre- dict then Propagate: Graph Neural Networks meet Personalized PageRank. In International Conference on Learning Representations (ICLR)
2019
-
[12]
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Ilia Kulikov, Peng Qi, Barlas Oguz, Wen-tau Yih, et al. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. InAdvances in Neural Information Processing Systems (NeurIPS)
2020
-
[13]
Jintang Li, Xiaonan Wang, and Kaixiong Zhou. 2024. Are Large Language Models In-Context Graph Learners?arXiv preprint arXiv:2502.13562(2024)
2024 arXiv
-
[15]
Zhenkun Li, Lingyao Li, Shuhang Lin, and Yongfeng Zhang. 2025. Know the Ropes: A Heuristic Strategy for LLM-based Multi-Agent System Design. arXiv:2505.16979 [cs.AI] https://arxiv.org/abs/2505.16979
2025 arXiv
-
[16]
Ao Liu, Wenshan Li, Tao Li, Beibei Li, Guangquan Xu, Pan Zhou, Wengang Ma, and Hanyuan Huang. 2024. Graph Agent Network: Empowering Nodes with Inference Capabilities for Adversarial Resilience.arXiv preprint arXiv:2306.06909 (2024)
2024 arXiv
-
[17]
Sitao Luan, Chenqing Hua, Qincheng Lu, Jiaqi Zhu, Mingde Zhao, Shuyuan Zhang, Xiao-Wen Chang, and Doina Precup. 2022. Revisiting heterophily for graph neural networks.Advances in neural information processing systems35 (2022), 1362–1375
2022
-
[18]
Karolis Martinkus, Pál András Papp, Benedikt Schesch, and Roger Wattenhofer
-
[19]
Kai Mei, Wujiang Xu, Shuhang Lin, and Yongfeng Zhang. 2025. OmniRouter: Bud- get and Performance Controllable Multi-LLM Routing. arXiv:2502.20576 [cs.DB] https://arxiv.org/abs/2502.20576
2025
-
[20]
InInternational Conference on Learn- ing Representations (ICLR)
Agent-based Graph Neural Networks. InInternational Conference on Learn- ing Representations (ICLR). arXiv:2206.11010 [cs.LG]
-
[21]
Fan Nie, Xiaotian Hou, Shuhang Lin, James Zou, Huaxiu Yao, and Linjun Zhang
-
[22]
Kai Mei, Xi Zhu, Wujiang Xu, Wenyue Hua, Mingyu Jin, Zelong Li, Shuyuan Xu, Ruosong Ye, Yingqiang Ge, and Yongfeng Zhang. 2025. AIOS: LLM Agent Operating System. arXiv:2403.16971 [cs.OS] https://arxiv.org/abs/2403.16971
2025 arXiv
-
[23]
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: language agents with verbal reinforcement learn- ing. InProceedings of the 37th International Conference on Neural Information Processing Systems(New Orleans, LA, USA)(NIPS ’23)...
2023
-
[24]
Dong Shu, Tianle Chen, Mingyu Jin, Chong Zhang, Mengnan Du, and Yongfeng Zhang. 2024. Knowledge graph large language model (KG-LLM) for link predic- tion.arXiv preprint arXiv:2403.07311(2024)
2024 arXiv
-
[25]
2010.Artificial intelligence: a modern approach
Stuart J Russell and Peter Norvig. 2010.Artificial intelligence: a modern approach. Pearson Education
2010
-
[26]
Guangyan Sun, Mingyu Jin, Zhenting Wang, Cheng-Long Wang, Siqi Ma, Qifan Wang, Tong Geng, Ying Nian Wu, Yongfeng Zhang, and Dongfang Liu. [n. d.]. Vi- sual Agents as Fast and Slow Thinkers. InThe Thirteenth International Conference on Learning Representations
-
[27]
DeepSeek-VL Team. 2024. DeepSeek-VL: Towards Real-World Vision-Language Understanding with Foundation Models.arXiv preprint arXiv:2402.03620(2024)
2024 arXiv
-
[28]
Weihang Su, Yichen Tang, Qingyao Ai, Junxi Yan, Changyue Wang, Hongning Wang, Ziyi Ye, Yujia Zhou, and Yiqun Liu. 2025. Parametric Retrieval Augmented Generation. InProceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval(...
2025
-
[29]
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. InInternational Conference on Learning Representations (ICLR)
2018
-
[30]
Haotian Wang et al. 2023. A Survey on Agentic AI: From Reflexion to Tool-Use Agents.arXiv preprint arXiv:2311.08850(2023)
2023 arXiv
-
[31]
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothee Lacroix, Baptiste Roziere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. LLaMA: Open and Efficient Foundation Language Models. arXiv preprint arXiv:2302.13971(2023)
2023 arXiv
-
[32]
Shinn Yao et al . 2022. ReAct: Synergizing reasoning and acting in language models.arXiv preprint arXiv:2210.03629(2022)
2022 arXiv
-
[33]
Ziwei Zhao, Fake Lin, Xi Zhu, Zhi Zheng, Tong Xu, Shitian Shen, Xueying Li, Zikai Yin, and Enhong Chen. 2024. DynLLM: When Large Language Models Meet Dynamic Graph Recommendation. arXiv:2405.07580 [cs.IR] https://arxiv. org/abs/2405.07580
2024 arXiv
-
[34]
Wujiang Xu, Kai Mei, Hang Gao, Juntao Tan, Zujie Liang, and Yongfeng Zhang
-
[35]
Xi Zhu, Pengfei Luo, Ziwei Zhao, Tong Xu, Aakas Lizhiyu, Yu Yu, Xueying Li, and Enhong Chen. 2023. Few-Shot Link Prediction for Event-Based Social Networks via Meta-learning. InDatabase Systems for Advanced Applications, Xin Wang, Maria Luisa Sapino, Wook-Shin Han, Amr El Abba...
2023
-
[36]
Adverse interaction with tree depth restriction
Xi Zhu, Haochen Xue, Ziwei Zhao, Wujiang Xu, Jingyuan Huang, Minghao Guo, Qifan Wang, Kaixiong Zhou, and Yongfeng Zhang. 2024. LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Models.arXiv preprint arXiv:2503.03313(2024). Minghao Guo et al. A Technica...
2024
-
[38]
Baizhou Zhou, Junxian He, Jiahui Wang, Xingyu Wang, Tongshuang Wu, Xianjun Ma, Weixin Zhang, Zhiliang Zhao, et al. 2024. Qwen2: Enhancing Open-Source Language Models with Grouped Query Attention and Instruction Tuning.arXiv preprint arXiv:2404.06092(2024)
2024 arXiv
-
[41]
Analyze the Current Node Text: - Identify primary topics and application domain - Determine the specific problem being solved - Note core methodologies and algorithms"
-
[42]
Analyze Memory Examples: - Understand application domains for each label - Identify types of problems addressed - Note under- lying methodologies
-
[43]
Compare and Weigh Evidence: - Prioritize domain and problem alignment - Evaluate methodological congruence - Consider both domain-specific techniques and general paradigms - Ensure holistic coherence in your decision
-
[44]
action_type
Avoid over-reliance on isolated keywords Please think step by step: First, analyze memory examples and their labels, then compare them to the input text. Identify the most semantically similar mem- ory items and explain why. Finally, decide which label best matches and explain...
-
[2023]
Cooperative graph neural networks.arXiv preprint arXiv:2310.01267(2023)
2023 arXiv
-
[2024]
Facttest: Factuality testing in large language models with statistical guar- antees. (2024)
2024
-
[2025]
arXiv:2502.12110 [cs.CL] https://arxiv.org/abs/2502.12110
A-MEM: Agentic Memory for LLM Agents. arXiv:2502.12110 [cs.CL] https://arxiv.org/abs/2502.12110
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.