Pith. sign in

REVIEW 3 major objections 5 minor 89 references

Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Task-trees turn node, edge, and graph tasks into one shared learning format

desk verdict A nice new learning unit and a broad empirical study, undermined by an order-invariance problem that makes the reported directed-edge results incompatible with the stated encoding. read the letter →

arxiv 2412.16441 v3 pith:PZ36SUNB submitted 2024-12-21 cs.LG cs.AIcs.SI

Zehong Wang , Zheyuan Zhang , Tianyi Ma , Nitesh V Chawla , Chuxu Zhang , Yanfang Ye This is my paper · ORCID
classification cs.LGcs.AIcs.SI
keywords graphfoundationmodelstask-treescross-taskgeneralizationneuralnetworkspretrainingtransferlearningin-contextzero-shot
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the reason graph foundation models are hard to build is task heterogeneity, and that this can be overcome by converting every learning instance—a node, an edge, or a whole graph—into a single shared object called a task-tree. A task-tree is formed by taking the computation trees of all task-relevant nodes and connecting them to a virtual root node. The authors argue that pretraining a graph neural network to reconstruct randomly corrupted task-trees makes it learn generalities that transfer across heterogeneous graphs and tasks. To test this, they build GIT, a GraphSAGE-based model pretrained on six graphs, and show it adapts to 32 graphs across five domains through fine-tuning, in-context learning, and zero-shot inference. If correct, the result would give graph learning a unified, sentence-like unit analogous to text tokens.

What carries the argument

The central object is the task-tree: for any node-, edge-, or graph-level instance, collect the L-layer computation trees of all task-relevant nodes and connect their roots to a new virtual node. Encoding a task-tree is done by averaging the GNN embeddings of those computation trees, which operationally means appending virtual nodes to the original graph and reading off their embeddings. This construction carries the argument because it turns every task into the same tree-encoding problem, and the theory shows that tree-structured information is what message-passing GNNs learn well. The pretraining mechanism is a corruption-invariant reconstruction loss: two randomly masked views of a task-tree are aligned with a stop-gradient target plus a KL regularizer that pulls instance embeddings toward a shared center.

What would settle it

Pretrain GIT exactly as described but replace each task-tree with a randomly rewired tree that keeps the same node features and degree distribution; if the downstream gains over no-pretraining remain largely intact, then task-tree structure is not the carrier of transfer and the paper's central explanation would be wrong.

Watch

Extended reading notes

Core claim

The paper's central claim is that task-trees preserve the generalities shared across graphs, so a GNN pretrained on diverse task-trees with a reconstruction objective acquires transferable knowledge. Formally, the paper states the Task-Tree Generality Assumption, then proves stability, transferability, and generalization bounds: similar subtrees yield similar task-tree embeddings, lower pretraining loss on task-trees implies better downstream transfer, and the generalization error of the fine-tuned model is controlled by the distribution gap between pretraining and downstream task-trees. The empirical counterpart, GIT, appends virtual task nodes to graphs, encodes them with a shared GNN, and uses a negative-free reconstruction loss with stop-gradient; after pretraining on six graphs and optional instruction tuning on a domain graph, it reports consistent gains over subgraph-based baselines on held-out graphs. The authors present this as evidence that task-trees are a valid unified learning unit for node-, edge-, and graph-level tasks.

Load-bearing premise

The result rests on the assumption that the generalities shared across different graphs survive inside their task-trees, and the paper does not prove this—it assumes it and then shows empirically that pretraining on task-trees works.

Editorial extensions

If this is right

  • One pretrained encoder can handle node, edge, and graph-level tasks by appending virtual task nodes, so task-specific readout architectures become unnecessary.
  • Domain specialization by instruction tuning on a single domain graph can bring a general pretrained model close to or above domain-expert models.
  • Task-trees are cheaper than subgraphs because they avoid explicit subgraph extraction, which matters at industry scale.
  • Pretraining on task-trees helps in few-shot in-context learning and zero-shot inference, not just full fine-tuning.
  • The generalization bound predicts that increasing pretraining diversity improves downstream transfer, guiding which graphs to pretrain on.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same virtual-node construction should extend to other relational structures such as hypergraphs and heterogeneous graphs without redesigning the objective, although the paper only tests homogeneous graphs.
  • The theory's dependence on the pretraining and fine-tuning distribution gap suggests a data-selection rule for building the pretraining corpus; the paper does not test whether selecting graphs to minimize that gap improves transfer.
  • A direct test of the Task-Tree Generality Assumption would be to compare GIT against the same model pretrained on random trees with matched features; if transfer persists, the explanation would need revision.
  • The reported scaling with hidden dimension hints at a scaling law for task-trees, but the authors note that they found no clear gain from adding pretraining data, leaving data scaling unresolved.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes task-trees, defined as virtual-node-rooted computation trees over task-relevant nodes, as unified learning instances for node-, edge-, and graph-level tasks. It states a Task-Tree Generality Assumption, presents stability, transferability, and generalization bounds (Theorems 3.1, 3.3, and 3.5), and instantiates the framework in GIT, a GraphSAGE-based model pretrained with a task-tree reconstruction objective and optionally specialized via instruction tuning. The evaluation spans 32 graphs in five domains under fine-tuning, in-context learning, and zero-shot settings, with held-out averages reported as favoring GIT over subgraph-based and self-supervised baselines.

Significance. The empirical study is the paper's main strength: it is broad (32 graphs, five domains), reports error bars, compares against standard baselines and domain experts, gives held-out averages that exclude pretraining/specialization data, and releases code. If the formal definitions and the implementation were aligned, task-trees would be a simple and efficient alternative to subgraph-based instance unification for graph foundation models. The theoretical contribution, however, is currently not established: the transferability proof contains algebraic gaps, and the stated bounds are generic rather than specific to task-trees. In addition, the formal edge-level task-tree encoding is invariant under endpoint exchange, which conflicts with the reported directed knowledge-graph results. These issues are fixable but require more than local editing, so I recommend major revision.

major comments (3)
  1. [Section 2.2, Eq. (2)] The formal task-tree encoding is order-invariant for edge-level tasks. Definition 2.3 connects the virtual node to the set of task-relevant endpoints, and Eq. (2) encodes the task-tree as the mean of the endpoint computation-tree embeddings, so the instances for directed edges (u,v) and (v,u) have identical representations and no linear head can separate them. The paper nevertheless reports strong relation-classification accuracy on directed knowledge graphs (Table 19, e.g., WN18RR 94.16 and FB15K237 80.39 under fine-tuning, and 89.78 for FB15K237 under 3-shot). Appendix B.4 acknowledges a related edge-isomorphism limitation of mean aggregation but does not address head/tail reversal. The authors need to specify the actual implementation (e.g., ordered concatenation, directional virtual edges, or separate head/tail tokens), make Definition 2.3 and Eq. (2) match it, and re-examine the claims in Section 3 that rely on the mean-pooling form.
  2. [Section 3, Eq. (5), Appendix D.2] The proof of Theorem 3.3 is not a valid derivation. The first step bounds a difference of minima by an expectation of the form E|theta^T phi(T) - theta'^T phi'(T)| with theta later chosen as (E[phi(T)phi(T)^T])^dagger E[phi(T)phi'(T)^T] theta', but that theta is not the minimizer of the downstream risk, and the inequality between the two minima and this expectation is not justified. The lower bound on the pretraining loss difference contains the equality min_W E||W phi(T_hat) - phi(T)||^2 - min_W' E||W' phi'(T_hat) - phi'(T)||^2 = min_W E||W phi(T_hat) - W' phi'(T_hat)||^2 + C_P, which is algebraically false because the left-hand side subtracts the optimal reconstruction loss for phi', not the representation loss relative to W' phi'(T_hat). Consequently Eq. (5) is unproven; Theorem 3.5 invokes Theorem 3.3 in term (c) and inherits the gap. The theorem also contains no assumption relating the pretraining distribution P to the downstream distribution T, so in the stated generality it cannot hold.
  3. [Section 2.2 and Section 3] The theoretical results are not specific to task-trees and therefore cannot establish the effectiveness of task-trees as claimed in the contributions. Theorems 3.1, 3.3, and 3.5 are formulated for arbitrary encoders and instance distributions; none of their quantities depends on the virtual-node construction in Definition 2.3, and none compares task-trees with subgraphs or other instance encodings. The only bridge from these generic bounds to task-trees is the Task-Tree Generality Assumption, which the paper explicitly treats as an assumption. The authors should either add a result that distinguishes task-tree instances from alternatives or explicitly reframe the contribution as empirical validation plus a generic transfer framework.
minor comments (5)
  1. [Theorem 3.5, Eq. (6)] The term sum_{x in X_phi} |T_phi(x) - P_phi(x)| sums over a distribution, which is not defined for continuous embeddings; this should be an integral or a total-variation-style distance.
  2. [Appendix D.3] In the proof of Theorem 3.5, term (c), the symbol h is used unintentionally in 'L_P(g composed with h)' where phi is meant; this makes the proof difficult to follow.
  3. [Section 3 and Appendix E.3] There are several minor language errors: 'boarder pretraining distribution' should be 'broader pretraining distribution', and 'we does not leverage edge features' should be 'we do not leverage edge features'.
  4. [Figure 4] The batch-size axis labels '29', '210', '211', etc. are missing superscript formatting and should read 2^9, 2^10, 2^11, etc.
  5. [Appendix B.4] The discussion of the edge-isomorphism limitation is useful, but it should be explicitly connected to Eq. (2) and to the directed knowledge-graph experiments in Table 19, since it currently reads as an isolated caveat.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular reduction found: the Task-Tree Generality Assumption is explicitly assumed, the theory is generic rather than circular, and the empirical validation is self-contained against external benchmarks.

full rationale

The paper's central premise, the Task-Tree Generality Assumption (Sec. 2.2), is explicitly labeled an assumption and is never disguised as a derived result; Theorems 3.1, 3.3, and 3.5 are generic Lipschitz, representation-transfer, and generalization bounds that would apply to any learning instances, and their conclusions are not identical to the assumption by construction. The pretraining objective LP and GIT's reconstruction loss (Eq. 7) are not fitted to downstream labels, and the reported fine-tuning, in-context, and zero-shot results are evaluated on held-out graphs against external baselines (GraphMAE, OFA, GCN/GAT/GIN, GraphPrompt+, All in One, OpenGraph, AnyGraph, ULTRA, GIMLET), so no fitted parameter is renamed as a prediction. The self-citation to GFT (Wang et al. 2024b) is contextual and non-load-bearing: the paper's own 32-graph benchmark and ablations carry the empirical claim. A formal consistency limitation exists in that Eq. (2)'s MEAN aggregation is order-invariant, making directed edge classification impossible under the stated encoding; this is acknowledged by the authors in App. B.4 for mean-aggregated edge embeddings and is a correctness/expressiveness risk rather than a circular derivation. Overall, the derivation does not reduce to its inputs by construction, so the circularity score is low.

Assumptions & free parameters 3 free parameters · 6 assumptions · 1 invented entities

The central claim rests on one explicit assumption (task-tree generality), a feature-alignment premise, and a distribution-similarity premise. Numerically, the main hand-tuned choices are the pretraining and SFT dataset selections and the regularizer weight lambda set to 10. The theoretical results use standard Lipschitz, trace-inequality, and Rademacher bounds.

free parameters (3)
  • Domain regularizer weight lambda = 10
    Set empirically in Appendix B.2; Table 7 shows lambda equals 10 yields the best average performance on the eight datasets used for pretraining and evaluation, so this is tuned on evaluation data.
  • Pretraining dataset selection = Arxiv, Products, WN18RR, FB15K237, Chemblpre, PCBA
    Chosen by hand in Section 5.1 as a diverse subset; the central transfer results depend on this choice.
  • Specialization (SFT) dataset selection = Arxiv, Products, FB15K237, PCBA, Products for temporal graphs
    Manually selected per domain in Section 5.1; Appendix H shows that the choice of SFT data affects zero-shot results.
assumptions (6)
  • domain assumption Task-Tree Generality Assumption: generalities shared across graphs are preserved within task-trees.
    Stated in Section 2.2 as an assumption; the paper says it evaluates this assumption but never proves it.
  • domain assumption Text-attributed graphs encoded by Sentence-BERT provide a shared feature space across domains.
    Used in Section 4 to isolate task heterogeneity; if the shared space is not truly aligned, cross-domain transfer could be confounded.
  • domain assumption Graphs from the same domain have similar task-tree distributions.
    Section 4.2 states this mild assumption to justify specialization via instruction tuning.
  • standard math GNN encoder functions are Lipschitz continuous and node features are bounded.
    Assumed in Theorem 3.1 and used throughout the stability proof.
  • standard math Ruhe's trace inequality for positive semidefinite matrices.
    Used in the proof of Theorem 3.3.
  • standard math Rademacher complexity and uniform convergence bounds.
    Used in the proof of Theorem 3.5.
invented entities (1)
  • Task-tree virtual node
    purpose: A virtual node connected to all task-relevant nodes, used as the root of a computation tree to unify node, edge, and graph tasks under one GNN.
    It is a modeling construct introduced in the paper; its only evidence is the in-paper empirical evaluation, not an external falsifiable prediction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees." pith.science (2026). https://pith.science/paper/PZ36SUNB

@misc{pith2026241216441,
  author       = {Pith},
  title        = {Pith review of: Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PZ36SUNB}},
  note         = {Machine review of arXiv:2412.16441}
}
read the original abstract

Foundation models are pretrained on large-scale corpora to learn generalizable patterns across domains and tasks -- such as contours, textures, and edges in images, or tokens and sentences in text. In contrast, discovering such generalities in graph-structured data, especially across heterogeneous graph tasks, remains an open challenge. To address this, we propose a novel approach to cross-task generalization in graphs via task-trees, which serve as unified learning instances aligning node-, edge-, and graph-level tasks. We theoretically analyze the stability, transferability, and generalization properties of task-trees, showing that pretraining a graph neural network (GNN) on diverse task-trees with a reconstruction objective induces transferable knowledge. This enables efficient adaptation to downstream tasks with minimal fine-tuning. To validate our framework, we introduce Graph Generality Identifier on Task-Trees (GIT), a graph foundation model that demonstrates strong performance on over 30 graphs across five domains via fine-tuning, in-context learning, and zero-shot generalization. Code and data are available at https://github.com/Zehong-Wang/GIT.

Figures

Figures reproduced from arXiv: 2412.16441 by the authors.

Figure 1
Figure 1. The formulation of task-trees. In contrast, our proposed task-trees offer both greater ef￾ficiency and improved learnability ( [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. subtree information examples. (with multiple subtrees). While the widths of these task-trees differ significantly, if their subtrees share some degree of similarity, they can produce similar representations. Thus, this theorem ensures that task-trees of nodes, edges, or graphs can potentially be similar, making it possible to use a GNN encoder to capture the shared patterns among them. We now examine the transferabi… view at source ↗
Figure 3
Figure 3. The model performance on all datasets in the fine-tuning setting. For GIT, the best result is selected between [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Training efficiency between task-tree and sub [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: The domain regularizer controls the distance between datasets while preserving the structure within each of them. [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Edge Isomorphic (Figure 1, ( [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: The impact of model sizes on performance. [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: The statistics of temporal graphs. GIMLET (Zhao et al., 2023). A foundation model for molecules that incorporates advanced models and instruction-based learning. E.3. Evaluation Protocol Pretraining Datasets. We select six datasets for pretraining, including Arxiv, Pro…
Figure 9
Figure 9. Figure 9: The impact of different SFT datasets used for specialization in academic networks. [PITH_FULL_IMAGE:figures/full_fig_p035_9.png]
Figure 10
Figure 10. Figure 10: The comprehensive results of the impact of hidden dimensions on model performance. [PITH_FULL_IMAGE:figures/full_fig_p037_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

89 extracted references · 73 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report. arXiv, 2023

    2023

  3. [3]

    When to pre-train graph neural networks? from data generation perspective! In KDD, 2023

    Cao, Y., Xu, J., Yang, C., Wang, J., Zhang, Y., Wang, C., Chen, L., and Yang, Y. When to pre-train graph neural networks? from data generation perspective! In KDD, 2023

    2023

  4. [4]

    K., Shah, N., and Wang, Z

    Chen, R., Zhao, T., JAISWAL, A. K., Shah, N., and Wang, Z. Llaga: Large language and graph assistant. In ICML, 2024 a

    2024

  5. [5]

    Can graph neural networks count substructures? In NeurIPS, 2020

    Chen, Z., Chen, L., Villar, S., and Bruna, J. Can graph neural networks count substructures? In NeurIPS, 2020

    2020

  6. [6]

    Text-space graph foundation models: Comprehensive benchmarks and new insights

    Chen, Z., Mao, H., Liu, J., Song, Y., Li, B., Jin, W., Fatemi, B., Tsitsulin, A., Perozzi, B., Liu, H., et al. Text-space graph foundation models: Comprehensive benchmarks and new insights. arXiv, 2024 b

    2024

  7. [7]

    Fastgas: Fast graph-based annotation selection for in-context learning

    Chen, Z., Wang, S., Shen, C., and Li, J. Fastgas: Fast graph-based annotation selection for in-context learning. In ACL 2024 (Findings), 2024 c

    2024

  8. [8]

    and Jegelka, S

    Chuang, C.-Y. and Jegelka, S. Tree mover's distance: Bridging graph metrics and stability of graph neural networks. In NeurIPS, 2022

    2022

Show all 89 references
  1. [9]

    On the generalization ability of unsupervised pretraining

    Deng, Y., Hong, J., Zhou, J., and Mahdavi, M. On the generalization ability of unsupervised pretraining. In AISTATS, 2024

    2024

  2. [10]

    Graph prototypical networks for few-shot learning on attributed networks

    Ding, K., Wang, J., Li, J., Shu, K., Liu, C., and Liu, H. Graph prototypical networks for few-shot learning on attributed networks. In CIKM, 2020

    2020

  3. [11]

    Learning theory can (sometimes) explain generalisation in graph neural networks

    Esser, P., Chennuru Vankadara, L., and Ghoshdastidar, D. Learning theory can (sometimes) explain generalisation in graph neural networks. In NeurIPS, 2021

    2021

  4. [12]

    Heterogeneous temporal graph neural network

    Fan, Y., Ju, M., Zhang, C., and Ye, Y. Heterogeneous temporal graph neural network. In SDM, 2022

    2022

  5. [13]

    Taglas: An atlas of text-attributed graph datasets in the era of large graph and language models

    Feng, J., Liu, H., Kong, L., Chen, Y., and Zhang, M. Taglas: An atlas of text-attributed graph datasets in the era of large graph and language models. arXiv, 2024

    2024

  6. [14]

    The development of social network analysis

    Freeman, L. The development of social network analysis. A Study in the Sociology of Science, 2004

    2004

  7. [15]

    Towards foundation models for knowledge graph reasoning

    Galkin, M., Yuan, X., Mostafa, H., Tang, J., and Zhu, Z. Towards foundation models for knowledge graph reasoning. In ICLR, 2024

    2024

  8. [16]

    Generalization and representational limits of graph neural networks

    Garg, V., Jegelka, S., and Jaakkola, T. Generalization and representational limits of graph neural networks. In ICML, 2020

    2020

  9. [17]

    Gpt4graph: Can large language models understand graph structured data? an empirical evaluation and benchmarking

    Guo, J., Du, L., and Liu, H. Gpt4graph: Can large language models understand graph structured data? an empirical evaluation and benchmarking. arXiv, 2023

    2023

  10. [18]

    Mirage: Model-agnostic graph distillation for graph classification

    Gupta, M., Manchanda, S., Ranu, S., and Kodamana, H. Mirage: Model-agnostic graph distillation for graph classification. In ICLR, 2024

    2024

  11. [19]

    Inductive representation learning on large graphs

    Hamilton, W., Ying, Z., and Leskovec, J. Inductive representation learning on large graphs. In NeurIPS, 2017

    2017

  12. [20]

    A., and Jin, W

    Hashemi, M., Gong, S., Ni, J., Fan, W., Prakash, B. A., and Jin, W. A comprehensive survey on graph reduction: Sparsification, coarsening, and condensation. In IJCAI, 2024

    2024

  13. [21]

    Masked autoencoders are scalable vision learners

    He, K., Chen, X., Xie, S., Li, Y., Doll \'a r, P., and Girshick, R. Masked autoencoders are scalable vision learners. In CVPR, 2022

    2022

  14. [22]

    and Hooi, B

    He, Y. and Hooi, B. Unigraph: Learning a cross-domain graph foundation model from natural language. arXiv preprint arXiv:2402.13630, 2024

    2024 arXiv

  15. [23]

    Graphmae: Self-supervised masked graph autoencoders

    Hou, Z., Liu, X., Cen, Y., Dong, Y., Yang, H., Wang, C., and Tang, J. Graphmae: Self-supervised masked graph autoencoders. In KDD, 2022

    2022

  16. [24]

    Strategies for pre-training graph neural networks

    Hu, W., Liu, B., Gomes, J., Zitnik, M., Liang, P., Pande, V., and Leskovec, J. Strategies for pre-training graph neural networks. In ICLR, 2020

    2020

  17. [25]

    Pre-training graph neural networks for generic structural feature extraction

    Hu, Z., Fan, C., Chen, T., Chang, K.-W., and Sun, Y. Pre-training graph neural networks for generic structural feature extraction. arXiv, 2019

    2019

  18. [26]

    S., and Leskovec, J

    Huang, Q., Ren, H., Chen, P., Kr z manc, G., Zeng, D., Liang, P. S., and Leskovec, J. Prodigy: Enabling in-context learning over graphs. In NeurIPS, 2023

    2023

  19. [27]

    Self-supervised learning on graphs: Deep insights and new direction

    Jin, W., Derr, T., Liu, H., Wang, Y., Wang, S., Liu, Z., and Tang, J. Self-supervised learning on graphs: Deep insights and new direction. arXiv, 2020

    2020

  20. [28]

    Ju, H., Li, D., Sharma, A., and Zhang, H. R. Generalization in graph neural networks: Improved pac-bayesian bounds on graph diffusion. In AISTATS, 2023 a

    2023

  21. [29]

    Multi-task self-supervised graph neural networks enable stronger task generalization

    Ju, M., Zhao, T., Wen, Q., Yu, W., Shah, N., Ye, Y., and Zhang, C. Multi-task self-supervised graph neural networks enable stronger task generalization. In ICLR, 2023 b

    2023

  22. [30]

    Kipf, T. N. and Welling, M. Semi-supervised classification with graph convolutional networks. In ICLR, 2017

    2017

  23. [31]

    On the transferability of spectral graph filters

    Levie, R., Isufi, E., and Kutyniok, G. On the transferability of spectral graph filters. In SampTA, 2019

    2019

  24. [32]

    Transferability of spectral graph convolutional neural networks

    Levie, R., Huang, W., Bucci, L., Bronstein, M., and Kutyniok, G. Transferability of spectral graph convolutional neural networks. JMLR, 2021

    2021

  25. [33]

    X., and Li, J

    Li, Y., Wang, P., Li, Z., Yu, J. X., and Li, J. Zerog: Investigating cross-dataset zero-shot transferability in graphs. In KDD, 2024

    2024

  26. [34]

    One for all: Towards training one graph model for all classification tasks

    Liu, H., Feng, J., Kong, L., Liang, N., Tao, D., Chen, Y., and Zhang, M. One for all: Towards training one graph model for all classification tasks. In ICLR, 2024 a

    2024

  27. [35]

    Graphprompt: Unifying pre-training and downstream tasks for graph neural networks

    Liu, Z., Yu, X., Fang, Y., and Zhang, X. Graphprompt: Unifying pre-training and downstream tasks for graph neural networks. In WWW, 2023

    2023

  28. [36]

    Liu, Z., He, X., Tian, Y., and Chawla, N. V. Can we soft prompt llms for graph learning tasks? In WWW, 2024 b

    2024

  29. [37]

    Revisiting heterophily for graph neural networks

    Luan, S., Hua, C., Lu, Q., Zhu, J., Zhao, M., Zhang, S., Chang, X.-W., and Precup, D. Revisiting heterophily for graph neural networks. In NeurIPS, 2022

    2022

  30. [38]

    Hypergraph contrastive learning for drug trafficking community detection

    Ma, T., Qian, Y., Zhang, C., and Ye, Y. Hypergraph contrastive learning for drug trafficking community detection. In ICDM, 2023

    2023

  31. [39]

    Is homophily a necessity for graph neural networks? In ICLR, 2022

    Ma, Y., Liu, X., Shah, N., and Tang, J. Is homophily a necessity for graph neural networks? In ICLR, 2022

    2022

  32. [40]

    Graph foundation models are already here

    Mao, H., Chen, Z., Tang, W., Zhao, J., Ma, Y., Zhao, T., Shah, N., Galkin, M., and Tang, J. Graph foundation models are already here. In ICML, 2024

    2024

  33. [41]

    L., Lenssen, J

    Morris, C., Ritzert, M., Fey, M., Hamilton, W. L., Lenssen, J. E., Rattan, G., and Grohe, M. Weisfeiler and leman go neural: Higher-order graph neural networks. In AAAI, 2019

    2019

  34. [42]

    Wl meet vc

    Morris, C., Geerts, F., T \"o nshoff, J., and Grohe, M. Wl meet vc. In ICML, 2023

    2023

  35. [43]

    Towards trustworthy retrieval augmented generation for large language models: A survey

    Ni, B., Liu, Z., Wang, L., Lei, Y., Zhao, Y., Cheng, X., Zeng, Q., Dong, L., Xia, Y., Kenthapadi, K., et al. Towards trustworthy retrieval augmented generation for large language models: A survey. arXiv, 2025

    2025

  36. [44]

    Dual-level hypergraph contrastive learning with adaptive temperature enhancement

    Qian, Y., Ma, T., Zhang, C., and Ye, Y. Dual-level hypergraph contrastive learning with adaptive temperature enhancement. In WWW, 2024

    2024

  37. [45]

    Adaptive graph enhancement for imbalanced multi-relation graph learning

    Qian, Y., Ma, T., Zhang, C., and Ye, Y. Adaptive graph enhancement for imbalanced multi-relation graph learning. In CIKM, 2025

    2025

  38. [46]

    Gcc: Graph contrastive coding for graph neural network pre-training

    Qiu, J., Chen, Q., Dong, Y., Zhang, J., Yang, H., Ding, M., Wang, K., and Tang, J. Gcc: Graph contrastive coding for graph neural network pre-training. In KDD, 2020

    2020

  39. [47]

    and Gurevych, I

    Reimers, N. and Gurevych, I. Sentence-bert: Sentence embeddings using siamese bert-networks. In EMNLP, 2019

    2019

  40. [48]

    Perturbation bounds for means of eigenvalues and invariant subspaces

    Ruhe, A. Perturbation bounds for means of eigenvalues and invariant subspaces. BIT Numerical Mathematics, 1970

    1970

  41. [49]

    Graphon neural networks and the transferability of graph neural networks

    Ruiz, L., Chamon, L., and Ribeiro, A. Graphon neural networks and the transferability of graph neural networks. In NeurIPS, 2020

    2020

  42. [50]

    A., Cubuk, E

    Sohn, K., Berthelot, D., Carlini, N., Zhang, Z., Zhang, H., Raffel, C. A., Cubuk, E. D., Kurakin, A., and Li, C.-L. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. In NeurIPS, 2020

    2020

  43. [51]

    Preference ranking optimization for human alignment

    Song, F., Yu, B., Li, M., Yu, H., Huang, F., Li, Y., and Wang, H. Preference ranking optimization for human alignment. In AAAI, 2024

    2024

  44. [52]

    and Ribeiro, B

    Srinivasan, B. and Ribeiro, B. On the equivalence between positional node embeddings and structural graph representations. In ICLR, 2020

    2020

  45. [53]

    A molecular multimodal foundation model associating molecule graphs with natural language

    Su, B., Du, D., Yang, Z., Zhou, Y., Li, J., Rao, A., Sun, H., Lu, Z., and Wen, J.-R. A molecular multimodal foundation model associating molecule graphs with natural language. arXiv, 2022

    2022

  46. [54]

    Gppt: Graph pre-training and prompt tuning to generalize graph neural networks

    Sun, M., Zhou, K., He, X., Wang, Y., and Wang, X. Gppt: Graph pre-training and prompt tuning to generalize graph neural networks. In KDD, 2022

    2022

  47. [55]

    All in one: Multi-task prompting for graph neural networks

    Sun, X., Cheng, H., Li, J., Liu, B., and Guan, J. All in one: Multi-task prompting for graph neural networks. In KDD, 2023

    2023

  48. [56]

    Fine-tuning graph neural networks by preserving graph generative patterns

    Sun, Y., Zhu, Q., Yang, Y., Wang, C., Fan, T., Zhu, J., and Chen, L. Fine-tuning graph neural networks by preserving graph generative patterns. In AAAI, 2024

    2024

  49. [57]

    Transductive linear probing: a novel framework for few-shot node classification

    Tan, Z., Wang, S., Ding, K., Li, J., and Liu, H. Transductive linear probing: a novel framework for few-shot node classification. In LoG, 2022

    2022

  50. [58]

    Graphgpt: Graph instruction tuning for large language models

    Tang, J., Yang, Y., Wei, W., Shi, L., Su, L., Cheng, S., Yin, D., and Huang, C. Graphgpt: Graph instruction tuning for large language models. In SIGIR, 2024

    2024

  51. [59]

    Galactica: A large language model for science

    Taylor, R., Kardas, M., Cucurull, G., Scialom, T., Hartshorn, A., Saravia, E., Poulton, A., Kerkez, V., and Stojnic, R. Galactica: A large language model for science. arXiv, 2022

    2022

  52. [60]

    G., Azabou, M., Dyer, E

    Thakoor, S., Tallec, C., Azar, M. G., Azabou, M., Dyer, E. L., Munos, R., Veli c kovi \'c , P., and Valko, M. Large-scale representation learning on graphs via bootstrapping. In ICLR, 2022

    2022

  53. [61]

    Llama: Open and efficient foundation language models

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozi \`e re, B., Goyal, N., Hambro, E., Azhar, F., et al. Llama: Open and efficient foundation language models. arXiv, 2023

    2023

  54. [62]

    Graph attention networks

    Veli c kovi \'c , P., Cucurull, G., Casanova, A., Romero, A., Liò, P., and Bengio, Y. Graph attention networks. In ICLR, 2018

    2018

  55. [63]

    Can language models solve graph problems in natural language? In NeurIPS, 2024 a

    Wang, H., Feng, S., He, T., Tan, Z., Han, X., and Tsvetkov, Y. Can language models solve graph problems in natural language? In NeurIPS, 2024 a

    2024

  56. [64]

    Graph few-shot learning with task-specific structures

    Wang, S., Chen, C., and Li, J. Graph few-shot learning with task-specific structures. NeurIPS, 2022 a

    2022

  57. [65]

    Task-adaptive few-shot node classification

    Wang, S., Ding, K., Zhang, C., Chen, C., and Li, J. Task-adaptive few-shot node classification. In KDD, 2022 b

    2022

  58. [66]

    V., Zhang, C., and Ye, Y

    Wang, Z., Zhang, Z., Chawla, N. V., Zhang, C., and Ye, Y. Gft: Graph foundation model with transferable tree vocabulary. In NeurIPS, 2024 b

    2024

  59. [67]

    Subgraph pooling: Tackling negative transfer on graphs

    Wang, Z., Zhang, Z., Zhang, C., and Ye, Y. Subgraph pooling: Tackling negative transfer on graphs. In IJCAI, 2024 c

    2024

  60. [68]

    Can llms convert graphs to text-attributed graphs? In NAACL, 2025 a

    Wang, Z., Liu, S., Zhang, Z., Ma, T., Zhang, C., and Ye, Y. Can llms convert graphs to text-attributed graphs? In NAACL, 2025 a

    2025

  61. [69]

    V., Zhang, C., and Ye, Y

    Wang, Z., Zhang, Z., Ma, T., Chawla, N. V., Zhang, C., and Ye, Y. Neural graph pattern machine. arXiv, 2025 b

    2025

  62. [70]

    Training mlps on graphs without supervision

    Wang, Z., Zhang, Z., Zhang, C., and Ye, Y. Training mlps on graphs without supervision. In WSDM, 2025 c

    2025

  63. [71]

    Y., Guu, K., Yu, A

    Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., and Le, Q. V. Finetuned language models are zero-shot learners. arXiv, 2021

    2021

  64. [72]

    From coarse to fine: enable comprehensive graph self-supervised learning with multi-granular semantic ensemble

    Wen, Q., Ju, M., Ouyang, Z., Zhang, C., and Ye, Y. From coarse to fine: enable comprehensive graph self-supervised learning with multi-granular semantic ensemble. In ICML, 2024

    2024

  65. [73]

    and Huang, C

    Xia, L. and Huang, C. Anygraph: Graph foundation model in the wild. arXiv, 2024

    2024

  66. [74]

    Opengraph: Towards open graph foundation models

    Xia, L., Kao, B., and Huang, C. Opengraph: Towards open graph foundation models. arXiv, 2024

    2024

  67. [75]

    M., Raghunathan, A., Liang, P., and Ma, T

    Xie, S. M., Raghunathan, A., Liang, P., and Ma, T. An explanation of in-context learning as implicit bayesian inference. In ICLR, 2022

    2022

  68. [76]

    How powerful are graph neural networks? In ICLR, 2019

    Xu, K., Hu, W., Leskovec, J., and Jegelka, S. How powerful are graph neural networks? In ICLR, 2019

    2019

  69. [77]

    Text-free multi-domain graph pre-training: Toward graph foundation models

    Yu, X., Zhou, C., Fang, Y., and Zhang, X. Text-free multi-domain graph pre-training: Toward graph foundation models. arXiv, 2024

    2024

  70. [78]

    Florence: A new foundation model for computer vision

    Yuan, L., Chen, D., Chen, Y.-L., Codella, N., Dai, X., Gao, J., Hu, H., Huang, X., Li, B., Li, C., et al. Florence: A new foundation model for computer vision. arXiv, 2021

    2021

  71. [79]

    A deep-learning system bridging molecule structure and biomedical text with comprehension comparable to human professionals

    Zeng, Z., Yao, Y., Liu, Z., and Sun, M. A deep-learning system bridging molecule structure and biomedical text with comprehension comparable to human professionals. Nature communications, 2022

    2022

  72. [80]

    Beyond weisfeiler-lehman: A quantitative framework for gnn expressiveness

    Zhang, B., Gai, J., Du, Y., Ye, Q., He, D., and Wang, L. Beyond weisfeiler-lehman: A quantitative framework for gnn expressiveness. In ICLR, 2024 a

    2024

  73. [81]

    Zhang, C., Song, D., Huang, C., Swami, A., and Chawla, N. V. Heterogeneous graph neural network. In KDD, 2019

    2019

  74. [82]

    Dtgb: A comprehensive benchmark for dynamic text-attributed graphs

    Zhang, J., Chen, J., Yang, M., Feng, A., Liang, S., Shao, J., and Ying, R. Dtgb: A comprehensive benchmark for dynamic text-attributed graphs. arXiv, 2024 b

    2024

  75. [83]

    Labeling trick: A theory of using graph neural networks for multi-node representation learning

    Zhang, M., Li, P., Xia, Y., Wang, K., and Jin, L. Labeling trick: A theory of using graph neural networks for multi-node representation learning. NeurIPS, 2021

    2021

  76. [84]

    V., Zhang, C., and Ye, Y

    Zhang, Z., Wang, Z., Hou, S., Hall, E., Bachman, L., White, J., Galassi, V., Chawla, N. V., Zhang, C., and Ye, Y. Diet-odin: A novel framework for opioid misuse detection with interpretable dietary patterns. In KDD, 2024 c

    2024

  77. [85]

    Gimlet: A unified graph-text model for instruction-based molecule zero-shot learning

    Zhao, H., Liu, S., Chang, M., Xu, H., Fu, J., Deng, Z., Kong, L., and Liu, Q. Gimlet: A unified graph-text model for instruction-based molecule zero-shot learning. In NeurIPS, 2023

    2023

  78. [86]

    All in one and one for all: A simple yet effective method towards cross-domain graph pretraining

    Zhao, H., Chen, A., Sun, X., Cheng, H., and Li, J. All in one and one for all: A simple yet effective method towards cross-domain graph pretraining. In KDD, 2024 a

    2024

  79. [87]

    Graphany: A foundation model for node classification on any graph

    Zhao, J., Mostafa, H., Galkin, M., Bronstein, M., Zhu, Z., and Tang, J. Graphany: A foundation model for node classification on any graph. arXiv, 2024 b

    2024

  80. [88]

    Transfer learning of graph neural networks with ego-graph information maximization

    Zhu, Q., Yang, C., Xu, Y., Wang, H., Zhang, C., and Han, J. Transfer learning of graph neural networks with ego-graph information maximization. In NeurIPS, 2021

    2021

  81. [89]

    Deep graph contrastive representation learning

    Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., and Wang, L. Deep graph contrastive representation learning. In ICML Workshop, 2020

    2020

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.