{"id":"27fe8fc9-9f5c-4d61-a16e-0a5ba7770f2a","arxiv_id":"2505.12027","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"REEF pre-trains a graph model using relation tokens and hypernetworks that generate relation-specific aggregators and classifiers, improving cross-dataset transfer on node classification and link prediction.","lead":"This paper introduces REEF, a graph foundation model that treats typed relations between nodes, encoded as text tokens, as the basic unit for pre-training across datasets. It reports large gains over prior graph pre-training methods on few-shot transfer tasks and link prediction.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper never cleanly tests whether relation tokens transfer: Table 2 uses only pretrained relation types, the 'out-of-domain' test is contaminated, and Figure 3 indicates the shared 'citation' token hurts cross-field transfer.","rationale":"The paper is coherent and the architectural components are clearly described; the ablations in Table 5 give real support for the claim that LM-initialized relation representations and the dataset-level modules matter, and the transfer results in Table 2 are consistently directionally favorable. My concern is not about the honesty of the experiments but about what they can plausibly attribute to the relation-vocabulary mechanism. Every transfer target in Table 2 uses a relation type already present during pretraining, the supposedly out-of-domain experiment is contaminated in its first row and near-chance in its second, and Figure 3 contains an internal test—shared 'citation' token across biomedical and CS domains—that goes the wrong way. These three observations jointly indicate that the central claim 'relation tokens serve as effective basic units' is not yet supported by the evidence. The proposed domain-split test would distinguish between two alternatives: (i) the LM embedding does not capture domain-invariant relation semantics, or (ii) the relation token is essentially a free parameter and the transfer gains come from the dataset projector/bias and fine-tuning. Either way, the paper would need to re-run or clarify the experiments to maintain the conditional acceptance. The existing CONDITIONAL verdict is therefore unchanged.","tokens_in":22518,"tokens_out":12290,"duration_ms":123484,"concrete_test":"Run controlled transfer to Cora with domain-split relation tokens: pretrain REEF on Pubmed and Citeseer using two separate tokens ('biomedical citation' and 'computer-science citation') instead of the shared 'citation' token, then fine-tune on Cora using the CS token. Compare against the shared-token numbers in Figure 3 (Citeseer-only 49.69; Pubmed+Citeseer 44.97). If the split-token version reaches or exceeds the Citeseer-only accuracy, the shared token is actively hurting transfer because citation semantics are not domain-invariant. If it does not, the relation-token mechanism is not responsible for the observed transfer.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that relation tokens are effective basic units for a graph foundation model because their semantics transfer across datasets. The main transfer experiments in Table 2 cannot establish this: Cora, Cornell and Computers reuse relation tokens that are already in the pretraining vocabulary (citation/paper classification, hyperlink/webpage classification, co-purchase/product classification), so they demonstrate reuse of seen tokens, not generalization of the vocabulary. The only experiment designed for unseen relations, Table 6 in Appendix F.3, is mislabeled: its first row uses the full 7-dataset pretraining set, which contains the target-domain relations, while its second row—the only genuinely unseen-relation condition—yields near-random results (Cornell Acc 34.58±12.87; Computers Acc 26.40±11.19, with AUC 47.65 and F1 6.10 on Computers). This is the regime the relation vocabulary is supposed to enable, and it does not transfer. Figure 3 provides a further internal check on the shared 'citation' token: Pubmed-only pretraining transfers far worse to Cora than Citeseer-only (37.60 vs 49.69), and adding Pubmed to Citeseer lowers Cora accuracy from 49.69 to 44.97. If the token captured domain-invariant citation semantics, this negative transfer should not occur. The ablations in Table 5 show that LM initialization helps, but they do not separate semantic transfer from the contributions of dataset-specific projectors, feature bias, and fine-tuning of the pretrained backbone. The load-bearing mechanism of the paper is therefore not demonstrated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes REEF, a graph foundation model built on relation tokens. Each edge or task type is mapped to a textual description, embedded with Sentence-BERT, and two hypernetworks generate relation-specific aggregator and classifier parameters; a third hypernetwork generates dataset-specific projectors and feature biases, and pretraining combines mixed datasets with edge-drop augmentation. The experiments evaluate pretraining accuracy on five datasets, C-way 1-shot transfer to Cora, Cornell, and Computers, link prediction on FB15K237 and WN18RR, ablations, and scaling analyses.","tokens_in":22816,"tokens_out":6025,"duration_ms":56818,"significance":"The relation-token design is original and the framework is coherent; the ablation study isolates the contributions of LM initialization, feature projector/bias, and augmentation, and the paper provides implementation details and code. If the claimed transferability held, this would be an important step toward graph foundation models with a shared relational vocabulary. However, as detailed below, the evidence for generalization to unseen relation types is negative, and the transfer experiments use relations already seen during pretraining, so the paper's strongest claims are currently not supported.","major_comments":[{"comment":"The out-of-domain experiment is mislabeled and, in the only genuinely unseen-relation condition, REEF performs at chance level. The first row uses the full seven-dataset pretraining set, which includes Wisconsin, Texas, and Photo, so the hyperlinks, webpage classification, co-purchase, and product classification tokens are all present in the relation vocabulary; this row is therefore an in-vocabulary transfer test. The second row (Pubmed, Citeseer, FB15K237, WN18RR) excludes those domains, but Cornell accuracy is 34.58±12.87 (chance is 20%) and Computers accuracy is 26.40±11.19 with AUC 47.65 and F1 6.10, which is chance-level. This directly contradicts the text's claim that REEF can still effectively transfer knowledge and adapt to new, unseen relations.","section":"Appendix F.3, Table 6"},{"comment":"The main transfer results do not test vocabulary generalization. Cora, Cornell, and Computers reuse the exact relation tokens (citation/paper classification, hyperlink/webpage classification, co-purchase/product classification) that were pretrained on Pubmed/Citeseer, Wisconsin/Texas, and Photo respectively, as stated in Section 4.1.2. At best, these results show that reusing a known relation token across datasets in the same domain helps; they say nothing about whether the relation vocabulary can be extended to unseen relations.","section":"Section 4.3, Table 2"},{"comment":"The scaling-law claim is not supported by the evidence. There are only six pretraining configurations, the trend is non-monotonic, and the citation-domain transfer results decrease when Pubmed is added to Citeseer (Cora accuracy drops from 49.69 to 44.97). The average curve is dominated by pretraining accuracy on the added datasets, so it does not demonstrate that larger pretraining sets improve transfer. A monotonic trend over more scales and repetitions, or a fitted model with error bars, would be needed to justify the term scaling law.","section":"Section 4.4, Figure 3"},{"comment":"No variance or significance information is reported for the pretraining comparisons, and the experimental setup is not commensurable: REEF's per-relation binary classifiers are averaged over binary tasks, while the joint GCN/GAT/RGCN baselines are trained with a single multi-class head over the union of label spaces. The headline highest average accuracy of 79.70% therefore conflates different prediction regimes. Please report standard deviations, significance tests, and a matched evaluation protocol, and note that REEF does not beat GCN or GAT on Pubmed and Citeseer in Table 1.","section":"Section 4.2, Table 1"}],"minor_comments":[{"comment":"The text says the pretraining configuration includes Pubmed, Citeseer, Wisconsin, Texas, Cornell, FB15K237, and WN18RR, but Cornell is a target dataset, not a pretraining dataset; this is inconsistent with Table 6 and should be corrected.","section":"Appendix F.3"},{"comment":"There are several typos and formatting issues, including pre-training performace in Section 4.4, Imporve. in Table 2, differenet in Appendix F, and T exas in Figure 3; a careful proofread is needed.","section":"Throughout"},{"comment":"The relation vocabulary size of 254 is stated without a derivation; please clarify whether it counts both aggregator and classifier tokens for each dataset and all 237 plus 11 knowledge-graph relation types, since Table 7 suggests the count should be 248 plus 2 plus 2 plus 2 equals 254.","section":"Section 4.1.2"},{"comment":"The limitation discussion focuses only on homogeneous graphs and does not address the core assumption that a shared relation description has consistent semantics across domains; the paper should acknowledge that the LM-based relation representation was not directly validated for cross-field consistency.","section":"Section 6"}],"recommendation":"major_revision","confidential_remarks":"The paper's central contribution is currently overstated. The mislabeled out-of-domain table is especially concerning because the genuinely unseen-relation row is at chance. If the authors cannot provide positive evidence for unseen-relation transfer, the manuscript should be reframed as a study of seen-relation reuse; the current abstract and introduction overstate the scope. The reported per-dataset results also suggest that the improvement is not uniform, so the claims need to be calibrated carefully."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Hi [Colleague],\n\nRead REEF. Here's the short version: the idea is genuinely interesting—encoding relation types with Sentence-BERT and using hypernetworks to generate relation-specific aggregators and classifiers is a new synthesis, and the authors make the components work together coherently. But the paper's central claim—that relation tokens form a transferable vocabulary for graph foundation models—is not actually demonstrated. The main transfer experiments reuse relation types that already appeared in pretraining, and the one experiment that tests unseen relations (Appendix F.3) shows near-random performance. That's the load-bearing claim, and it doesn't hold as written.\n\nWhat's good: the architecture is sensible, the ablations show the LM initialization helps, and the pre-training results on the five source datasets are respectable. The dataset-specific projectors and feature bias are a reasonable way to handle feature distribution shift across datasets sharing the same relation. If the contribution is framed as \"cross-dataset transfer when relation types are shared,\" then the paper is a useful building block.\n\nWhere it falls short: First, Table 2's target datasets (Cora, Cornell, Computers) all have relation types present in the pretraining set—citation, hyperlink, co-purchase—so it is not evidence of vocabulary generalization. Second, the \"out-of-domain\" experiment is mislabeled: the first row of Table 6 includes Wisconsin, Texas, and Photo, which contain the target relations, and the second row—the only genuinely unseen-relation condition—gives Cornell 34.58±12.87 and Computers 26.40±11.19 with AUC 47.65 and F1 6.10. Those numbers are close to chance and contradict the claim that REEF can \"effectively transfer knowledge and adapt to new, unseen relations.\" Third, Figure 3 shows Pubmed-only pretraining transfers to Cora worse than Citeseer-only (37.60 vs 49.69), and adding Pubmed lowers accuracy. That is negative transfer despite identical \"citation\" tokens, undercutting the claim that the token captures domain-invariant semantics. Fourth, Table 1 has no error bars and compares REEF's per-relation binary classifiers against multi-class joint baselines without explaining the conversion—an apples-to-oranges risk. Fifth, the \"scaling law\" in Section 4.4 is a non-monotonic curve, and the authors admit the initial drop; calling it a scaling law is overreach. Finally, the checklist claims an anonymized code link in the abstract, but no such link appears.\n\nNone of these are unfixable. A revision that reframes the contribution to cross-dataset transfer with shared relations, reports error bars, cleanly relabels the out-of-domain experiment, and stops calling the curve a scaling law would make this a solid paper. As is, it deserves a serious referee because the architecture is worth discussing, but the authors need to run the missing experiments and clarify the claims before it is citable as a foundation model.","headline":"Good architecture, but the load-bearing transfer claim isn't supported: unseen relations fail, and the 'out-of-domain' experiment is mislabeled.","tokens_in":23385,"tokens_out":4475,"would_cite":false,"duration_ms":42743,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Typed relations, not nodes or datasets, can serve as the shared vocabulary of a graph foundation model, enabling cross-domain pretraining and transfer.","keywords":["graph foundation models","relation tokens","hypernetworks","cross-domain graph pretraining","few-shot transfer","link prediction","Sentence-BERT relation embeddings","graph data augmentation"],"falsifier":"Take a citation graph whose edges are structurally identical but whose relation description is replaced with a semantically different phrase, and test transfer to Cora; if accuracy does not change, the language-model semantics are not load-bearing, and if it collapses, the consistency assumption is doing the work. A second check: compare PubMed-only versus Citeseer-only pretraining on Cora, where the paper itself predicts a gap because the two citation networks come from different fields.","tokens_in":22299,"feed_emoji":"🔗","tokens_out":8397,"duration_ms":79332,"temperature":0.7,"pith_summary":"The paper claims that typed relations between nodes—citation, hyperlink, co-purchase—are the right basic units for a graph foundation model, playing the role that tokens play in large language models. It proposes REEF, which converts textual descriptions of relation types into language-model embeddings, collects those embeddings into a shared relation vocabulary, and uses hypernetworks to generate relation-specific GNN aggregators and classifiers from each token. Because the same relation type recurs across datasets in a domain, pretraining on several graphs transfers to an unseen graph of that domain. The paper reports that this design beats GNN, contrastive, and existing graph-foundation baselines on pretraining and few-shot transfer, and that adding more pretraining datasets with a richer relation vocabulary improves average accuracy. The core claim is that relational semantics, not node identity or dataset identity, is what carries transferable graph knowledge.","feed_headline":"Relations become the tokens that let graph models transfer knowledge","feed_subtitle":"REEF turns typed edges like citation and co-purchase into shared tokens that transfer across graphs.","key_machinery":"The central machinery is the relation vocabulary plus the hypernetworks that consume it. A relation token is the text description of an edge or task type embedded by Sentence-BERT and stored in one shared vocabulary over all pretraining datasets. From each token embedding, the aggregator hypernetwork $F_{\\mathrm{Agg}}$ generates layerwise message-passing parameters $\\Phi_r^{(l)}$, and the classifier hypernetwork $F_{\\mathrm{Cls}}$ generates relation-specific classifier weights $\\Psi_r$, so the GNN's behavior is conditioned on the semantics of the relation rather than on the dataset it came from. A third hypernetwork $F_{\\mathrm{Proj}}$ maps the dataset description embedding to a projector $\\Phi_G$, and a feature bias $h_G$ is added to each node's initial representation, allowing relation-level sharing while adapting to dataset-specific feature distributions. The pretraining objective is binary relation prediction on subgraph pairs, trained with mixed-dataset batching and edge-drop augmentation.","core_discovery":"REEF's central claim is that a graph foundation model can be built on a vocabulary of relation tokens instead of node or dataset tokens. Each relation type is described in words, embedded with Sentence-BERT, and the embedding is passed to two hypernetworks that produce the weight matrices of that relation's message-passing aggregator and its task classifier; a third hypernetwork produces a dataset-specific feature projector and a feature bias from the dataset's text description. Pretraining is formulated as binary relation prediction between two subgraphs—does relation $r$ hold between $s_i$ and $s_j$?—with seven pretraining datasets spanning citation, WebKB, Amazon, and knowledge-graph domains and a shared vocabulary of 254 relation tokens. The same aggregator and classifier tokens are reused when fine-tuning on target datasets from the same domain. Reported results include a pretraining average accuracy of 79.70% on five benchmark datasets, accuracy gains over the best transfer baseline of 10.07–24.05% on three $C$-way-1-shot target graphs, and link-prediction accuracies of 91.04% on FB15K237 and 94.70% on WN18RR.","pith_inferences":["Editorial inference: if relation embeddings are the reusable unit, the vocabulary is open-ended, so a new relation type could be added by embedding its description without retraining; the paper does not run this zero-shot relation-extension experiment.","Editorial inference: the architecture splits transfer cleanly—relation tokens control aggregator and classifier behavior while dataset tokens control feature projection—so node-level tokenization could be added to the same backbone to combine both transfer mechanisms.","Editorial inference: the scaling results imply that relation vocabulary size, not node count or parameter count, may be the natural capacity measure for a graph foundation model, which a dedicated vocabulary-scaling study could test directly."],"forward_implications":["Pretraining on more datasets with a richer relation vocabulary raises average downstream accuracy, a scaling trend the paper documents in Section 4.4.","A relation token learned on one dataset can be reused at fine-tuning time on an unseen dataset in the same domain, so citation and hyperlink knowledge transfers without retraining.","Relation-token pretraining extends beyond node classification to link prediction, outperforming the tree-vocabulary graph foundation baseline on both FB15K237 and WN18RR.","The ablation study shows that removing any of the three components—LM-initialized relation embeddings, dataset projector and feature bias, or edge-drop augmentation—lowers average performance, indicating each carries part of the transfer.","Out-of-domain transfer improves with relation diversity in pretraining: dropping WebKB and Amazon relations from the pretraining mix sharply degrades transfer to Cornell and Computers."],"supporting_citations":[{"why":"Supplies the Sentence-BERT embeddings that turn textual relation and dataset descriptions into the token representations the hypernetworks consume.","marker":"[23]"},{"why":"RGCN's relation-conditioned message passing provides the aggregator pattern that REEF replaces with hypernetwork-generated parameters.","marker":"[24]"},{"why":"Introduces hypernetworks, the mechanism used to generate aggregator, classifier, and projector weights from token embeddings.","marker":"[11]"},{"why":"GCOPE defines the cross-domain pretraining and C-way-1-shot transfer protocol and serves as the main transfer baseline REEF compares against.","marker":"[45]"},{"why":"MDGPT supplies the domain-token dataset-level pretraining baseline that REEF's relation-token approach is designed to outperform.","marker":"[44]"},{"why":"GFT provides the tree-vocabulary graph foundation baseline whose link-prediction setup REEF extends in Table 4.","marker":"[35]"}],"fun_headline_variants":["REEF makes relations the tokens of graph foundation models","Graph transfer with a vocabulary of relation tokens","Hypernetworks craft relation tokens for graph pretraining","From edges to tokens: REEF's recipe for graph transfer","Relation tokens: the unit that lets graphs share knowledge"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the language-model embedding of a textual relation description is semantically consistent enough across datasets and domains that one relation token can generate useful aggregator and classifier weights for all of them, as when a citation token pretrained on PubMed and Citeseer is reused for Cora.","fun_headline_variants_meta":{"raw":{"variants":["REEF makes relations the tokens of graph foundation models","Graph transfer with a vocabulary of relation tokens","Hypernetworks craft relation tokens for graph pretraining","From edges to tokens: REEF's recipe for graph transfer","Relation tokens: the unit that lets graphs share knowledge"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000638,"raw_usage":{"total_tokens":2977,"prompt_tokens":1020,"completion_tokens":1957,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":636,"completion_tokens_details":{"reasoning_tokens":1880}},"tokens_in":636,"tokens_out":1957,"duration_ms":12567,"temperature":1.0,"reasoning_tokens":1880,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:42:11.713012+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a citation graph whose edges are structurally identical but whose relation description is replaced with a semantically different phrase, and test transfer to Cora; if accuracy does not change, the language-model semantics are not load-bearing, and if it collapses, the consistency assumption is doing the work. A second check: compare PubMed-only versus Citeseer-only pretraining on Cora, where the paper itself predicts a gap because the two citation networks come from different fields.","supporting_citations":[{"cited_title":"Schlichtkrull, T","cited_arxiv_id":null,"evidence_quote":"RGCN's relation-conditioned message passing provides the aggregator pattern that REEF replaces with hypernetwork-generated parameters."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"GCOPE defines the cross-domain pretraining and C-way-1-shot transfer protocol and serves as the main transfer baseline REEF compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"GFT provides the tree-vocabulary graph foundation baseline whose link-prediction setup REEF extends in Table 4."}],"review_version":1}