{"id":"40aa62fc-9247-45e2-801e-3bb5da616a11","arxiv_id":"2508.17690","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"The authors introduce TextTopoOOD, a four-scenario benchmark for out-of-distribution detection in text-rich networks, and TNT-OOD, a cross-attention and HyperNetwork model that improves detection over several baselines.","lead":"This paper builds a new test suite for spotting unusual nodes in networks where both text and connections matter, and proposes a model that combines both signals to flag anomalies. It evaluates the model on 11 networks and four types of shift.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"TNT-OOD's own per-scenario results show AUROC 3.77 on Reddit text shifts, contradicting the paper's claim of consistent improvement and revealing that Table 2's aggregation masks a central failure.","rationale":"The reader's weakest assumption was the unstated full-graph access at inference, which is a real concern about the inductive framing. However, the most load-bearing problem is more direct: the paper's own Appendix I contains a central scenario where TNT-OOD catastrophically underperforms all baselines, and the main text's aggregated Table 2 obscures this. Because the strongest claim is explicitly about consistent improvement across diverse OOD scenarios, a single counterexample in a scenario that is central to the paper's motivation is sufficient to falsify that claim. The Reddit text-shift failure is not a disagreement with the broader community; it is an internal inconsistency between the stated claim and the reported numbers. The benchmark contribution may still have value, and the failure may be repairable with a score calibration or a different alignment objective, but the paper as written cannot support the claim that TNT-OOD consistently outperforms or matches baselines. A conditional acceptance would require the authors to substantially revise the central claim, report per-scenario results prominently, and either fix or explain the inversion. Given that the current framing is contradicted by the data, the appropriate stress-test verdict is REJECT rather than CONDITIONAL.","tokens_in":63526,"tokens_out":5404,"duration_ms":59578,"concrete_test":"Run a focused reproduction of the Reddit Text (Synonym) and Text (Antonym) rows of Table 20 with 20 seeds, reporting per-seed AUROC and FPR95 for TNT-OOD, GNNSafe, and NodeSafe, and plot the OOD score distributions. Also compute max(AUROC, 1 - AUROC) to check whether a score-sign flip explains the near-zero value. If TNT-OOD's AUROC remains below 50 while the baselines remain above 80, the 'consistent outperforms' claim should be withdrawn. If flipping the sign of the alignment term in Eq. 26 restores AUROC above 90, the method requires scenario-dependent score calibration and cannot be presented as a single fixed detector.","verdict_should_be":"REJECT","load_bearing_attack":"The paper's strongest claim is that TNT-OOD 'consistently outperforms or matches baseline methods across diverse network domains and OOD scenarios' (Section 5.1). This claim is contradicted by the paper's own per-scenario results in Appendix I, not merely by missing baselines. In Table 20, on the Reddit social network, TNT-OOD obtains AUROC 3.77 on the Text (Synonym) shift and AUROC 4.72 on Text (Antonym), with FPR95 near 98-99%. For the same shifts, GNNSafe reaches 84.52 and 88.52 AUROC, and NodeSafe reaches 87.80 and 97.79. An AUROC of 3.77 is not a small underperformance; it is an inverted ranking, far worse than random and worse than every reported baseline. These are attribute-level textual shifts that TextTopoOOD is explicitly designed to expose and that the paper's motivating social-bot example invokes. Table 2 hides this failure by aggregating over scenarios (Reddit AUROC 70.61 with standard deviation 37.62). Therefore, the 'consistently outperforms or matches' claim is false as stated. The honest conclusion would be that TNT-OOD is brittle on text-only shifts in social networks, not that it consistently improves over baselines.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces two contributions: TextTopoOOD, a benchmark framework for out-of-distribution detection in text-rich networks covering attribute-level text/feature shifts, structural shifts, thematically guided label shifts, and domain-based splits across 11 datasets; and TNT-OOD, a detection method that fuses textual and structural information through cross-attention and uses a HyperNetwork to generate node-specific projections, with an OOD score combining energy and text-topology alignment. The empirical study compares TNT-OOD against post-hoc and graph-specific baselines and reports aggregated AUROC/AUPR/FPR95 over the proposed scenarios.","tokens_in":63812,"tokens_out":4688,"duration_ms":51271,"significance":"If the benchmark is adopted by the community, TextTopoOOD would fill a real gap: existing graph OOD benchmarks focus on label or degree/temporal splits and largely ignore the coupled text-topology shifts that arise in social, e-commerce, and citation networks. The paper ships code, uses 11 datasets across four shift families, and reports three-seed means with ablations and hyperparameter sensitivity, which supports reproducibility. The method itself is reasonable and the alignment-score idea is clearly motivated. However, the central empirical claim of consistent superiority is contradicted by the paper's own per-scenario tables, and the test-time assumption of full graph access is not discussed as a limitation.","major_comments":[{"comment":"The claim that TNT-OOD 'consistently outperforms or matches baseline methods' is not supported by the per-scenario results. On Reddit, for Text (Synonym), TNT-OOD obtains AUROC 3.77 ± 9.45 and FPR95 98.70 ± 28.31, while GNNSafe obtains 84.52 and NodeSafe obtains 87.80 AUROC; for Text (Antonym), TNT-OOD obtains 4.72 AUROC versus 88.52 and 97.79 for the two propagation baselines. An AUROC near 4 is worse than random and worse than every reported baseline, and it occurs on exactly the attribute-level text shifts that the paper's social-network motivation identifies as central. The aggregate Reddit row in Table 2 (70.61 ± 37.62) hides this failure because the standard deviation is enormous. The consistency claim should be removed or replaced with an explicit per-scenario discussion of where TNT-OOD fails and why.","section":"Section 5.1 and Appendix I, Table 20"},{"comment":"The final OOD score is smoothed by K-layer propagation over the adjacency matrix, and Appendix D.1 states that 'the full graph structure remains accessible during inference with its associated OOD nodes.' This is a strong inductive assumption: if OOD nodes arrive without observed edges, or with only partially observed topology, Eq. (27) is undefined or degenerates to propagation over an ID-only graph. The manuscript does not list this as a limitation and does not evaluate a variant where OOD-node edges are withheld at test time. Since several real-world deployments of node-level OOD detection involve newly arriving nodes, this modeling choice should be explicitly acknowledged and ideally stress-tested.","section":"Section 4.2, Eq. (27), and Appendix D.1"},{"comment":"On Arxiv, TNT-OOD achieves AUROC 47.65, which is substantially below Energy (76.96) and NECO (75.19); the paper's statement that TextTopoOOD 'reveals challenging scenarios' explains the difficulty but does not address the fact that the proposed method is among the weakest on that dataset. The aggregated presentation in Table 2 should be supplemented with per-scenario counts of wins and losses against the best baseline, and the phrase 'significant improvements' should be limited to datasets and scenarios where the improvements actually hold.","section":"Section 5.1, Table 2"}],"minor_comments":[{"comment":"The caption says Best and Runner-up are highlighted, but the table does not visually distinguish the two; please specify the formatting used for each.","section":"Table 2 caption"},{"comment":"The row labels 'TNT-OODFPR@ 4o mini TPR(↓)' and 'TNT-OODFPR@ Gemini TPR(↓)' are garbled, and the metric description 'FPR at equivalent TPR levels' should be defined more precisely.","section":"Table 5"},{"comment":"There are typos: 'TEXTAUGMEN T' in Table 1's caption and 'a TnR' in the sentence below Eq. (2) should read 'a TrN'.","section":"Table 1 and Eq. (2)"},{"comment":"For text augmentation the paper sets alpha_text = 1 and p_char = 1, meaning every eligible word is replaced and every replacement also receives character-level edits; this is a very strong perturbation, and it would help to report the actual coverage statistics for different datasets.","section":"Appendix D.1"}],"recommendation":"major_revision","confidential_remarks":"The benchmark and the method are both worth publishing, but the current framing overstates the method's performance. The paper would be more convincing if the title and abstract emphasized TextTopoOOD as the main contribution and presented TNT-OOD as a strong but brittle baseline, with per-scenario analyses of its failure modes. The full-graph-at-test-time assumption should also be disclosed prominently."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The benchmark is the real contribution here. TextTopoOOD covers four shift families (attribute, structural, thematically-guided label, domain) across 11 datasets, with code and construction details released. That is a step up from prior OOD-TAG/GLIP/GSyncOOD, which mostly stick to label or degree-based splits. If you work on graph or text-attributed OOD detection, this is a worthwhile evaluation suite to have on hand.\n\nThe method part is more ordinary. TNT-OOD is a reasonable composition of cross-attention, a HyperNetwork projection, and the energy-plus-propagation score from Wu et al. The ablation shows the cross-attention and contrastive loss help; the HyperNetwork is mixed. That is fine, but it does not change the fact that the central claim is over-stated. Section 5.1 says TNT-OOD 'consistently outperforms or matches baseline methods.' The paper's own Appendix I shows that on Reddit, Text (Synonym) and Text (Antonym), TNT-OOD gets AUROC 3.77 and 4.72, while GNNSafe gets 84.52 and NodeSafe gets 87.80/97.79. That is not a small underperformance; it is an inverted ranking. Table 2 hides it by aggregating across scenarios with a 37.62 standard deviation. The honest summary is that TNT-OOD is brittle on text-only shifts in at least one social network, not that it consistently beats baselines.\n\nThe full-graph-at-inference assumption is a second soft spot. Appendix D.1 says label-shift OOD nodes keep the complete edge set so the full graph is accessible during inference. The scoring function in Eq. 26-27 propagates over the adjacency matrix. If OOD nodes arrive without observed connections, the propagation step is undefined or silently becomes ID-only. The paper never flags this as a limitation, though the Limitations section does admit to approximation strategies and limited configuration search. Those admissions are honest but do not fix the main reporting problem.\n\nCitation pattern is fine; the method credits Wu et al. for propagation and builds on prior graph OOD work. The LLM-as-detector comparison is methodologically muddy but clearly secondary.\n\nBottom line: worth a serious referee, but the authors need to revise the 'consistent' claim, report per-scenario results without hiding the Reddit failure, and discuss the inductive setting honestly. I would bring the benchmark to a reading group, and I would cite it for the benchmark, not for the method.","headline":"TextTopoOOD is a genuinely useful benchmark, but the paper's 'consistently outperforms' claim is contradicted by its own per-scenario results on Reddit text shifts.","tokens_in":64393,"tokens_out":1216,"would_cite":true,"duration_ms":15453,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fusing text and topology with cross-attention and node-specific projections catches out-of-distribution nodes in text-rich networks where post-hoc and graph-only baselines miss.","keywords":["out-of-distribution detection","text-rich networks","graph neural networks","cross-attention","HyperNetwork","energy-based scoring","distribution shift","benchmark"],"falsifier":"Compare TNT-OOD against its own no-propagation variant on a split where OOD nodes and all incident edges are withheld from the adjacency matrix at inference. If the AUROC and FPR95 gap over energy-alone shrinks to zero once OOD edges disappear, the reported gains depend on seeing the very connections that an inductive deployment would not have.","tokens_in":63259,"feed_emoji":"🕸️","tokens_out":7073,"duration_ms":71683,"temperature":0.7,"pith_summary":"The paper sets out to show that out-of-distribution detection in text-rich networks is a two-modality problem: a node can drift because its language changed, because its connections changed, or because the two no longer agree. To test this it introduces TextTopoOOD, an evaluation suite covering attribute-level, structural, thematically-guided label, and domain-based shifts across 11 datasets. It then proposes TNT-OOD, which fuses neighborhood structure into text embeddings with cross-attention and projects each node through HyperNetwork-generated weights, scoring nodes by combining classifier energy with a text-topology alignment term. The reported results claim consistent gains over propagation-based and post-hoc baselines, with the largest improvements on citation and social networks.","feed_headline":"Fusing text with topology catches out-of-distribution nodes","feed_subtitle":"Four shift types across 11 networks test detectors on wording and wiring, not just labels.","key_machinery":"The load-bearing mechanism is the combination of neighborhood cross-attention fusion and node-specific HyperNetwork projection. Cross-attention uses structure embeddings as queries over neighboring text, injecting local graph context into each node's text representation; the HyperNetwork then emits per-node low-rank projection weights so that ID nodes can be aligned in a contrastive space while OOD nodes expose misalignment. The final detector score couples the energy of the classifier logits with an alignment score and propagates the result across the graph.","core_discovery":"The central claim is that TNT-OOD consistently outperforms or matches existing graph-OOD detectors across diverse network domains and OOD scenarios. The discovery is a design rather than a single identity: because OOD can surface as textual, structural, or joint text-structure misalignment, the detector builds a fused representation from a GCN-derived structure query and text keys and values via cross-attention, then generates node-specific projection matrices from a low-rank HyperNetwork, aligning text and structure embeddings with a symmetric contrastive loss. At test time the score subtracts a temperature-scaled alignment term from the energy logit score and propagates the smoothed scores over the adjacency matrix. The paper also argues that TextTopoOOD is the first benchmark to cover this range of shifts, and that the variance across scenarios shows the framework's difficulty.","pith_inferences":[],"forward_implications":["In citation networks, TNT-OOD reduces the false-positive rate at 95% true-positive rate by up to roughly 10 percentage points over the compared baselines.","On knowledge and social networks such as Reddit and WikiCS, the method raises average AUROC markedly, for example from about 57.43 to 70.61 on Reddit.","Score propagation helps detection on Cora but hurts it on Arxiv, so the value of graph-based score spreading is dataset-dependent rather than universal.","Coupled text and structure shifts are detected more accurately than either shift alone, suggesting that real-world joint shifts may be easier to flag than isolated ones.","TextTopoOOD exposes genuinely hard cases: on Arxiv even TNT-OOD underperforms relative to its results elsewhere, marking the temporal and feature-shift scenarios as open challenges.","Beyond the paper: the full-graph inductive setting means the reported numbers assume OOD nodes' edges are visible at inference; a cold-start evaluation with disconnected OOD nodes would isolate whether the alignment score alone carries the signal.","Beyond the paper: because the alignment score measures agreement between projected text and structure-aware embeddings, replacing the frozen SBERT encoder with a different sentence encoder is a direct test of whether the gain depends on one embedding geometry.","Beyond the paper: node-specific projections from a HyperNetwork could serve as a learned calibration layer for any two-modality model, so the idea transfers to image-text or tabular networks even though the paper does not claim this."],"supporting_citations":[{"why":"Supplies the energy-based score-propagation baseline GNNSafe that TNT-OOD is compared against and whose propagation scheme the paper adapts.","marker":"(Wu et al., 2023)"},{"why":"Supplies NODESafe, the bounded-uniform-energy propagation baseline used for comparison.","marker":"(Yang et al., 2024)"},{"why":"Defines the energy score that forms the logit-level component of TNT-OOD's final scoring function.","marker":"(Liu et al., 2020)"},{"why":"Provides the GCN layer used as the structure encoder for neighborhood-aware node representations.","marker":"(Kipf and Welling, 2017)"},{"why":"Provides the frozen SBERT text encoder that produces the node text embeddings.","marker":"(Reimers and Gurevych, 2019)"},{"why":"Introduces HyperNetworks, the concept behind TNT-OOD's node-specific projection generation.","marker":"(Ha et al., 2017)"},{"why":"OOD-TAG, an earlier text-attributed-graph OOD benchmark that TextTopoOOD is designed to go beyond.","marker":"(Wang et al., 2024c)"},{"why":"GLIP-OOD, a zero-shot label-shift OOD method whose narrower scenario coverage motivates TextTopoOOD.","marker":"(Xu et al., 2025b)"},{"why":"Provides a benchmark suite that supplies several text-rich network datasets used in the evaluation.","marker":"(Chen et al., 2024a)"},{"why":"Provides another benchmark source of datasets and splits used in the evaluation.","marker":"(Li et al., 2024)"}],"fun_headline_variants":["TNT-OOD fuses text and topology to catch OOD nodes","Cross-attention and HyperNetwork improve graph OOD detection","Four shift types, 11 networks: text-topology OOD benchmark","Detecting OOD nodes by aligning text and structure embeddings","TextTopoOOD: first OOD benchmark with 4 shift types"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The detector assumes that at test time the full graph, including edges attached to out-of-distribution nodes, is already known and can be used to spread scores; if out-of-distribution nodes arrive without any recorded connections, the propagation step has nothing to operate on.","fun_headline_variants_meta":{"raw":{"variants":["TNT-OOD fuses text and topology to catch OOD nodes","Cross-attention and HyperNetwork improve graph OOD detection","Four shift types, 11 networks: text-topology OOD benchmark","Detecting OOD nodes by aligning text and structure embeddings","TextTopoOOD: first OOD benchmark with 4 shift types"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000875,"raw_usage":{"total_tokens":3783,"prompt_tokens":938,"completion_tokens":2845,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":554,"completion_tokens_details":{"reasoning_tokens":2754}},"tokens_in":554,"tokens_out":2845,"duration_ms":20804,"temperature":1.0,"reasoning_tokens":2754,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:01:22.824855+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare TNT-OOD against its own no-propagation variant on a split where OOD nodes and all incident edges are withheld from the adjacency matrix at inference. If the AUROC and FPR95 gap over energy-alone shrinks to zero once OOD edges disappear, the reported gains depend on seeing the very connections that an inductive deployment would not have.","supporting_citations":[],"review_version":1}