{"id":"c6ae2279-38dc-4d90-96c5-532ab725325c","arxiv_id":"2412.16447","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"GeneralDyG combines ego-graph sampling, an alternating node/edge GNN, and a transformer to detect anomalies in dynamic graphs, reporting strong results on four benchmarks, though the claimed generalizability is not actually tested.","lead":"GeneralDyG is a new method for detecting unusual nodes and edges in graphs that change over time, tested on four real-world datasets. It combines small graph sampling, a graph neural network, and a transformer to capture structure and time, but the paper's evidence for its headline 'generalizable' claim is weak.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Training objective in Eq. 10 is internally inconsistent: labels are stated to be unavailable in training, but BCE loss uses them, and with all training labels zero it drives all scores to zero, leaving no mechanism for anomaly ranking.","rationale":"The reader identified the same weakest assumption, and I agree. This concern is load-bearing because every reported result depends on the model having learned something. The paper's own appendix supplies the objective that contradicts the problem setting. It is not a matter of disagreement with consensus; it is an internal inconsistency. I considered whether Eq. 10 might be a typo for a contrastive or reconstruction loss, but no such loss appears anywhere in the paper, and the ablation study removes components (ego-graph, GNN, Transformer) without ever changing the loss. The code release is the natural place to resolve this: the published implementation either uses labels during training, leaks test labels, or trains with a degenerate objective and the results would not reproduce. Because the central claim depends on an unspecified and internally contradictory training mechanism, the appropriate verdict is REJECT rather than UNVERDICTED: the paper as written cannot be accepted without a fundamental rewrite of the objective and re-running all experiments.","tokens_in":15045,"tokens_out":3353,"duration_ms":31459,"concrete_test":"Run the released implementation (github.com/YXNTU/GeneralDyG) and trace the exact loss used for training. If Eq. 10 is used with labels, check whether labels are truly absent from training: if test labels are used to supervise, the 'unsupervised' claim is refuted by test leakage; if all training labels are zero, verify that the BCE loss collapses to predicting all zeros and held-out anomalous events receive no higher scores than normal events. A minimal control: train on a normal-only dynamic graph, inject a small set of anomalous edges at evaluation, and measure whether the trained model's scores separate them (AUC > 0.5). If the model cannot separate them, the central claim is unsupported as stated.","verdict_should_be":"REJECT","load_bearing_attack":"The paper's central claim is that GeneralDyG 'significantly outperforms state-of-the-art methods on four real-world datasets.' The most load-bearing condition for that claim is that the method has a well-defined training signal that produces scores ranking anomalies above normal events. As written, it does not. The Problem Definition states that training is unsupervised: 'During training, all edges and nodes are considered normal. Binary labels indicating anomalies are provided during the testing process.' Yet Appendix Eq. 10 defines a binary cross-entropy objective over node and edge labels: L = α[-Σ_v (yn log f(v) + (1-yn) log(1-f(v)))] + β[-Σ_e (ye log f(e) + (1-ye) log(1-f(e)))]. With every training label set to 0, each summand reduces to -log(1-f), minimized at f=0; the network is trained to output zero for every event. No reconstruction error, density estimate, contrastive loss, or pseudo-labeling mechanism is described that would make zero-labeled BCE produce useful anomaly scores. For edge-level tasks the paper states that the final output embedding is 'directly used as the anomaly score,' so after this training the score is degenerate. This is not a missing ablation or a tuning issue; it is an incoherent training objective. The reported WADI result (F1 60.43 vs FuSAGNet 60.70) additionally contradicts the abstract's 'significantly outperforms,' but the objective problem is more fundamental: as specified, the method cannot learn to detect anomalies at all.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GeneralDyG, an unsupervised anomaly detection method for continuous-time dynamic graphs that aims to handle both edge-level and node-level anomalies in a single architecture. The method samples temporal k-hop ego-graphs around each event, embeds them with a node/edge alternating GNN called TensGNN, and feeds the resulting structural and temporal features into a Transformer with hierarchical special tokens. The authors report state-of-the-art or competitive results on four real-world datasets (Bitcoin-Alpha, Bitcoin-OTC, SWaT, WADI) and include ablations, parameter analyses, and a generalizability study.","tokens_in":15418,"tokens_out":3027,"duration_ms":28239,"significance":"If the claimed results were sound, GeneralDyG would be a practically valuable contribution: a single architecture that handles both edge- and node-level anomalies, with a lightweight ego-graph sampling scheme that could reduce computational cost and a TensGNN design that explicitly alternates node and edge message passing. The paper also provides a public code link and compares against a broad set of 20 baselines. However, the central training mechanism described in the paper is internally inconsistent, and one of the headline results contradicts the abstract's claim of significant improvement. These issues prevent the paper from currently supporting its main claims.","major_comments":[{"comment":"The training objective is internally inconsistent with the stated unsupervised setting. The Problem Definition states that all edges and nodes are considered normal during training and that binary labels are only provided for testing; yet Eq. (10) defines a binary cross-entropy loss over node and edge labels y_n and y_e. With every training label set to 0, each summand reduces to -log(1 - f(·)), which is minimized by f(·) = 0 for every event. The paper does not specify any reconstruction error, density estimate, contrastive objective, or pseudo-labelling mechanism that could turn this zero-label BCE loss into a meaningful anomaly score. Since the text says that for edge-level tasks the final output embedding is directly used as the anomaly score, the trained model would assign a degenerate score to all edges, leaving no mechanism for ranking anomalies above normal events. This is not a tuning or ablation issue; as written, the method cannot learn to detect anomalies.","section":"Problem Definition; Appendix, Objective Function (Eq. 10)"},{"comment":"The abstract and introduction claim GeneralDyG 'significantly outperforms state-of-the-art methods on four real-world datasets,' but Table 2 shows that on WADI GeneralDyG achieves F1=60.43, slightly lower than FuSAGNet's 60.70. The main text itself acknowledges this as 'slightly below.' The central claim of significant improvement is therefore false on one of the four datasets, and the claim should be qualified or the experiments revisited.","section":"Abstract; Table 2 (Node-Level results)"},{"comment":"Table 4 and the surrounding text do not present a valid comparison for the generalizability claim. The table's headers are confusing ('Node-Level Method' combined with 'Bitcoin-Alpha Edge-Level' and 'WADI'), and the text says baselines were 'evaluated in a mismatched dataset context' while the table appears to compare GeneralDyG's performance on one dataset with baselines' performance on another. Without a clear protocol—same task, same dataset, same anomaly injection, and standard evaluation metrics—this table does not support the conclusion that GeneralDyG generalizes across tasks better than the baselines. The authors should either redesign this experiment or remove this analysis.","section":"Generalizable Analysis, Table 4"}],"minor_comments":[{"comment":"The manuscript contains several typos and formatting inconsistencies, including 'Tabel 5' for Table 5, inconsistent spacing in 'W ADI', and 'LSTM-V AE' with an unwanted space.","section":"Throughout"},{"comment":"Equation (2) is difficult to parse; the notation '⟨|KHS |⟩, ai, ⟨|KHS |⟩, ai; 11, a2_i;1, ...' appears garbled and should be rewritten to clearly show how special tokens and hierarchical event sets are concatenated.","section":"Eq. (2)"},{"comment":"The caption and table layout should be revised so that columns unambiguously indicate which method is evaluated on which dataset/task; currently the table is not self-contained.","section":"Generalizable Analysis, Table 4"},{"comment":"The hyperparameters k=2 and K=2 are selected based on Figure 2 on Bitcoin-Alpha and then used for all datasets; the authors should state whether performance is robust to these settings on the other datasets, since a single-dataset choice of a key structural hyperparameter weakens the generality claim.","section":"Experiment Description (Appendix)"},{"comment":"The phrase 'they often lack of generalizability' is grammatically incorrect; it should be 'they often lack generalizability.'","section":"Introduction"}],"recommendation":"reject","confidential_remarks":"The central flaw in the training objective (all-zero labels with BCE) is not a local fix; it requires redesigning the learning signal or providing a concrete self-supervised mechanism. The contradiction on WADI and the unusable generalizability table further weaken the paper. The authors should be encouraged to resubmit a substantially revised version that specifies a well-defined unsupervised objective and reruns the experiments, but in its current form the manuscript's central claims are not supported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper has a load-bearing flaw in its training objective, and the abstract overclaims. In the appendix, Eq. (10) defines a binary cross-entropy loss over node and edge labels. But the problem definition states that during training all edges and nodes are normal, i.e., every label is 0. With all labels zero, the BCE loss reduces to -log(1-f), minimized at f=0. The authors then say the final output embedding is used directly as the anomaly score for edge-level tasks. That gives a degenerate score that cannot rank anything. No reconstruction error or density estimate is described to create a meaningful anomaly signal. This is not a missing ablation; as written, the method has no training signal for anomaly detection.\n\nThat said, the paper is not without value. The architecture is clearly presented: temporal ego-graph sampling, an alternating node/edge GNN (TensGNN), and a Transformer with GraphIT-style kernel attention. The special tokens for hierarchy are a small but sensible touch. The experimental section is thorough—20 baselines, four datasets, with an ablation study that isolates each component. The code is available. The WADI result is honestly reported in the text as slightly below FuSAGNet, but the abstract claims 'significantly outperforms' on all four datasets, which is contradicted by that same table.\n\nThe 'generalizable analysis' in Table 4 is weak. It compares unadapted baselines on mismatched task types and calls that a generalizability test. That is not evidence of transfer; it is a strawman comparison. The hyperparameter sensitivity analysis on k and K is reasonable, and choosing k=2, K=2 is standard model selection, not circular.\n\nIf the training objective issue can be resolved—for example, if the code uses a different loss or labels are not actually all zero—the empirical results might be salvageable. But as the paper stands, the central mechanism is incoherent. I would not cite it, but I'd send it to peer review because the flaw is specific, and the empirical study is substantial enough that the authors deserve a chance to clarify.","headline":"The training objective in Eq. (10) is incoherent as written—all-zero labels during training drive scores to zero—so the method cannot work as described, despite a solid experimental study.","tokens_in":15892,"tokens_out":2763,"would_cite":false,"duration_ms":25848,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims GeneralDyG, one unsupervised pipeline, detects anomalous edges and nodes in continuous-time dynamic graphs and outperforms state-of-the-art baselines on four real-world datasets.","keywords":["dynamic graph anomaly detection","continuous-time dynamic graphs","temporal ego-graph sampling","graph neural network","transformer","node anomaly detection","edge anomaly detection","generalization"],"falsifier":"Train GeneralDyG on the normal split of a benchmark such as Bitcoin-Alpha, score held-out test events with injected anomalies at 1%, 5%, and 10%, and compute AUC and AP; if the scores rank anomalous events no better than chance (AUC near 0.5), the generalizable-detection claim is refuted. The paper's Table 1 reports the opposite outcome, so this check is directly reproducible with the published protocol.","tokens_in":14857,"feed_emoji":"🕸️","tokens_out":10699,"duration_ms":88048,"temperature":0.7,"pith_summary":"GeneralDyG is a single unsupervised pipeline for anomaly detection in continuous-time dynamic graphs, designed to cover edge-level and node-level anomalies at once. Rather than consuming whole graphs or snapshot sequences, it samples a temporal $k$-hop ego-graph around each event, orders those historical interactions by time, and inserts special tokens that mark their hop hierarchy. A GNN extractor called TensGNN embeds nodes and edges together by alternating node and edge message passing, and a temporal-aware transformer scores each event. The paper reports that this pipeline outperforms twenty state-of-the-art baselines on Bitcoin-Alpha and Bitcoin-OTC (edge anomalies) and on SWaT and WADI (node anomalies), and that it degrades less than baselines when a model built for one task type is applied to the other. If the claim holds, one architecture can cover multiple dynamic-graph anomaly tasks without task-specific redesign.","feed_headline":"One model spots both bad edges and bad nodes in evolving graphs","feed_subtitle":"Sampling ego-graphs plus a node-edge GNN and a temporal transformer covers both anomaly tasks on four datasets.","key_machinery":"Temporal ego-graph sampling converts a large evolving graph into small, chronologically ordered subgraphs centered on each event; the $\\langle|KHS|\\rangle$ special tokens tell the model which hop layer each historical interaction belongs to. TensGNN alternately applies node-level propagation $H_v^{(K+1)} = \\sigma(T^T H_e^{(K)} W'_e \\odot \\bar{A}_v H_v^{(K)} W_v)$ and edge-level propagation $H_e^{(K+1)} = \\sigma(T^T H_v^{(K)} W'_v \\odot \\bar{A}_e H_e^{(K)} W_e)$, so node and edge embeddings refine each other inside a shared feature space. The temporal-aware transformer then rewrites self-attention as kernel smoothing, with queries and keys built from the GNN structure embedding $\\varphi(z_i)$ and values from the original event features, preserving the raw signal while attending over structure. Together these components are what the paper credits for handling data diversity, capturing local and global dynamics, and keeping computational cost low.","core_discovery":"The central claim is that generalizability follows from treating every edge and node as an 'anomaly event' and encoding each event through its own temporally ordered $k$-hop ego-graph. The mechanism is: temporal ego-graph sampling so the model sees local neighborhoods rather than entire graphs; TensGNN, which alternates node-level and edge-level propagation rules built on Laplacian-adjacency matrices $\\bar{A}_v$ and $\\bar{A}_e$; and a temporal-aware transformer whose attention kernel uses structural embeddings for queries and keys while keeping the original event features as values, with $\\langle|KHS|\\rangle$ tokens preserving hop hierarchy. Trained with binary cross-entropy under the convention that all training events are normal, the model assigns anomaly scores that the authors report beat state-of-the-art edge-level and node-level detectors on four benchmarks. The result is a single score function for both anomalous edges and anomalous nodes.","pith_inferences":["A stricter test of the paper's generalizability claim would be zero-shot transfer: train once on one domain and score anomalies in an unseen domain with different semantics, such as network-intrusion logs, without retraining or retuning.","The training objective in Eq. (10) uses binary cross-entropy on labels that are all normal during training, so the method is effectively one-class; the anomaly-ranking behavior must come from an implicit inductive bias, such as anomalies having atypical ego-graph neighborhoods, which the paper does not name.","If the mechanism is the portable part, swapping TensGNN for a different node-and-edge GNN should preserve most of the benefit, which gives a cheap experimental check of where the generalization actually comes from."],"forward_implications":["A single trained model can score both anomalous edges and anomalous nodes, so separate pipelines for, say, fraudulent transactions and compromised users would not be needed.","Because training sees $k$-hop ego-graphs around events rather than whole graphs, the method scales more cheaply to large dynamic graphs.","The model does not depend on rich raw event attributes; when attributes are sparse or absent, topology and timing alone can still form the event encoding.","On the reported benchmarks, the largest gains are in average precision on edge-level datasets, so the method's main practical value may be ranking rare anomalous events ahead of many normal ones."],"supporting_citations":[{"why":"Supplies the Bitcoin-Alpha and Bitcoin-OTC experimental settings and is the strongest edge-level baseline GeneralDyG compares against.","marker":"Liu et al. 2021"},{"why":"Supplies the SWaT and WADI node-level benchmark settings and a baseline, and motivates the need for explicit timestamps.","marker":"Deng and Hooi 2021"},{"why":"Provides the kernel-smoothing self-attention formulation that the Temporal-Aware Transformer adapts.","marker":"Mialon et al. 2021"},{"why":"Defines the base Transformer attention machinery that the temporal module builds on.","marker":"Vaswani et al. 2017"},{"why":"StrGNN is a subgraph-based edge-level baseline whose closed-subgraph idea is closely related to ego-graph sampling.","marker":"Cai et al. 2021"},{"why":"SimpleDyG is the baseline and motivating contrast for why edge and topology information must not be discarded.","marker":"Wu, Fang, and Liao 2024"},{"why":"TGAT is a temporal graph embedding baseline that GeneralDyG must outperform in the edge-level comparison.","marker":"Xu et al. 2020"}],"fun_headline_variants":["GeneralDyG: one model for both edge and node anomalies","Ego-graph sampling makes anomaly detection generalize","Unified anomaly score for dynamic graph edges and nodes","Generalizable anomaly detection via temporal ego-graphs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Training uses binary cross-entropy on data in which every training event is labeled normal, so the model's ability to rank unseen anomalies above normal events depends on an unstated inductive bias rather than on an explicit reconstruction, density, or contrastive signal.","fun_headline_variants_meta":{"raw":{"variants":["GeneralDyG: one model for both edge and node anomalies","Ego-graph sampling makes anomaly detection generalize","Unified anomaly score for dynamic graph edges and nodes","Generalizable anomaly detection via temporal ego-graphs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000269,"raw_usage":{"total_tokens":1570,"prompt_tokens":841,"completion_tokens":729,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":457,"completion_tokens_details":{"reasoning_tokens":674}},"tokens_in":457,"tokens_out":729,"duration_ms":6672,"temperature":1.0,"reasoning_tokens":674,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:34:10.170153+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train GeneralDyG on the normal split of a benchmark such as Bitcoin-Alpha, score held-out test events with injected anomalies at 1%, 5%, and 10%, and compute AUC and AP; if the scores rank anomalous events no better than chance (AUC near 0.5), the generalizable-detection claim is refuted. The paper's Table 1 reports the opposite outcome, so this check is directly reproducible with the published protocol.","supporting_citations":[{"cited_title":"G.; Xiong, F.; Wang, L.; Chen, Q.; and Lee, V","cited_arxiv_id":null,"evidence_quote":"Supplies the Bitcoin-Alpha and Bitcoin-OTC experimental settings and is the strongest edge-level baseline GeneralDyG compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"StrGNN is a subgraph-based edge-level baseline whose closed-subgraph idea is closely related to ego-graph sampling."}],"review_version":1}