{"id":"5d7b5adc-53af-4fae-9d5d-c6901c538f38","arxiv_id":"2505.07508","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A graph autoencoder with meta-path-level contrastive learning is proposed for unsupervised heterogeneous graph anomaly detection, reporting top AUC on DBLP, Aminer, and Yelp.","lead":"EAGLE is a contrastive learning method for spotting unusual nodes in heterogeneous networks, tested on three citation and review datasets. It claims faster detection with higher accuracy than existing graph anomaly detectors, but the evidence is weakened by synthetic test anomalies and inconsistent timing reports.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported SOTA superiority rests on synthetic anomalies constructed to match EAGLE's own local-context-distance assumption, so the empirical claim is not yet established for real anomaly distributions.","rationale":"The paper's contribution is a contrastive anomaly score that assumes anomalous nodes are more distant from their local meta-path context. That assumption appears in Instance Pair Sampling and in the claimed behavior of Eq. (11). The only experimental evidence for the contribution is on synthetic anomalies injected by replacing a node's attributes with those of a far-away node; this is exactly the assumption made by the method. Hence the benchmark is not independent evidence. The load-bearing question is not whether EAGLE is correctly implemented or whether the math is internally consistent; it is whether the empirical claim generalizes. A real heterogeneous-network anomaly (e.g., a fraudulent Yelp review, a citation-venue violation, an account hijacking) need not be \"attribute farthest from its neighbors,\" and the paper gives no evidence for that premise. The concrete test would settle this: run the same comparison on labels that are not constructed from the method's own assumption. If EAGLE's edge persists, this concern is retired; if it does not, the reported superiority is a benchmark artifact. This aligns with the reader's weakest-assumption analysis, though I would frame it as an external-validity problem with the central empirical claim rather than as an internal inconsistency. I am not raising authorship, code-release, or baseline-selection concerns as the primary issue; they are secondary to the fact that the only anomaly signal evaluated is the one the model was designed to detect.","tokens_in":9967,"tokens_out":8278,"duration_ms":80563,"concrete_test":"Re-run the Table 2 comparison using organic anomaly labels that are not produced by farthest-attribute substitution, e.g., the fraudulent-review labels in the Yelp dataset or structural anomalies created by edge rewiring on the same three graphs. Use the same train/fine-tune split, hyperparameters, and AUC metric, and report EAGLE against DOMINANT, AnomalyDAE, and FRAUDRE. If EAGLE's AUC margin disappears or reverses, the headline \"outperforms SOTA\" is an artifact of the synthetic injection protocol.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is Table 2's \"EAGLE outperforms the state-of-the-art methods.\" The only anomaly labels used to produce that table are synthetic, generated in the Experiments section by taking a target node, sampling k=50 nodes, and replacing the target's attribute embedding with the attribute of the farthest sampled node. This is explicitly a \"contextual anomaly based on the assumption that the node attribute embedding deviating from its neighbors is anomalous.\" That construction instantiates exactly the premise on which EAGLE's contrastive objective and Eq. (11) are built: anomalous nodes are distant from their local context. Thus the evaluation is constructed to satisfy the method's core assumption and cannot falsify it. It provides no evidence that EAGLE detects real heterogeneous-graph anomalies, which need not follow the local-context-distance pattern. The reported 28.4% average improvement and sub-second times are therefore not established for real anomaly distributions; the same benchmark could reward any method that detects attribute substitution. This is a validity problem with the central claim, not a disagreement with consensus.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes EAGLE, a self-supervised contrastive learning framework for anomaly detection on heterogeneous graphs. The method samples positive and negative instance pairs at the meta-path level, trains a graph autoencoder with a contrastive discriminator, and computes an anomaly score as a weighted combination of reconstruction errors and a discriminator-based term. The authors report AUC and timing results on three heterogeneous network datasets (DBLP, Aminer, Yelp), claiming that EAGLE outperforms state-of-the-art baselines and is efficient enough for embedded devices. The empirical evaluation, however, uses synthetic anomalies that are generated by replacing a target node's attribute vector with that of the farthest node among k=50 sampled neighbors, which directly instantiates the paper's core assumption that anomalous nodes are distant from their local context. Because the evaluation cannot falsify this assumption, the central claim of state-of-the-art accuracy is not established for real anomaly distributions.","tokens_in":10101,"tokens_out":6630,"duration_ms":63795,"significance":"If the reported accuracy and efficiency results held on realistic anomaly distributions, EAGLE would be a useful contribution to heterogeneous-graph anomaly detection, particularly for resource-constrained deployment. The idea of contrasting a node with its local meta-path context is a reasonable inductive bias, and combining a graph autoencoder with a contrastive discriminator is a sensible architecture. However, the paper provides no code, no error bars, no comparison with heterogeneous-graph-specific anomaly detectors, and an anomaly injection procedure that matches the model's own scoring assumption. The significance of the claimed SOTA superiority therefore depends entirely on an evaluation loop that is circular with respect to the model's core premise. The efficiency claim is also not yet credible because the timing protocol is undefined and the reported times for pre-training and no-pre-training configurations are inconsistent.","major_comments":[{"comment":"The evaluation is circular. The synthetic anomalies are created exactly as 'the node attribute embedding deviating from its neighbors is regarded as anomalous' by replacing a target node's attributes with the farthest of k=50 sampled nodes, while EAGLE's anomaly score in Eq. (11) and its contrastive objective explicitly assume that anomalous nodes are more distant from their local meta-path context than normal nodes. Table 2 therefore measures the model's ability to detect the particular attribute-substitution pattern used to generate labels, not its ability to detect real heterogeneous-graph anomalies. The 28.4% average improvement and the sub-second times are not established for anomalies that do not follow the local-context-distance pattern, and the paper provides no experiment with real anomalous labels or with injected anomalies that violate this assumption.","section":"Experiments, Dataset; Eq. (11)"},{"comment":"All AUC values in Table 2 are reported as single point estimates with no standard deviations, number of repeated runs, or statistical tests. The claimed improvements over AnomalyDAE, DOMINANT, and FRAUDRE (e.g., 0.9502 vs. 0.8018 on DBLP) cannot be distinguished from run-to-run variability. In addition, the timing comparison is ambiguous: EAGLE Pre (without pre-training) is reported as slower than EAGLE (with pre-training) on all datasets, which is only possible if the reported time excludes the pre-training cost. The paper does not state what is included in the reported timing, so the efficiency claim is not comparable across methods.","section":"Table 2; Experimental Setup"},{"comment":"The baseline set does not include any recent heterogeneous-graph-specific anomaly or fraud detection method, despite several being cited in the introduction, such as the heterogeneous malicious-account detector of Liu et al. [7] and the metagraph-aggregated illicit-product detector of Fan et al. [5]. A claim of outperforming the state of the art on heterogeneous networks requires at least one such heterogeneous-graph-specific detector in the comparison; without it, the SOTA claim is not supported.","section":"Table 2; Baselines"},{"comment":"The method is not reproducible from the text. The paper does not specify how meta-path instances are enumerated, how the set P of meta-paths is chosen for each dataset, how the target node is selected, how the meta-path adjacency matrices (PAP and PVP in Figure 2) are constructed, or how the pre-trained GAE is fine-tuned for the anomaly detection task. Without these details or a code release, the architecture cannot be reimplemented and the results in Table 2 are not independently checkable.","section":"Instance Pair Sampling; Graph AutoEncoder-based Contrastive Learning Model"}],"minor_comments":[{"comment":"There are typos in the equations: Eq. (4) says 'digonal matrix' instead of 'diagonal matrix', and Eq. (6) uses 'sigmod' instead of 'sigmoid'.","section":"Eq. (4), Eq. (6)"},{"comment":"The caption says the best results in each function are in bold, but no entries in the printed table are bold. The accompanying text also states that min pooling has the worst results on all three datasets, yet Table 3 shows min pooling outperforming max pooling on DBLP (0.9425 vs. 0.9409); the caption and the reported comparison need to be corrected.","section":"Table 3; Parameter Study"},{"comment":"Figure 3 appears garbled in the manuscript, with fragments such as '*OSKTYOUT' and '!(YKS' visible, making the embedding-dimension study difficult to read; the figure and its labels should be replaced.","section":"Figure 3"},{"comment":"The pre-training split (30% for DBLP and Aminer, 70% for Yelp) and the hyperparameters alpha=0.8, beta=0.2, and gamma=0.3 are stated without justification or sensitivity analysis, even though these free parameters directly affect the anomaly score in Eq. (11).","section":"Experimental Setup"},{"comment":"The name 'EAGLE Pre' for the variant without pre-training is confusing, since 'Pre' usually denotes pre-training; a name such as 'EAGLE-NoPretrain' would be clearer.","section":"Table 2"}],"recommendation":"reject","confidential_remarks":"The core issue is not a disagreement with the community's preferred anomaly definitions; it is that the evaluation injects exactly the pattern the model is designed to detect, so Table 2 cannot support the stated SOTA claim. The problems are load-bearing and would require new experiments with non-circular anomaly labels or injection schemes, additional heterogeneous-graph baselines, repeated runs with error bars, and a defined timing protocol. These changes go beyond a minor revision, though the underlying idea could potentially be salvageable in a future submission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take on EAGLE: the method is a sensible combination of old pieces, and the paper is clearly written, but the experimental validation is too weak to support the headline claim.\n\nWhat's new: the specific pairing of meta-path-based positive/negative instance sampling with a graph autoencoder and a contrastive discriminator for heterogeneous graph anomaly detection. Each component is established—CoLA [16] does contrastive anomaly detection on attributed networks, DOMINANT/AnomalyDAE use autoencoder reconstruction, meta-path sampling is standard in heterogeneous GNNs—but this particular assembly for this task hasn't been published, as far as I know. The model itself is coherent: Eq. (11) combines reconstruction error with the discriminator's [s- - s+] term in a way that follows from their assumption. The complexity analysis is a nice touch, though it's a high-level bound.\n\nWhere it gets shaky is the evaluation. The anomalies are injected by replacing a target node's attribute embedding with the farthest of k=50 random nodes (k=50 is fixed for all datasets). That's exactly the \"local-context deviation\" assumption EAGLE is built on. So the test set is constructed to satisfy the method's core premise, and the reported gains over baselines (28.4% average) only demonstrate that EAGLE can detect attribute substitution, not that it works on real heterogeneous-graph anomalies. That's not a fatal flaw by itself—many papers in this area use synthetic anomalies—but it severely limits what the numbers mean.\n\nThe baseline set also misses heterogeneous-graph anomaly detectors; HeGAN and DGI are representation learning methods, not anomaly detectors, and FRAUDRE is focused on fraud. A comparison against a metapath-based or heterogeneous GNN anomaly detector (e.g., an HAN-based reconstruction method) would be more informative. No error bars or multiple runs are reported, which is a problem for a deep model with as many hyperparameters as this one. And the timing comparison is confusing: \"EAGLE Pre\" is claimed to be without pre-training, yet it takes longer than the pre-trained EAGLE (0.165s vs 0.098s on DBLP). If pre-training is counted, it should be slower; if only fine-tuning is measured, that needs to be stated. No code or data is provided either, so the numbers can't be checked.\n\nBottom line: the idea is publishable in principle, but the current evidence doesn't establish SOTA. The paper needs real or at least more varied anomaly labels, a fairer baseline suite, error bars, and transparent timing. As it stands, I'd reject it in current form but invite a major revision. It deserves peer review because the concept is interesting and the flaws are fixable.","headline":"EAGLE combines familiar components sensibly, but the evaluation is circular: synthetic anomalies match its own local-context assumption, so the SOTA claim isn't established.","tokens_in":10666,"tokens_out":3705,"would_cite":false,"duration_ms":30795,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A self-supervised graph model that contrasts each node with its local meta-path context can rank anomalies in heterogeneous networks without labels and, on three benchmarks, does so in under half a second.","keywords":["graph anomaly detection","heterogeneous graphs","contrastive learning","meta-path","graph autoencoder","unsupervised learning","node embeddings","efficiency"],"falsifier":"Inject anomalies by replacing a target node's attribute vector with that of a nearby node rather than the most distant of 50 candidates, and measure whether EAGLE's AUC drops; alternatively, apply the model to a heterogeneous network with verified real-world anomalies and check whether anomalous nodes actually show $|s^{-}-s^{+}|$ near zero while normal nodes show it near one. If the gap distributions overlap, the central claim fails.","tokens_in":1532,"feed_emoji":"🕸️","tokens_out":5097,"duration_ms":127000,"temperature":0.7,"pith_summary":"This paper sets out to show that anomaly detection on heterogeneous graphs can be both accurate and cheap when the detector is built around one idea: compare every node with the meta paths that make up its local context. EAGLE does this without labels by sampling positive and negative meta-path instance pairs, learning node embeddings with a graph autoencoder, and then scoring nodes from reconstruction errors plus a discriminator's confidence gap. The reported payoff is that on three benchmark heterogeneous networks the method reaches AUC values of 0.9502, 0.9592, and 0.9826, with running times below 0.5 seconds, outperforming five existing methods. Embedded and large-scale monitoring systems would gain a practical way to flag abnormal nodes with little compute and no labeled data.","feed_headline":"Graph anomalies detected in under a second, no labels needed","feed_subtitle":"A self-supervised autoencoder scores nodes by local-context distance and tops five baselines on three networks.","key_machinery":"The load-bearing machinery is meta-path-level instance pair sampling combined with a graph autoencoder and a bilinear discriminator. For each target node $u$, the positive set $\\mathcal{P}^{mepa}_u$ collects meta paths containing $u$, while the negative set $\\mathcal{N}_u$ collects local meta paths that exclude $u$; the discriminator scores each pair by $\\sigma(h_{tn_i}^T W h_i)$. The autoencoder uses two graph convolutional encoders, one per relation type, so the embeddings carry both attribute and structural information. The anomaly score is $f(v_i)=\\alpha\\lVert a-\\hat a\\rVert^2 + \\beta\\lVert x-\\hat x\\rVert^2 + \\gamma(s_i^{-}-s_i^{+})$, fusing reconstruction error and contrastive discrimination into a single ranking criterion.","core_discovery":"The paper's central idea is that a meta path—a sequence of node types tracing a high-order relation such as Author-Paper-Author—defines the local context of a node, and an anomalous node can be detected by contrasting it with that context. EAGLE builds positive instance pairs from meta paths that contain the target node and negative pairs from meta paths that touch its direct neighbors but exclude it, then trains a graph autoencoder whose discriminator must tell the two kinds of pairs apart. For a normal node the discriminator should output near 1 for positive pairs and near 0 for negative pairs, while for an anomalous node both scores hover near 0.5, so the signed gap $s^{-}-s^{+}$ is strongly negative for normal nodes and near zero for anomalies. The final anomaly score adds that gap to attribute and structure reconstruction errors, and the paper reports that ranking nodes by this score yields AUC values of 0.9502, 0.9592, and 0.9826 on the three benchmark networks, with running times below 0.5 seconds.","pith_inferences":["Because the anomaly score hinges on the distance of a node from its local meta-path context, EAGLE may also flag structurally ambiguous nodes such as community-boundary hubs, not only genuinely malicious behavior; separating 'outlier' from 'fraud' would need extra validation.","The pre-training stage learns a task-agnostic heterogeneous-graph encoder, so the same embeddings could plausibly be transferred to other downstream tasks on the same graph, such as node classification or link prediction.","The evaluation injects anomalies by swapping a node's attributes with the most distant of 50 random candidates, which guarantees the distance-from-context pattern the method assumes; testing on real labeled anomalies, or on anomalies injected to stay close to their neighbors, would show whether the reported advantage persists."],"forward_implications":["Anomaly detection on heterogeneous graphs can run without any labeled anomalies: the contrastive signal from meta-path pairs plus reconstruction error is sufficient to rank nodes by abnormality.","The discriminator's score gap $s^{-}-s^{+}$ is a complementary signal to reconstruction error, so the final score fuses structural, attribute, and contrastive evidence in one formula.","Pre-training on part of the graph and fine-tuning on the rest improves both AUC and runtime relative to training from scratch, supporting deployment when labels or compute are scarce.","The reported time complexity $O(edFP)$ and sub-second runtimes on three datasets suggest the method fits the resource budget of embedded devices and large networks."],"supporting_citations":[{"why":"Supplies the reconstruction-error basis for anomaly scoring and is the main autoencoder baseline EAGLE must beat.","marker":"[11]"},{"why":"Dual-autoencoder baseline on attributed networks that EAGLE compares against.","marker":"[12]"},{"why":"Provides the bilinear discriminator design and the contrastive self-supervised anomaly detection scheme that EAGLE adapts.","marker":"[16]"},{"why":"Contrastive pre-training on heterogeneous graphs at relation and metagraph level, the precedent for meta-path-level instance sampling.","marker":"[14]"},{"why":"Unsupervised graph representation baseline that EAGLE outperforms.","marker":"[19]"},{"why":"Adversarial heterogeneous network embedding baseline used for comparison.","marker":"[18]"},{"why":"GNN-based fraud detection baseline used for comparison.","marker":"[9]"}],"fun_headline_variants":["EAGLE detects graph anomalies in 0.5s, no labels","Graph anomalies caught in a flash without any labels","Contrastive learning speeds graph anomaly detection to 0.5s","No labels? EAGLE still finds graph anomalies in under a second"],"cache_read_input_tokens":12800,"weakest_assumption_plain":"The load-bearing premise is that abnormal nodes sit measurably farther from their local graph context than normal nodes do; if a real anomaly blends into its neighborhood, the contrastive scores will not separate it.","fun_headline_variants_meta":{"raw":{"variants":["EAGLE detects graph anomalies in 0.5s, no labels","Graph anomalies caught in a flash without any labels","Contrastive learning speeds graph anomaly detection to 0.5s","No labels? EAGLE still finds graph anomalies in under a second"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000821,"raw_usage":{"total_tokens":3575,"prompt_tokens":907,"completion_tokens":2668,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":523,"completion_tokens_details":{"reasoning_tokens":2593}},"tokens_in":523,"tokens_out":2668,"duration_ms":22964,"temperature":1.0,"reasoning_tokens":2593,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:15:02.546382+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inject anomalies by replacing a target node's attribute vector with that of a nearby node rather than the most distant of 50 candidates, and measure whether EAGLE's AUC drops; alternatively, apply the model to a heterogeneous network with verified real-world anomalies and check whether anomalous nodes actually show $|s^{-}-s^{+}|$ near zero while normal nodes show it near one. If the gap distributions overlap, the central claim fails.","supporting_citations":[{"cited_title":"Adversarial learning on heterogeneous information networks,","cited_arxiv_id":null,"evidence_quote":"Adversarial heterogeneous network embedding baseline used for comparison."},{"cited_title":"Fraudre: fraud detection dual- resistant to graph inconsistency and imbalance,","cited_arxiv_id":null,"evidence_quote":"GNN-based fraud detection baseline used for comparison."},{"cited_title":"Deep anomaly detection on attributed networks,","cited_arxiv_id":null,"evidence_quote":"Supplies the reconstruction-error basis for anomaly scoring and is the main autoencoder baseline EAGLE must beat."},{"cited_title":"Anomalydae: Dual autoen- coder for anomaly detection on attributed networks,","cited_arxiv_id":null,"evidence_quote":"Dual-autoencoder baseline on attributed networks that EAGLE compares against."},{"cited_title":"Anomaly detection on attributed networks via con- trastive self-supervised learning,","cited_arxiv_id":null,"evidence_quote":"Provides the bilinear discriminator design and the contrastive self-supervised anomaly detection scheme that EAGLE adapts."},{"cited_title":"Contrastive pre- training of gnns on heterogeneous graphs,","cited_arxiv_id":null,"evidence_quote":"Contrastive pre-training on heterogeneous graphs at relation and metagraph level, the precedent for meta-path-level instance sampling."},{"cited_title":"Deep graph infomax","cited_arxiv_id":null,"evidence_quote":"Unsupervised graph representation baseline that EAGLE outperforms."}],"review_version":1}