{"id":"9708f893-5060-4d7b-8a2a-62c616830f5c","arxiv_id":"2501.07078","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"A dual-channel training method with KL consistency loss is proposed for knowledge graph anomaly detection, but reported state-of-the-art claims are contradicted by the paper's own comparisons.","lead":"This paper presents ADKGD, a method for detecting wrong facts in knowledge graphs by training two parallel views of the data, one focused on entities and one on triplets, and forcing them to agree via a KL-style loss. It reports gains over several graph-based detectors, but its own tables show text-based detectors are still more accurate, undercutting the headline claim.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Abstract claims ADKGD 'outperforms state-of-the-art anomaly detection algorithms,' but Table 2 shows CCA and SeSICL beat ADKGD on most metrics; the paper admits this in §5.3, so the central claim is not supported as stated.","rationale":"The reader's stated weakest_assumption concerns whether the injected noisy triplets are generated the same way as the training negatives, which would align the test set with the training objective. That is a real and important methodological concern, but it is secondary to a more direct problem: the paper's own Table 2 contradicts the abstract's unqualified claim that ADKGD outperforms state-of-the-art anomaly detection algorithms. SeSICL and CCA, which the paper itself identifies as state-of-the-art text-based methods, beat ADKGD on most reported metrics, and the paper concedes this in §5.3. The strongest claim therefore cannot be accepted as written. The reader's rationale does mention this contradiction, which is why I mark agreement as partial rather than full: the nominated weakest_assumption is not the same as the load-bearing concern, but the overall rejection rationale overlaps. The other issues noted by the reader, such as hyperparameter tuning without a held-out validation set, absence of error bars, and the time-efficiency inconsistency, reinforce the rejection but are not needed to establish that the central claim is unsupported. The verdict of REJECT remains appropriate; the paper could become acceptable only with a substantially revised, scoped claim and a proper evaluation protocol.","tokens_in":39488,"tokens_out":3394,"duration_ms":33429,"concrete_test":"Parse Table 2 and count, for each of the six datasets and each of the fifteen metric cells (Precision@K and Recall@K for K=1% to 5%), whether ADKGD exceeds or falls below CCA and SeSICL. If ADKGD wins fewer than half of the thirty comparisons against either baseline, the unqualified abstract claim is unsupported. As a stronger check, run the released code for one dataset, e.g., WN18RR at the 5% anomaly ratio, and verify that the ADKGD, CCA, and SeSICL rows reproduce the values in Table 2; if they do, no amount of hyperparameter tuning changes the conclusion that the headline claim needs to be narrowed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is empirical superiority, and it fails on the paper's own reported numbers. Section 5.3's Table 2 includes SeSICL and CCA, described as state-of-the-art text-based error detection methods. Across the six datasets, in nearly every Precision@K and Recall@K cell, CCA and SeSICL exceed ADKGD. For example, on WN18RR at K=5%, ADKGD Precision is 0.560 versus CCA's 0.660 and SeSICL's 0.612; on FB15K at K=5%, ADKGD is 0.659 versus CCA's 0.709 and SeSICL's 0.696. Recall follows the same pattern. The paper's own §5.3 states that ADKGD 'still falls short compared to text-based methods such as SeSICL and CCA.' Therefore the abstract's unqualified 'outperforms the state-of-the-art' is internally contradicted, not merely challenged by external consensus. If the intended claim is 'outperforms graph-structure-only KG anomaly detection baselines,' that may be credible, since ADKGD beats CAGED and other graph-only methods on the reported numbers, but that is not what the abstract says. This is the load-bearing concern because the paper's contribution is evaluated by that headline; a scope correction or claim revision is required.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ADKGD, a dual-channel anomaly detection method for knowledge graphs that combines entity-view and triplet-view representations via BI-LSTM encoders, neighbor aggregation, cross-layer learning, and a KL-divergence consistency loss. The method is evaluated on WN18RR, FB15K, NELL-995, and three additional datasets with injected noisy triplets at 5%, 10%, and 15% ratios, and compared against KG-embedding baselines, graph-based anomaly detection methods (CKRL, KGTtm, KGIst, CAGED), and text-based methods (SeSICL, CCA). The abstract and introduction claim that ADKGD outperforms state-of-the-art anomaly detection algorithms; the paper also reports an ablation study, hyperparameter analysis, and time-efficiency experiments.","tokens_in":39770,"tokens_out":3239,"duration_ms":32384,"significance":"If the headline claim were supported, ADKGD would be a meaningful contribution to KG error detection and to improving the reliability of KG-augmented LLM pipelines. The method itself is a reasonable extension of CAGED that introduces a dual-channel architecture and a KL-based consistency loss, and the paper includes an ablation study, hyperparameter analysis, and complexity analysis that partially help assess the design. However, the paper's own reported numbers contradict the central claim: in Table 2, the text-based baselines SeSICL and CCA achieve higher Precision@K and Recall@K than ADKGD on virtually every dataset and K value, and the text in §5.3 explicitly admits that ADKGD 'still falls short compared to text-based methods such as SeSICL and CCA.' In addition, the experimental protocol has circularity issues: hyperparameters are tuned by grid search on the test datasets, and the injected anomalies are likely generated by the same random head/tail corruption used to create training negatives. These issues undermine the validity of the reported performance as an independent measure of generalization.","major_comments":[{"comment":"The central claim that ADKGD 'outperforms the state-of-the-art anomaly detection algorithms' is contradicted by the paper's own results. In Table 2, SeSICL and CCA achieve higher Precision@K and Recall@K than ADKGD on all three main datasets (FB15K, WN18RR, NELL-995-h25) at nearly every K; for example, at K=5% on FB15K, ADKGD Precision@K is 0.659 while CCA attains 0.709, and on WN18RR ADKGD attains 0.560 versus CCA's 0.660. Section 5.3 itself states that ADKGD 'still falls short compared to text-based methods such as SeSICL and CCA.' The abstract and introduction therefore overstate the findings. If the intended claim is that ADKGD outperforms graph-structure-only KG anomaly detection baselines, that claim is plausible from the table, but it is not what the paper states.","section":"Abstract and §5.3, Table 2"},{"comment":"The experimental setup performs grid search for the hyperparameters α, β, and γ on the test datasets (WN18RR, FB15K, NELL-995) and then reports the best Precision@K and Recall@K values on those same datasets. Selecting hyperparameters on the test set means the reported metrics are fitted values, not independent predictions, and the comparison against baselines is not a fair evaluation of generalization. The paper needs a separate validation split or nested cross-validation, or it must clearly state that the reported numbers are oracle-tuned and thus not directly comparable to baseline numbers obtained under their own tuning protocols.","section":"§5.2, Hyper-parameter analysis (§5.5)"},{"comment":"The paper does not specify how the injected noisy triplets used as ground truth are generated, but §4.1 describes creating negative samples by replacing the head or tail entity with a random entity. If the noisy triplets are generated by the same random corruption process, then the test distribution exactly matches the training negative distribution by construction, so the reported detection performance does not measure generalization to real-world error patterns. The authors should state the anomaly injection mechanism explicitly and, if it matches the negative-sampling procedure, discuss why the evaluation is not circular.","section":"§4.1 and §5.1"},{"comment":"The KL divergence is applied to quantities that are not probability distributions: L_KL, score = KL(f_entity || f_triplet), where f_entity and f_triplet are raw score vectors. KL divergence is defined for normalized probability distributions; using it on arbitrary non-negative scores is not justified. In addition, the scoring convention is inconsistent: Definition 3.4 defines the score via φ(e_h,e_r,e_t) = -||e_h+e_r-e_t||_2 and states that higher values indicate anomalies, yet Eq. (28) and Eq. (32) use the positive norm ||h+r-t||_2 as the score, meaning higher values now indicate anomalies. The signs and definitions of the loss and scoring functions need to be reconciled.","section":"Definition 3.5, Eq. (20), Definition 3.4, Eq. (28)-(32)"}],"minor_comments":[{"comment":"In the ablation table, the row labeled 'Entity-view w/o internal learning' is duplicated; the second occurrence should presumably be for the triplet-view component, but as printed it repeats the entity-view configuration.","section":"Table 4"},{"comment":"The dataset name is given as FB15K-237 in §5.1 and Table 1, but as FB15K in Table 2 and the surrounding text; the notation should be consistent. Also, the baseline name appears as both 'KGttm' (Table 2) and 'KGTtm' (text and references); please unify.","section":"Table 2 and §5.1"},{"comment":"The hyperparameter analysis reports that γ=0.5 is optimal across datasets and anomaly ratios, but the text in §5.5 says 'adjusting γ to 0.5 yields the best results,' without a clear connection to the earlier claim that γ ranges from 0 to 1; please clarify how γ relates to the margin in Eq. (28), which is described in §5.2 as ranging from 0 to 1.","section":"§5.5, Figure 7"},{"comment":"The claim that precision improves with anomaly ratio is partly mechanical because Precision@K measures the fraction of anomalies in the top-K, and a higher anomaly ratio increases the expected precision even for a random ranking; the paper should discuss this baseline effect.","section":"§5.3, Table 3"}],"recommendation":"reject","confidential_remarks":"The paper's central empirical claim is not supported by its own Table 2, and the evaluation protocol has circularity in both hyperparameter tuning on the test set and the likely generation of injected anomalies via the same corruption process used for negative sampling. The method may have value as an incremental extension of CAGED with dual-channel learning, but the current presentation does not establish a sound contribution. The authors would need to correct the headline claim, redo the evaluation with a proper validation split and independent anomaly generation, and address the KL-divergence technical issue before the paper could be considered for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing you should know up front is that the paper's headline claim does not survive its own results. The abstract says ADKGD outperforms state-of-the-art anomaly detection algorithms, but Table 2 shows SeSICL and CCA beating ADKGD on nearly every Precision@K and Recall@K cell across FB15K, WN18RR, and NELL-995. The text in §5.3 admits this, so the contradiction is internal, not a matter of external consensus. A re-scoped claim like \"outperforms graph-structure-only KG anomaly detection methods\" would be credible, because ADKGD does beat CAGED and the other graph-only baselines on the reported numbers. As written, the abstract overclaims.\n\nWhat is genuinely new: the entity-view channel is CAGED's approach, but the triplet-view channel (BI-LSTM-D with dimensionality reduction) plus the KL consistency loss across channels is a legitimate extension, and the ablation study shows each component pulling its weight. The paper also ships code and data, which is real evidence and makes the work reproducible. The comparison against six datasets, including three additional ones for ADKGD vs. CAGED, is more than many papers in this area bother with.\n\nThe soft spots are mostly around evaluation hygiene. Hyperparameters (alpha, beta, gamma) were chosen by grid search on the test datasets without a validation split, so the final numbers are partly fitted values. There are no error bars despite averaging over ten runs. The time-efficiency section contains an internal contradiction: the text says ADKGD takes more time than CAGED, but the example reports CAGED at 14.33 minutes per epoch versus ADKGD at 12.83 minutes. Also, the injected anomalies are generated by the same random head/tail corruption used to create training negatives, so the test set is aligned with the training objective by construction; the paper does not address how the method would transfer to real-world error patterns. These are fixable, but they are not nitpicks — they collectively weaken the empirical claim.\n\nWho gets value from this: researchers working on graph-structure-only KG error detection who want a baseline with public code and a clean ablation. The citation pattern looks fine; the self-citations in the LLM section are contextual and not used to prop up the method.\n\nMy recommendation: send it to peer review, but with a clear expectation of major revision. The claim needs to be narrowed, the evaluation needs a validation split and variance reporting, and the time-efficiency contradiction needs fixing. If those are addressed, it could be a solid incremental contribution.","headline":"The dual-channel extension over CAGED is real and the ablations are careful, but the 'outperforms SOTA' claim is contradicted by the paper's own Table 2, so the paper needs a claim revision and a proper validation split before it is publishable.","tokens_in":40319,"tokens_out":1746,"would_cite":false,"duration_ms":20398,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A dual-channel scoring model ranks erroneous knowledge-graph triplets above correct ones more sharply than prior detectors.","keywords":["knowledge graph anomaly detection","dual-channel learning","entity-view and triplet-view","cross-layer aggregation","KL divergence consistency loss","negative sampling","knowledge graph error detection"],"falsifier":"Run ADKGD on a benchmark of naturally occurring KG errors, such as human-annotated wrong statements in Wikidata or YAGO, where corruptions are not produced by uniform random head or tail replacement. If Precision@K on the top-K scores falls to near the base rate or drops below rule-based or text-based detectors while remaining high on random head/tail corruption, the paper's claim of general state-of-the-art anomaly detection would be contradicted.","tokens_in":39239,"feed_emoji":"🕸️","tokens_out":6276,"duration_ms":49465,"temperature":0.7,"pith_summary":"The paper tries to establish that a dual-channel anomaly detector, ADKGD, outperforms existing knowledge-graph anomaly detection methods on benchmark graphs. The motivation is that LLMs increasingly rely on external knowledge graphs, and errors in those graphs degrade downstream performance, so reliable triplet-level error detection matters. ADKGD learns representations from an entity-view and a triplet-view, aggregates neighbor context at two layers, and aligns both views with a KL-divergence consistency loss. Experiments on WN18RR, FB15K, and NELL-995 report higher Precision@K and Recall@K than embedding baselines and the CAGED, CKRL, KGTtm, and KGIst detectors at 5% injected noise.","feed_headline":"Dual-channel model beats KG anomaly detection baselines","feed_subtitle":"Entity and triplet views, aligned by a KL loss, caught injected noise better on three benchmark graphs.","key_machinery":"The central mechanism is a dual-channel cross-layer encoder. Channel I, the entity-view, applies a BI-LSTM that preserves input dimensions and concatenates the outputs into $q_i = [\\tilde{e}_h; \\tilde{e}_r; \\tilde{e}_t]$; Channel II, the triplet-view, applies a dimension-reducing BI-LSTM-D whose last hidden state $q'_i = \\tilde{e}_{h,r,t}$ represents the entire triplet. Both channels aggregate over head and tail neighbor triplets using softmax-weighted dot-product similarities to produce $z_1,z_2$ and $z_3,z_4$. A KL-divergence consistency loss $\\mathcal{L}_{\\mathrm{KL,score}}+\\mathcal{L}_{\\mathrm{KL,head}}+\\mathcal{L}_{\\mathrm{KL,tail}}$ aligns the channel scores, and a margin loss $\\max(0, \\mathrm{loss}_n - \\mathrm{loss}_p + \\gamma)$ separates positive from negative triplets, where negatives are generated by random head or tail replacement. At inference the model ranks triplets by the weighted sum above, without needing negatives or the KL term.","core_discovery":"On the paper's own terms, the discovery is that viewing each triplet simultaneously as an entity sequence and as a compressed triplet-level sequence, while tying the two score distributions together with a KL-divergence consistency loss, produces a scoring function that separates injected noisy triplets from correct ones more sharply than a single-view model or a two-view model without alignment. The entity-view BI-LSTM preserves per-entity structure in $q_i$, while the dimension-reducing BI-LSTM-D summarizes the whole triplet in $q'_i$, and both channels aggregate head and tail neighbor triplets through softmax-weighted similarities. The KL loss enforces agreement between the channels' scores on the triplet itself and on its neighbor aggregations, and a margin loss then ranks positive triplets below their random-corruption negatives. The final anomaly score is $\\alpha(f_{\\mathrm{BI-LSTM}}+f_{\\mathrm{BI-LSTM-D}})+(1-\\alpha)\\frac{1}{2}(\\mathrm{sim}(z_1,z_2)+\\mathrm{sim}(z_3,z_4))$.","pith_inferences":["Because the injected test anomalies are generated by the same random head/tail replacement that builds training negatives, the reported numbers most plausibly measure ranking within that single corruption class; an unaligned test set with semantically plausible corruptions (same entity type or relation) would be a stricter generalization check.","The two views are likely encoding different inductive biases, entity-local structure versus whole-triplet semantics, so a probing study of which view drives detection on low-degree versus high-degree entities would clarify the source of the gain.","Following the paper's own future-work suggestion, a hybrid that feeds textual embeddings into the neighbor-aggregation step could recover the recall gap at higher K values; this is an editorial projection, not a result the paper claims."],"forward_implications":["If the claim holds, ADKGD is the best structure-only scorer among the compared methods for ranking erroneous triplets under the 5%-noise benchmark protocol.","The KL consistency loss is a direct source of gain: ablations on WN18RR show the full two-view model with KL surpasses the entity-view alone and the two-view model without KL by about 14% precision and 16% recall on average.","Precision at K improves as the injected noise ratio rises from 5% to 15%, suggesting the method is robust to increasingly corrupted graphs, while recall at K declines.","The structure-only approach trails text-based methods (SeSICL, CCA) on recall at larger K on some datasets, so the claimed superiority is specifically over graph-structure baselines in that regime.","Training cost is dominated by $O(|T|\\,n^2)$, so with the embedding dimension fixed at 100 the framework remains feasible on the tested graphs; scalability to very large KGs is left open."],"supporting_citations":[{"why":"CAGED is the cross-layer framework and entity-view baseline that ADKGD extends with dual-channel training and the KL consistency loss.","marker":"[45]"},{"why":"TransE supplies the distance-based scoring function, negative-sampling scheme, and a key embedding baseline.","marker":"[4]"},{"why":"CKRL is a state-of-the-art KG anomaly detection baseline that ADKGD must beat in experiments.","marker":"[31]"},{"why":"KGTtm is a state-of-the-art KG anomaly detection baseline used in the comparison.","marker":"[15]"},{"why":"KGIst is a state-of-the-art KG anomaly detection baseline used in the comparison.","marker":"[3]"},{"why":"WN18RR is one of the three primary real-world datasets on which ADKGD is evaluated.","marker":"[6]"},{"why":"FB15K-237 is one of the three primary real-world datasets on which ADKGD is evaluated.","marker":"[34]"},{"why":"NELL-995 is one of the three primary real-world datasets on which ADKGD is evaluated.","marker":"[2]"}],"fun_headline_variants":["KL-aligned dual views sharpen KG anomaly scores","Entity and triplet views aligned by KL loss catch KG noise","Dual-channel plus KL loss improves KG anomaly detection","ADKGD: dual-encoder KG anomaly detector outshines baselines","KL-loss alignment boosts dual-view KG anomaly detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that the noisy triplets used as ground truth are representative of the random head/tail corruption used to create training negatives, so the reported detection performance does not measure generalization to real-world error patterns if those patterns differ.","fun_headline_variants_meta":{"raw":{"variants":["KL-aligned dual views sharpen KG anomaly scores","Entity and triplet views aligned by KL loss catch KG noise","Dual-channel plus KL loss improves KG anomaly detection","ADKGD: dual-encoder KG anomaly detector outshines baselines","KL-loss alignment boosts dual-view KG anomaly detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001282,"raw_usage":{"total_tokens":5292,"prompt_tokens":1053,"completion_tokens":4239,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":669,"completion_tokens_details":{"reasoning_tokens":4159}},"tokens_in":669,"tokens_out":4239,"duration_ms":30136,"temperature":1.0,"reasoning_tokens":4159,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:49:07.268368+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run ADKGD on a benchmark of naturally occurring KG errors, such as human-annotated wrong statements in Wikidata or YAGO, where corruptions are not produced by uniform random head or tail replacement. If Precision@K on the top-K scores falls to near the base rate or drops below rule-based or text-based detectors while remaining high on random head/tail corruption, the paper's claim of general state-of-the-art anomaly detection would be contradicted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CAGED is the cross-layer framework and entity-view baseline that ADKGD extends with dual-channel training and the KL consistency loss."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"TransE supplies the distance-based scoring function, negative-sampling scheme, and a key embedding baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CKRL is a state-of-the-art KG anomaly detection baseline that ADKGD must beat in experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"KGTtm is a state-of-the-art KG anomaly detection baseline used in the comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"KGIst is a state-of-the-art KG anomaly detection baseline used in the comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"WN18RR is one of the three primary real-world datasets on which ADKGD is evaluated."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"FB15K-237 is one of the three primary real-world datasets on which ADKGD is evaluated."}],"review_version":1}