Pith. sign in

REVIEW 4 major objections 5 minor 43 references

Debate-on-Graph: Reliable and Adaptive Reasoning of Large Language Model on Uncertain Knowledge Graph

T0 review · 4 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Uncertain knowledge graph confidence scores, paired with an information-asymmetric agent debate, let large language models answer questions more reliably and reach state-of-the-art accuracy on four QA benchmarks.

desk verdict DoG cleanly combines confidence-weighted graph search with multi-agent debate, but the paper's own DoG-E ablation shows confidence labels are nearly irrelevant, undercutting the central UKG claim. read the letter →

arxiv 2607.17266 v1 pith:NGUDZ7LH submitted 2026-07-19 cs.CL cs.AI

Peiji Yu , Xin Chen , Tianxing Wu This is my paper
classification cs.CLcs.AI
keywords uncertainknowledgegraphlargelanguagemodelquestionansweringknowledge-graph-enhancedreasoningmulti-agentdebateconfidence-awareretrievaladaptivehallucinationmitigation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the confidence scores carried by uncertain knowledge graphs can be turned into a reliable reasoning signal for large language models. It proposes Debate-on-Graph, a two-step framework that first extracts a question-relevant, confidence-weighted subgraph using a best-first heuristic search, then runs an information-asymmetric multi-agent debate to answer the question. On four QA benchmarks spanning commonsense and medical reasoning, it reports accuracy above existing LLM reasoning methods and knowledge-graph baselines, with its largest gains on medical QA. The paper's own limitations section notes that the benchmarks are not closely aligned with the underlying uncertain graphs, so the headline gains may reflect the debate mechanism more than the graph's uncertainty labels.

What carries the argument

The reasoning-path score Score(p, Q) = SR(p, Q) * (Conf(p))^α, where SR is a semantic similarity between path and question and Conf(p) is the product of triple confidences along the path. This score drives the priority-queue expansion, the top-W selective pruning, and the dynamic threshold filtering in subgraph extraction. The other load-bearing piece is the Multi-Agent Debate with four agents and an information-asymmetric setting, which adaptively decides whether to expand the subgraph (by increasing the retain-top-K parameter) or terminate. Together these mechanisms operationalize the trade-off between knowledge utility and evidence reliability.

What would settle it

Construct a QA dataset where each question is automatically generated from a ground-truth subgraph of an uncertain knowledge graph, guaranteeing that the correct answer is reachable via high-confidence paths. Then run DoG against a variant that ignores confidence scores (all set to 1) and against strong LLM-only baselines. If the full method does not clearly outperform both, the confidence-aware extraction is not doing the claimed work.

Watch

Extended reading notes

Core claim

The central claim is that jointly optimizing for semantic relevance and path confidence during subgraph retrieval, followed by an adversarial debate in which only the proponent sees the retrieved evidence, yields answers that are more accurate and less hallucinated than either LLM-only reasoning or graph-augmented baselines. The paper defines a reasoning-path score as the product of a semantic-similarity term and a path-confidence term, and uses this score to prune low-quality paths dynamically. It then lets a decision-making agent expand the subgraph on demand, so the evidence scope adapts to each question. If the claim holds, the confidence labels that uncertain knowledge graphs already ca

Load-bearing premise

That the four QA datasets are answerable from the associated uncertain knowledge graphs, so that the confidence scores and retrieved paths actually carry the information needed to answer the question; the paper itself concedes these datasets are not highly relevant to the graphs, which weakens the link between method and measured accuracy.

Editorial extensions

If this is right

  • DoG achieves state-of-the-art accuracy on all four tested benchmarks, with the largest gains on medical QA (about 10 percentage points over the second-best baseline).
  • The framework is plug-and-play: it requires no additional training of the language model and works with both open and proprietary LLMs.
  • Adaptive subgraph expansion makes the method faster in runtime than prior agent-based knowledge-graph reasoning methods, while still using many LLM calls.
  • Ablations show that removing the debate mechanism causes a substantial performance drop, indicating that the debate is central to filtering noise.
  • When confidence scores are flattened to 1.0 (a variant called DoG-E), performance remains strong though slightly lower, so the method does not strictly depend on the confidence labels.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A dedicated QA benchmark that is actually generated from an uncertain knowledge graph is needed to separate the contribution of confidence-aware retrieval from the debate mechanism; the paper itself concedes the current benchmarks are poorly aligned.
  • The same confidence-weighted path scoring could be applied to deterministic knowledge graphs by estimating a pseudo-confidence (from embedding scores, frequency, or rule support), widening the method's applicability.
  • The information-asymmetric debate is a general retrieval-augmentation technique that could be transferred to unstructured document retrieval, not just graph-structured knowledge.
  • If future experiments show that confidence scores rarely alter the extracted subgraph, the measured gains on these datasets would mostly be attributable to the debate module rather than to uncertainty modelling.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Debate-on-Graph (DoG), a training-free framework for question answering over uncertain knowledge graphs (UKGs). DoG first extracts a question-relevant, confidence-weighted subgraph via a best-first heuristic search that scores reasoning paths by semantic relevance times path confidence (Eq. 5), then runs a multi-agent debate (proponent, challenger, decision-maker, judge) with information asymmetry and adaptive subgraph expansion. The authors claim state-of-the-art accuracy on CSQA, CSQA 2.0, OBQA, and MedQA with Qwen3-8B and GPT-4o-mini, and report ablations showing that both the subgraph extractor and the adaptive debate contribute to performance. They also analyze hyperparameters, adaptive behavior, and efficiency.

Significance. If the central claim is correct, DoG is a meaningful contribution to KG-enhanced LLM reasoning: it is model-agnostic, requires no training, and explicitly targets noise in uncertain knowledge graphs—an underexplored problem. The framework is clearly described, and the public code release is a strength. The adaptive-K mechanism is a sensible way to balance evidence sufficiency against reliability, and the information-asymmetric debate design is a reasonable antidote to over-reliance on retrieved context. However, the paper's own DoG-E ablation and its stated benchmark-relevance limitation substantially weaken the evidence for the load-bearing role of UKG confidence scores. The headline 'state-of-the-art' claim is not yet supported by the reported evaluation, which lacks error bars, significance tests, and a held-out validation protocol for hyperparameters.

major comments (4)
  1. [§5.2, Table 2] The core novelty—using UKG confidence scores to filter noisy triples—is not shown to be load-bearing. DoG-E, which sets all confidences to 1.0, is within 0–3.5 accuracy points of DoG on every dataset/backbone pair (e.g., CSQA 2.0 Qwen: 77.4 vs 77.4; MedQA Qwen: 79.0 vs 80.0; OBQA GPT: 93.6 vs 96.2). With only 200–500 test items and no error bars or significance tests, these gaps are plausibly sampling noise. The paper even states 'DoG does not heavily depend on the confidence labels within UKGs.' This undercuts the abstract's claim that confidence-aware extraction is what enables reliable reasoning. Please report confidence intervals or paired significance tests (e.g., McNemar), and, if possible, a variant that demonstrably fails when confidence is removed, or reposition the central claim to focus on subgraph retrieval plus debate rather than UKG confidence.
  2. [§5.1 and Fig. 4] Hyperparameters α, W, Dmax, and K (Θ={1,3,5}, rmax=3) appear to be selected directly on the same four benchmark test sets used for the headline numbers. Fig. 4 shows accuracy peaking at α=1.0 and W=3 across all datasets, which suggests tuning on the reported data. No held-out validation split or cross-validation procedure is described. Consequently, the reported numbers are optimistic and the 'state-of-the-art' claim is under-supported. Please add a validation protocol (e.g., a development set or nested CV) and report variance across multiple runs or seeds.
  3. [§6, first limitation] The manuscript explicitly concedes: 'The QA datasets utilized in our experiments are not highly relevant to the corresponding UKGs, which leads to limited performance gains for DoG on these datasets.' This is a serious threat to external validity for the UKGQA task: if the benchmark questions are not answerable from the associated UKGs, then the accuracy differences in Table 2 may reflect LLM/debate behavior rather than the proposed UKG reasoning capability. Please quantify the actual relevance (e.g., fraction of questions with at least one high-scoring extracted path that supports the correct answer), and either use benchmarks with tighter KG–question alignment or temper the SOTA claim accordingly.
  4. [§5.2/Table 2] The headline gains on MedQA (10.3% and 7.2% over the second-best baseline) are based on only 200 test questions and a single run. With that sample size, a 7–10 point difference may be meaningful if paired, but no statistical evidence is provided. Please report per-question standard errors, bootstrap confidence intervals, or McNemar tests, and consider averaging over multiple runs since the MAD agents are stochastic (temperature 0.5).
minor comments (5)
  1. [§2.2] Typo: 'agent-based paradigms are prune to errors' should be 'prone to errors'.
  2. [§5.6] Grammar: 'DoG are most efficient in terms of runtime' should be 'DoG is most efficient'.
  3. [Table 3] The ablation table is reported only for GPT-4o-mini, but the caption does not state this. Please clarify in the caption or main text.
  4. [§4.1/§4.2] The pseudo-code of subgraph extraction and the prompt templates are deferred to supplementary material. For reproducibility, at least the prompts and the exact algorithm steps should be included in an appendix in the submission version.
  5. [§5.4, Fig. 3] The distributions of K and debate rounds are presented as proportions, but the number of questions behind each bar is not given. Since MedQA has only 200 questions, small proportion differences may be unstable. Please report counts or confidence intervals.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DoG is an empirical pipeline whose components are explicit modeling choices rather than fitted predictions disguised as derivations.

full rationale

The paper does not contain a derivation chain that reduces to its inputs. The path scoring function (Eq. 5) is an explicitly stated modeling choice combining semantic relevance and confidence; it is not fitted to benchmark answers and then used to predict those same answers. Subgraph extraction is a heuristic best-first search over the UKG, and the multi-agent debate is an LLM prompting procedure; neither is derived from the benchmarks in a way that would make the final accuracies tautological. Hyperparameters (alpha, W, K, rmax) are tuned on the evaluation benchmarks, which is a standard empirical fitting concern rather than logical circularity. The paper's self-citations to the corresponding author's prior UKG work are confined to related work and background; they do not supply a load-bearing premise for DoG's reported gains. The DoG-E ablation showing near-parity when confidence scores are set to 1.0 is a substantive challenge to the causal role of UKG confidence labels, but that is a validity/correctness concern, not a circularity in the derivation. The paper's own limitation statement about dataset-UKG relevance further supports reading the reported numbers as empirical results rather than as consequences of the framework's definitions.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The method's operation rests on UKG confidence semantics and on evaluation validity. There are four hand-set hyperparameters, with alpha and W explicitly selected from benchmark performance curves. No new physical or ontological entities are postulated; the 'agents' are roles instantiated by the same LLM, not independent evidence sources.

free parameters (4)
  • alpha (trade-off factor) = 1.0
    Controls the confidence weight in Score(p,Q)=SR*Conf^alpha; selected because accuracy peaks at 1.0 across all datasets in Fig. 4a.
  • W (expansion width) = 3
    Top-W neighbor pruning parameter; selected because accuracy peaks at W=3 in Fig. 4b.
  • Dmax (max entity distance) = 3
    Maximum search distance; hand-set with no sensitivity analysis reported.
  • Theta = {1,3,5} and rmax=3 = K candidate set {1,3,5}, rmax=3
    Candidate subgraph expansion levels and maximum debate rounds; fixed by the authors without a described validation procedure.
assumptions (5)
  • domain assumption Path confidence is the product of triple confidences (Eq. 3), assuming the confidences are independent and honest probabilities.
    Definition 5 defines Conf(p)=product ci. No independence or calibration justification is given; if ConceptNet weights are not calibrated probabilities, ranking by this product is ungrounded.
  • domain assumption ConceptNet and MedKGent can be treated as uncertain knowledge graphs with meaningful confidence scores.
    Section 5.1 uses ConceptNet and MedKGent as UKGs, but no validation is provided that their scores satisfy Definition 1's 'probability that the triple is true' semantics.
  • domain assumption Topic entities can be correctly obtained via NER and entity linking.
    Section 4 states topic entities are obtained via NER and entity linking, but no error analysis of linking is provided; linking errors would break retrieval and the downstream claim.
  • domain assumption The four benchmark QA datasets are suitable UKGQA tests whose answers are reachable from the corresponding UKGs.
    This is directly challenged by the authors' own Section 6 limitation: the datasets 'are not highly relevant to the corresponding UKGs, which leads to limited performance gains.'
  • ad hoc to paper The heuristic score Score(p,Q)=SR(p,Q)*Conf(p)^alpha is a valid criterion for evidence quality.
    Defined in Eq. 5 and tuned via alpha; no theoretical or independent empirical calibration beyond downstream accuracy on the same benchmarks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Debate-on-Graph: Reliable and Adaptive Reasoning of Large Language Model on Uncertain Knowledge Graph." pith.science (2026). https://pith.science/paper/NGUDZ7LH

@misc{pith2026260717266,
  author       = {Pith},
  title        = {Pith review of: Debate-on-Graph: Reliable and Adaptive Reasoning of Large Language Model on Uncertain Knowledge Graph},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NGUDZ7LH}},
  note         = {Machine review of arXiv:2607.17266}
}
read the original abstract

Large language models (LLMs) have demonstrated remarkable capabilities in natural language processing. However, LLMs often suffer from hallucinations and lack of relevant knowledge when dealing with question answering (QA) tasks. To mitigate these issues, knowledge graphs (KGs) have been utilized to enhance LLM reasoning. Nevertheless, KGs often contain noise and errors, while existing KG-enhanced LLM approaches are generally unable to identify and filter such noisy and erroneous content, which can instead amplify hallucinations and pose challenges for reliable reasoning. Uncertain knowledge graphs (UKGs), which associate each triple with a confidence score to quantify uncertainty, offer a promising direction to address this challenge. Compared with prior work, we investigate how to leverage UKGs to support LLMs for QA. We propose Debate-on-Graph (DoG), a new framework that enables LLMs and UKGs to collaborate adaptively for reliable reasoning. Specifically, we first design a heuristic search algorithm tailored for UKGs to extract reliable and question-relevant subgraphs, thereby reducing noise and errors in retrieved knowledge. We then introduce a Multi-Agent Debate mechanism, which yields reliable answers through adaptive adversarial debates, aiming to fully exploit the knowledge in UKGs while preserving the reliability of retrieved evidence. Extensive experiments on four benchmark QA datasets show that DoG achieves state-of-the-art performance over existing LLM reasoning methods and KG-based baselines, while enabling reliable and adaptive reasoning. Our code is available at https://github.com/seucoin/Debate-on-Graph.

Figures

Figures reproduced from arXiv: 2607.17266 by the authors.

Figure 1
Figure 1. The overview of Debate-on-Graph framework. [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. The overview of the subgraph extraction algorithm. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Distribution of K and debate round of DoG. the debate, without expanding the subgraph. We employ GPT-4o-mini as the backbone LLM. As shown in [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Hyperparameter analysis of DoG on Qwen3-8B. [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

43 extracted references · 6 linked inside Pith

  1. [1]

    In: Proceedings of the 2008 ACM SIGMOD international conference on management of data

    Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collabo- ratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD international conference on management of data. pp. 1247–1250 (2008)

    2008

  2. [2]

    Advances in neural information processing systems33, 1877–1901 (2020)

    Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Nee- lakantan, A., Shyam, P., Sastry, G., Askell, A., et al.: Language models are few-shot learners. Advances in neural information processing systems33, 1877–1901 (2020)

    1901

  3. [3]

    In: Proceedings of the AAAI conference on artificial intelligence

    Carlson, A., Betteridge, J., Kisiel, B., Settles, B., Hruschka, E., Mitchell, T.: To- ward an architecture for never-ending language learning. In: Proceedings of the AAAI conference on artificial intelligence. vol. 24, pp. 1306–1313 (2010)

    2010

  4. [4]

    Data Intelligence 6(3), 587–620 (2024)

    Chen, H.: Large Knowledge Model: Perspectives and Challenges. Data Intelligence 6(3), 587–620 (2024)

    2024

  5. [5]

    In: Proc

    Chen, X., Chen, M., Shi, W., Sun, Y., Zaniolo, C.: Embedding Uncertain Knowl- edge Graphs. In: Proc. of AAAI. vol. 33, pp. 3363–3370 (2019)

    2019

  6. [6]

    In: Proc

    Chen, Y., Wu, T., Liu, Y., Wang, Y., Qi, G.: Uncertain Knowledge Graph Com- pletion with Rule Mining. In: Proc. of WISA. pp. 100–112 (2024)

    2024

  7. [7]

    IEEE Transactions on Knowledge and Data Engineering35(8), 8343–8357 (2022)

    Chen, Y., Li, H., Qi, G., Wu, T., Wang, T.: Outlining and Filling: Hierarchi- cal Query Graph Generation for Answering Complex Questions Over Knowledge Graphs. IEEE Transactions on Knowledge and Data Engineering35(8), 8343–8357 (2022)

    2022

  8. [8]

    In: Proc

    Chen, Z.M., Yeh, M.Y., Kuo, T.W.: PASSLEAF: A Pool-bAsed Semi-Supervised LEArning Framework for Uncertain Knowledge Graph Embedding. In: Proc. of AAAI. vol. 35, pp. 4019–4026 (2021)

    2021

Show all 43 references
  1. [9]

    In: Forty-first International Conference on Machine Learning (2023)

    Du, Y., Li, S., Torralba, A., Tenenbaum, J.B., Mordatch, I.: Improving factual- ity and reasoning in language models through multiagent debate. In: Forty-first International Conference on Machine Learning (2023)

    2023

  2. [10]

    In: Findings of the Association for Computa- tional Linguistics: ACL 2025

    Fang, S., Ma, K., Zheng, T., Du, X., Lu, N., Zhang, G., Tang, Q.: KARPA: A training-free method of adapting knowledge graph as references for large language model’s reasoning path aggregation. In: Findings of the Association for Computa- tional Linguistics: ACL 2025. pp. 24724...

    2025

  3. [11]

    Guo, Q., Zhuang, F., Qin, C., Zhu, H., Xie, X., Xiong, H., He, Q.: A Survey on KnowledgeGraph-basedRecommenderSystems.IEEETransactionsonKnowledge and Data Engineering34(8), 3549–3568 (2020)

    2020

  4. [12]

    ACM Computing Surveys54(4), 1–37 (2021)

    Hogan, A., Blomqvist, E., Cochez, M., d’Amato, C., Melo, G.D., Gutierrez, C., Kirrane, S., Gayo, J.E.L., Navigli, R., Neumaier, S., et al.: Knowledge graphs. ACM Computing Surveys54(4), 1–37 (2021)

    2021

  5. [13]

    In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    Hu, W., Zhang, W., Jiang, Y., Zhang, C.J., Wei, X., Qing, L.: Removal of hallu- cination on hallucination: Debate-Augmented RAG. In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 15839–15853 (2025)

    2025

  6. [14]

    arXiv preprint arXiv:2410.21276 (2024)

    Hurst, A., Lerer, A., Goucher, A.P., Perelman, A., Ramesh, A., Clark, A., Ostrow, A., Welihinda, A., Hayes, A., Radford, A., et al.: GPT-4o system card. arXiv preprint arXiv:2410.21276 (2024)

    2024 arXiv

  7. [15]

    ACM Computing Surveys55(12), 1–38 (2023)

    Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y.J., Madotto, A., Fung, P.: Survey of hallucination in natural language generation. ACM Computing Surveys55(12), 1–38 (2023)

    2023

  8. [16]

    In: Proceedings of the International Conference on Learning Representations (ICLR) (2024) Debate-on-Graph 17

    Luo, L., Li, Y.F., Haffari, G., Pan, S.: Reasoning on Graphs: Faithful and Inter- pretable Large Language Model Reasoning. In: Proceedings of the International Conference on Learning Representations (ICLR) (2024) Debate-on-Graph 17

    2024

  9. [17]

    In: Forty-second International Conference on Machine Learning (2025)

    Luo, L., Zhao, Z., Haffari, G., Li, Y.F., Gong, C., Pan, S.: Graph-constrained Reasoning:FaithfulReasoningonKnowledgeGraphswithLargeLanguageModels. In: Forty-second International Conference on Machine Learning (2025)

    2025

  10. [18]

    In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing

    Ma, C., Chen, Y., Wu, T., Khan, A., Wang, H.: Large language models meet knowl- edge graphs for question answering: Synthesis and opportunities. In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. pp. 24578–24597 (2025)

    2025

  11. [19]

    In: Findings of the As- sociation for Computational Linguistics: ACL 2025

    Mavromatis, C., Karypis, G.: GNN-RAG: Graph Neural Retrieval for Efficient Large Language Model Reasoning on Knowledge Graphs. In: Findings of the As- sociation for Computational Linguistics: ACL 2025. pp. 16682–16699 (2025)

    2025

  12. [20]

    In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing

    Mihaylov, T., Clark, P., Khot, T., Sabharwal, A.: Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. pp. 2381–2391 (2018)

    2018

  13. [21]

    IEEE Transactions on Knowledge and Data Engineering36(7), 3580–3599 (2024)

    Pan, S., Luo, L., Wang, Y., Chen, C., Wang, J., Wu, X.: Unifying large language models and knowledge graphs: A roadmap. IEEE Transactions on Knowledge and Data Engineering36(7), 3580–3599 (2024)

    2024

  14. [22]

    In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)

    Reimers, N., Gurevych, I.: Sentence-BERT: Sentence Embeddings using Siamese BERT-Network. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). p. 3982 (2019)

    2019

  15. [23]

    JMIR Medical Informatics13(1), e75279 (2025)

    Sha, H., Gong, F., Liu, B., Liu, R., Wang, H., Wu, T.: Leveraging retrieval- augmented large language models for dietary recommendations with traditional chinese medicine’s medicine food homology: algorithm development and valida- tion. JMIR Medical Informatics13(1), e75279 (2025)

    2025

  16. [24]

    In: Proceedings of the AAAI conference on artificial intelli- gence

    Speer, R., Chin, J., Havasi, C.: ConceptNet 5.5: An Open Multilingual Graph of General Knowledge. In: Proceedings of the AAAI conference on artificial intelli- gence. vol. 31 (2017)

    2017

  17. [25]

    The Knowledge Engineering Re- view39, e4 (2024)

    Stathopoulos,E.A.,Vassiliades,A.,Diplaris,S.,Vrochidis,S.,Kompatsiaris,I.:Ap- plied logic and semantics on indoor and urban adaptive design through knowledge graphs, reasoning and explainable argumentation. The Knowledge Engineering Re- view39, e4 (2024)

    2024

  18. [26]

    In: The Twelfth International Conference on Learning Representations (ICLR) (2023)

    Sun, J., Xu, C., Tang, L., Wang, S., Lin, C., Gong, Y., Ni, L., Shum, H.Y., Guo, J.: Think-on-graph: Deep and responsible reasoning of large language model on knowl- edge graph. In: The Twelfth International Conference on Learning Representations (ICLR) (2023)

    2023

  19. [27]

    In: Findings of the Association for Computational Linguistics: ACL 2024

    Sun, L., Tao, Z., Li, Y., Arakawa, H.: ODA: Observation-driven agent for integrat- ing LLMs and knowledge graphs. In: Findings of the Association for Computational Linguistics: ACL 2024. pp. 7417–7431 (2024)

    2024

  20. [28]

    In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)

    Talmor, A., Herzig, J., Lourie, N., Berant, J.: CommonsenseQA: A Question An- swering Challenge Targeting Commonsense Knowledge. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, ...

    2019

  21. [29]

    arXiv preprint arXiv:2201.05320 (2022)

    Talmor,A.,Yoran,O.,Bras,R.L.,Bhagavatula,C.,Goldberg,Y.,Choi,Y.,Berant, J.: CommonsenseQA 2.0: Exposing the Limits of AI through Gamification. arXiv preprint arXiv:2201.05320 (2022)

    2022 arXiv

  22. [30]

    In: Proceedings of the ACM on Web Conference 2025

    Tan, X., Wang, X., Liu, Q., Xu, X., Yuan, X., Zhang, W.: Paths-over-Graph: Knowledge graph empowered large language model reasoning. In: Proceedings of the ACM on Web Conference 2025. pp. 3505–3522 (2025) 18 P. Yu et al

    2025

  23. [31]

    arXiv preprint arXiv:2312.11805 (2023)

    Team, G., Anil, R., Borgeaud, S., Alayrac, J.B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A.M., Hauth, A., Millican, K., et al.: Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 (2023)

    2023 arXiv

  24. [32]

    Com- munications of the ACM57(10), 78–85 (2014)

    Vrandečić, D., Krötzsch, M.: Wikidata: a free collaborative knowledgebase. Com- munications of the ACM57(10), 78–85 (2014)

    2014

  25. [33]

    In: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval

    Wang, J., Wu, T., Chen, S., Liu, Y., Zhu, S., Li, W., Xu, J., Qi, G.: unKR: A python library for uncertain knowledge graph reasoning by representation learning. In: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval....

    2024

  26. [34]

    In: Proc

    Wang, J., Wu, T., Zhang, J.: Incorporating Uncertainty of Entities and Relations into Few-Shot Uncertain Knowledge Graph Embedding. In: Proc. of CCKS. pp. 16–28 (2022)

    2022

  27. [35]

    arXiv preprint arXiv:2203.11171 (2022)

    Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., Zhou, D.: Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171 (2022)

    2022 arXiv

  28. [36]

    Advances in neural information processing systems35, 24824–24837 (2022)

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q.V., Zhou, D., et al.: Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems35, 24824–24837 (2022)

    2022

  29. [37]

    In: Proceedings of the 62nd Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers)

    Wen, Y., Wang, Z., Sun, J.: MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Models. In: Proceedings of the 62nd Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 10370–10388 (2024)

    2024

  30. [38]

    Health Information Science and Systems13(1), 63 (2025)

    Wu, T., Wu, X., Li, J., Sha, H., Wang, H., Chen, Y., Jiang, Y., Liu, L., Lan, Y., Lu, X.: Supervised prediction of post-stroke upper limb motor recovery with uncertain knowledge graph and large language model. Health Information Science and Systems13(1), 63 (2025)

    2025

  31. [39]

    Advances in Neu- ral Information Processing Systems38, 171548–171567 (2025)

    Wu, T., Zhu, S., Wang, J., Xu, N., Qi, G., Wang, H.: Uncertain knowledge graph completion via semi-supervised confidence distribution learning. Advances in Neu- ral Information Processing Systems38, 171548–171567 (2025)

    2025

  32. [40]

    arXiv preprint arXiv:2505.09388 (2025)

    Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al.: Qwen3 technical report. arXiv preprint arXiv:2505.09388 (2025)

    2025 arXiv

  33. [41]

    Information Sciences609, 679–690 (2022)

    Yang, S., Zhang, W., Tang, R., Zhang, M., Huang, Z.: Approximate inferring with confidence predicting based on uncertain knowledge graph embedding. Information Sciences609, 679–690 (2022)

    2022

  34. [42]

    arXiv preprint arXiv:2508.12393 (2025)

    Zhang, D., Wang, Z., Li, Z.Z., Yu, Y., Jia, S., Dong, J., Xu, H., Wu, X., Zhang, Y., Zhang, T., et al.: MedKGent: A Large Language Model Agent Framework for Constructing Temporally Evolving Medical Knowledge Graph. arXiv preprint arXiv:2508.12393 (2025)

    2025 arXiv

  35. [43]

    In: International conference on database systems for ad- vanced applications

    Zhang, J., Wu, T., Qi, G.: Gaussian metric learning for few-shot uncertain knowl- edge graph completion. In: International conference on database systems for ad- vanced applications. pp. 256–271. Springer (2021)

    2021

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.