Pith. sign in

REVIEW 3 major objections 4 minor 69 references

BYOKG-RAG: Multi-Strategy Graph Retrieval for Knowledge Graph Question Answering

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read BYOKG-RAG claims that multi-strategy graph retrieval—LLM-generated entities, paths, queries, and draft answers handed to specialized graph tools—improves KGQA by 4.5 percentage points over the strongest baseline and generalizes to custom…

desk verdict A real system and a real evaluation leak: the draft-answer feedback loop and MedQA H@2 protocol make the 4.5-point gain uninterpretable as evidence for graph retrieval. read the letter →

arxiv 2507.04127 v1 pith:B4HAUJI4 submitted 2025-07-05 cs.CL

Costas Mavromatis , Soji Adeshina , Vassilis N. Ioannidis , Zhen Han , Qi Zhu , Ian Robinson , Bryan Thompson , Huzefa Rangwala
show 1 more author
George Karypis
This is my paper
classification cs.CL
keywords knowledgegraphquestionansweringretrieval-augmentedgenerationretrievalentitylinkingOpenCypherqueryagentictraversalzero-shotKGQAiterativerefinement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

BYOKG-RAG claims that question answering over 'bring-your-own' knowledge graphs should not ask the LLM to traverse the graph; instead, the LLM should propose graph artifacts and specialized graph tools should do the linking and retrieval. The paper shows that this division of labor—entity linking, path retrieval, query execution, and triplet retrieval run in parallel and then iteratively refined—beats the second-best graph retrieval method by 4.5 percentage points across five benchmarks, with no task-specific training data. This matters because existing approaches force a choice between fine-tuned retrievers that do not transfer to new graphs and agentic traversal that is brittle to initialization errors; BYOKG-RAG claims a middle path that is both zero-shot and general across general, temporal, medical, and enterprise graphs.

What carries the argument

The load-bearing machinery is the KG-Linker prompt and the graph toolkit built around it. KG-Linker is a single LLM call that produces entities, paths, an OpenCypher query, and draft answers in tagged sections; the toolkit consumes those artifacts through four retrieval routes—entity linking (string plus bge-m3 embedding similarity, top-m=3), path retrieval (follow-paths BFS plus Dijkstra shortest paths between entities and draft answers), graph query execution, and triplet retrieval (agentic or scoring-based). The refinement loop collects the verbalized union of these retrievals as context, feeds it back into KG-Linker, and repeats until no new entities are produced, which is what lets the framework correct early linking errors.

What would settle it

Run BYOKG-RAG with the draft-answer pathway ablated: do not entity-link draft answers, do not use them as shortest-path anchors, and score MedQA without augmenting H@2 with the original prediction. If the 4.5-point lead over the second-best method and the MedQA gain shrink or vanish, the claim that multi-strategy graph retrieval drives the improvement would be falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that KGQA over custom graphs should treat the LLM as a generator of graph anchors rather than as a graph traverser. Given a question and schema, the KG-Linker LLM emits four artifacts: topic entities, candidate answer entities, plausible relation paths, and an OpenCypher query. A graph toolkit then performs fuzzy-string and embedding-based entity linking, follows and validates the generated paths, executes the query, and retrieves triplets either by agentic traversal or by scoring; the union of these results is verbalized, fed back into KG-Linker for up to two refinement iterations, and finally given to the LLM for answer generation. The paper reports that this framework raises Hit rates to 86.6 on WebQSP-IH, 73.6 on CWQ-IH, and 65.5 on CronQuestions, lifts MedQA H@2 to 65.0, and reaches 64.9 on the Northwind Cypher benchmark, beating the second-best graph retrieval method by 4.5 percentage points on average and matching or exceeding state-of-the-art KG agents with fewer LLM calls.

Load-bearing premise

The load-bearing premise is that the reported gains come from graph evidence rather than from the LLM confirming its own draft answers, since draft answers are entity-linked and used as shortest-path anchors, and MedQA's H@2 metric explicitly augments the final generation with the model's original prediction.

Editorial extensions

If this is right

  • Zero-shot KGQA becomes competitive with trained retrievers: BYOKG-RAG reaches 70.5% Recall@10 on CWQ against 54.5–64.1% for RoG, SubgraphRAG, and GNN-RAG, using no training data.
  • No single retrieval strategy dominates: the best component varies by graph—agentic traversal on Freebase multi-hop, text-based retrieval on temporal questions, graph query execution on enterprise Cypher graphs—so combining them is what carries the accuracy gain.
  • LLM call efficiency improves over agent baselines: the agentic variant uses 4.5 calls on WebQSP and 6.3 on CWQ versus 11.2 and 14.3 for ToG, and the scoring variant uses 2–3 calls.
  • Iterative refinement with self-termination contributes on top of each base retriever: on CWQ-IH it lifts Graph Reranker from 63.0 to 68.8, and on Northwind it lifts Graph-Query from 55.3 to 64.9.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Ablating the draft-answer channel would test self-confirmation: because draft answers are entity-linked and used as shortest-path anchors, and MedQA's H@2 augments the model's own prediction, some of the measured gain may come from answer leakage rather than graph evidence.
  • The artifact-driven design should transfer to hybrid sources: the paper lists text databases as future work, and the same KG-Linker prompt could emit entities and queries that link into a text corpus or a combined text+KG index.
  • Context length is the main scaling constraint: the paper's own limitation notes that the union of retrieval results can grow long, so token-budgeted pruning is a natural next experiment to keep the method usable for small-context models.
  • Entity-linking hyperparameters are a sensitivity knob: the default top-m=3 with the union of string and embedding matching could be tuned per graph, and downstream accuracy's dependence on linking recall and precision is not isolated in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces BYOKG-RAG, a zero-shot KGQA framework that combines an LLM-based KG-Linker with specialized graph retrieval tools. The LLM generates question entities, draft answer entities, relation paths, and OpenCypher queries; a graph toolkit then performs entity linking, path retrieval, graph query execution, and triplet retrieval, with an iterative refinement loop before final answer generation. Experiments on WebQSP(-IH), CWQ(-IH), CronQuestions, MedQA, and Northwind report that BYOKG-RAG outperforms the second-best graph retrieval baseline by 4.5 percentage points on average and generalizes to custom KGs, while using fewer LLM calls than several competing KGQA agents. The authors also provide an open-source implementation.

Significance. The paper addresses a practically important problem: bringing zero-shot KGQA to custom knowledge graphs without task-specific training data. The framework is well engineered, open-sourced, and evaluated across multiple backbone LLMs and several benchmark types, with consistent qualitative improvements and moderate context sizes. The multi-strategy retrieval design and the use of specialized graph tools are sensible and could be a valuable contribution to the Graph RAG literature if the reported gains are attributable to graph retrieval. However, the current experimental protocol contains a draft-answer feedback loop that may inflate the headline numbers, and several core evaluations rely on in-house or filtered subsets. The central claim that multi-strategy graph retrieval is responsible for the 4.5-point gain is therefore not yet established by the evidence presented.

major comments (3)
  1. [§3.1–3.6, Eqs. (4), (6), (9), Algorithm 2] The draft-answer feedback loop is a load-bearing confound. In §3.1, Eq. (4), the LLM generates a draft answer from its parametric knowledge before any graph context is available (Algorithm 2, line 4). In §3.2, those draft answers are entity-linked together with question entities, and in §3.3, Eq. (6), they are used as endpoints for shortest-path retrieval. Algorithm 2, line 9, also includes them in the entity set for triplet retrieval, and line 10 unions the resulting paths and triplets into the final context C. The final answer generation (Algorithm 2, line 12) therefore sees graph context that is anchored at the LLM's own guess, so it can recover that guess even when the graph provides no additional evidence. The baselines in Table 1 (Vanilla LLM, LLM+graph-query, Text-based Retrieval, Graph Reranker, Agentic Traversal) do not have this draft-answer channel, making the comparison asymmetric. The ablations in Table 3 compare "Entity Linking only" against full KG-Linker, but the latter bundles draft answers together with paths and queries, so they do not isolate the effect of the draft channel. Please rerun the evaluation with the draft-answer component removed from entity linking, shortest-path retrieval, and triplet retrieval, or otherwise show that the reported gains persist without this channel.
  2. [§4.1, Metrics; Table 1, MedQA row] The MedQA Hit@2 metric explicitly includes the LLM's original prediction. Section 4.1 states: "we report Hit@2 (H@2) by augmenting the LLM generation based on the retrieved KG context with its original prediction." This means that even a retrieval method that contributes nothing can achieve at least the Vanilla LLM score, which is 57.9 for Claude-Sonnet-3.5 in Table 1. The reported BYOKG-RAG score of 65.0 on MedQA is therefore not a clean measure of the value added by graph retrieval, especially because the draft-answer feedback loop already places the original prediction into the retrieved context. Please report Hit@1 or Hit without augmentation for all methods on MedQA, and provide a breakdown of how often the correct answer comes from the original prediction versus from graph-derived context.
  3. [§4.1, Dataset descriptions; Table 6] The main results rely heavily on in-house and filtered subsets whose representativeness is not established. WebQSP-IH and CWQ-IH are described only as "in-house subsets, containing 500 questions each from the original set," with no description of how the questions were sampled or whether the subsets preserve the difficulty distribution of the full benchmarks. MedQA is filtered to 227 questions whose answer candidates appear in the KG via entity matching, and Text2cypher is filtered to questions whose ground-truth answers can be retrieved; the effect of these filters on the reported accuracy is not analyzed. Since the abstract claims improvements across "five benchmarks" and "better generalization to custom KGs," the paper should either report full-benchmark numbers for WebQSP and CWQ, or justify that the in-house subsets and filtering criteria do not bias the comparison.
minor comments (4)
  1. [§3.5, Eq. (10)] Equation (10) writes the text-based retrieval score as Embed(q,h)+Embed(q,r)+Embed(q,t); the notation should clarify that these are cosine-similarity scores from a pretrained embedding model and that the sum is over the decomposed triplet components.
  2. [Table 9, Northwind row] The sentence "BYOKG-RAG generates an executable cypehr query" contains a typo: "cypehr" should be "Cypher."
  3. [Appendix A.2] The transition from Eq. (12) to Eq. (13) contains a typo: "Next, we collect triplets of which relations are present in Rq t9 T(L)r" should read something like "Next, we collect triplets whose relations are present in Rq."
  4. [§5.1, Table 1] The phrase "outperforming the strongest baseline across benchmarks ... by 4.5% points" is ambiguous because the strongest baseline differs per benchmark; please state whether the average is computed over the per-benchmark best baseline and report the standard deviation or per-benchmark deltas.

Circularity Check

2 steps flagged · score 8.0 of 10

Draft-answer feedback loop and MedQA H@2 union make the central KGQA gain partially circular: the LLM's own parametric guess is entity-linked, path-retrieved, and folded into final context, so the final answer can confirm the guess rather than graph evidence.

  1. self definitional [Section 3.1 (Eq. 4), Section 3.2, Section 3.3 (Eq. 6), Section 3.5 / Algorithm 2 (line 9), Section 3.6 (Eq. 9)]
    "A~ = LLM(task = "Draft Answering"), (4) ... we retrieve the shortest paths connecting the extracted entities E and candidate answers A generated by the LLM (if non-empty). ... Ps = Shortest-Paths(E, A, G) (6) ... Tq = TripletRetrieve(q, E ∪ A, C) ... C = Tq ∪ Pf ∪ Ps ∪ Qa, (9) ... A = LLM(q, C). (Algorithm 2, line 12)"

    The draft answer is generated before any retrieval (Eq. 4); at the first iteration the graph context is empty (Algorithm 2, C ← ∅), so A~ comes from parametric memory. That draft answer is then entity-linked with the extracted entities (Section 3.2: 'Entity linking maps LLM-generated entities Ê = Ē ∪ Ã'), used as the target of Shortest-Paths (Eq. 6), added to the seed set for agentic/triplet retrieval (Algorithm 2, line 9: E ∪ A), and unioned into the final context (Eq. 9). The final answer A = LLM(q,C) can therefore recover the model's own pre-retrieval guess from context that was constructed around that guess. The reported gain over baselines that lack this draft-answer channel reduces to this loop by construction; no ablation removes A from the retrieval set.

  2. other [Section 4.1 (Metrics), combined with Algorithm 2 and Section 3.6]
    "In MedQA, we report Hit@2 (H@2) by augmenting the LLM generation based on the retrieved KG context with its original prediction, and measure if correct answers are found."

    The H@2 metric is defined as the union of the model's context-based generation and its original prediction, so the reported MedQA score is satisfied even when graph retrieval contributes nothing. For BYOKG-RAG, the original prediction is exactly the draft answer produced by the same LLM (Eq. 4), and that draft answer is entity-linked, path-retrieved, triplet-retrieved, and placed into C (Eq. 9, Algorithm 2 line 12). Thus the MedQA column can be carried by the model's parametric guess alone, while the baselines in Table 1 do not have this draft-answer feedback channel. The paper's central 4.5 percentage point average gain is therefore partially forced by the evaluation protocol, not by multi-strategy graph retrieval.

full rationale

The paper's own equations expose a self-referential retrieval loop. KG-Linker generates a draft answer from parametric memory (Eq. 4) before graph context exists; that draft is entity-linked (Section 3.2), used as a shortest-path target (Eq. 6), included in triplet-retrieval seeding (Algorithm 2, line 9: E ∪ A), and unioned into the final context (Eq. 9). The final answer (Algorithm 2, line 12) can then simply restate the draft answer, with graph context built around the model's own guess. For MedQA, Section 4.1 further unions the original prediction into H@2, so the headline metric can be met without any graph evidence. The ablations do not close this gap: Table 3 compares 'Entity Linking only' with full KG-Linker, which bundles the draft answer with paths and queries, and Table 7 tests refinement, not removal of the draft-answer channel. This is not a self-citation issue; it is a by-construction leakage of the LLM's own prediction into the retrieved context and into the metric. The framework does contain components with independent value (query execution on Northwind, path retrieval on Freebase), so the circularity is partial rather than total; however, the central comparative claim is substantially inflated by this feedback loop.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The framework introduces no new scientific entities, but it depends on several hand-chosen hyperparameters and domain assumptions about schema availability, entity linking reliability, and query executability. The most consequential assumption is that candidate answers generated by the LLM can be used as retrieval anchors without biasing the final answer, which is the source of the circularity concern.

free parameters (7)
  • m (entity linking top-k) = 3
    Number of candidate entities retrieved per LLM-generated mention; set by default, not tuned per dataset.
  • k (triplet retrieval top-k) = 50 for Freebase, 10 for other KGs
    Number of triplets retrieved by text-based and scoring-based retrieval; chosen per benchmark.
  • TA (agentic iterations) = 3
    Maximum number of agentic traversal iterations; default.
  • TR (refinement iterations) = 2
    Number of KG-Linker refinement rounds; default with self-termination.
  • L (hop limit) = 2
    Number of hops for graph reranker neighborhood.
  • kr (relation count in reranker) = 20
    Top-k relations selected by reranker in first step.
  • kt (triplet count in reranker) = 100
    Top-k triplets selected in second step of reranker.
assumptions (5)
  • domain assumption KG schema is available and accurate for all target graphs.
    The framework relies on schema for prompting and for path generation; if schema is incomplete, artifacts are invalid. Invoked in Section 3.1.
  • domain assumption Entity linking via fuzzy string matching and bge-m3 embeddings reliably maps generated mentions to graph entities.
    All downstream retrieval depends on linked entities E; errors propagate. Invoked in Section 3.2.
  • domain assumption LLMs can generate executable openCypher queries given a schema and the graph store can execute them.
    Graph query retrieval (Section 3.4) assumes the LLM can write correct queries and the store supports the language.
  • ad hoc to paper The in-house subsets and filtered test sets are representative of the original benchmarks and filtering does not bias results.
    The paper does not describe how WebQSP-IH and CWQ-IH were sampled, nor report how many questions were filtered from MedQA and Text2cypher. This is critical for the reported generalization claims.
  • domain assumption The exact-match Hit metric and LLM-as-a-judge are reliable across diverse KGQA outputs.
    Hit requires exact string match, which may be brittle for synonyms; LLM-as-a-judge is used for Northwind without human validation reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BYOKG-RAG: Multi-Strategy Graph Retrieval for Knowledge Graph Question Answering." pith.science (2026). https://pith.science/paper/B4HAUJI4

@misc{pith2026250704127,
  author       = {Pith},
  title        = {Pith review of: BYOKG-RAG: Multi-Strategy Graph Retrieval for Knowledge Graph Question Answering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/B4HAUJI4}},
  note         = {Machine review of arXiv:2507.04127}
}
read the original abstract

Knowledge graph question answering (KGQA) presents significant challenges due to the structural and semantic variations across input graphs. Existing works rely on Large Language Model (LLM) agents for graph traversal and retrieval; an approach that is sensitive to traversal initialization, as it is prone to entity linking errors and may not generalize well to custom ("bring-your-own") KGs. We introduce BYOKG-RAG, a framework that enhances KGQA by synergistically combining LLMs with specialized graph retrieval tools. In BYOKG-RAG, LLMs generate critical graph artifacts (question entities, candidate answers, reasoning paths, and OpenCypher queries), and graph tools link these artifacts to the KG and retrieve relevant graph context. The retrieved context enables the LLM to iteratively refine its graph linking and retrieval, before final answer generation. By retrieving context from different graph tools, BYOKG-RAG offers a more general and robust solution for QA over custom KGs. Through experiments on five benchmarks spanning diverse KG types, we demonstrate that BYOKG-RAG outperforms the second-best graph retrieval method by 4.5% points while showing better generalization to custom KGs. BYOKG-RAG framework is open-sourced at https://github.com/awslabs/graphrag-toolkit.

Figures

Figures reproduced from arXiv: 2507.04127 by the authors.

Figure 1
Figure 1. While agentic retrieval (top) may be prone to [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. BYOKG-RAG prompts an LLM to generate critical graph artifacts for graph linking. Then, a toolkit of graph retrievers operates on the underlying graph and, based on the generated artifacts and query’s context, retrieves relevant information for final KGQA or linking refinement. work of using context originating from a graph or a KG within RAG (Edge et al., 2024; He et al., 2024; Mavromatis and Karypis, 2024). Recentl… view at source ↗
Figure 3
Figure 3. The prompt template used in KG-Linker. Full [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (5 more)
Figure 5
Figure 5. Figure 5: The path generation prompt in KG-Linker. [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 4
Figure 4. Figure 4: The entity extraction prompt in KG-Linker. [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]
Figure 6
Figure 6. Figure 6: The graph query generation prompt in KG [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 9
Figure 9. Figure 9: The entity selection prompt template used in [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 8
Figure 8. Figure 8: The relation selection prompt template used [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

69 extracted references · 31 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Dhruv Agarwal, Rajarshi Das, Sopan Khosla, and Rashmi Gangadharaiah. 2023. Bring your own kg: Self-supervised program synthesis for zero-shot kgqa. arXiv preprint arXiv:2311.07850

    work page arXiv 2023

  4. [4]

    Anthropic. 2024. https://www-cdn.anthropic.com/de8ba9b01c9ab7cbabf5c33b80b7bbc618857627/Model_Card_Claude_3.pdf The claude 3 model family: Opus, sonnet, haiku

    2024

  5. [5]

    Tu Ao, Yanhua Yu, Yuling Wang, Yang Deng, Zirui Guo, Liang Pang, Pinghui Wang, Tat-Seng Chua, Xiao Zhang, and Zhen Cai. 2025. Lightprof: A lightweight reasoning framework for large language model on knowledge graph. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 23424--23432

    work page 2025

  6. [6]

    Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pages 1247--1250

    work page 2008

  7. [7]

    Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, and 1 others. 2021. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258

    arXiv 2021

  8. [8]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

Show all 69 references
  1. [9]

    Boyu Chen, Zirui Guo, Zidan Yang, Yuluo Chen, Junze Chen, Zhenghao Liu, Chuan Shi, and Cheng Yang. 2025. Pathrag: Pruning graph-based retrieval augmented generation with relational paths. arXiv preprint arXiv:2502.14902

    2025

  2. [10]

    Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. arXiv preprint arXiv:2402.03216

    2024 arXiv

  3. [11]

    Zixuan Dong, Baoyun Peng, Yufei Wang, Jia Fu, Xiaodong Wang, Xin Zhou, Yongxue Shan, Kangchen Zhu, and Weiguo Chen. 2025. E ffi QA : Efficient question-answering with strategic multi-model collaboration on knowledge graphs. In Proceedings of the 31st International Conference o...

    2025

  4. [12]

    Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazar \'e , Maria Lomeli, Lucas Hosseini, and Herv \'e J \'e gou. 2024. The faiss library. arXiv preprint arXiv:2401.08281

    2024 arXiv

  5. [13]

    Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, and Jonathan Larson. 2024. From local to global: A graph rag approach to query-focused summarization. arXiv preprint arXiv:2404.16130

    2024 arXiv

  6. [14]

    Jianqi Gao, Jian Cao, Ranran Bu, Nengjun Zhu, Wei Guan, and Hang Yu. 2025. Promoting knowledge base question answering by directing llms to generate task-relevant logical forms. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 23914--23922

    2025

  7. [15]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. 2023. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997

    2023 arXiv

  8. [16]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  9. [17]

    Yu Gu, Xiang Deng, and Yu Su. 2023. Don ' t generate, discriminate: A proposal for grounding language models to real-world environments. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Toronto, Canada. Associa...

    2023

  10. [18]

    Yu Gu and Yu Su. 2022. Arcaneqa: Dynamic program induction and contextualized encoding for knowledge base question answering. arXiv preprint arXiv:2204.08109

    2022 arXiv

  11. [19]

    Zirui Guo, Lianghao Xia, Yanhua Yu, Tu Ao, and Chao Huang. 2024. Lightrag: Simple and fast retrieval-augmented generation

    2024

  12. [20]

    Bernal Jim \'e nez Guti \'e rrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, and Yu Su. 2024. Hipporag: Neurobiologically inspired long-term memory for large language models. arXiv preprint arXiv:2405.14831

    2024 arXiv

  13. [21]

    Gaole He, Yunshi Lan, Jing Jiang, Wayne Xin Zhao, and Ji-Rong Wen. 2021. Improving multi-hop knowledge base question answering by learning intermediate supervision signals. In Proceedings of the 14th ACM international conference on web search and data mining, pages 553--561

    2021

  14. [22]

    Xiaoxin He, Yijun Tian, Yifei Sun, Nitesh V Chawla, Thomas Laurent, Yann LeCun, Xavier Bresson, and Bryan Hooi. 2024. G-retriever: Retrieval-augmented generation for textual graph understanding and question answering. arXiv preprint arXiv:2402.07630

    2024 arXiv

  15. [23]

    Jinhao Jiang, Kun Zhou, Zican Dong, Keming Ye, Wayne Xin Zhao, and Ji-Rong Wen. 2023 a . Structgpt: A general framework for large language model to reason over structured data. arXiv preprint arXiv:2305.09645

    2023 arXiv

  16. [24]

    Jinhao Jiang, Kun Zhou, Wayne Xin Zhao, Yang Song, Chen Zhu, Hengshu Zhu, and Ji-Rong Wen. 2024. Kg-agent: An efficient autonomous agent framework for complex reasoning over knowledge graph. arXiv preprint arXiv:2402.11163

    2024 arXiv

  17. [25]

    Jinhao Jiang, Kun Zhou, Wayne Xin Zhao, and Ji-Rong Wen. 2023 b . Unikgqa: Unified retrieval and reasoning for solving multi-hop question answering over knowledge graph. In International Conference on Learning Representations

    2023

  18. [26]

    Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. 2021. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences

    2021

  19. [27]

    Sumin Jo, Junseong Choi, Jiho Kim, and Edward Choi. 2025. R2-kg: General-purpose dual-agent framework for reliable reasoning on knowledge graphs. arXiv preprint arXiv:2502.12767

    2025

  20. [28]

    Jiho Kim, Yeonsu Kwon, Yohan Jo, and Edward Choi. 2023. KG - GPT : A general framework for reasoning on knowledge graphs using large language models. In Findings of the Association for Computational Linguistics: EMNLP 2023

    2023

  21. [29]

    Timoth \'e e Lacroix, Guillaume Obozinski, and Nicolas Usunier. 2020. Tensor decompositions for temporal knowledge base completion. arXiv preprint arXiv:2004.04926

    2020 arXiv

  22. [30]

    Yunshi Lan and Jing Jiang. 2020 a . Query graph generation for answering multi-hop complex questions from knowledge bases. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 969--974. Association for Computational Linguistics

    2020

  23. [31]

    Yunshi Lan and Jing Jiang. 2020 b . Query graph generation for answering multi-hop complex questions from knowledge bases. Association for Computational Linguistics

    2020

  24. [32]

    Meng-Chieh Lee, Qi Zhu, Costas Mavromatis, Zhen Han, Soji Adeshina, Vassilis N Ioannidis, Huzefa Rangwala, and Christos Faloutsos. 2024. Hybgrag: Hybrid retrieval-augmented generation on textual and relational knowledge bases. arXiv preprint arXiv:2412.16311

    2024 arXiv

  25. [33]

    u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K \"u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \"a schel, and 1 others. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Informati...

    2020

  26. [34]

    Belinda Z Li, Sewon Min, Srinivasan Iyer, Yashar Mehdad, and Wen-tau Yih. 2020. Efficient one-pass end-to-end entity linking for questions. arXiv preprint arXiv:2010.02413

    2020 arXiv

  27. [35]

    Kun Li, Tianhua Zhang, Xixin Wu, Hongyin Luo, James Glass, and Helen Meng. 2024 a . Decoding on graphs: Faithful and sound reasoning on knowledge graphs through generation of well-formed chains. arXiv preprint arXiv:2410.18415

    2024 arXiv

  28. [36]

    Mufei Li, Siqi Miao, and Pan Li. 2024 b . Simple is effective: The roles of graphs and large language models in knowledge-graph-based retrieval-augmented generation. arXiv preprint arXiv:2410.20724

    2024 arXiv

  29. [37]

    Tianle Li, Xueguang Ma, Alex Zhuang, Yu Gu, Yu Su, and Wenhu Chen. 2023. Few-shot in-context learning on knowledge base question answering. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6966--6980. Ass...

    2023

  30. [38]

    Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2023. Lost in the middle: How language models use long contexts. arXiv preprint arXiv:2307.03172

    2023 arXiv

  31. [39]

    Haoran Luo, Yikai Guo, Qika Lin, Xiaobao Wu, Xinyu Mu, Wenhao Liu, Meina Song, Yifan Zhu, Luu Anh Tuan, and 1 others. 2025 a . Kbqa-o1: Agentic knowledge base question answering with monte carlo tree search. arXiv preprint arXiv:2501.18922

    2025 arXiv

  32. [40]

    Linhao Luo, Yuan-Fang Li, Gholamreza Haffari, and Shirui Pan. 2024 a . Reasoning on graphs: Faithful and interpretable large language model reasoning. In International Conference on Learning Representations

    2024

  33. [41]

    Linhao Luo, Zicheng Zhao, Chen Gong, Gholamreza Haffari, and Shirui Pan. 2024 b . Graph-constrained reasoning: Faithful reasoning on knowledge graphs with large language models. arXiv preprint arXiv:2410.13080

    2024 arXiv

  34. [42]

    Linhao Luo, Zicheng Zhao, Gholamreza Haffari, Dinh Phung, Chen Gong, and Shirui Pan. 2025 b . Gfm-rag: Graph foundation model for retrieval augmented generation. arXiv preprint arXiv:2502.01113

    2025

  35. [43]

    Costas Mavromatis and George Karypis. 2022. https://aclanthology.org/2022.findings-emnlp.181 R ea R ev: Adaptive reasoning for question answering over knowledge graphs . In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 2447--2458, Abu Dhabi, Unit...

    2022

  36. [44]

    Costas Mavromatis and George Karypis. 2024. G NN - RAG : Graph neural retrieval for large language model reasoning. arXiv preprint arXiv:2405.20139

    2024 arXiv

  37. [45]

    Armin Oliya, Amir Saffari, Priyanka Sen, and Tom Ayoola. 2021. End-to-end entity resolution and question answering using differentiable knowledge graphs. arXiv preprint arXiv:2109.05817

    2021 arXiv

  38. [46]

    Makbule Gulcin Ozsoy, Leila Messallem, Jon Besga, and Gianandrea Minneci. 2024. Text2cypher: Bridging natural language and graph databases. arXiv preprint arXiv:2412.10064

    2024 arXiv

  39. [47]

    Shirui Pan, Linhao Luo, Yufei Wang, Chen Chen, Jiapu Wang, and Xindong Wu. 2024. Unifying large language models and knowledge graphs: A roadmap. IEEE Transactions on Knowledge and Data Engineering

    2024

  40. [48]

    Boci Peng, Yun Zhu, Yongchao Liu, Xiaohe Bo, Haizhou Shi, Chuntao Hong, Yan Zhang, and Siliang Tang. 2024. Graph retrieval-augmented generation: A survey. arXiv preprint arXiv:2408.08921

    2024 arXiv

  41. [49]

    Bhaskarjit Sarmah, Dhagash Mehta, Benika Hall, Rohan Rao, Sunil Patel, and Stefano Pasquali. 2024. Hybridrag: Integrating knowledge graphs and vector retrieval augmented generation for efficient information extraction. In Proceedings of the 5th ACM International Conference on ...

    2024

  42. [50]

    Apoorv Saxena, Soumen Chakrabarti, and Partha Talukdar. 2021. Question answering over temporal knowledge graphs. arXiv preprint arXiv:2106.01515

    2021 arXiv

  43. [51]

    Apoorv Saxena, Aditay Tripathi, and Partha Talukdar. 2020. Improving multi-hop question answering over knowledge graphs using knowledge base embeddings. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

    2020

  44. [52]

    Jiaxin Shi, Shulin Cao, Lei Hou, Juanzi Li, and Hanwang Zhang. 2021. Transfernet: An effective and transparent framework for multi-hop question answering over relation graph. arXiv preprint arXiv:2104.07302

    2021 arXiv

  45. [53]

    Yiheng Shu, Zhiwei Yu, Yuhan Li, B \"o rje F Karlsson, Tingting Ma, Yuzhong Qu, and Chin-Yew Lin. 2022. Tiara: Multi-grained retrieval for robust question answering over large knowledge bases. arXiv preprint arXiv:2210.12925

    2022 arXiv

  46. [54]

    Yuan Sui, Yufei He, Nian Liu, Xiaoxin He, Kun Wang, and Bryan Hooi. 2024. Fidelis: Faithful reasoning in large language model for knowledge graph question answering. arXiv preprint arXiv:2405.13873

    2024 arXiv

  47. [55]

    Haitian Sun, Bhuwan Dhingra, Manzil Zaheer, Kathryn Mazaitis, Ruslan Salakhutdinov, and William Cohen. 2018. Open domain question answering using early fusion of knowledge bases and text. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing...

    2018

  48. [56]

    Jiashuo Sun, Chengjin Xu, Lumingyuan Tang, Saizhuo Wang, Chen Lin, Yeyun Gong, Heung-Yeung Shum, and Jian Guo. 2024. Think-on-graph: Deep and responsible reasoning of large language model with knowledge graph. In International Conference on Learning Representations

    2024

  49. [57]

    Yawei Sun, Lingling Zhang, Gong Cheng, and Yuzhong Qu. 2020. Sparqa: skeleton-based semantic parsing for complex questions over knowledge bases. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 8952--8959

    2020

  50. [58]

    Alon Talmor and Jonathan Berant. 2018. The web as a knowledge-base for answering complex questions. In Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computational Linguistics

    2018

  51. [59]

    Denny Vrande c i \'c and Markus Kr \"o tzsch. 2014. Wikidata: a free collaborative knowledgebase. Communications of the ACM, 57(10):78--85

    2014

  52. [60]

    Keheng Wang, Feiyu Duan, Sirui Wang, Peiguang Li, Yunsen Xian, Chuantao Yin, Wenge Rong, and Zhang Xiong. 2023. Knowledge-driven cot: Exploring faithful reasoning in llms for knowledge-intensive question answering. arXiv preprint arXiv:2308.13259

    2023 arXiv

  53. [61]

    David S Wishart, Yannick D Feunang, An C Guo, Elvis J Lo, Ana Marcu, Jason R Grant, Tanvir Sajed, Daniel Johnson, Carin Li, Zinat Sayeeda, and 1 others. 2018. Drugbank 5.0: a major update to the drugbank database for 2018. Nucleic acids research, 46(D1):D1074--D1082

    2018

  54. [62]

    Shirley Wu, Shiyu Zhao, Michihiro Yasunaga, Kexin Huang, Kaidi Cao, Qian Huang, Vassilis Ioannidis, Karthik Subbian, James Y Zou, and Jure Leskovec. 2024. Stark: Benchmarking llm retrieval on textual and relational knowledge bases. Advances in Neural Information Processing Sys...

    2024

  55. [63]

    Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. 2023. https://arxiv.org/abs/2309.07597 C-pack: Packaged resources to advance general chinese embedding . Preprint, arXiv:2309.07597

    2023 arXiv

  56. [64]

    Vikas Yadav and Steven Bethard. 2019. A survey on recent advances in named entity recognition from deep learning models. arXiv preprint arXiv:1910.11470

    2019 arXiv

  57. [65]

    Michihiro Yasunaga, Hongyu Ren, Antoine Bosselut, Percy Liang, and Jure Leskovec. 2021. Qa-gnn: Reasoning with language models and knowledge graphs for question answering. In North American Chapter of the Association for Computational Linguistics (NAACL)

    2021

  58. [66]

    Xi Ye, Semih Yavuz, Kazuma Hashimoto, Yingbo Zhou, and Caiming Xiong. 2022. RNG - KBQA : Generation augmented iterative ranking for knowledge base question answering. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pap...

    2022

  59. [67]

    Scott Wen-tau Yih, Ming-Wei Chang, Xiaodong He, and Jianfeng Gao. 2015. Semantic parsing via staged query graph generation: Question answering with knowledge base. In Proceedings of the Joint Conference of the 53rd Annual Meeting of the ACL and the 7th International Joint Conf...

    2015

  60. [68]

    Donghan Yu, Sheng Zhang, Patrick Ng, Henghui Zhu, Alexander Hanbo Li, Jun Wang, Yiqun Hu, William Wang, Zhiguo Wang, and Bing Xiang. 2022. Decaf: Joint decoding of answers and logical forms for question answering over knowledge bases. arXiv preprint arXiv:2210.00063

    2022 arXiv

  61. [69]

    Han Zhang, Langshi Zhou, and Hanfang Yang. 2025. Learning to retrieve and reason on knowledge graph through active self-reflection. arXiv preprint arXiv:2502.14932

    2025 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.