REVIEW 6 cited by
Generate-on-Graph: Treat LLM as both Agent and KG in Incomplete Knowledge Graph Question Answering
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
To address the issues of insufficient knowledge and hallucination in Large Language Models (LLMs), numerous studies have explored integrating LLMs with Knowledge Graphs (KGs). However, these methods are typically evaluated on conventional Knowledge Graph Question Answering (KGQA) with complete KGs, where all factual triples required for each question are entirely covered by the given KG. In such cases, LLMs primarily act as an agent to find answer entities within the KG, rather than effectively integrating the internal knowledge of LLMs and external knowledge sources such as KGs. In fact, KGs are often incomplete to cover all the knowledge required to answer questions. To simulate these real-world scenarios and evaluate the ability of LLMs to integrate internal and external knowledge, we propose leveraging LLMs for QA under Incomplete Knowledge Graph (IKGQA), where the provided KG lacks some of the factual triples for each question, and construct corresponding datasets. To handle IKGQA, we propose a training-free method called Generate-on-Graph (GoG), which can generate new factual triples while exploring KGs. Specifically, GoG performs reasoning through a Thinking-Searching-Generating framework, which treats LLM as both Agent and KG in IKGQA. Experimental results on two datasets demonstrate that our GoG outperforms all previous methods.
Forward citations
Cited by 6 Pith papers
-
EarthSE: A Benchmark for Evaluating Earth Scientific Exploration Capability of LLMs
EarthSE provides a two-level QA benchmark and an open-ended dialogue benchmark for Earth science and shows current LLMs perform poorly on both.
-
A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data
A metric-oriented survey that classifies intrinsic quality and trustworthiness metrics for LLM-generated data across six modalities and documents systematic evaluation gaps in the current literature.
-
Search-on-Graph: Iterative Informed Navigation for Large Language Model Reasoning on Knowledge Graphs
An LLM that iteratively inspects 1-hop neighbors of a knowledge-graph entity and chooses the next relation achieves state-of-the-art KGQA scores on six Freebase/Wikidata benchmarks without fine-tuning.
-
SlimRAG: Retrieval without Graphs via Entity-Aware Context Selection
SlimRAG shows that an entity-aware inverted index without graphs can match or beat graph-based RAG retrieval on HotpotQA while using far fewer index tokens.
-
KG2Code: Bridging Knowledge Graphs and Large Language Models via Executable Code for Question Answering
KG2Code-QA represents knowledge-graph subgraphs as executable Python/NetworkX code and trains LLMs to complete the code, improving KGQA accuracy and cross-KG transfer under oracle retrieval.
-
LLM Inference Enhanced by External Knowledge: A Survey
A survey of methods that enhance LLM inference by integrating external structured knowledge from tables and knowledge graphs.
Discussion (0). Continue with ORCID to comment.