Pith. sign in

REVIEW 4 major objections 4 minor 13 references

What External Knowledge is Preferred by LLMs? Characterizing and Exploring Chain of Evidence in Imperfect Context for Multi-Hop QA

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that LLMs prefer external knowledge that forms a chain of evidence — relevant to the question and internally interconnected — and that this preference improves multi-hop QA, poisoning, and defense pipelines.

desk verdict A useful practical reranking heuristic for RAG, but the central 'CoE preference' claim is undercut by a confounded control that removes the answer-bearing evidence. read the letter →

arxiv 2412.12632 v3 pith:LSLHRQDM submitted 2024-12-17 cs.CL cs.AI

Zhiyuan Chang , Mingyang Li , Xiaojun Jia , Junjie Wang , Yuekai Huang , Qing Wang , Yihao Huang , Yang Liu This is my paper · ORCID
classification cs.CLcs.AI
keywords chainofevidencemulti-hopquestionansweringretrieval-augmentedgenerationknowledgepoisoningLLMpreferencecontextrobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks what makes external knowledge usable by LLMs in multi-hop QA when the context is noisy or misleading. It characterizes the preferred knowledge as forming a Chain of Evidence (CoE): relevant to the question and internally interconnected through three textual features — intent, evidence nodes, and evidence relations. Using an automated discriminator built on GPT-4o, the authors label knowledge as CoE or Non-CoE and show that CoE-aligned knowledge substantially outperforms perturbed alternatives across five LLMs and two multi-hop datasets. They then integrate CoE features into RAG, knowledge-poisoning, and poisoning-defense pipelines and report consistent improvements over strong baselines.

What carries the argument

The carrying mechanism is the CoE discrimination pipeline: for each question, GPT-4o extracts an intent (the noun phrase describing the desired answer type), evidence nodes (key entities for reasoning), and evidence relations (logical predicates linking pairs of nodes). A second GPT-4o stage checks whether a given piece of external knowledge satisfies all three features — intent via textual entailment, nodes and relations via containment classification — and only then is the knowledge labeled CoE. This operationalization of relevance plus interconnectivity is what generates the CoE versus Non-CoE comparison and the downstream task variants.

What would settle it

Re-label the 1,336 samples' CoE status with human annotators following the paper's own definitions, then rerun the significance, deceptiveness, and robustness comparisons; if the CoE advantage (for instance, the 22.5% accuracy gap) disappears or reverses under human labels, the effect is an artifact of GPT-4o's self-consistent labeling rather than a preference LLMs generally hold.

Watch

Extended reading notes

Core claim

The paper's central claim is that external knowledge exhibiting CoE features is preferred by LLMs: it resists extraneous and conflicting information in multi-hop QA better than knowledge that is merely relevant but lacks interconnectivity. Empirically, CoE knowledge achieves an average accuracy of 92.0% across five LLMs and two datasets, outperforming the Non-CoE sentence-perturbed and word-perturbed variants by 22.5% and 16.3% respectively, with the gap widening as extraneous information increases. When the CoE contains factual errors, it is also more deceptive, reaching an average attack success rate of 85.4%. Structuring knowledge as CoE improves a standard RAG pipeline by 10.4% accuracy, a poisoning attack by 11.0% attack success rate, and a defense framework by 27.2% accuracy while lowering attack success by 35.8%.

Load-bearing premise

The whole study assumes that GPT-4o, given hand-crafted prompts, reliably extracts evidence nodes and relations from questions and correctly judges whether external knowledge exhibits them, with no human annotation study to validate these labels.

Editorial extensions

If this is right

  • Retrievers and rerankers that prioritize CoE features should improve multi-hop QA accuracy in noisy retrieval settings, since CoE-aligned context degrades far less as extraneous information is added.
  • Knowledge poisoning attacks become more potent when false documents are structured as CoE, so RAG defenses must check factual correctness of evidence chains, not just their structural fit.
  • Defense frameworks can be strengthened by generating or selecting CoE-structured supporting knowledge: the paper's InstructRAG variant improves accuracy and sharply reduces attack success rate.
  • The CoE preference appears consistent across closed- and open-source LLMs and across both HotpotQA and 2WikiMultihopQA, suggesting a general behavioral tendency rather than a model-specific artifact.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that CoE preference may unify previously reported single-hop biases — coherence bias, completeness bias, and confirmation bias — into a single structural factor: relevance plus mutual support among evidence pieces.
  • Because the same model family (GPT-4o) both extracts the CoE features and evaluates the downstream answers, part of the measured effect could reflect GPT-4o's self-consistent labeling style; a human-annotated CoE label set would test this directly.
  • Since CoE-structured false knowledge is highly deceptive, adding CoE-aware reranking to a RAG system without verifying answer correctness could increase vulnerability to poisoning — a risk the paper acknowledges but does not quantify beyond the deceptiveness results.
  • The CoE features could plausibly be used as a reward signal to train LLMs to ignore irrelevant context, but the paper does not test this training-based extension.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes Chain of Evidence (CoE) as a characterization of external knowledge that LLMs prefer in multi-hop QA under imperfect contexts. CoE is operationalized through three features extracted from the question: intent, evidence nodes, and evidence relations. The authors use GPT-4o to extract these features and to classify whether given external knowledge exhibits them, then construct a dataset from HotpotQA and 2WikiMultihopQA by keeping only CoE-compliant supporting facts. Non-CoE controls are created by sentence-level removal of evidence-bearing sentences (SenP) or word-level replacement of evidence nodes with hypernyms (WordP), iterating until the GPT-4o discriminator says CoE is absent. The paper reports comparative experiments across significance (accuracy under extraneous information), deceptiveness (attack success rate for incorrect CoE), and robustness (accuracy under conflicting information), as well as three usability tasks: RAG-based multi-hop QA, knowledge poisoning attacks, and poisoning defense. In all settings, CoE-aligned knowledge outperforms or more strongly affects LLMs than Non-CoE knowledge, and CoE-enhanced variants improve over the original baselines.

Significance. If the central claim holds, the paper offers a concrete, operationalizable account of what makes external knowledge effective for multi-hop QA, going beyond single-feature relevance or coherence biases studied earlier. The empirical scope is substantial: five LLMs spanning closed and open families, two multi-hop datasets, and three downstream tasks, with consistently large effect sizes (10--20+ percentage point gaps in Tables 1--4). The authors also provide a reproduction package and openly discuss limitations, including extraction-error sensitivity and the textual-level constraint of their reranking approach. The main risk is that the evidence for the characterization is entangled with how the Non-CoE controls and the IR+CoE defense are constructed, both of which may introduce answer-presence or extra-knowledge confounds that are not intrinsic to the CoE notion.

major comments (4)
  1. [Section 4.2 and Appendix E] The Non-CoE controls are constructed by removing sentences containing question evidence nodes (SenP) or replacing evidence nodes with hypernyms (WordP), iterating until the GPT-4o discriminator says the CoE is broken. This means Non-CoE contexts differ from CoE contexts not only in interconnectivity but also in whether the correct answer can be derived from the context at all. The 16--22 point ACC/ASR gaps in Tables 1--3 may therefore reflect presence versus absence of answer-bearing evidence rather than a preference for coherent evidence chains. A control that preserves all answer facts while breaking the chain (e.g., reordering sentences, substituting relations with relationally false but still answer-bearing statements, or adding unrelated but answer-independent text) is needed to isolate interconnectivity.
  2. [Section 3.2 and Section 4.1] All CoE labels--feature extraction, feature discrimination, and the selection of which supporting facts count as CoE--are produced by GPT-4o, with no human annotation or independent model validation reported. Since the same model family (GPT-4/GPT-4o) is also used to judge ACC and ASR in Sections 5.1 and 6.1, the measured 'preference' for CoE could partly reflect the labeler's own text preferences. The open-source models in the evaluation mitigate this concern somewhat, but the paper should include a human agreement study on a sample of the CoE/Non-CoE labels, or reproduce the discrimination with a different model family, to show the effect is not an artifact of the labeler.
  3. [Section 8.3 and Table 4] The IR+CoE defense is confounded: the variant injects CoE-structured knowledge into the knowledge base, so IR+CoE has access to additional correct documents that the IR baseline does not. The reported 27.2% ACC improvement and 35.8% ASR reduction may therefore come from the extra injected knowledge rather than from the CoE-based validation mechanism. An ablation where IR receives the same number and type of extra documents without CoE structuring, or where the CoE validation step is applied to the original retrieved set, is necessary to support the defense claim.
  4. [Section 5.1 and Section 6.1] The paper states in Section 5.1 that GPT-4 is used as the judge for accuracy, but Section 6.1 and Section 7.1 state GPT-4o. Aside from the inconsistency, both are from the same model family used to define and label CoE. Reporting agreement with human evaluation or an independent judge on a random sample would strengthen the validity of all ACC/ASR comparisons.
minor comments (4)
  1. [Section 8] The typo 'chodse' in the opening sentence of Section 8 should be corrected to 'chose'.
  2. [Section 7.1] The citation 'Chen et al.;' appears incomplete; the reference is not resolved in the bibliography.
  3. [Appendix D] The text says accuracy drops from 90.2% to 89.3%/89.4%, but Table 9 shows 91.9% at proportion 0 for the 'Our' condition. Please make the values consistent.
  4. [Section 5.2] The abbreviation 'COE' is used in several places (e.g., 'the COE group' and 'COE completeness') instead of the established 'CoE'. Please unify the capitalization.

Circularity Check

2 steps flagged · score 4.0 of 10

The central CoE-vs-Non-CoE comparison is partially circular by construction: CoE is the gold supporting-fact chain and Non-CoE is the same chain with answer-bearing evidence nodes removed or abstracted until the GPT-4o discriminator fails, so the reported preference gaps partly reflect answer presence rather than interconnectivity; the RAG+CoE and poisoning/defense applications are less…

  1. self definitional [Section 4.1-4.2 and Appendix E]
    "SenP simulates incomplete knowledge by removing key evidence pieces, while WordP replaces specific evidence nodes with their higher-level expressions. These strategies ensure fair comparison by maintaining the same question context while only varying the CoE completeness."

    CoE knowledge is built from the datasets' gold supporting facts, which by construction contain the evidence nodes and relations needed to derive the answer. Non-CoE is produced by deleting exactly those evidence-bearing sentences or by replacing the evidence nodes with hypernyms, iterating until the GPT-4o discriminator reports that no complete CoE remains. The independent variable is therefore not 'interconnectivity' versus 'no interconnectivity' but 'answer-derivable evidence present' versus 'answer-derivable evidence removed/abstracted.' The reported ACC/ASR gaps in Tables 1-3 are substantially forced: any model answers worse when the sentences containing the entities required to derive the answer are deleted.

  2. other [Section 3.2, Section 6.1 and Section 7.1]
    "First, for a user question, we extract its intent, the evidence nodes and the evidence relations using GPT-4o with a hand-crafted prompt. Second, with the extracted CoE features, we discriminate whether external knowledge exhibits them using GPT-4o."

    The same GPT-4o model that defines and labels CoE is also used, in the deceptiveness and robustness assessments, as the judge that decides whether a subject LLM's output is the 'successfully misled' target or is accurate. This creates a partial labeler-evaluator overlap: the deceptiveness/robustness metrics partly measure GPT-4o's consistency with its own CoE concept rather than an independently anchored preference of the five subject LLMs. It is not a full tautology because the answers are generated by other models, including open-source families, but the lack of human validation of the CoE labels and the use of GPT-4o as both labeler and evaluator mean the measured 'preference' is not fully self-contained against the labeling model.

full rationale

The strongest circularity concern is the construction of the control condition. The paper defines CoE knowledge as the datasets' supporting-fact chain (Section 4.1) and then defines Non-CoE as the same chain with evidence sentences or evidence nodes removed until the GPT-4o discriminator says CoE is absent (Section 4.2, Appendix E). The 22.5%/16.3% ACC gaps and the parallel ASR gaps therefore primarily demonstrate that removing answer-bearing evidence hurts performance, which is a known and near-tautological result, not evidence for a novel property called 'interconnectivity.' A second, milder self-reference is that GPT-4o both labels what counts as CoE and, in Sections 6 and 7, judges whether answers were successfully misled or accurate, so the labeler and evaluator are the same model family. That said, the paper is not wholly circular: the RAG+CoE reranking experiment (Table 4) selects externally retrieved snippets using GPT-4o judgments and still improves open-source LLMs, and the poisoning/defense variants are independent applications. There is no load-bearing self-citation chain or imported uniqueness theorem. The score of 4 reflects a central comparative analysis that is partially circular by construction while leaving some independent empirical content in the usability studies.

Assumptions & free parameters 2 free parameters · 4 assumptions · 2 invented entities

The paper introduces CoE as a model-based construct and relies on GPT-4o for labeling, judging, and selection. This creates a circularity burden and several domain assumptions that are not independently validated.

free parameters (2)
  • CoE completeness threshold (all three features required) = all features
    External knowledge is classified as CoE only if intent, evidence nodes, and evidence relations are all judged present by GPT-4o; this strict conjunction is chosen by the authors without tuning or ablation against other thresholds.
  • Coverage selection priority in RAG+CoE = intent -> evidence relations -> evidence nodes
    Algorithm 1 first selects snippets covering intent, then relations, then nodes; this ordering is hand-specified and not compared to alternative priority orders.
assumptions (4)
  • domain assumption GPT-4o provides reliable extraction and discrimination of CoE features.
    The entire CoE labeling pipeline uses GPT-4o with hand-crafted prompts; no human agreement study is reported (Section 3.2, Appendix D).
  • domain assumption HotpotQA and 2WikiMultihopQA supporting facts are valid CoE chains.
    The paper assumes the datasets' supporting knowledge contains the necessary logical chains, then filters with GPT-4o (Section 4.1).
  • domain assumption Sentence-level and word-level perturbations produce representative Non-CoE samples.
    SenP removes key evidence sentences and WordP generalizes evidence nodes; the paper asserts these simulate imperfect knowledge but does not validate that they exhaust the space of relevant imperfections (Section 4.2).
  • domain assumption LLM judges (GPT-4/GPT-4o) give accurate accuracy and ASR scores.
    Evaluation uses model-based judges following Adlakha et al. (2024); no human spot-check is reported beyond the judge itself (Sections 5.1, 6.1, 7.1).
invented entities (2)
  • Chain of Evidence (CoE) as a latent knowledge property
    purpose: To explain and predict which external knowledge LLMs follow in multi-hop QA.
    CoE is defined by the authors via three features (intent, evidence nodes, evidence relations) and detected by GPT-4o; there is no external ground truth or falsifiable measurement beyond the model's own judgments.
  • CoE feature components (intent, evidence nodes, evidence relations)
    purpose: To operationalize relevance and interconnectivity.
    These components are extracted using GPT-4o from questions; they are not independently validated against human annotations or established ontologies.

how reviews work

0 comments
Cite this review

Pith. "Pith review of What External Knowledge is Preferred by LLMs? Characterizing and Exploring Chain of Evidence in Imperfect Context for Multi-Hop QA." pith.science (2026). https://pith.science/paper/LSLHRQDM

@misc{pith2026241212632,
  author       = {Pith},
  title        = {Pith review of: What External Knowledge is Preferred by LLMs? Characterizing and Exploring Chain of Evidence in Imperfect Context for Multi-Hop QA},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LSLHRQDM}},
  note         = {Machine review of arXiv:2412.12632}
}
read the original abstract

Incorporating external knowledge has emerged as a promising way to mitigate outdated knowledge and hallucinations in LLM. However, external knowledge is often imperfect, encompassing substantial extraneous or even inaccurate content, which interferes with the LLM's utilization of useful knowledge in the context. This paper seeks to characterize the features of preferred external knowledge and perform empirical studies in imperfect contexts. Inspired by the chain of evidence (CoE), we characterize that the knowledge preferred by LLMs should maintain both relevance to the question and mutual support among the textual pieces. Accordingly, we propose a CoE discrimination approach and conduct a comparative analysis between CoE and Non-CoE samples across significance, deceptiveness, and robustness, revealing the LLM's preference for external knowledge that aligns with CoE features. Furthermore, we selected three representative tasks (RAG-based multi-hop QA, external knowledge poisoning and poisoning defense), along with corresponding SOTA or prevalent baselines. By integrating CoE features, the variants achieved significant improvements over the original baselines.

Figures

Figures reproduced from arXiv: 2412.12632 by the authors.

Figure 1
Figure 1. Example of CoE and the CoE features. Next, we will characterize relevance and inter￾connectivity from the following three perspectives of textual features presented by external knowledge. • Intent is a noun or noun phrase representing the user’s desired answer to their question, and it aims to align the purpose of the user’s question with the ultimate facts derived from external knowledge. • Evidence Nodes are the k… view at source ↗
Figure 2
Figure 2. The overview of CoE discrimination. 4 Subject Dataset 4.1 CoE Knowledge Construction Oriented to multi-hop QA, we selected two com￾monly used datasets, HotpotQA and 2WikiMulti￾hopQA as sources. Both datasets contain samples comprising not just QA pairs, but also the sup￾porting knowledge required to derive each answer. During construction, the supporting knowledge was specifically designed to capture the necessary l… view at source ↗
Figure 3
Figure 3. Examples of CoE and two types of Non-CoE. [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: The overview of RAG+CoE. H Details of Information Extraction Prompts The details of the information extraction prompts are illustrated below. In pipeline, we replace the placeholders in the following prompts with the question and evidence nodes [PITH_FULL_IMAGE:figure…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 8 canonical work pages

  1. [1]

    As for intent, please indicate the content intent of the evidence that the question expects, without going into specific details

  2. [2]

    state"from the intent/question is incorrectly included in the evidencenodes,whileMissingErrorswouldhappen when essential evidence node like

    Missing Errors: failing to extract essential evidence nodes. For example, as shown in Figure 1,ExtractionErrorswouldoccurwhen"state"from the intent/question is incorrectly included in the evidencenodes,whileMissingErrorswouldhappen when essential evidence node like "CEO" are not extracted, both of which could affect the accuracy of CoE identification. To ...

    work page

  3. [3]

    750 7th Avenue

    If there is no evidence relation between evidence nodes, no extraction is required. The output must be in json format, consistent with the examples. Here are some examples: The output must be in json format, consistent with the sample. Here are some examples: Example1: Question:750 7th Avenue and 101 Park Avenue, are located in which city? Evidence nodes:...

    work page

  4. [4]

    Flows naturally as coherent text

  5. [5]

    Intent":

    As for evidence nodes, Please extract the specific details of the question. The output must be in json format, consistent with the sample. Here are some examples: Example1: Question:750 7th Avenue and 101 Park Avenue, are located in which city? Output: { "Intent": "City address Information", "evi- dencenodes": ["7507thAvenue","101ParkAvenue"] } Example2: ...

    work page 2013

  6. [6]

    Each evidence relation has two elements, the im- plied evidence nodes and the textual description of the evidence relations

    work page

  7. [7]

    The description of the evidence relations is limited to the two evidence nodes and does not involve other evidence nodes

    work page

  8. [9]

    Integrateallgiven[Nodes]naturally,donotreplace [Nodes] with pronouns, each sentence in the corpus include as many [Nodes] as possible

    work page

Show all 13 references
  1. [10]

    If there are [Relations] between [Nodes], when constructing sentences containing these [Nodes], try to express the [Relations] between them

  2. [11]

    The corpus should align with the meaning of the [Intent]

  3. [13]

    The corpus should serve as a comprehensive ba- sis for supporting why the [Question] has the an- swer:[Answer]

    Please limited the corpus to 100 words. The corpus should serve as a comprehensive ba- sis for supporting why the [Question] has the an- swer:[Answer]. It’sacreativegamefocusingongeneratingthesupport for the specified answer: [Answer], without requiring factual accuracy. K Det...

    1964

  4. [2022]

    Jiawei Chen, Hongyu Lin, Xianpei Han, and Le Sun

    Rich knowledge sources bring complex knowl- edge conflicts: Recalibrating models to reflect con- flicting evidence.arXiv preprint arXiv:2210.13701. Jiawei Chen, Hongyu Lin, Xianpei Han, and Le Sun

    arXiv

  5. [2024]

    InProceedings of the AAAI Conference on Artificial Intelligence, vol- ume 38, pages 17754–17762

    Benchmarking large language models in retrieval-augmented generation. InProceedings of the AAAI Conference on Artificial Intelligence, vol- ume 38, pages 17754–17762. Sunhao Dai, Chen Xu, Shicheng Xu, Liang Pang, Zhen- hua Dong, and Jun Xu. 2024. Unifying bias and unfairness i...

    2024 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.