Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Removal of Hallucination on Hallucination: Debate-Augmented RAG

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Structured multi-agent debate at both retrieval and generation stages can cut RAG's compounding hallucinations, with the largest gains on multi-hop question answering.

desk verdict Plausible new configuration of debate plus RAG, with real gains on multi-hop QA, but the abstract overclaims and the judge's reliability is unexamined—worth a serious referee with major revisions. read the letter →

arxiv 2505.18581 v1 pith:RQCKRM6O submitted 2025-05-24 cs.CL cs.AI

Wentao Hu , Wengyu Zhang , Yiyang Jiang , Chen Jason Zhang , Xiaoyong Wei , Qing Li This is my paper · ORCID
classification cs.CLcs.AI
keywords Retrieval-AugmentedGenerationMulti-AgentDebateHallucinationQuestionAnsweringFactualAccuracyTraining-freeframeworkMulti-hopQAInformationAsymmetry
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper identifies a failure mode it calls 'Hallucination on Hallucination': when retrieval returns biased, incomplete, or wrong documents, the generation stage can compound those errors instead of correcting them. It proposes Debate-Augmented RAG (DRAG), a training-free framework that inserts structured multi-agent debate into both stages — a retrieval debate among proponent, challenger, and judge agents to refine the query pool, and a response debate in which one agent reasons only from retrieved documents, a second only from internal knowledge, and a judge selects the final answer. The reported results show DRAG achieving the best Exact Match scores on the two multi-hop QA benchmarks (28.80 on 2WikiMultihopQA and 30.80 on HotpotQA), and ablations indicate both debate stages and the information asymmetry contribute. The claim is that adversarial role-structured deliberation can repair RAG's two-stage error cascade without any additional training.

What carries the argument

The mechanism is a two-stage multi-agent debate with fixed role asymmetry. In the Retrieval Debate, three agents — proponent, challenger, and judge — argue over whether the current query pool is sufficient; the judge applies Eq. (7) to select between keeping or refining the pool, and the loop terminates when the pool stops changing or the round limit (r = 3) is reached. In the Response Debate, the proponent generates answers from retrieved documents (Eq. 9), the challenger from internal knowledge only (Eq. 11), then they iteratively revise against each other's answers (Eqs. 10, 12), and the judge aggregates the final responses into the selected answer (Eq. 13). The information asymmetry is the load-bearing design: giving both agents the same documents lets them uncritically echo retrieved bias, and the ablation shows removing asymmetry hurts all six datasets.

What would settle it

On the same 500-question subsets, measure the judge agent's selected answers against the gold labels for every Response Debate; if the judge's Exact Match is no better than randomly choosing between the proponent's and challenger's final answers — especially on 2WikiMultihopQA and HotpotQA, where DRAG claims its largest gains — then the judge stage is not the source of the improvement and the central claim that response debate removes errors is undermined.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that the RAG pipeline's two error sources should be treated symmetrically, each with its own structured debate. In the Retrieval Debate, a proponent argues the current query set already yields sufficient evidence, a challenger proposes query optimization or expansion, and a judge selects which action to take, iterating until the query pool converges. In the Response Debate, the proponent answers from retrieved documents while the challenger answers from parametric knowledge alone — the 'asymmetric information' setting — then the two exchange and refine their answers under adversarial prompts, and a judge picks the final response. The paper reports that this two-stage debate improves retrieval reliability, reduces RAG-induced hallucinations, and yields the best Exact Match scores on 2WikiMultihopQA and HotpotQA among the compared methods, and that the response debate alone raises Exact Match on questions where the gold answer is absent from the retrieved documents.

Load-bearing premise

The load-bearing premise is that the judge agent reliably selects the correct final answer from the debaters' responses, since the paper reports no judge-accuracy statistic and its own error case shows the judge choosing a wrong answer that the debate had already corrected.

Editorial extensions

If this is right

  • On multi-hop benchmarks, DRAG's Exact Match exceeds the best compared baselines by 6 points (2WikiMultihopQA) and 3 points (HotpotQA), suggesting debate-driven retrieval and generation beats single-agent iterative retrieval.
  • When the gold answer is missing from retrieved evidence, adding the response debate raises average EM on the affected questions (e.g., from 12.50 to 28.98 on TriviaQA), implying internal knowledge can be marshaled to cover retrieval gaps.
  • Removing information asymmetry degrades results across all six datasets, so keeping debaters on different information diets is a necessary ingredient, not a side detail.
  • Debate round counts adapt to task difficulty: single-hop QA converges in roughly 1.2–1.3 retrieval debate rounds, multi-hop in about 1.5–1.6 rounds, and more rounds beyond the optimum cause problem drift.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the judge agent is a hidden single point of failure: its own error case shows the judge selecting 'Oxford Castle, England' while the correct answer 'Westminster' appears in the challenger's responses, and no judge-selection accuracy is reported. A natural test is to replace the judge with a rule that picks the answer repeated most often across debate rounds, or to me
  • Because DRAG is evaluated with one LLM (Llama-3.1-8B) and one retriever (E5-base-v2), the framework's payoff at larger scales or with stronger retrievers is unspecified; the debate may help more when the base model is weaker, or less if both debaters share similar biases.
  • The 'Hallucination on Hallucination' framing suggests a design rule beyond RAG: any component that injects information into an LLM (query expansion, re-ranking, tool output) should be contested by an agent with independent access, a pattern that could be tested in tool-use or agentic pipelines with the same three-role structure.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Debate-Augmented RAG (DRAG), a training-free framework that applies multi-agent debate to both the retrieval and generation stages of retrieval-augmented generation. In the retrieval stage, proponent, challenger, and judge agents iteratively refine the query pool; in the response stage, a proponent grounded in retrieved documents and a challenger using only parametric knowledge debate, with a judge selecting the final answer. The method is evaluated on six QA benchmarks (NQ, TriviaQA, PopQA, 2WikiMultihopQA, HotpotQA, StrategyQA) using Llama-3.1-8B-Instruct, reporting EM and F1 scores against several RAG baselines. The main claims are that DRAG reduces 'hallucination on hallucination' (the compounding of errors when flawed retrieval misleads generation) and significantly improves overall factual accuracy, with the strongest gains on multi-hop reasoning tasks.

Significance. If the headline claim were fully supported, the paper would make a useful contribution by coupling multi-agent debate with RAG and demonstrating that adversarial interaction can correct retrieval-induced errors without training. The method is training-free, built on an open-source toolkit, and the code is released, which aids reproducibility. The multi-hop results on 2WikiMultihopQA (28.80 EM) and HotpotQA (30.80 EM) are the most convincing part of the empirical case. However, the claim of 'significantly enhances overall factual accuracy' is not supported across the full benchmark suite: on NQ, TriviaQA, and PopQA the method is not better than strong baselines, and the final judge-selection step is not evaluated for reliability despite being the mechanism that converts debate into the reported answer.

major comments (4)
  1. [Table 1 and Abstract] The abstract states that DRAG 'significantly enhances overall factual accuracy,' but Table 1 shows that DRAG is best only on 2WikiMultihopQA, HotpotQA, and StrategyQA. On NQ it is below Naive RAG (36.80 vs 38.20 EM) and far below Self-RAG (44.00); on TriviaQA it ties Naive RAG (60.80); on PopQA it is below SuRe (38.60 vs 41.80 EM). No error bars, confidence intervals, or significance tests are reported anywhere in the paper. The empirical support for the global claim is therefore incomplete, and the paper should either restrict the claim to the tasks where DRAG actually leads or provide statistical evidence of improvement.
  2. [Section 3.3, Eq. (13), and Figure 6] The judge agent is the final decision mechanism, but its reliability is not analyzed. Figure 6 shows an error case in which the opponent agent correctly answers 'Westminster' in every round, yet the judge selects 'Oxford Castle, England.' The paper reports no aggregate judge accuracy, no comparison between judge selections and the better of the two debaters' answers, and no oracle analysis. Without this, the response-debate improvement could stem from debater self-correction or from retrieval-stage changes rather than from the judge's selection. The paper should report how often the judge chooses the correct answer when it appears in the debate, and how often the judge overrules a correct debater.
  3. [Section 3.2 vs. Appendix A.2] The formalization in Section 3.2 defines the debate termination by a convergence criterion on the query pool, ||Q_{j+1} - Q_j|| ≤ ε, with a predefined threshold. However, Appendix A.2 states that the implementation does not calculate convergence conditions but instead 'directly use[s] the judge agent to evaluate the arguments and determine the winning agent.' This is a mismatch between the stated mechanism and the actual implementation. The reported debate rounds and query counts in Table 3 therefore reflect the judge's stopping decisions, not the formal convergence criterion, and the paper should clarify which mechanism is actually evaluated.
  4. [Section 5.5, Table 5] The ablation results do not uniformly support the claim that both debate stages help. Compared with Naive RAG, adding only fRetDebate drops TriviaQA from 60.80 to 57.20, and adding only fResDebate drops NQ from 38.20 to 34.40. The paper attributes single-hop degradation to 'problem drift,' but the component-level ablations show specific harms that are not explained by this general phenomenon. The discussion should address why the retrieval debate is harmful on TriviaQA and why the response debate is harmful on NQ, or qualify the claim that each stage improves performance.
minor comments (5)
  1. [Table 3] The text below Table 3 says 'higher query counts (1.17 rounds)' for multi-hop QA, but the table column is 'Query Counts' and the value 1.17 refers to queries, not rounds. Please correct this mislabeling.
  2. [Figure 4 caption] The caption of Figure 4 contains a stray path and timestamp ('5/20/25, 7:08 PM StrategyQA2.svg file:///Users/huwt/Downloads/StrategyQA2.svg'), which appears to be an artifact of the figure-generation process. This should be removed.
  3. [Appendix A.1] The appendix says baselines were reproduced with their 'few-shot setting' while DRAG runs zero-shot. This is a notable difference in inference conditions; please state the exact number of shots for each baseline and whether DRAG was also tried with few-shot prompts.
  4. [Section 4.3] The random selection of 500 examples per dataset is not reproducible without the random seed. Please provide the seed or the exact example indices used in the evaluation.
  5. [Throughout] The notation in Figure 2 and Eq. (3) mixes script symbols (fRetDebate, fResDebate) and calligraphic symbols inconsistently; a unified notation would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DRAG's claims are supported by external benchmark evaluations rather than by definition or self-citation.

full rationale

DRAG is an empirical pipeline paper. Its reported gains are benchmark comparisons on external datasets (NQ, TriviaQA, PopQA, 2WikiMultihopQA, HotpotQA, StrategyQA) in Tables 1, 5, and 6, and there is no formal derivation chain whose conclusion is assumed in its inputs. The only self-citations are incidental related-work references (Zheng et al. 2024; Ngo et al. 2007; Wei and Ngo 2008), and none supplies a load-bearing theorem, uniqueness argument, or fitted parameter. The judge-selection concern raised by Figure 6 and Eq. (13) is an empirical reliability issue, not a circularity: the judge's output is not defined as the quantity being predicted, and the paper reports the error case as evidence rather than using it to derive the method's success. The choice of r=3 after ablations is post-hoc configuration selection, which may weaken generalization claims, but it is not a fitted-input-called-prediction reduction because no reported result is constructed to equal its own input by definition. The framework is therefore self-contained against external benchmarks, with no significant circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

No new physical or ontological entities are postulated. The phrase 'Hallucination on Hallucination' is a framing term, not a new mechanism. The framework relies on free parameters such as debate rounds, agent count, top-k, and convergence threshold, plus assumptions about agent role adherence and judge reliability.

free parameters (5)
  • Maximum debate rounds r = 3
    Set by hand before experiments; Section 5.2 ablates 0-4 and Section 4.3 fixes r=3 for final results, so the reported headline numbers use a setting informed by the same test benchmarks.
  • Convergence threshold epsilon = 0
    Defined in Section 3.2, but Appendix A.2 says the judge decides termination instead of computing the convergence condition; parameter is effectively unused.
  • Number of agents per stage = 3 (one per role)
    Chosen in Section 4.3 without ablation; affects debate dynamics and cost.
  • Top-k retrieved paragraphs = 3
    Fixed for DRAG and iterative baselines in Section 4.3; retrieval recall depends strongly on this choice.
  • Test subset size = 500 examples per dataset
    Random sample from each benchmark in Section 4.1; all reported metrics are on this subsample, not full test sets.
assumptions (4)
  • domain assumption Wikipedia dump from December 2018 contains sufficient evidence for all six benchmarks
    Section 4.3 uses this corpus; if evidence is missing or outdated, retrieval debate cannot compensate.
  • domain assumption LLM agents follow their assigned debate roles and produce coherent arguments
    The whole framework relies on role prompts in Appendices A.2 and A.3; no reliability metric for role adherence is reported.
  • domain assumption Judge agent answer selection is a valid aggregation of debate outcomes
    Eq. 13 selects the final answer; Figure 6 shows a case where the judge picks a wrong answer while the correct one is present in the debate.
  • domain assumption Dataset labels are correct ground truth for EM/F1 evaluation
    Evaluation uses gold answers from NQ, TriviaQA, PopQA, 2WikiMultihopQA, HotpotQA, and StrategyQA.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Removal of Hallucination on Hallucination: Debate-Augmented RAG." pith.science (2026). https://pith.science/paper/RQCKRM6O

@misc{pith2026250518581,
  author       = {Pith},
  title        = {Pith review of: Removal of Hallucination on Hallucination: Debate-Augmented RAG},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RQCKRM6O}},
  note         = {Machine review of arXiv:2505.18581}
}
read the original abstract

Retrieval-Augmented Generation (RAG) enhances factual accuracy by integrating external knowledge, yet it introduces a critical issue: erroneous or biased retrieval can mislead generation, compounding hallucinations, a phenomenon we term Hallucination on Hallucination. To address this, we propose Debate-Augmented RAG (DRAG), a training-free framework that integrates Multi-Agent Debate (MAD) mechanisms into both retrieval and generation stages. In retrieval, DRAG employs structured debates among proponents, opponents, and judges to refine retrieval quality and ensure factual reliability. In generation, DRAG introduces asymmetric information roles and adversarial debates, enhancing reasoning robustness and mitigating factual inconsistencies. Evaluations across multiple tasks demonstrate that DRAG improves retrieval reliability, reduces RAG-induced hallucinations, and significantly enhances overall factual accuracy. Our code is available at https://github.com/Huenao/Debate-Augmented-RAG.

Figures

Figures reproduced from arXiv: 2505.18581 by the authors.

Figure 1
Figure 1. Demonstration of Hallucination in Retrieval [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An overview of our Debate-Augmented RAG (DRAG) framework. It iteratively refines the retrieval [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Case study of the response debate. Method NQ TriviaQA PopQA 2wiki HotpotQA StrategyQA EM F1 EM F1 EM F1 EM F1 EM F1 EM Naive RAG 38.20 50.08 60.80 69.55 37.60 45.69 14.80 24.27 25.80 35.80 62.60 + fRetDebate 38.20 50.79 57.20 65.31 39.60 47.04 16.00 25.91 28.20 37.48 64.60 + fResDebate 34.40 47.61 60.40 69.60 37.60 46.43 26.00 34.30 30.20 41.00 68.80 DRAG 36.80 50.38 60.80 69.93 38.60 46.50 28.80 36.97 30.80 41.74 6… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Average LLM and Retriever calls for DRAG [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Case study of the Retrieval Debate [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Error Case of the Response Debate [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. GOSU: Retrieval-Augmented Generation with Global-Level Optimized Semantic Unit-Centric Framework

    cs.CL 2025-08 reject novelty 5.0 of 10

    GOSU globally merges semantic units from text chunks into a unit-centric knowledge graph and uses three-tier keyword retrieval to improve RAG generation quality, according to LLM-judge win rates.

Reference graph

Works this paper leans on

40 extracted references · 5 canonical work pages · cited by 1 Pith paper

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. https://arxiv.org/abs/2303.08774 Gpt-4 technical report . arXiv preprint arXiv:2303.08774

    arXiv 2023

  2. [2]

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2023. https://arxiv.org/abs/2310.11511 Self-rag: Learning to retrieve, generate, and critique through self-reflection . arXiv preprint arXiv:2310.11511

    arXiv 2023

  3. [3]

    Jonas Becker. 2024. https://arxiv.org/abs/2410.22932 Multi-agent large language models for conversational task-solving . arXiv preprint arXiv:2410.22932

    arXiv 2024

  4. [4]

    Tenenbaum, and Igor Mordatch

    Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. 2024. https://dl.acm.org/doi/10.5555/3692070.3692537 Improving factuality and reasoning in language models through multiagent debate . In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org

    arXiv 2024

  5. [5]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. https://arxiv.org/abs/2407.21783 The llama 3 herd of models . arXiv preprint arXiv:2407.21783

    arXiv 2024

  6. [6]

    Zhangyin Feng, Xiaocheng Feng, Dezhi Zhao, Maojin Yang, and Bing Qin. 2024. https://ieeexplore.ieee.org/document/10448015 Retrieval-generation synergy augmented large language models . In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 11661--11665. IEEE

    arXiv 2024

  7. [7]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. 2023. https://arxiv.org/abs/2312.10997 Retrieval-augmented generation for large language models: A survey . arXiv preprint arXiv:2312.10997

    arXiv 2023

  8. [8]

    Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. 2021. https://doi.org/10.1162/tacl_a_00370 Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies . Transactions of the Association for Computational Linguistics, 9:346--361

    doi:10.1162/tacl_a_00370 2021

Show all 40 references
  1. [9]

    Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020. https://dl.acm.org/doi/abs/10.5555/3524938.3525306 Retrieval augmented language model pre-training . In International conference on machine learning, pages 3929--3938. PMLR

    2020

  2. [10]

    Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. https://doi.org/10.18653/v1/2020.coling-main.580 Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps . In Proceedings of the 28th International Conference on Computational Li...

    2020 doi

  3. [11]

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. 2024. https://dl.acm.org/doi/10.1145/3703155 A survey on hallucination in large language models: Principles, taxonomy, challenges, and o...

    2024 doi

  4. [12]

    Gautier Izacard and Edouard Grave. 2021. https://doi.org/10.18653/v1/2021.eacl-main.74 Leveraging passage retrieval with generative models for open domain question answering . In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Li...

    2021 doi

  5. [13]

    Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2022. https://arxiv.org/abs/2208.03299 Few-shot learning with retrieval augmented language models . arXiv preprint arX...

    2022 arXiv

  6. [14]

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. https://dl.acm.org/doi/10.1145/3571730 Survey of hallucination in natural language generation . ACM Computing Surveys, 55(12):1--38

    2023 doi

  7. [15]

    Zhengbao Jiang, Frank Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.495 Active retrieval augmented generation . In Proceedings of the 2023 Conference on Empirical Methods in...

    2023 doi

  8. [16]

    Jiajie Jin, Yutao Zhu, Xinyu Yang, Chenghao Zhang, and Zhicheng Dou. 2024. https://arxiv.org/abs/2405.13576 Flashrag: A modular toolkit for efficient retrieval-augmented generation research . CoRR, abs/2405.13576

    2024 arXiv

  9. [17]

    Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. https://doi.org/10.18653/v1/P17-1147 T rivia QA : A large scale distantly supervised challenge dataset for reading comprehension . In Proceedings of the 55th Annual Meeting of the Association for Computational...

    2017 doi

  10. [18]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.550 Dense passage retrieval for open-domain question answering . In Proceedings of the 2020 Conference on Empiric...

    2020 doi

  11. [19]

    Jaehyung Kim, Jaehyun Nam, Sangwoo Mo, Jongjin Park, Sang-Woo Lee, Minjoon Seo, Jung-Woo Ha, and Jinwoo Shin. 2024. https://arxiv.org/abs/2404.13081 Sure: Summarizing retrievals using answer candidates for open-domain qa of llms . arXiv preprint arXiv:2404.13081

    2024 arXiv

  12. [20]

    Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav...

    2019 doi

  13. [21]

    u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K \"u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \"a schel, et al. 2020. https://arxiv.org/abs/2005.11401 Retrieval-augmented generation for knowledge-intensive nlp tasks . A...

    2020 arXiv

  14. [22]

    Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.992 Encouraging divergent thinking in large language models through multi-agent debate . In Proceedings of the 2024 ...

    2024 doi

  15. [23]

    Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. https://doi.org/10.18653/v1/2023.acl-long.546 When not to trust language models: Investigating effectiveness of parametric and non-parametric memories . In Proceedings of the 6...

    2023 doi

  16. [24]

    Chong-Wah Ngo, Yu-Gang Jiang, Xiaoyong Wei, Feng Wang, Wanlei Zhao, Hung-Khoon Tan, and Xiao Wu. 2007. https://www.academia.edu/2716391/Experimenting_vireo_374_Bag_of_visual_words_and_visual_based_ontology_for_semantic_video_indexing_and_search Experimenting vireo-374: Bag-of-...

    2007

  17. [25]

    Timo Schick, Jane Dwivedi-Yu, Roberto Dess \` , Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. https://dl.acm.org/doi/10.5555/3666122.3669119 Toolformer: Language models can teach themselves to use tools . Advances in ...

    2023

  18. [26]

    Zhihong Shao, Yeyun Gong, Yelong Shen, Minlie Huang, Nan Duan, and Weizhu Chen. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.620 Enhancing retrieval-augmented large language models with iterative retrieval-generation synergy . In Findings of the Association for Comput...

    2023 doi

  19. [27]

    Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Richard James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. 2024. https://doi.org/10.18653/v1/2024.naacl-long.463 REPLUG : Retrieval-augmented black-box language models . In Proceedings of the 2024 Conference of the Nor...

    2024 doi

  20. [28]

    Weihang Su, Yichen Tang, Qingyao Ai, Zhijing Wu, and Yiqun Liu. 2024. https://doi.org/10.18653/v1/2024.acl-long.702 DRAGIN : Dynamic retrieval augmented generation based on the real-time information needs of large language models . In Proceedings of the 62nd Annual Meeting of ...

    2024 doi

  21. [29]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. https://arxiv.org/abs/2307.09288 Llama 2: Open foundation and fine-tuned chat models . arXiv preprint arXiv:...

    2023 arXiv

  22. [30]

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2023. https://doi.org/10.18653/v1/2023.acl-long.557 Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions . In Proceedings of the 61st Annual Meeting of th...

    2023 doi

  23. [31]

    Keheng Wang, Feiyu Duan, Peiguang Li, Sirui Wang, and Xunliang Cai. 2025. https://aclanthology.org/2025.coling-main.163/ LLM s know what they need: Leveraging a missing information guided framework to empower retrieval-augmented generation . In Proceedings of the 31st Internat...

    2025

  24. [32]

    Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. https://arxiv.org/abs/2212.03533 Text embeddings by weakly-supervised contrastive pre-training . arXiv preprint arXiv:2212.03533

    2022 arXiv

  25. [33]

    Xiao-Yong Wei and Chong-Wah Ngo. 2008. https://dl.acm.org/doi/10.1145/1459359.1459371 Fusing semantics, observability, reliability and diversity of concept detectors for video search . In Proceedings of the 16th ACM international conference on Multimedia, pages 81--90

    2008

  26. [34]

    Fangyuan Xu, Weijia Shi, and Eunsol Choi. 2023. https://arxiv.org/abs/2310.04408 Recomp: Improving retrieval-augmented lms with compression and selective augmentation . arXiv preprint arXiv:2310.04408

    2023 arXiv

  27. [35]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. https://doi.org/10.18653/v1/D18-1259 H otpot QA : A dataset for diverse, explainable multi-hop question answering . In Proceedings of the 2018 Conference...

    2018 doi

  28. [36]

    Zhangyue Yin, Qiushi Sun, Cheng Chang, Qipeng Guo, Junqi Dai, Xuanjing Huang, and Xipeng Qiu. 2023. https://aclanthology.org/2023.emnlp-main.936 Exchange-of-thought: Enhancing large language model capabilities through cross-model communication . In Proceedings of the 2023 Conf...

    2023

  29. [37]

    Tian Yu, Shaolei Zhang, and Yang Feng. 2024. https://arxiv.org/abs/2411.19443 Auto-rag: Autonomous retrieval-augmented generation for large language models . arXiv preprint arXiv:2411.19443

    2024 arXiv

  30. [38]

    Changmeng Zheng, Dayong Liang, Wengyu Zhang, Xiao-Yong Wei, Tat-Seng Chua, and Qing Li. 2024. https://dl.acm.org/doi/10.1145/3664647.3681102 A picture is worth a graph: A blueprint debate paradigm for multimodal reasoning . In Proceedings of the 32nd ACM International Conferen...

    2024

  31. [39]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  32. [40]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.