Pith. sign in

REVIEW 4 major objections 6 minor 6 cited by

R-Search: Empowering LLM Reasoning with Search via Multi-Reward Reinforcement Learning

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read R-Search trains an LLM to interleave reasoning and search, rewards both the final answer and the factual quality of the evidence it distills, and reports gains of up to 32.2% in-domain and 25.1% out-of-domain over advanced…

desk verdict A solid, incremental RL-for-RAG result that is slightly oversold in the abstract and under-analyzed in the ablation, but worthy of review. read the letter →

arxiv 2506.04185 v1 pith:ETOL6PWB submitted 2025-06-04 cs.CL

Qingfei Zhao , Ruobing Wang , Dingling Xu , Daren Zha , Limin Liu This is my paper · ORCID
classification cs.CL
keywords R-Searchretrieval-augmentedgenerationreinforcementlearningmulti-hopquestionansweringevidencerewardreasoning-searchinteractionGRPOopen-domainQA
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

R-Search is a reinforcement learning method for teaching a large language model to solve knowledge-heavy questions by interleaving its own step-by-step reasoning with calls to a search tool. The paper's central claim is that rewarding not only the final answer but also the factual quality of the evidence the model distills from retrieved passages lets the model learn better reasoning–search trajectories than prompt-based retrieval-augmented generation methods. On seven open-domain question-answering datasets, spanning multi-hop and single-hop questions, the paper reports gains over advanced RAG baselines of up to 32.2% in-domain and 25.1% out-of-domain. The key design is a multi-reward signal: answer F1, an evidence reward computed from a frozen different-family model's answer to the same evidence, and a format check. If correct, this is a practical recipe for making retrieval-augmented models less hallucination-prone and more autonomous in deciding when to search.

What carries the argument

The central object is the evidence-augmented reasoning–search trajectory together with its multi-stage, multi-type reward. At rollout, the policy writes a chain of thought, may emit a marked search query at any point, receives retrieved passages inside observation tags, and before answering must distill all observations into an original-evidence block. The reward has three terms: answer F1 against the gold answer; an evidence reward computed as the F1 of a frozen different-family model's answer when it is given only the question and the policy's evidence; and a format reward enforcing exactly one evidence box and one answer box. Retrieved tokens are masked from the policy gradient so the model is not credited for copying search output, while evidence tokens are not masked, so the gradient pushes the model to learn how to select, understand, and integrate external knowledge.

What would settle it

Take a trained R-Search policy and give it retrieved passages where the gold answer appears verbatim, then check whether the evidence reward is high almost exactly when the evidence quote-copies those sentences; if so, the reward is capturing copying rather than integration. A cleaner causal test is to retrain with the policy's own answer F1 as the evidence reward: if the gains disappear, the cross-family scorer is the active ingredient.

Watch

Extended reading notes

Core claim

R-Search treats retrieval-augmented question answering as a trajectory-optimization problem: at every token the policy may continue reasoning, emit a search query, or, once it has enough information, produce a distilled evidence block and a final answer. The discovery is that adding an evidence-quality reward to the usual answer reward is what makes the trajectory learnable. The evidence reward works by freezing a different-family language model, feeding it only the question and the policy's distilled evidence, and scoring its answer against the gold answer with F1; a policy whose evidence lets an independent model answer correctly is rewarded. Combined with a format reward and with masking that excludes retrieved tokens from the policy gradient while leaving evidence tokens in it, this multi-reward scheme pushes the model to search at the right moments and to synthesize rather than copy what it retrieves. The paper reports consistent gains over prior iterative and adaptive RAG baselines, with the largest margins on multi-hop datasets, and shows that the trained evidence can be transferred to other models as a standalone tool.

Load-bearing premise

The load-bearing premise is that the frozen cross-family model's F1 score on the policy's evidence is a faithful measure of evidence quality; if the scorer can be satisfied by copying answer-like text, or if genuinely useful evidence scores low, the reward trains the wrong behavior.

Editorial extensions

If this is right

  • The policy learns retrieval timing without a separate router: the model itself decides at any token whether to search, replacing hand-crafted when-to-retrieve heuristics with an RL-trained policy.
  • Evidence distilled after all searches becomes a global integration step, and the paper's ablation indicates this component is responsible for large gains on multi-hop datasets.
  • The trained evidence segments can be packaged as a transferable tool: feeding them to a different or larger model improves that model's answers, allowing expensive reasoning–search interaction to be offloaded to local deployments.
  • Group-relative policy optimization converges faster and reaches a higher reward ceiling than actor-critic PPO in this setting, so the paper recommends a larger model combined with GRPO.
  • Because the model is trained only on one dataset but evaluated on seven, the reported gains indicate the learned search-and-integration behavior generalizes out of domain.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An untested extension would isolate the cross-family scorer: retrain everything identically but score evidence with the policy's own answer, and see whether the reported gains shrink; the paper does not run this comparison.
  • The mask/non-mask design implies the model is being pushed to synthesize retrieved text into its own evidence rather than reproduce it; measuring n-gram overlap between evidence and retrieved passages after training would confirm whether that behavior is actually learned.
  • Because only one dataset is used for training, the out-of-domain gains suggest the learned search-and-integration policy is fairly general; adding a second training domain would test whether the policy improves further or plateaus.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes R-Search, a reinforcement learning framework that trains an LLM to interleave multi-step reasoning with search actions and to emit a consolidated evidence block before the final answer. Training uses GRPO (and PPO in a comparison) with a multi-reward signal composed of an answer F1 reward, an evidence reward computed by feeding the policy-generated evidence to a frozen cross-family model (Llama-3.2-3B) and scoring its answer with F1 against the gold answer, and a format reward. The model is trained only on the 2WikiMQA training set and evaluated on seven QA datasets (four multi-hop, three single-hop), with comparisons against naive generation, vanilla RAG, branching, multi-step, adaptive RAG baselines, and Search-R1. The paper reports consistent gains over most baselines, a PPO/GRPO comparison, an ablation removing the evidence component, and an RSTool experiment transferring generated evidence to downstream models.

Significance. If the central claim were fully established, the work would make a useful contribution: it provides a simple, RL-based recipe for training LLMs to decide when to search and how to integrate retrieved evidence, and it compares directly with the concurrent Search-R1 method under aligned training settings. The decision to train only on 2WikiMQA and test on seven datasets is a clean generalization setup, the code and data are promised, and the case studies illustrate the intended behavior. However, the core methodological novelty is the cross-family evidence reward, and the current evidence for its effectiveness is undermined by a confounded ablation, a gameable reward proxy, and the absence of variance estimates. The significance is therefore real but not yet demonstrated at the level claimed in the abstract.

major comments (4)
  1. [Abstract; Section 5.1; Table 2] The headline improvement percentages are not traceable to the reported results. The abstract claims gains 'by up to 32.2% (in-domain) and 25.1% (out-of-domain)', and Section 5.1 claims 'up to a 37.2% improvement on complex multi-hop QA tasks' and 'up to a 45.2% improvement over branching RAG'. Using Table 2, R-Search (Qwen-2.5-3B) has a multi-hop Avg of 47.9 versus 13.5 for LLM w/o search and 24.4 for Vanilla RAG, which corresponds to relative gains of roughly 255% and 96%, not 37.2%. No aggregation formula or selection criterion is given, and the Avg columns average EM and F1 together without stating this. In addition, the Table 2 footnote labels † as in-domain and ∗ as out-of-domain, yet 2WikiMQA—the only dataset whose training split is used (Section 4.3 and Limitations)—is marked with ∗. The abstract's in-domain/out-of-domain split is therefore ambiguous, and the headline numbers need to be recomputed with an explicit formula and a clearly reported comparison baseline.
  2. [Section 5.2; Table 3; Eq. (4)] The ablation does not isolate the evidence reward. The variant 'R-Search w/o Evidence' simultaneously removes the <original_evidence> block from the rollout, disables the evidence reward r_e, disables the evidence-related format term in Eq. (5), and changes the token-level masking behavior described in Section 3.2.3. The observed F1 drop could be caused by any of these factors. Since r_e is presented as the key methodological contribution, this confound is load-bearing. A clean ablation should keep the evidence block in the rollout and vary only the reward, e.g., compare r_e against a same-model answer reward or against no evidence reward, while keeping all format and masking conditions fixed. Without such an experiment, the paper does not establish that the cross-family evidence reward drives the gains.
  3. [Section 3.2.2; Eq. (4); Table 1] The evidence reward is structurally gameable and its fidelity to genuine evidence quality is unmeasured. The reward is F1(α_cf, α_gold), where α_cf is produced by a frozen model from the question and the policy-generated evidence e. Because the policy writes e before the final answer, and because retrieved passages in these QA datasets frequently contain the gold answer verbatim, the policy can maximize the reward by copying answer-bearing fragments into <original_evidence>. The system template in Table 1 even instructs the model to preserve original wording without modification, which may encourage such copying rather than faithful evidence integration. No analysis reports evidence–gold overlap, evidence–observation fidelity, or the correlation between r_e and human-judged evidence quality. The claim that r_e 'more objectively reflects the underlying factual quality of the evidence' (Section 3.2.2) is therefore unsupported. I would like to see measurements of how often the evidence contains the gold answer and a manual or automated evaluation of evidence quality for high-reward rollouts.
  4. [Tables 2, 3, and 6] No variance estimates, confidence intervals, or significance tests are reported, and several decisive comparisons are small in absolute terms. For example, with Qwen-2.5-3B the R-Search versus Search-R1 comparison on TriviaQA F1 is 64.0 vs 63.2 and on PopQA F1 is 44.9 vs 43.5, while R-Search is worse than Search-R1 on HotpotQA F1 (54.4 vs 57.8) and Bamboogle F1 (49.8 vs 53.9). Without multiple seeds or a bootstrap procedure, the statement that R-Search 'consistently outperforms' Search-R1 cannot be separated from evaluation noise. Reporting per-seed results or at least standard deviations across seeds is necessary to support the empirical claims.
minor comments (6)
  1. [Table 2 footnote] The in-domain/out-of-domain symbols should be corrected or explained. As printed, 2WikiMQA is marked with the out-of-domain symbol even though Section 4.3 states that training uses only the 2WikiMQA training set.
  2. [Table 3; Appendix A.2] The color legend text 'Blue' and 'purple' appears to be a formatting artifact; the table as rendered contains no colored values and the legend should be removed or fixed.
  3. [Table 1] The template text contains 'box<original_evidence>' where the formatting tag should be '<original_evidence>'.
  4. [Figure 2] The RSTool bar chart has no numeric values on the vertical axis and no values above the bars, so the claimed improvements of RSTool over Vanilla cannot be quantified from the figure.
  5. [Section 5.1] The relative improvement percentages should state whether they are computed from EM, F1, or the reported Avg columns, and should specify the baseline against which each percentage is calculated.
  6. [Appendix A.3] The inference temperature is reported as 0.1, but the sampling temperature used during training rollouts is not stated, and no random seed or number of evaluation runs is given; reporting these details would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: R-Search's claims are empirical and its rewards are anchored to external gold answers.

full rationale

R-Search's central claim is empirical: RL training with reward signals (answer F1, cross-model evidence F1, format) improves reasoning-search trajectories, evaluated on held-out QA sets. No step in the derivation reduces to its inputs by construction. The answer reward (Eq. 3) computes F1 against gold answers; the evidence reward (Eq. 4) passes the policy-written evidence to a frozen cross-family model and scores that model's answer against the same gold answers; the format reward (Eq. 5) checks structural tags. These are reward definitions, not fitted parameters renamed as predictions. Training uses the 2WikiMQA training split and evaluation uses separate test sets, so the headline gains are not a re-statement of the training objective. The only self-citation (Zhao et al. 2024, LongRAG) appears in the related-work survey of RAG and is not load-bearing for any claim. The evidence reward is a proxy that may be gameable, and the 'w/o Evidence' ablation removes both the evidence block and its rewards (a confound), but these are correctness and robustness concerns, not circularity: the paper does not derive a result from a premise that already contains it. No uniqueness theorem or ansatz is imported from the authors' prior work, and no fitted input is relabeled as a prediction. Accordingly, the paper is self-contained against external benchmarks and receives a score of 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The method relies on standard RL and retrieval components plus hand-set reward weights and design choices. The cross-family evidence reward is the most assumption-heavy piece. No new physical or structural entities are introduced.

free parameters (5)
  • gamma_e (evidence format reward weight) = 0.2
    Hand-set in Section 4.3; no sensitivity analysis or ablation varying this weight.
  • gamma_alpha (answer format reward weight) = 0.2
    Hand-set in Section 4.3; shares the same fixed value as gamma_e.
  • KL coefficient beta = 0.001
    Set in Section 4.3; controls the balance between reward and reference policy.
  • retrieval top-k = 3 during training, 5 during evaluation
    Chosen in Section 4.3; affects how much evidence the policy observes.
  • rollout samples per prompt = 5
    Set in Section 4.3; serves as the GRPO group size and affects policy updates.
assumptions (5)
  • domain assumption The POMDP and RL objective in Eq. (2) is an appropriate model of reasoning-search interaction and optimizing it improves QA performance.
    The paper formalizes the RAG pipeline as an action sequence and assumes RL on this objective transfers to better answers, Section 3.1.
  • domain assumption The E5 dense retriever over Wikipedia corpora returns top-k documents that contain the facts needed to answer each question.
    All training and evaluation rely on retrieved documents being sufficient; no retrieval recall analysis is provided, Section 4.1.
  • domain assumption F1 token overlap between a cross-family model's answer and the gold answer is a valid measure of evidence quality.
    This is the evidence reward in Eq. (4); if F1 is gameable, the reward misleads the policy, Section 3.2.2.
  • domain assumption Training on 2WikiMQA only provides enough diversity for the learned reasoning-search policy to generalize to six other datasets.
    The entire out-of-domain evaluation depends on this transfer; the Limitations section acknowledges the narrow training source, Section 4.3 and Limitations.
  • ad hoc to paper Masking retrieved tokens and not masking evidence tokens is the correct gradient assignment for learning evidence integration.
    This is a design choice specific to R-Search with no theoretical or empirical analysis of alternatives, Section 3.2.3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of R-Search: Empowering LLM Reasoning with Search via Multi-Reward Reinforcement Learning." pith.science (2026). https://pith.science/paper/ETOL6PWB

@misc{pith2026250604185,
  author       = {Pith},
  title        = {Pith review of: R-Search: Empowering LLM Reasoning with Search via Multi-Reward Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ETOL6PWB}},
  note         = {Machine review of arXiv:2506.04185}
}
read the original abstract

Large language models (LLMs) have notably progressed in multi-step and long-chain reasoning. However, extending their reasoning capabilities to encompass deep interactions with search remains a non-trivial challenge, as models often fail to identify optimal reasoning-search interaction trajectories, resulting in suboptimal responses. We propose R-Search, a novel reinforcement learning framework for Reasoning-Search integration, designed to enable LLMs to autonomously execute multi-step reasoning with deep search interaction, and learn optimal reasoning search interaction trajectories via multi-reward signals, improving response quality in complex logic- and knowledge-intensive tasks. R-Search guides the LLM to dynamically decide when to retrieve or reason, while globally integrating key evidence to enhance deep knowledge interaction between reasoning and search. During RL training, R-Search provides multi-stage, multi-type rewards to jointly optimize the reasoning-search trajectory. Experiments on seven datasets show that R-Search outperforms advanced RAG baselines by up to 32.2% (in-domain) and 25.1% (out-of-domain). The code and data are available at https://github.com/QingFei1/R-Search.

Figures

Figures reproduced from arXiv: 2506.04185 by the authors.

Figure 1
Figure 1. Overview of R-Search. pabilities of LLMs. Recent studies (Guo et al., 2025; Shao et al., 2024; Jaech et al., 2024) have shown that RL with rule-based reward functions enables models to acquire complex task reasoning and self-correction abilities (Weng et al., 2022; Ku￾mar et al., 2024) without explicit intermediate su￾pervision. However, most existing RL approaches focus mainly on internal reasoning, with limited in… view at source ↗
Figure 2
Figure 2. R-Search-as-a-Tool. models, substantially reducing potential overhead. 5.2 Ablation Study In [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Analysis of training reward and number of valid searches on Qwen-2.5-7B/3B-Instruct models. The semi-transparent and the solid lines indicate raw samples and the smoothed trend. Method Multi-Hop QA Single-Hop QA Hot. 2Wiki MuSi. Bamb. NQ Tri. Pop. Qwen2.5-3B-Instruct R-Search w/o Evidence 53.4 66.6 33.3 42.3 44.6 62.3 41.6 R-Search 54.4 72.6 34.8 49.8 46.0 64.0 44.9 Qwen2.5-7B-Instruct R-Search w/o Evidence 61.9 77.… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Fetch-then-Explore: Decoupling Selection from Extraction over a Persistent Workspace for Search Agents

    cs.AI 2026-08 conditional novelty 6.0 of 10

    Fetch-then-Explore, which stores fetched pages in a per-question workspace and extracts evidence on demand with grep/read, beats visit-and-read and browsing baselines on BrowseComp across three LLM backbones.

  2. Fishing Out Free Riders: Shapley-Based Reward Attribution for Parallel Reasoning via Reinforcement Learning

    cs.AI 2026-07 conditional novelty 6.0 of 10

    Rewarding each parallel reasoning path by Monte-Carlo-Shapley marginal contribution, scored by a generative reward model, lifts Pass@16 on AIME24/AIME25/AMC23 by 4-90% relative over Parallel-R1 with a fifth of the tra...

  3. ARCO: Adaptive Rubrics with Co-Evolution for Multi-Step LLM-Based Agents

    cs.AI 2026-06 unverdicted novelty 6.0 of 10

    ARCO introduces a co-evolving rubric model with generation and scoring heads plus a trajectory decomposition constraint that improves exact-match scores on multi-hop QA tasks over outcome, rubric, and process reward b...

  4. Beyond Correctness: Rewarding Faithful Reasoning in Retrieval-Augmented Generation

    cs.CL 2025-10 reject novelty 6.0 of 10

    A turn-level faithfulness reward improves a Search-R1-style agent's Information-Think and Think-Answer faithfulness as judged by the same reward model used for training, while task accuracy is roughly unchanged.

  5. ParallelSearch: Train your LLMs to Decompose Query and Search Sub-queries in Parallel with Reinforcement Learning

    cs.CL 2025-08 unverdicted novelty 6.0 of 10

    ParallelSearch applies reinforcement learning with parallel-decomposition rewards to teach LLM search agents to issue independent sub-queries concurrently, lifting QA accuracy by 2.9% on average and 12.7% on paralleli...

  6. C-MIG: Multi-view Information Gain-based Retrieval-Augmented Generation for Clinical Diagnosis Reasoning

    cs.AI 2026-05 unverdicted novelty 5.0 of 10

    C-MIG uses multi-view information gain from retrieved documents and refinements to supervise RAG-RL for clinical diagnosis, claiming top performance on four medical benchmarks.

Reference graph

Works this paper leans on

44 extracted references · 11 canonical work pages · cited by 6 Pith papers

  1. [1]

    Akari Asai and Hannaneh Hajishirzi. 2020. https://doi.org/10.18653/V1/2020.ACL-MAIN.499 Logic-guided data augmentation and regularization for consistent question answering . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020 , pages 5642--5650. Association for Computational Linguistics

    work page doi:10.18653/v1/2020.acl-main.499 2020

  2. [2]

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2024. https://openreview.net/forum?id=hSyW5go0v8 Self-rag: Learning to retrieve, generate, and critique through self-reflection . In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net

    2024

  3. [3]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert - Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, and 12 others. 2020. https://proceedings.neurips.cc/pap...

    work page 2020

  4. [4]

    Chi - Min Chan, Chunpu Xu, Ruibin Yuan, Hongyin Luo, Wei Xue, Yike Guo, and Jie Fu. 2024. https://doi.org/10.48550/ARXIV.2404.00610 RQ-RAG: learning to refine queries for retrieval augmented generation . CoRR, abs/2404.00610

    arXiv doi:10.48550/arxiv.2404.00610 2024

  5. [5]

    Qinyuan Cheng, Xiaonan Li, Shimin Li, Qin Zhu, Zhangyue Yin, Yunfan Shao, Linyang Li, Tianxiang Sun, Hang Yan, and Xipeng Qiu. 2024. https://aclanthology.org/2024.findings-emnlp.999 Unified active retrieval for retrieval augmented generation . In Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 20...

    work page 2024

  6. [6]

    Christoph Dann, Yishay Mansour, and Mehryar Mohri. 2023. Reinforcement learning can be more efficient with multiple rewards. In International Conference on Machine Learning, pages 6948--6967. PMLR

    work page 2023

  7. [7]

    DeepSeek - AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 81 others. 2025. https://doi.org/10.48550/ARXIV.2501.12948 Deepseek-r1: Incentivizing reasoning capability in llms via reinfor...

    arXiv doi:10.48550/arxiv.2501.12948 2025

  8. [8]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Qianyu Guo, Meng Wang, and Haofen Wang. 2023. https://doi.org/10.48550/ARXIV.2312.10997 Retrieval-augmented generation for large language models: A survey . CoRR, abs/2312.10997

    arXiv doi:10.48550/arxiv.2312.10997 2023

Show all 44 references
  1. [9]

    Xinyan Guan, Jiali Zeng, Fandong Meng, Chunlei Xin, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun, and Jie Zhou. 2025. https://doi.org/10.48550/ARXIV.2502.01142 Deeprag: Thinking to retrieval step by step for large language models . CoRR, abs/2502.01142

    2025 doi arXiv

  2. [10]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    2025 arXiv

  3. [11]

    Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming - Wei Chang. 2020. https://arxiv.org/abs/2002.08909 REALM: retrieval-augmented language model pre-training . CoRR, abs/2002.08909

    2020 arXiv

  4. [12]

    Xanh Ho, Anh - Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. https://doi.org/10.18653/V1/2020.COLING-MAIN.580 Constructing A multi-hop QA dataset for comprehensive evaluation of reasoning steps . In Proceedings of the 28th International Conference on Computational ...

    2020 doi

  5. [13]

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El - Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, Ally Bennett, Ananya...

    2024 doi arXiv

  6. [14]

    Soyeong Jeong, Jinheon Baek, Sukmin Cho, Sung Ju Hwang, and Jong Park. 2024. https://doi.org/10.18653/V1/2024.NAACL-LONG.389 Adaptive-rag: Learning to adapt retrieval-augmented large language models through question complexity . In Proceedings of the 2024 Conference of the Nor...

    2024 doi

  7. [15]

    Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi - Yu, Yiming Yang, Jamie Callan, and Graham Neubig

    Zhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi - Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. https://doi.org/10.18653/V1/2023.EMNLP-MAIN.495 Active retrieval augmented generation . In Proceedings of the 2023 Conference on Empirical Metho...

    2023 doi

  8. [16]

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516

    2025 arXiv

  9. [17]

    Jiajie Jin, Yutao Zhu, Xinyu Yang, Chenghao Zhang, and Zhicheng Dou. 2024. https://doi.org/10.48550/ARXIV.2405.13576 Flashrag: A modular toolkit for efficient retrieval-augmented generation research . CoRR, abs/2405.13576

    2024 doi arXiv

  10. [18]

    Weld, and Luke Zettlemoyer

    Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. 2017. https://doi.org/10.18653/V1/P17-1147 Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension . In Proceedings of the 55th Annual Meeting of the Association for Computational...

    2017 doi

  11. [19]

    Leslie Pack Kaelbling, Michael L Littman, and Andrew W Moore. 1996. Reinforcement learning: A survey. Journal of artificial intelligence research, 4:237--285

    1996

  12. [20]

    Jaehyung Kim, Jaehyun Nam, Sangwoo Mo, Jongjin Park, Sang - Woo Lee, Minjoon Seo, Jung - Woo Ha, and Jinwoo Shin. 2024. https://openreview.net/forum?id=w4DW6qkRmt Sure: Summarizing retrievals using answer candidates for open-domain QA of llms . In The Twelfth International Con...

    2024

  13. [21]

    Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, and 1 others. 2024. Training language models to self-correct via reinforcement learning. arXiv preprint arXiv:2409.12917

    2024 arXiv

  14. [23]

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. https://doi.org/10.1145/3600006.3613165 Efficient memory management for large language model serving with pagedattention . In Proceedings of the 29...

    2023

  15. [24]

    u ttler, Mike Lewis, Wen - tau Yih, Tim Rockt \

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K \" u ttler, Mike Lewis, Wen - tau Yih, Tim Rockt \" a schel, Sebastian Riedel, and Douwe Kiela. 2020. https://proceedings.neurips.cc/paper/2020/hash/6b493230205f780e1bc269...

    2020

  16. [25]

    Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. https://doi.org/10.18653/V1/2023.ACL-LONG.546 When not to trust language models: Investigating effectiveness of parametric and non-parametric memories . In Proceedings of the 6...

    2023 doi

  17. [26]

    Meta. 2024. Llama 3.2: Revolutionizing edge ai and vision with open, customizable models

    2024

  18. [27]

    Tong Mu, Alec Helyar, Johannes Heidecke, Joshua Achiam, Andrea Vallone, Ian Kivlichan, Molly Lin, Alex Beutel, John Schulman, and Lilian Weng. 2024. https://proceedings.neurips.cc/paper_files/paper/2024/file/c4e380fb74dec9da9c7212e834657aa9-Paper-Conference.pdf Rule based rewa...

    2024

  19. [28]

    OpenAI. 2023. https://doi.org/10.48550/ARXIV.2303.08774 GPT-4 technical report . CoRR, abs/2303.08774

    2023 doi arXiv

  20. [29]

    Aske Plaat, Annie Wong, Suzan Verberne, Joost Broekens, Niki van Stein, and Thomas B \" a ck. 2024. https://doi.org/10.48550/ARXIV.2407.11511 Reasoning with large language models, a survey . CoRR, abs/2407.11511

    2024 doi

  21. [30]

    Smith, and Mike Lewis

    Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A. Smith, and Mike Lewis. 2023. https://doi.org/10.18653/V1/2023.FINDINGS-EMNLP.378 Measuring and narrowing the compositionality gap in language models . In Findings of the Association for Computational Linguistics: EMNLP...

    2023 doi

  22. [32]

    Hamilton, Chris Dyer, and Dani Yogatama

    Devendra Singh Sachan, Siva Reddy, William L. Hamilton, Chris Dyer, and Dani Yogatama. 2021. https://proceedings.neurips.cc/paper/2021/hash/da3fde159d754a2555eaa198d2d105b2-Abstract.html End-to-end training of multi-document reader and retriever for open-domain question answer...

    2021

  23. [33]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347

    2017 arXiv

  24. [34]

    Zhihong Shao, Yeyun Gong, Yelong Shen, Minlie Huang, Nan Duan, and Weizhu Chen. 2023. https://doi.org/10.18653/V1/2023.FINDINGS-EMNLP.620 Enhancing retrieval-augmented large language models with iterative retrieval-generation synergy . In Findings of the Association for Comput...

    2023 doi

  25. [35]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300

    2024 arXiv

  26. [36]

    Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Richard James, Mike Lewis, Luke Zettlemoyer, and Wen - tau Yih. 2024. https://doi.org/10.18653/V1/2024.NAACL-LONG.463 REPLUG: retrieval-augmented black-box language models . In Proceedings of the 2024 Conference of the No...

    2024 doi

  27. [37]

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. https://doi.org/10.1162/TACL\_A\_00475 Musique: Multihop questions via single-hop question composition . Trans. Assoc. Comput. Linguistics, 10:539--554

    2022 doi

  28. [38]

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2023. https://doi.org/10.18653/V1/2023.ACL-LONG.557 Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions . In Proceedings of the 61st Annual Meeting of th...

    2023 doi

  29. [39]

    Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Shengping Liu, Bin Sun, Kang Liu, and Jun Zhao. 2022. Large language models are better reasoners with self-verification. arXiv preprint arXiv:2212.09561

    2022 arXiv

  30. [40]

    Marco A Wiering and Martijn Van Otterlo. 2012. Reinforcement learning. Adaptation, learning, and optimization, 12(3):729

    2012

  31. [41]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, and 1 others. 2024. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115

    2024 arXiv

  32. [42]

    Cohen, Ruslan Salakhutdinov, and Christopher D

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. https://doi.org/10.18653/V1/D18-1259 Hotpotqa: A dataset for diverse, explainable multi-hop question answering . In Proceedings of the 2018 Conference...

    2018 doi

  33. [43]

    Ori Yoran, Tomer Wolfson, Ori Ram, and Jonathan Berant. 2024. https://openreview.net/forum?id=ZS4m74kZpH Making retrieval-augmented language models robust to irrelevant context . In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, M...

    2024

  34. [44]

    Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang, Jiadai Sun, Jiajie Zhang, Jiale Cheng, Jiayi Gui, Jie Tang, Jing Zhang, Juanzi Li, Lei Zhao, and 36 others. 2024. https://doi.org/10.48550/ARXIV.2406...

    2024 doi arXiv

  35. [45]

    Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, Longyue Wang, Anh Tuan Luu, Wei Bi, Freda Shi, and Shuming Shi. 2023. https://doi.org/10.48550/ARXIV.2309.01219 Siren's song in the AI ocean: A survey on hallucin...

    2023 doi arXiv

  36. [46]

    Qingfei Zhao, Ruobing Wang, Yukuo Cen, Daren Zha, Shicheng Tan, Yuxiao Dong, and Jie Tang. 2024. https://aclanthology.org/2024.emnlp-main.1259 Longrag: A dual-perspective retrieval-augmented generation paradigm for long-context question answering . In Proceedings of the 2024 C...

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.