REVIEW 6 cited by
RAG-Star: Enhancing Deliberative Reasoning with Retrieval Augmented Verification and Refinement
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Existing large language models (LLMs) show exceptional problem-solving capabilities but might struggle with complex reasoning tasks. Despite the successes of chain-of-thought and tree-based search methods, they mainly depend on the internal knowledge of LLMs to search over intermediate reasoning steps, limited to dealing with simple tasks involving fewer reasoning steps. In this paper, we propose \textbf{RAG-Star}, a novel RAG approach that integrates the retrieved information to guide the tree-based deliberative reasoning process that relies on the inherent knowledge of LLMs. By leveraging Monte Carlo Tree Search, RAG-Star iteratively plans intermediate sub-queries and answers for reasoning based on the LLM itself. To consolidate internal and external knowledge, we propose an retrieval-augmented verification that utilizes query- and answer-aware reward modeling to provide feedback for the inherent reasoning of LLMs. Our experiments involving Llama-3.1-8B-Instruct and GPT-4o demonstrate that RAG-Star significantly outperforms previous RAG and reasoning methods.
Forward citations
Cited by 6 Pith papers
-
Fishing Out Free Riders: Shapley-Based Reward Attribution for Parallel Reasoning via Reinforcement Learning
Rewarding each parallel reasoning path by Monte-Carlo-Shapley marginal contribution, scored by a generative reward model, lifts Pass@16 on AIME24/AIME25/AMC23 by 4-90% relative over Parallel-R1 with a fifth of the tra...
-
Lessons from Training Grounded LLMs with Verifiable Rewards
A two-stage GRPO reward scheme improves citation-grounded answering and refusal in RAG models, with reasoning models benefiting more than instruction-tuned ones.
-
Self-Critique Guided Iterative Reasoning for Multi-hop Question Answering
SiGIR trains a language model to decompose multi-hop questions, self-critique each retrieval and reasoning step, and use cumulative self-rewards in a beam search, beating prior methods by up to 14.4 F1 points on MuSiQue.
-
Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learning
Tool-Star combines cold-start supervised fine-tuning with a multi-tool self-critic reinforcement learning algorithm and hierarchical rewards to improve LLM tool-use reasoning.
-
R1-Searcher++: Incentivizing the Dynamic Knowledge Acquisition of LLMs via Reinforcement Learning
R1-Searcher++ uses SFT cold-start plus reinforcement learning with group and memorization rewards to teach Qwen-2.5-7B to balance internal knowledge and external retrieval, improving accuracy and reducing retrieval calls.
-
Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMs
A survey organizing RAG-reasoning systems into three stages: reasoning-enhanced RAG, RAG-enhanced reasoning, and synergized agentic RAG-reasoning.
Discussion (0). Continue with ORCID to comment.