REVIEW 11 cited by
Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large Language Models (LLMs) have demonstrated impressive capability in many natural language tasks. However, the auto-regressive generation process makes LLMs prone to produce errors, hallucinations and inconsistent statements when performing multi-step reasoning. In this paper, by casting multi-step reasoning of LLMs as a heuristic search problem, we aim to alleviate the pathology by introducing Q*, a general, versatile and agile framework for guiding LLMs decoding process with deliberative planning. By learning a plug-and-play Q-value model as heuristic function for estimating expected future rewards, our Q* can effectively guide LLMs to select the most promising next reasoning step without fine-tuning LLMs for the current task, which avoids the significant computational overhead and potential risk of performance degeneration on other tasks. Extensive experiments on GSM8K, MATH and MBPP demonstrate the superiority of our method, contributing to improving the reasoning performance of existing open-source LLMs.
Forward citations
Cited by 11 Pith papers
-
Reasoning LLMs are Wandering Solution Explorers
Six current reasoning LLMs, including commercial systems, exhibit structured-search failures on verifiable computation tasks and degrade as the solution space grows.
-
Self-Critique Guided Iterative Reasoning for Multi-hop Question Answering
SiGIR trains a language model to decompose multi-hop questions, self-critique each retrieval and reasoning step, and use cumulative self-rewards in a beam search, beating prior methods by up to 14.4 F1 points on MuSiQue.
-
Engineering Trustworthy Agentic AI for Critical Systems
A survey claiming that agentic AI trustworthiness is a single cross-domain problem and outlining a framework for graded, certifiable assurance.
-
DeepResearch-9K: A Challenging Benchmark Dataset of Deep-Research Agent
A synthetic benchmark of 9,000 multi-hop web-research questions with difficulty tiers and teacher-generated search trajectories, plus an open-source RL training framework that reportedly lets 3B-parameter agents beat ...
-
Let's Revise Step-by-Step: A Unified Local Search Framework for Code Generation with LLMs
The abstract claims a new local search framework for code generation, but the manuscript body is a different mathematics paper.
-
FreePRM: Training Process Reward Models Without Ground Truth Process Labels
A weakly supervised PRM training method using outcome-only pseudo-labels plus a buffer probability reaches 53.0% F1 on ProcessBench, beating supervised baselines in the paper's comparisons.
-
One Missing Piece for Open-Source Reasoning Models: A Dataset to Mitigate Cold-Starting Short CoT LLMs in RL
A pipeline converts short-CoT LLM outputs into o1-style long chain-of-thought rationales using 1K seed reasoning flows, and SFT on the resulting dataset improves downstream RLVR cold-start.
-
On the Emergence of Thinking in LLMs I: Searching for the Right Intuition
A post-training framework called RLSP, decoupling exploration from correctness rewards, induces backtracking and self-correction in LLMs and improves math reasoning performance.
-
Holistically Guided Monte Carlo Tree Search for Intricate Information Seeking
HG-MCTS guides Monte Carlo tree search with an adaptive checklist and self-generated rewards to improve multi-hop information seeking, claiming better benchmark accuracy.
-
Bag of Tricks for Inference-time Computation of LLM Reasoning
An empirical study showing that inference-time LLM reasoning methods are highly sensitive to sampling hyperparameters and reward-model choices, with recommended defaults of temperature 0.8 and top-p 0.9.
-
Large Language Models for Planning: A Comprehensive and Systematic Survey
A structured survey of LLM planning methods, benchmarks, and interpretability work, organized around a three-way taxonomy.
Discussion (0). Continue with ORCID to comment.