REVIEW 7 cited by
Revisiting the Test-Time Scaling of o1-like Models: Do they Truly Possess Test-Time Scaling Capabilities?
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The advent of test-time scaling in large language models (LLMs), exemplified by OpenAI's o1 series, has advanced reasoning capabilities by scaling computational resource allocation during inference. While successors like QwQ, Deepseek-R1 (R1) and LIMO replicate these advancements, whether these models truly possess test-time scaling capabilities remains underexplored. This study found that longer CoTs of these o1-like models do not consistently enhance accuracy; in fact, correct solutions are often shorter than incorrect ones for the same questions. Further investigation shows this phenomenon is closely related to models' self-revision capabilities - longer CoTs contain more self-revisions, which often lead to performance degradation. We then compare sequential and parallel scaling strategies on QwQ, R1 and LIMO, finding that parallel scaling achieves better coverage and scalability. Based on these insights, we propose Shortest Majority Vote, a method that combines parallel scaling strategies with CoT length characteristics, significantly improving models' test-time scalability compared to conventional majority voting approaches.
Forward citations
Cited by 7 Pith papers
-
Test-Time Scaling with Reflective Generative Model
MetaStone-S1 combines a shared policy and self-supervised process reward head to select high-quality reasoning traces, reaching o3-mini-level scores at 32B parameters.
-
AlphaOne: Reasoning Models Thinking Slow and Fast at Test Time
A stochastic scheduling framework that modulates slow-to-fast reasoning in large reasoning models at test time, improving accuracy while reducing token usage.
-
Reasoning LLMs are Wandering Solution Explorers
Six current reasoning LLMs, including commercial systems, exhibit structured-search failures on verifiable computation tasks and degrade as the solution space grows.
-
OpenCodeReasoning-II: A Simple Test Time Scaling Approach via Self-Critique
A 2.5M-example code reasoning dataset with critique traces enables Qwen2.5-based models to surpass prior open-weight distilled models on LiveCodeBench via test-time self-critique selection.
-
Adaptive Termination for Multi-round Parallel Reasoning: An Universal Semantic Entropy-Guided Framework
A semantic entropy-guided stopping rule for multi-round parallel LLM reasoning improves accuracy while reducing inference steps on five benchmarks.
-
Reasoning or Not? A Comprehensive Evaluation of Reasoning LLMs for Dialogue Summarization
Explicit reasoning LLMs do not improve dialogue summarization quality and tend to be more verbose and less faithful than their non-reasoning counterparts.
-
Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey
A comprehensive review that categorizes methods for shortening and adaptively triggering chain-of-thought reasoning in large language models.
Discussion (0). Continue with ORCID to comment.