REVIEW 37 cited by
Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) such as OpenAI's o1 have demonstrated remarkable abilities in complex reasoning tasks by scaling test-time compute and exhibiting human-like deep thinking. However, we identify a phenomenon we term underthinking, where o1-like LLMs frequently switch between different reasoning thoughts without sufficiently exploring promising paths to reach a correct solution. This behavior leads to inadequate depth of reasoning and decreased performance, particularly on challenging mathematical problems. To systematically analyze this issue, we conduct experiments on three challenging test sets and two representative open-source o1-like models, revealing that frequent thought switching correlates with incorrect responses. We introduce a novel metric to quantify underthinking by measuring token efficiency in incorrect answers. To address underthinking, we propose a decoding strategy with thought switching penalty TIP that discourages premature transitions between thoughts, encouraging deeper exploration of each reasoning path. Experimental results demonstrate that our approach improves accuracy across challenging datasets without requiring model fine-tuning. Our findings contribute to understanding reasoning inefficiencies in o1-like LLMs and offer a practical solution to enhance their problem-solving capabilities.
Forward citations
Cited by 37 Pith papers
-
CUSUM-Shaped Inference-Time Monitoring and Targeted Re-Decoding for Quantized Small Language Model Reasoning
A monitoring-guided rollback controller produced a small and statistically uncertain accuracy gain on first-seen MATH-500 problems, and a larger exploratory gain on a historical-coverage set contaminated by manual thr...
-
CAM: A Causality-based Analysis Framework for Multi-Agent Code Generation Systems
CAM ranks how much each intermediate output of a multi-agent code-generation system causally matters for final correctness, then uses that ranking to repair failures and prune features.
-
Controllable LLM Reasoning via Sparse Autoencoder-Based Steering
SAE-Steering finds, via keyword-logit recall plus effectiveness ranking, sparse-autoencoder features that steer a reasoning model into a chosen reasoning strategy, beating baseline steering by ~15% on a judge-based me...
-
Reconsidering Overthinking: Penalizing Internal and External Redundancy in CoT Reasoning
A dual-penalty RL method that compresses chain-of-thought traces by separately penalizing internal semantic stagnation and external post-answer continuation reduces reasoning length by about 40% while preserving accur...
-
LAPO: Internalizing Reasoning Efficiency via Length-Adaptive Policy Optimization
LAPO is a two-stage RL method that reduces reasoning token usage by up to 40.9% while slightly improving accuracy, by learning per-problem length targets from the model's own correct answers and prompting the model to...
-
It's Not That Simple. An Analysis of Simple Test-Time Scaling
The apparent scaling behavior of the s1 model is mostly an artifact of truncating long outputs, while appending 'Wait' does not reliably improve answers.
-
NaturalThoughts: Selecting and Distilling Reasoning Traces for General Reasoning Tasks
Selecting difficult, strategy-diverse reasoning traces from DeepSeek-R1 on NaturalReasoning questions improves supervised distillation of reasoning into 7B-70B student models.
-
HCNQA: Enhancing 3D VQA with Hierarchical Concentration Narrowing Supervision
HCNQA supervises three intermediate reasoning phases (BoI, OoI, OoT) derived from ScanQA annotations, improving EM@1 from 25.94 to 27.01 over 3D-VisTA.
-
Do Thinking Tokens Help or Trap? Towards More Efficient Large Reasoning Model
Suppressing 'thinking tokens' in a 1.5B reasoning model preserves accuracy while cutting tokens, and the proposed DuP-PO RL method improves both accuracy and efficiency over GRPO.
-
AdapThink: Adaptive Thinking Preferences for Reasoning Language Model
AdapThink is an RL post-training framework that adaptively reduces overthinking and underthinking in reasoning language models by rewarding confidence-appropriate reasoning depth and diverse training samples.
-
Excessive Reasoning Attack on Reasoning LLMs
A 10-token adversarial suffix, optimized with three custom losses, raises reasoning length 3x to 9x on open reasoning LLMs without hurting accuracy, and partly transfers to closed models.
-
Towards Understanding the Cognitive Habits of Large Reasoning Models
A new benchmark shows large reasoning models exhibit human-like cognitive habits in their chain-of-thought, and some habits correlate with unsafe responses.
-
Overclocking LLM Reasoning: Monitoring and Controlling Thinking Path Lengths in LLMs
A fitted progress vector in LLM hidden states can monitor and, when added during decoding, shorten the thinking phase, often improving accuracy and cutting tokens.
-
Unlocking Recursive Thinking of LLMs: Alignment via Refinement
An offline alignment pipeline using reward-filtered self-refinement data and long chain-of-thought SFT raises an 8B model's AlpacaEval 2 win rate from 25.0% to 51.0% with roughly 14k training examples.
-
Sample Complexity and Representation Ability of Test-time Scaling Paradigms
Best-of-n sampling provably needs about 1/Δ samples versus 1/Δ² for self-consistency, and a constructed Transformer can route among experts using verifier feedback to reach near-optimal final responses.
-
EPiC: Towards Lossless Speedup for Reasoning Training through Edge-Preserving CoT Condensation
Keeping only the first and last parts of chain-of-thought training traces preserves math reasoning accuracy on MATH500 at half the sequence length, but not on harder benchmarks.
-
ReasoningFlow: Semantic Structure of Complex Reasoning Traces
ReasoningFlow is a schema that labels the semantic roles and relationships in LLM reasoning traces as directed graphs, plus a small hand-annotated dataset and a pattern-query tool.
-
AlphaOne: Reasoning Models Thinking Slow and Fast at Test Time
A stochastic scheduling framework that modulates slow-to-fast reasoning in large reasoning models at test time, improving accuracy while reducing token usage.
-
What Makes a Good Reasoning Chain? Uncovering Structural Patterns in Long Chain-of-Thought Reasoning
Reasoning trees parsed from long chain-of-thought text predict answer correctness better than length and expose failure patterns such as over-branching.
-
Are Large Language Models Reliable AI Scientists? Assessing Reverse-Engineering of Black-Box Systems
LLMs struggle to use passive observations for reverse engineering, but active intervention improves performance, largely through the process of generating queries rather than the data obtained.
-
Failure Cases Are Better Learned But Boundary Says Sorry: Facilitating Smooth Perception Change for Accuracy-Robustness Trade-Off in Adversarial Training
A paper whose abstract describes new adversarial training experiments, but whose full text is a different paper on CoT compression, leaving the claims unsupported.
-
REST: Stress Testing Large Reasoning Models by Asking Multiple Problems at Once
Asking a reasoning model several problems at once reveals large accuracy drops and exposes differences that single-question benchmarks miss.
-
OpenCodeReasoning-II: A Simple Test Time Scaling Approach via Self-Critique
A 2.5M-example code reasoning dataset with critique traces enables Qwen2.5-based models to surpass prior open-weight distilled models on LiveCodeBench via test-time self-critique selection.
-
Adaptive Termination for Multi-round Parallel Reasoning: An Universal Semantic Entropy-Guided Framework
A semantic entropy-guided stopping rule for multi-round parallel LLM reasoning improves accuracy while reducing inference steps on five benchmarks.
-
Activation Steering for Chain-of-Thought Compression
A single steering vector extracted from paired verbose and concise rationales compresses chain-of-thought output at inference time without retraining.
-
Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMs
The survey's L1/L2 taxonomy and benchmark show that current reasoning models waste compute on easy problems and underthink hard ones, motivating more adaptive inference.
-
Large Reasoning Models are not thinking straight: on the unreliability of thinking trajectories
Large reasoning models frequently ignore explicitly injected ground-truth solutions mid-trajectory and continue to overthink, often ending on incorrect answers.
-
Efficient Reasoning Through Suppression of Self-Affirmation Reflections in Large Reasoning Models
Suppressing low-probability 'wait' tokens that trigger self-affirmation reflections shortens reasoning-model output by 8-18% with roughly unchanged accuracy.
-
Atomic-to-Compositional Generalization for Mobile Agents with A New Benchmark and Scheduling System
AI phone agents succeed on single-step commands but fail on multi-step compositions; a new benchmark measures this gap and a scheduler that decomposes tasks recovers most of it.
-
One Missing Piece for Open-Source Reasoning Models: A Dataset to Mitigate Cold-Starting Short CoT LLMs in RL
A pipeline converts short-CoT LLM outputs into o1-style long chain-of-thought rationales using 1K seed reasoning flows, and SFT on the resulting dataset improves downstream RLVR cold-start.
-
Control-R: Towards controllable test-time scaling
Control-R-32B is a fine-tuned 32B model that takes 11 control-field scores in the prompt, claiming to let users trade reasoning depth for efficiency and reaching SOTA on AIME2024 and MATH500 at that scale.
-
Two Experts Are All You Need for Steering Thinking: Reinforcing Cognitive Effort in MoE Reasoning Models Without Additional Training
Reinforcing the two experts most correlated with thinking tokens improves reasoning accuracy and efficiency in MoE large reasoning models, with gains of up to 10 points on AIME benchmarks.
-
Self-Reasoning Language Models: Unfold Hidden Reasoning Chains with Few Reasoning Catalyst
Self-Reasoning Language Models use 1,000 seed catalyst rationales to teach a model to expand its own reasoning chains and iteratively self-improve, gaining about 2.5 points on average across five benchmarks.
-
From Long to Short: LLMs Excel at Trimming Own Reasoning Chains
EDIT searches across step-count prompts to find the shortest reasoning path the model consistently answers correctly.
-
A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models
A structured literature survey concluding that reasoning capabilities do not automatically make LLMs more trustworthy and can introduce new vulnerabilities in safety, robustness, and privacy.
-
First Finish Search: Efficient Test-Time Scaling in Large Language Models
First Finish Search launches n parallel reasoning traces and returns the shortest one, improving accuracy on AIME benchmarks while cutting token use.
-
A Survey on Large Language Models for Mathematical Reasoning
Recent advances in LLM mathematical reasoning are organized into comprehension and generation phases, covering methods from prompting to test-time scaling.
Discussion (0). Continue with ORCID to comment.