REVIEW 18 cited by
Missing Premise exacerbates Overthinking: Are Reasoning Models losing Critical Thinking Skill?
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We find that the response length of reasoning LLMs, whether trained by reinforcement learning or supervised learning, drastically increases for ill-posed questions with missing premises (MiP), ending up with redundant and ineffective thinking. This newly introduced scenario exacerbates the general overthinking issue to a large extent, which we name as the MiP-Overthinking. Such failures are against the ``test-time scaling law'' but have been widely observed on multiple datasets we curated with MiP, indicating the harm of cheap overthinking and a lack of critical thinking. Surprisingly, LLMs not specifically trained for reasoning exhibit much better performance on the MiP scenario, producing much shorter responses that quickly identify ill-posed queries. This implies a critical flaw of the current training recipe for reasoning LLMs, which does not encourage efficient thinking adequately, leading to the abuse of thinking patterns. To further investigate the reasons behind such failures, we conduct fine-grained analyses of the reasoning length, overthinking patterns, and location of critical thinking on different types of LLMs. Moreover, our extended ablation study reveals that the overthinking is contagious through the distillation of reasoning models' responses. These results improve the understanding of overthinking and shed novel insights into mitigating the problem.
Forward citations
Cited by 18 Pith papers
-
Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction
Structured cognitive-episode features from LRM reasoning traces, combined with item semantics, improve human item-difficulty prediction and show harder items drive more implementation-centered, iterative solving.
-
Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression
Extra-CoT trains a semantic compressor on math CoT data, applies mixed-ratio SFT, and uses CHRPO reinforcement learning to achieve over 73% token reduction on MATH-500 with 0.6% accuracy gain on Qwen3-1.7B.
-
Reasoning While Asking: Transforming Reasoning Large Language Models from Passive Solvers to Proactive Inquirers
Proactive Interactive Reasoning (PIR) teaches LLMs to insert clarification questions into their chain-of-thought, improving simulated task success while cutting reasoning tokens roughly in half.
-
Are Large Reasoning Models Interruptible?
Interrupting large reasoning models or changing the problem mid-thought sharply degrades accuracy, with up to 60% drops and distinct failure modes: reasoning leakage, panic, and self-doubt.
-
ParaThinker: Native Parallel Thinking as a New Paradigm to Scale LLM Test-time Compute
ParaThinker trains LLMs for native parallel reasoning and reports 7 to 12 percent higher accuracy on math benchmarks over sequential thinking with modest latency overhead.
-
Can Large Multimodal Models Actively Recognize Faulty Inputs? A Systematic Evaluation Framework of Their Input Scrutiny Ability
Large multimodal models mostly fail to proactively detect flawed textual premises, and their performance depends on error type and on how they weight text versus images.
-
VLM4D: Towards Spatiotemporal Awareness in Vision Language Models
VLM4D benchmarks spatiotemporal reasoning in VLMs and finds large gaps versus humans, with proposed methods showing partial improvement.
-
"Harmless to You, Hurtful to Me!": Investigating the Detection of Toxic Languages Grounded in the Perspective of Youth
The authors construct the first Chinese youth-toxicity dataset, show that youth and adult perceptions of toxic language diverge, and report that adding contextual meta information improves detection accuracy.
-
Beyond Passive Critical Thinking: Fostering Proactive Questioning to Enhance Human-AI Collaboration
A training method using reinforcement learning and answerability heuristics lets small language models actively ask for missing math details and then solve problems, raising accuracy on the new GSM-MC benchmark from 0...
-
BMMR: A Large-Scale Bilingual Multimodal Multi-Discipline Reasoning Dataset
BMMR provides a 110k-question bilingual, multimodal, college-level dataset across 300 subjects where state-of-the-art models score at most about 50%.
-
What makes Reasoning Models Different? Follow the Reasoning Leader for Efficient Decoding
FoReaL-Decoding lets a strong reasoning model generate the first few tokens of each sentence and a weaker model complete the sentence, cutting theoretical FLOPs by 30-55% while retaining 86-100% of accuracy on four ma...
-
From Token to Action: State Machine Reasoning to Mitigate Overthinking in Information Retrieval
A state-machine framework that replaces token-level chain-of-thought with discrete query-refinement and reranking actions reduces token use by 74% while improving nDCG@10 on retrieval benchmarks.
-
Answer-Centric or Reasoning-Driven? Uncovering the Latent Memory Anchor in LLMs
Across ten LLMs, masking the final answer inside a complete reasoning chain causes a 26.9-point accuracy drop, evidence that models anchor to answers, not reasoning templates.
-
THINK-Bench: Evaluating Thinking Efficiency and Chain-of-Thought Quality of Large Reasoning Models
Most large reasoning models overthink easy questions, and Think-Bench provides a benchmark and metrics to quantify this inefficiency and the quality of their chain-of-thought.
-
A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models
A structured literature survey concluding that reasoning capabilities do not automatically make LLMs more trustworthy and can introduce new vulnerabilities in safety, robustness, and privacy.
-
When LLMs Copy to Think: Uncovering Copy-Guided Attacks in Reasoning LLMs
Hidden strings in code exploit a reasoning model's tendency to copy tokens into its own thinking, enabling output length and result manipulation.
-
Revisiting Overthinking in Long Chain-of-Thought from the Perspective of Self-Doubt
A 'check the question, then answer concisely' prompt reduces token use and often improves accuracy in reasoning LLMs, but the self-doubt measure is based on an unvalidated LLM judge.
-
Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey
A comprehensive review that categorizes methods for shortening and adaptively triggering chain-of-thought reasoning in large language models.
Discussion (0). Sign in to comment.