REVIEW 12 cited by
Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recent advancements in Large Language Models (LLMs) have significantly enhanced their ability to perform complex reasoning tasks, transitioning from fast and intuitive thinking (System 1) to slow and deep reasoning (System 2). While System 2 reasoning improves task accuracy, it often incurs substantial computational costs due to its slow thinking nature and inefficient or unnecessary reasoning behaviors. In contrast, System 1 reasoning is computationally efficient but leads to suboptimal performance. Consequently, it is critical to balance the trade-off between performance (benefits) and computational costs (budgets), giving rise to the concept of reasoning economy. In this survey, we provide a comprehensive analysis of reasoning economy in both the post-training and test-time inference stages of LLMs, encompassing i) the cause of reasoning inefficiency, ii) behavior analysis of different reasoning patterns, and iii) potential solutions to achieve reasoning economy. By offering actionable insights and highlighting open challenges, we aim to shed light on strategies for improving the reasoning economy of LLMs, thereby serving as a valuable resource for advancing research in this evolving area. We also provide a public repository to continually track developments in this fast-evolving field.
Forward citations
Cited by 12 Pith papers
-
The Vision Wormhole: Latent-Space Communication in Heterogeneous Multi-Agent Systems
Reasoning messages between heterogeneous VLMs can be routed through the image-token span: a distilled universal codec plus affine alignment transmits latent traces across model families, cutting wall-clock time in sma...
-
Exploring and Exploiting the Inherent Efficiency within Large Reasoning Models for Self-Guided Efficiency Enhancement
Large reasoning models already contain the ability to reason concisely; steering their internal activations or rewarding brevity can cut reasoning length by up to half without hurting accuracy.
-
From Token to Action: State Machine Reasoning to Mitigate Overthinking in Information Retrieval
A state-machine framework that replaces token-level chain-of-thought with discrete query-refinement and reranking actions reduces token use by 74% while improving nDCG@10 on retrieval benchmarks.
-
Thinking Fast and Right: Balancing Accuracy and Reasoning Length with Adaptive Rewards
An adaptive length-penalty reward for RL-trained LLMs reduces reasoning length by over 50% with small accuracy loss by automatically tightening and relaxing the penalty based on the model's current accuracy.
-
Learn to Reason Efficiently with Adaptive Length-based Reward Shaping
An adaptive, difficulty-aware length reward for RL training makes large reasoning models produce shorter chains of thought while matching or improving math accuracy.
-
Is Your Model Thinking or Just Stagnating? PUMA: Diagnosing Reasoning Pathology via Phase-Momentum Alignment
PUMA detects when a reasoning model's entropy drop aligns with hidden-state momentum, truncates at that point, and reports improved accuracy-efficiency on 1.5B-32B reasoning models.
-
CoRE: Enhancing Metacognition with Label-free Self-evaluation in LRMs
A training-free and label-free detector of cyclic hidden-state patterns triggers early exit during chain-of-thought reasoning, reducing token length while mostly preserving or improving accuracy.
-
Efficient Reasoning Through Suppression of Self-Affirmation Reflections in Large Reasoning Models
Suppressing low-probability 'wait' tokens that trigger self-affirmation reflections shortens reasoning-model output by 8-18% with roughly unchanged accuracy.
-
Revisiting Test-Time Scaling: A Survey and a Diversity-Aware Method for Efficient Reasoning
ADAPT, a diversity-aware prefix fine-tuning method, improves best-of-N sampling efficiency for a 1.5B reasoning model, reaching 80% accuracy at N=32 versus N=256 for the baseline.
-
Reasoning Can Hurt the Inductive Abilities of Large Language Models
Chain-of-thought reasoning can hurt LLMs' ability to infer hidden rules from gameplay transcripts, and structured interventions recover the lost accuracy.
-
Don't Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models
DTO constructs compressed, ground-truth-curated reasoning trajectories from LRM outputs and uses them as preferred responses in SimPO, yielding up to 12% accuracy gains and roughly 40% token reduction on math benchmarks.
-
Two Experts Are All You Need for Steering Thinking: Reinforcing Cognitive Effort in MoE Reasoning Models Without Additional Training
Reinforcing the two experts most correlated with thinking tokens improves reasoning accuracy and efficiency in MoE large reasoning models, with gains of up to 10 points on AIME benchmarks.
Discussion (0). Sign in to comment.