Pith. sign in

REVIEW 4 major objections 6 minor 3 cited by

Thinking Fast and Right: Balancing Accuracy and Reasoning Length with Adaptive Rewards

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An adaptive length penalty in RL training cuts LLM reasoning tokens by more than half while losing less than 0.04 accuracy.

desk verdict Adaptive length-penalty reward is a good idea, but the evidence here is too fragile to justify the headline claims without revision. read the letter →

arxiv 2505.18298 v1 pith:OQFWU423 submitted 2025-05-23 cs.CL

Jinyan Su , Claire Cardie This is my paper
classification cs.CL
keywords adaptiverewardshapinglengthpenaltyreinforcementlearningreasoningLLMtokenefficiencyGRPOoverthinking
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Reasoning models trained with reinforcement learning often emit thousands of tokens even for simple math questions, and fixed length penalties are hard to tune. This paper proposes a self-tuning reward shaper: when the model stays accurate, the length penalty strengthens and pushes output length down; when accuracy slips, the penalty relaxes so correctness is not traded away. The claimed result is an average token reduction of more than 50% with an accuracy loss under 0.04 on five math benchmarks, plus stable convergence instead of the collapse that fixed penalties trigger. That matters because inference cost and latency, not just benchmark accuracy, determine whether long-reasoning LLMs are practical to deploy.

What carries the argument

The central object is a reward signal, not a new model or architecture: the adaptive length penalty coefficient $\lambda_t$ inside $R_{\lambda_t}(x,y) = \mathbb{I}\{y=y^*\} - \lambda_t \, \mathrm{len}(y)$. The control law in Eq. (3), $\lambda_{t+1} = \max(0, \lambda_t + \eta(\mathrm{acc}_t - \mathrm{acc}_{\mathrm{ref}}))$, turns the difference between observed and reference accuracy into a proportional adjustment to the penalty. That feedback loop is what carries the argument: it makes length compression aggressive early in training, when the model is verbose but still accurate, and gentle later, when further shortening would begin to cost correctness. The non-negativity clamp lets the penalty turn off completely, which the paper identifies as the mechanism that prevents training collapse.

What would settle it

Run A-DLP on a different RL-trained reasoning model with $\mathrm{acc}_{\mathrm{ref}}$ estimated from its first training batch. If the trained model does not show the predicted coupling—token length falling by over 50% while accuracy drops less than 0.04, with the static-penalty curve below and to the right—then the claim fails to transfer. A more direct mechanistic check: deliberately set $\mathrm{acc}_{\mathrm{ref}}$ 0.1 above the true base accuracy; the paper predicts $\lambda_t$ reaches zero within about 10 steps and length reduction plateaus, while 0.1 below predicts over-penalization and collapse. A run that avoids both failure modes would contradict the proposed control law.

Watch

Extended reading notes

Core claim

The central discovery is that replacing a fixed length penalty with a penalty coefficient that tracks the model's own accuracy produces a better accuracy–length frontier. The reward is $R_{\lambda_t}(x,y) = \mathbb{I}\{y=y^*\} - \lambda_t \, \mathrm{len}(y)$, and the coefficient moves by the update $\lambda_{t+1} = \max(0, \lambda_t + \eta(\mathrm{acc}_t - \mathrm{acc}_{\mathrm{ref}}))$. When the model is more accurate than the reference, the penalty grows and drives length down; when accuracy slips, the penalty decays so compression does not destroy correctness. In the paper's experiments this yields more than 50% token reduction with less than 0.04 accuracy loss on five math benchmarks, places the adaptive curve above and to the left of the static-penalty curve, and avoids the collapse that fixed-penalty training shows around step 100.

Load-bearing premise

The load-bearing premise is that a single reference accuracy estimated from the first training batch is close enough to the model's true accuracy to drive the penalty update; Section 5.3 shows that an error of 0.1 in either direction either stalls length reduction or collapses the model.

Editorial extensions

If this is right

  • A-DLP can be integrated into an existing RL pipeline through the reward function alone, requiring no architectural or inference-time changes.
  • Token cost on the tested math benchmarks drops by more than half while accuracy stays within 0.04 of the base model.
  • The method stabilizes naturally as training continues, removing the need to carefully choose a stopping checkpoint to avoid over-compression.
  • The penalty shortens both correct and incorrect responses roughly in proportion to their original lengths, with reported reductions exceeding 55% in both categories.
  • Compared with fixed-penalty baselines, the adaptive curve lies above and to the left on the accuracy–length plot, meaning comparable or better accuracy is reached with fewer tokens.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The feedback law in Eq. (3) is generic: any observable that should trade off against cost could replace accuracy, so the same update could shape other efficiency axes such as tool-use count or refusal rate.
  • A rolling estimate of the reference accuracy from recent training batches might remove the calibration sensitivity that makes $\mathrm{acc}_{\mathrm{ref}}$ a fragile choice in Section 5.3.
  • Because correct and incorrect responses shrink proportionally, a version with difficulty-aware penalties could compress easy questions more aggressively while preserving hard-question accuracy.
  • The results are demonstrated on a 1.5B-parameter model; the method's practical value would increase substantially if the accuracy loss stays below 0.04 when scaled to larger reasoning models.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Adaptive Direct Length Penalty (A-DLP), a reward-shaping method for RL fine-tuning of reasoning LLMs. The reward is R(x,y)=I{y=y*}-lambda_t*len(y), where the penalty coefficient is updated as lambda_{t+1}=max(0, lambda_t + eta*(acc_t - acc_ref)), with acc_ref estimated from the base model's first training batch. The paper compares A-DLP against a static length penalty (S-DLP), L1-Exact, and L1-Max on DeepScaleR-1.5B using GRPO, evaluating on AIME2025, MATH, AMC, Olympiad-Bench, and Minerva. The central claims are that A-DLP reduces token length by more than 50% while sacrificing less than 0.04 accuracy, that it avoids the collapse observed under S-DLP, and that it does so without manual tuning of the penalty coefficient. Section 5.3 reports sensitivity experiments showing that the method degrades when acc_ref is 0.1 above or below the value used in the main experiments, and that small eta can lead to collapse while large eta leads to oscillatory lambda_t.

Significance. If the empirical claims hold, A-DLP is an attractive drop-in intervention: it requires no architectural change, integrates into existing GRPO pipelines, and its core update rule is simple and interpretable. The paper also provides a useful negative result by documenting that the static S-DLP baseline collapses after prolonged training, and the sensitivity analysis in Section 5.3 is more candid than is typical. However, the general significance is currently conditional: the headline result rests on a single model scale, a single training run, figures without numeric tables or error bars, and an acc_ref anchor whose noise is comparable to the perturbation that the paper shows can make the method fail. The evaluation is genuinely external to the method's own parameters, so there is no circularity, but the robustness evidence is not yet sufficient to support the broad 'consistently and dramatically' claim in the abstract and conclusion.

major comments (4)
  1. [§3.2 and §5.3] The reference accuracy acc_ref is the load-bearing hyperparameter of Eq. (3), but the paper estimates it from a single GRPO batch of 64 prompts (Section 4). For a Bernoulli accuracy near 0.62, the standard error of that estimate is about 0.06, and Section 5.3 shows that setting acc_ref just 0.1 too high (0.7) drives lambda_t to zero within about 10 steps with length reduction plateauing, while setting it 0.1 too low (0.5) causes the penalty to overgrow and the model to collapse. The tested miss of ±0.1 is only about 1.6 standard errors away from the point estimate, yet one direction stalls and the other collapses; the paper gives no bootstrap, multi-batch averaging, or other robust estimation procedure. This makes the headline >50% length reduction a conditional outcome rather than a robust property of A-DLP.
  2. [§5.1 and Figures 1–3] The central quantitative claims—more than 50% token-length reduction with less than 0.04 accuracy degradation, and reduction rates exceeding 55%—are stated only in prose and supported only by plotted curves. No numeric table reports per-dataset accuracy and length values, no error bars or confidence intervals are given, and no multiple-seed results are reported. Because the adaptive update in Eq. (3) is driven by a per-step accuracy estimate that Section 5.3 itself describes as noisy, the absence of any variance reporting is particularly consequential; the reader cannot tell whether the single displayed run is typical or favorable. The main results should be reported as tables with means and standard deviations over at least three seeds.
  3. [§4, §5.1, and Figure 1] The comparison to S-DLP is based on a single training trajectory under one static penalty value, plotted as a fitted curve over checkpoints, rather than a sweep over lambda. The paper asserts monotonic length reduction to justify reading this trajectory as the full accuracy-length trade-off curve, but Figure 2 shows S-DLP undergoing a sharp collapse around step 100; a curve that includes a collapsed policy does not represent the attainable frontier of static penalties, because a smaller static lambda might avoid the collapse and yield better trade-offs. The value of lambda used for S-DLP is not explicitly reported, so the baseline comparison is underspecified; the claim that A-DLP lies above and to the left of S-DLP should be established against multiple static lambda values.
  4. [§7 and §4] All experiments use a single backbone, DeepScaleR-1.5B-Preview, with one training run per configuration. The limitations section acknowledges that generalization to larger models is untested, but the abstract and conclusion assert that A-DLP consistently and dramatically reduces length and is broadly applicable. Without evidence across model scales, model families, or datasets with substantially different base accuracy levels, that consistency claim is not established; the acc_ref sensitivity in Section 5.3 suggests the method's behavior could change materially on a model whose accuracy distribution differs from the one used here.
minor comments (6)
  1. [Abstract] The phrase 'math mathematical reasoning' contains a duplicated word and should be corrected.
  2. [Section 7] The sentence beginning 'Although we expect A-DLP to generalize to larger models... scaling, it remains an important next step' is grammatically incomplete and should be rewritten.
  3. [Appendix Figure 8 caption] The caption reads 'For A-LDP' but should read 'For A-DLP'.
  4. [Section 5.3] The rule of thumb equating lambda_0 and eta is supported only by Figure 7, which varies both together; the independent variations in Figures 9–11 show a more complex interaction, so the guidance should be presented as an empirical heuristic rather than as a derived principle.
  5. [Section 4] The evaluation metric is described only as 'accuracy' over 16 sampled completions per question; the paper should state explicitly whether this is pass@1, majority vote, or some other aggregation.
  6. [Figure 4] The validation-accuracy panel for AIME2024 is not anchored to the base model's AIME2024 accuracy in the text, so the reader cannot assess the magnitude of the observed drop from the figure alone.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported length and accuracy outcomes are external measurements, not constructed from the update rule or reference accuracy.

full rationale

The paper's derivation chain is self-contained. Equation (3) defines an update rule for the penalty coefficient lambda_t from a measured accuracy gap, and Equation (4) is the reward function used during training; neither equation defines the evaluation metrics. The claimed >50% token-length reduction and <0.04 accuracy loss are measured on held-out benchmarks (AIME2025, MATH, AMC, Olympiad-Bench, Minerva) against the base model, not computed from acc_ref or lambda_t. The reference accuracy acc_ref=0.62 is estimated from the base model's first training batch, which serves as an external anchor, and Section 5.3 explicitly studies misspecification of this value. That sensitivity is an empirical robustness limitation, not circularity, because the reported result is not equivalent to the input by construction. The only author self-citation, [26], is used as one of several references for the 'overthinking' phenomenon in the introduction and related work; it is motivational rather than load-bearing for the proposed method. No fitted quantity is relabeled as a prediction, and no uniqueness theorem or ansatz is imported from prior work. The evaluation is benchmark-based and independent of the adaptive penalty parameters, so the central claim does not reduce to its own inputs.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

No new physical or model entities are introduced; the adaptive lambda_t is a scalar training parameter, not an invented entity. The central claim rests on the calibration of acc_ref and on the reliability of batch-level accuracy estimates, both of which the paper shows are sensitive knobs.

free parameters (3)
  • lambda_0 (initial length penalty coefficient) = 1e-3
    Initial value of the penalty coefficient in Eq. 2/4, chosen by hand; Appendix B and Figure 15 explore sensitivity.
  • eta (penalty learning rate) = 1e-3
    Update rate in Eq. 3; Section 5.3 and Figures 5, 12, 13 show that eta too large causes oscillation and eta too small causes collapse.
  • acc_ref (reference accuracy) = 0.62
    Set to the base model's first-batch training accuracy; Section 5.3 shows that +/-0.1 changes cause failure modes (no compression or model collapse).
assumptions (3)
  • domain assumption Per-step accuracy acct from a GRPO batch of 64 prompts is a reliable signal for adjusting the penalty coefficient.
    Eq. 3 updates lambda_t using batch-level accuracy; Section 5.3 notes that batch size 64 injects noise that affects the method's stability.
  • domain assumption Exact-match answer accuracy is a valid correctness measure for math reasoning.
    The reward in Eqs. 1-4 uses the indicator I{y=y*}; standard for math QA but ignores partial credit and answer-equivalence issues.
  • ad hoc to paper The S-DLP training trajectory is monotonic in generation length, so checkpoint sampling captures the full trade-off curve.
    Section 5.1 states this to justify fitting a curve through S-DLP checkpoints; if false, the comparison between A-DLP and S-DLP is not a fair Pareto comparison.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Thinking Fast and Right: Balancing Accuracy and Reasoning Length with Adaptive Rewards." pith.science (2026). https://pith.science/paper/OQFWU423

@misc{pith2026250518298,
  author       = {Pith},
  title        = {Pith review of: Thinking Fast and Right: Balancing Accuracy and Reasoning Length with Adaptive Rewards},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OQFWU423}},
  note         = {Machine review of arXiv:2505.18298}
}
read the original abstract

Large language models (LLMs) have demonstrated strong reasoning abilities in mathematical tasks, often enhanced through reinforcement learning (RL). However, RL-trained models frequently produce unnecessarily long reasoning traces -- even for simple queries -- leading to increased inference costs and latency. While recent approaches attempt to control verbosity by adding length penalties to the reward function, these methods rely on fixed penalty terms that are hard to tune and cannot adapt as the model's reasoning capability evolves, limiting their effectiveness. In this work, we propose an adaptive reward-shaping method that enables LLMs to "think fast and right" -- producing concise outputs without sacrificing correctness. Our method dynamically adjusts the reward trade-off between accuracy and response length based on model performance: when accuracy is high, the length penalty increases to encourage faster length reduction; when accuracy drops, the penalty is relaxed to preserve correctness. This adaptive reward accelerates early-stage length reduction while avoiding over-compression in later stages. Experiments across multiple datasets show that our approach consistently and dramatically reduces reasoning length while largely maintaining accuracy, offering a new direction for cost-efficient adaptive reasoning in large-scale language models.

Figures

Figures reproduced from arXiv: 2505.18298 by the authors.

Figure 1
Figure 1. Performance comparison of A-DLP with baseline methods. For the S-DLP, we plot checkpoints sampled every 20 training steps and fit a curve through them. Since both accuracy and generation length change monotonically during training under S-DLP, this trajectory captures the full accuracy–length trade-off. A-DLP consistently achieves better trade-offs, lying above and to the left of the S-DLP curve [PITH_FULL_IMAGE:fi… view at source ↗
Figure 2
Figure 2. Accuracy and average token length across training steps for A-DLP and S-DLP. The dotted line is the accuracy and token length for the base model before length reduction. For S-DLP, performance remains stable during the early training phase, but both accuracy and token length drop sharply around step 100, indicating model collapse due to excessive length penalization. In contrast, A-DLP exhibits stable convergence, w… view at source ↗
Figure 3
Figure 3. Token length of correct and incorrect responses before and after applying A-DLP. The reduction rates for both categories consistently exceed 55%. Correspondingly, the average response length steadily decreases and converges to the range of 1500–2000 tokens—representing more than a 50% reduction from the initial average of 5000 tokens. Meanwhile, the validation accuracy initially drops slightly as the model aggressiv… view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Training dynamics of A-DLP showing the accuracy gap between the current model and [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Training dynamics of λt and response length under different learning rates (η ∈ {10−2 , 10−3 , 10−4}). A larger learning rate causes λt to fluctuate more sharply due to sensitivity to noisy accuracy estimates, resulting in slower length reduction but eventually converg…
Figure 6
Figure 6. Figure 6: Training dynamics under different reference accuracies (accref ∈ {0.5, 0.62, 0.7}). When the reference accuracy is set too high (e.g., 0.7), λt quickly drops to zero and remains there, causing only a brief period of length reduction in the early steps and leading to co…
Figure 7
Figure 7. Figure 7: Training dynamics of using λ0 = η = 1e − 2, 1e − 3, 1e − 4 respectively. 6 Conclusion In this paper, we presented Adaptive Direct Length Penalty (A-DLP), a simple yet effective reward￾shaping method that dynamically adjusts the trade-off between reasoning accuracy and …
Figure 8
Figure 8. Figure 8 [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Accuracy and token length trade-off comparison of A-DLP on two sets of parameters [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Accuracy and token length comparison of A-DLP on two sets of parameters varying on the training steps [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: Training behavior using a larger learning rate η = 1e − 2. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_11.png]
Figure 12
Figure 12. Figure 12: Training behavior using a smaller learning rate η = 1e − 4 [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]
Figure 13
Figure 13. Figure 13: Training behavior using a reference accuracy accref = 0.5 that is much smaller than the accuracy of the base model [PITH_FULL_IMAGE:figures/full_fig_p014_13.png]
Figure 14
Figure 14. Figure 14: Training behavior using a reference accuracy accref = 0.7 that is much larger than the accuracy of the base model [PITH_FULL_IMAGE:figures/full_fig_p014_14.png]
Figure 15
Figure 15. Figure 15: Training dynamics of using different λ0 while keeping learning rate η = 1e − 3. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_15.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Learning to Reason Efficiently with Discounted Reinforcement Learning

    cs.LG 2025-10 conditional novelty 6.0 of 10

    Discounted correctness rewards in GRPO post-training shorten LLM reasoning traces by roughly 13-40% while matching pass@1 accuracy, supported by Blackwell-optimality analysis for restricted policy classes.

  2. Do Thinking Tokens Help or Trap? Towards More Efficient Large Reasoning Model

    cs.CL 2025-06 conditional novelty 6.0 of 10

    Suppressing 'thinking tokens' in a 1.5B reasoning model preserves accuracy while cutting tokens, and the proposed DuP-PO RL method improves both accuracy and efficiency over GRPO.

  3. Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey

    cs.AI 2025-07 conditional novelty 3.0 of 10

    A comprehensive review that categorizes methods for shortening and adaptively triggering chain-of-thought reasoning in large language models.

Reference graph

Works this paper leans on

35 extracted references · 4 canonical work pages · cited by 3 Pith papers

  1. [1]

    L1: Controlling how long a reasoning model thinks with reinforcement learning.arXiv preprint arXiv:2503.04697, 2025

    Pranjal Aggarwal and Sean Welleck. L1: Controlling how long a reasoning model thinks with reinforcement learning.arXiv preprint arXiv:2503.04697, 2025. 9

    arXiv 2025

  2. [2]

    Training language models to reason efficiently.arXiv preprint arXiv:2502.04463, 2025

    Daman Arora and Andrea Zanette. Training language models to reason efficiently.arXiv preprint arXiv:2502.04463, 2025

    arXiv 2025

  3. [3]

    Do not think that much for 2+ 3=? on the overthinking of o1-like llms.arXiv preprint arXiv:2412.21187, 2024

    Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, et al. Do not think that much for 2+ 3=? on the overthinking of o1-like llms.arXiv preprint arXiv:2412.21187, 2024

    arXiv 2024

  4. [4]

    Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    arXiv

  5. [5]

    competitive programming platform.https://codeforces.com/, 2025

    Codeforces. competitive programming platform.https://codeforces.com/, 2025

    work page 2025

  6. [6]

    Efficient reasoning models: A survey.arXiv preprint arXiv:2504.10903, 2025

    Sicheng Feng, Gongfan Fang, Xinyin Ma, and Xinchao Wang. Efficient reasoning models: A survey.arXiv preprint arXiv:2504.10903, 2025

    arXiv 2025

  7. [7]

    Omni-math: A universal olympiad level mathematic benchmark for large language models.arXiv preprint arXiv:2410.07985, 2024

    Bofei Gao, Feifan Song, Zhe Yang, Zefan Cai, Yibo Miao, Qingxiu Dong, Lei Li, Chenghao Ma, Liang Chen, Runxin Xu, et al. Omni-math: A universal olympiad level mathematic benchmark for large language models.arXiv preprint arXiv:2410.07985, 2024

    arXiv

  8. [8]

    Scaling laws for reward model overoptimization

    Leo Gao, John Schulman, and Jacob Hilton. Scaling laws for reward model overoptimization. InInternational Conference on Machine Learning, pages 10835–10866. PMLR, 2023

    2023

Show all 35 references
  1. [9]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  2. [10]

    Token-budget-aware llm reasoning.arXiv preprint arXiv:2412.18547, 2024

    Tingxu Han, Zhenting Wang, Chunrong Fang, Shiyu Zhao, Shiqing Ma, and Zhenyu Chen. Token-budget-aware llm reasoning.arXiv preprint arXiv:2412.18547, 2024

    2024 arXiv

  3. [11]

    Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems

    Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, et al. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. InProceedings of the 62nd Annua...

    2024

  4. [12]

    Measuring mathematical problem solving with the math dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021

    2021 arXiv

  5. [13]

    Thinkprune: Pruning long chain-of-thought of llms via reinforcement learning.arXiv preprint arXiv:2504.01296, 2025

    Bairu Hou, Yang Zhang, Jiabao Ji, Yujian Liu, Kaizhi Qian, Jacob Andreas, and Shiyu Chang. Thinkprune: Pruning long chain-of-thought of llms via reinforcement learning.arXiv preprint arXiv:2504.01296, 2025

    2025 arXiv

  6. [14]

    C3ot: Generating shorter chain-of- thought without compromising effectiveness

    Yu Kang, Xianghui Sun, Liangyu Chen, and Wei Zou. C3ot: Generating shorter chain-of- thought without compromising effectiveness. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 24312–24320, 2025

    2025

  7. [15]

    Training language models to self-correct via reinforcement learning.arXiv preprint arXiv:2409.12917, 2024

    Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, et al. Training language models to self-correct via reinforcement learning.arXiv preprint arXiv:2409.12917, 2024

    2024 arXiv

  8. [16]

    Let’s verify step by step

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. InThe Twelfth International Conference on Learning Representations, 2023

    2023

  9. [17]

    O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning.arXiv preprint arXiv:2501.12570, 2025

    Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning.arXiv preprint arXiv:2501.12570, 2025

    2025 arXiv

  10. [18]

    Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Li Erran Li, Raluca Ada Popa, and Ion Stoica

    Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y . Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepscaler: Sur- passing o1-preview with a 1.5b model by scaling rl. https://pretty-radio-b75.notion. site/...

    2025

  11. [19]

    Imitate, explore, and self-improve: A reproduction report on slow-thinking reasoning systems.arXiv preprint arXiv:2412.09413, 2024

    Yingqian Min, Zhipeng Chen, Jinhao Jiang, Jie Chen, Jia Deng, Yiwen Hu, Yiru Tang, Jiapeng Wang, Xiaoxue Cheng, Huatong Song, et al. Imitate, explore, and self-improve: A reproduction report on slow-thinking reasoning systems.arXiv preprint arXiv:2412.09413, 2024

    2024 arXiv

  12. [20]

    Self-training elicits concise reasoning in large language models.arXiv preprint arXiv:2502.20122, 2025

    Tergel Munkhbat, Namgyu Ho, Seo Hyun Kim, Yongjin Yang, Yujin Kim, and Se-Young Yun. Self-training elicits concise reasoning in large language models.arXiv preprint arXiv:2502.20122, 2025

    2025 arXiv

  13. [21]

    Concise thoughts: Impact of output length on llm reasoning and cost.arXiv preprint arXiv:2407.19825, 2024

    Sania Nayab, Giulio Rossolini, Marco Simoni, Andrea Saracino, Giorgio Buttazzo, Nicolamaria Manes, and Fabrizio Giacomelli. Concise thoughts: Impact of output length on llm reasoning and cost.arXiv preprint arXiv:2407.19825, 2024

    2024 arXiv

  14. [22]

    Learning to reason with llms.https://openai.com/index/learning-to-reason-with-llms/, 2024

    OpenAI. Learning to reason with llms.https://openai.com/index/learning-to-reason-with-llms/, 2024

    2024

  15. [23]

    Gpqa: A graduate-level google-proof q&a benchmark

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark. InFirst Conference on Language Modeling, 2024

    2024

  16. [24]

    Code llama: Open foundation models for code.arXiv preprint arXiv:2308.12950, 2023

    Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, et al. Code llama: Open foundation models for code.arXiv preprint arXiv:2308.12950, 2023

    2023 arXiv

  17. [25]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  18. [26]

    Between underthinking and overthinking: An empirical study of reasoning length and correctness in llms.arXiv preprint arXiv:2505.00127, 2025

    Jinyan Su, Jennifer Healey, Preslav Nakov, and Claire Cardie. Between underthinking and overthinking: An empirical study of reasoning length and correctness in llms.arXiv preprint arXiv:2505.00127, 2025

    2025 arXiv

  19. [27]

    Stop overthinking: A survey on efficient reasoning for large language models.arXiv preprint arXiv:2503.16419, 2025

    Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Hanjie Chen, et al. Stop overthinking: A survey on efficient reasoning for large language models.arXiv preprint arXiv:2503.16419, 2025

    2025 arXiv

  20. [28]

    Qwq-32b-preview.https://qwenlm.github.io/blog/qwq-32b-preview/, 2025

    Qwen Team. Qwq-32b-preview.https://qwenlm.github.io/blog/qwq-32b-preview/, 2025

    2025

  21. [29]

    Harnessing the reasoning economy: A survey of efficient reasoning for large language models.arXiv preprint arXiv:2503.24377, 2025

    Rui Wang, Hongru Wang, Boyang Xue, Jianhui Pang, Shudong Liu, Yi Chen, Jiahao Qiu, Derek Fai Wong, Heng Ji, and Kam-Fai Wong. Harnessing the reasoning economy: A survey of efficient reasoning for large language models.arXiv preprint arXiv:2503.24377, 2025

    2025 arXiv

  22. [30]

    Self-consistency improves chain of thought reasoning in language models.arXiv preprint arXiv:2203.11171, 2022

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models.arXiv preprint arXiv:2203.11171, 2022

    2022 arXiv

  23. [31]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  24. [32]

    Unlocking efficient long-to-short llm reasoning with model merging.arXiv preprint arXiv:2503.20641, 2025

    Han Wu, Yuxuan Yao, Shuqi Liu, Zehua Liu, Xiaojin Fu, Xiongwei Han, Xing Li, Hui-Ling Zhen, Tao Zhong, and Mingxuan Yuan. Unlocking efficient long-to-short llm reasoning with model merging.arXiv preprint arXiv:2503.20641, 2025

    2025 arXiv

  25. [33]

    Tokenskip: Controllable chain-of-thought compression in llms.arXiv preprint arXiv:2502.12067, 2025

    Heming Xia, Yongqi Li, Chak Tou Leong, Wenjie Wang, and Wenjie Li. Tokenskip: Controllable chain-of-thought compression in llms.arXiv preprint arXiv:2502.12067, 2025

    2025

  26. [34]

    Towards thinking-optimal scaling of test-time compute for llm reasoning.arXiv preprint arXiv:2502.18080, 2025

    Wenkai Yang, Shuming Ma, Yankai Lin, and Furu Wei. Towards thinking-optimal scaling of test-time compute for llm reasoning.arXiv preprint arXiv:2502.18080, 2025

    2025

  27. [35]

    Shorterbetter: Guiding reasoning models to find optimal inference length for efficient reasoning.arXiv preprint arXiv:2504.21370, 2025

    Jingyang Yi and Jiazheng Wang. Shorterbetter: Guiding reasoning models to find optimal inference length for efficient reasoning.arXiv preprint arXiv:2504.21370, 2025. 11 A Additional Experiments In Figure 8, we plot the A-DLP, L1-Max and L1-Exact with more configurations and f...

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.