Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

How Far Are We from Optimal Reasoning Efficiency?

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The efficiency gap between current reasoning models and the best achievable accuracy-per-token trade-off can be cut by more than half with a sparsely budgeted RL objective.

desk verdict A credible efficiency metric and training method built on a large study, but the 50% REG reduction leans on an uncounted fallback answer stage. read the letter →

arxiv 2506.07104 v2 pith:IDZ7AHMZ submitted 2025-06-08 cs.CL cs.AI

Jiaxuan Gao , Shu Yan , Qixin Tan , Lu Yang , Shusheng Xu , Wei Fu , Zhiyu Mei , Kaifeng Lyu
show 1 more author
Yi Wu
This is my paper
classification cs.CLcs.AI
keywords reasoningefficiencytokenbudgetslargemodelschain-of-thoughtreinforcementlearningaccuracy-lengthtrade-offfrontierGap
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the accuracy-versus-token trade-off of fine-tuned large reasoning models can be measured by a single number, the Reasoning Efficiency Gap (REG), and that this gap can be cut by more than half with a reinforcement-learning objective evaluated at only a handful of token budgets. The authors build empirical reasoning efficiency frontiers from many fine-tuned variants of four model families and show that current methods sit far from these frontiers: they either shorten responses at the cost of accuracy or stay verbose under tight budgets. Their REO-RL algorithms optimize a length-constrained reward averaged over exponentially spaced budgets, approximating the full budget integral via the trapezoidal rule. Across DeepSeek-R1-Distill-Qwen-1.5B/7B and Qwen3-4B/8B, REO-RL reduces REG by at least 50 percent while holding accuracy within a few points, and for Qwen3 models it reaches the empirical frontier under a 16K budget. If true, this means much of the verbosity of current reasoning models is removable without a real accuracy penalty.

What carries the argument

The load-bearing objects are the reasoning efficiency frontier, the pointwise maximum over fine-tuned models of accuracy at each token budget, and REG, the area between a model's budget-accuracy curve and that frontier. To optimize REG cheaply, REO-RL rewrites the sum over all budgets as an RL objective with dense rewards, then approximates it by numerical integration: coefficients $c_i = (L_{i+1}-L_{i-1})/2$ weight the reward at each selected budget $L_i$, following the trapezoidal rule. Budgets are chosen either greedily from the known frontier (Oracle) or by exponential spacing $L_i = L_{\min} (L_{\max}/L_{\min})^{(i-1)/N}$ (Exp), after observing that the oracle-selected budgets roughly follow an exponential pattern. The exponential scheme with $N=5$ budgets approximates the full objective with error below 1% and makes the RL training tractable.

What would settle it

Take a fixed test set and compare REG computed under the paper's strict forced-answer prompt with REG computed under a looser protocol that lets the model write a short summary after the cutoff. If rankings change substantially, or if a model adversarially trained to use the answer phase for extra reasoning shows a much smaller gap under the strict protocol, the frontier measurement is not robust. A simpler check: run REO-RL with N=5 exponential budgets but vary the answer-phase prompt text and see whether the reported at-least-50% REG reduction survives.

Watch

Extended reading notes

Core claim

The paper's central claim is that reasoning efficiency has a measurable frontier: for each token budget there is a best accuracy achievable by any fine-tuned model from a family, and current methods deviate from this frontier in a systematic, quantifiable way. The paper introduces the Reasoning Efficiency Gap (REG) as the summed difference between the frontier accuracy and the model's accuracy over all budgets, making REG a single number that captures both length and accuracy. It then claims that REG can be optimized directly: REO-RL approximates the full-budget objective with the trapezoidal rule over a few token budgets, and a sparse set of exponentially spaced budgets suffices. The empirical claim is that this training recipe reduces REG by at least 50% on all four model families while keeping accuracy nearly unchanged, and for Qwen3 models it reaches the empirical frontier under a 16K budget. The paper also reports that perfectly matching the frontier remains an open problem, since visible gaps remain under tight budgets on the DeepSeek-distilled families.

Load-bearing premise

The entire efficiency frontier and every REG number depend on the fallback rule that a model cut off at budget L is forced to answer straight from the truncated trace; if that forced answer secretly lets the model do extra thinking, the measured frontier and REG are systematically off.

Editorial extensions

If this is right

  • Any fine-tuned reasoning model can now be scored by one number, REG, that combines accuracy and length; low-REG methods are precisely those that keep accuracy while shortening responses.
  • Training on a sparse set of exponentially spaced token budgets is sufficient: raising the number of budgets from 5 to 10 slows convergence and destabilizes training, so the sparse choice is not merely a compute saving.
  • Across DeepSeek-R1-Distill-Qwen-1.5B/7B and Qwen3-4B/8B, REO-RL cuts REG by at least 50 percent relative to baselines, with Qwen3-4B/8B reaching the empirical frontier under a 16K token budget at the cost of only a few accuracy points.
  • Methods that shorten aggressively, such as hard token budgets or preference pairs built from shortest correct answers, fail on the frontier criterion because they sacrifice accuracy, and length-reward methods can even enlarge the gap on Qwen3 models.
  • Matching the empirical frontier exactly remains open; the largest residual inefficiency sits at tight token budgets for the DeepSeek-distilled model families.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: REG could serve as a portable reporting standard for reasoning-efficiency studies, since it is anchored to a frontier built from each model family rather than to one benchmark's absolute numbers.
  • Beyond the paper: the exponential budget-selection rule suggests the frontier is smooth and roughly concave; the same trapezoidal approximation may transfer to other settings where reward evaluations are expensive, such as test-time compute allocation or multi-agent reasoning.
  • Beyond the paper: a natural next experiment is to use REG as a decoder-time selection criterion, picking among candidate traces at different lengths; the paper's frontier construction provides exactly the data such a controller would need.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces the reasoning efficiency frontier, an empirical upper envelope of accuracy under token budgets obtained by fine-tuning base LRMs with diverse methods, and a Reasoning Efficiency Gap (REG) metric that integrates the difference between a model's budget-accuracy curve and the frontier. It then proposes REO-RL, a family of RL algorithms that optimize a trapezoidal approximation of the integrated length-constrained reward over a small set of token budgets, with exponentially spaced budgets as the practical variant. Experiments on four LRMs (DeepSeek-R1-Distill-Qwen-1.5B/7B and Qwen3-4B/8B) on AMC/AIME/Minerva benchmarks report that REO-RL preserves accuracy while shortening responses and reduces REG by at least 50% relative to baselines, and that Qwen3 models trained with REO-RL approach the empirical frontier under a 16K budget.

Significance. If the headline result holds, the paper makes a useful contribution: REG is a plausible single-number summary of the accuracy-length trade-off, and REO-RL's use of exponentially spaced token budgets is a practical way to approximate dense budget-aware RL training. The paper's empirical scope is broad, covering four model families, several baseline families, and four benchmarks, and the derivation of the trapezoidal integration objective is coherent. At the same time, the main quantitative claims are conditional on an unmeasured property of the fallback evaluation and on single-seed point estimates, so the significance is contingent on resolving those issues.

major comments (3)
  1. [Sec. 4.1, Eq. (2); Appendix A; Sec. 5.2, Eq. (10)] The fallback answer generation Answer(pi_theta, x, y:L) is not counted in the reported lengths or in REG, and REO-RL's training objective is exactly the expected value of this fallback reward (Eq. 10). The paper asserts in Appendix A that its prompt forces the model to produce a plausible answer with minimal additional reasoning, but it provides no measurement of fallback token counts and no no-reasoning control. Because REO-RL is the only method whose policy gradient directly optimizes the fallback reward, it has the strongest incentive to defer reasoning into the uncounted continuation; if it does, the headline length reductions and the >=50% REG reductions are inflated. Please report the distribution of fallback continuation lengths for all methods, recompute REG and length with those tokens included, and add a control where the answer is extracted from the truncated trace without further generation.
  2. [Sec. 4.2, Eq. (5); Sec. 6.2, Table 1] The reasoning efficiency frontier is an upper envelope over single-seed runs of the authors' own fine-tuning configurations, and every REG value in Table 1 is a point estimate with no error bars, confidence intervals, or significance statements. The central claim that REO-RL consistently reduces REG by at least 50% across all evaluated LRMs is therefore not statistically supported. Please provide multiple seeds for the main comparisons and for the frontier construction, or at minimum bootstrap confidence intervals over the 32 sampled responses used for accuracy.
  3. [Sec. 5.2, Eqs. (8)-(10)] The claimed equivalence between the trapezoidal approximation and the REO-RL objective is not exact as written. Equation (8) includes the endpoint term (L_1/2) * J(D, theta, 0), but Eq. (10) sums only i = 1,...,N+1 and omits this term. The text says the objective follows Eq. (9), so either the L=0 term is intentionally dropped (which should be stated and justified numerically) or the equation needs correction. This is a local derivation issue, but it should be fixed before publication.
minor comments (5)
  1. [Definition 4.1, Eq. (5)] Equation (5) uses D_t in J(D_t, theta, L) while the definition statement and Eq. (4) use D; please clarify whether the frontier is evaluated on the training distribution, the test benchmarks, or both.
  2. [Table 1] In the average columns, REG is reported only as a percentage change relative to vanilla RL or the base LRM, but the absolute REG value is not shown; please state explicitly which baseline each relative change uses and include the absolute REG in the average row.
  3. [Table 3] The naming is inconsistent: the method is called REO-RL (Q-Spec) in the main text and Table 1, but REO-RL (Task-Specific) in Table 3; please unify the terminology.
  4. [Table 1, Qwen3 rows] Several 'RL w. Token Budget' rows for Qwen3-4B/8B report average lengths much larger than the training budget (e.g., Qwen3-4B with Token Budget=1K has length 22992.1), which appears to contradict the hard-budget training protocol; please explain how these lengths are produced and whether the reported length is measured with or without truncation.
  5. [Sec. 6.3, Table 2] The comparison with frontier LRMs uses only 71 questions and reports only correct-response lengths; please state the accuracy on that subset for each model, since differences in accuracy could confound the length comparison.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation; the REG reduction is an empirical optimization outcome, though the metric/frontier are in-paper constructions and one training framework is self-cited.

full rationale

Walking the claimed derivation chain: the reasoning efficiency frontier (Eq. 5) is an empirical upper envelope over fine-tuned baseline models; REG (Eq. 6) is the area between that envelope and a model's budget-accuracy curve. REO-RL's objective (Eq. 10) is a trapezoidal approximation of sum_L J(D,theta,L), which is exactly the theta-dependent part of REG, so minimizing the training objective is equivalent to minimizing an approximation of REG. The reported >=50% REG reduction is therefore an outcome of optimizing the metric, not an independent first-principles prediction. However, the reduction is not forced by construction: training data (DeepScaleR/AReaL) differs from evaluation benchmarks (AMC/AIME/Minerva), the N=5-budget approximation has nonzero error, and the frontier is constructed from baseline methods, not from REO-RL itself. The main near-circularity risks are external-validity issues rather than formal circularity. First, the fallback Answer() in Eq. 2 is asserted to introduce only 'a minor additional token cost' and to prevent budget-aware reasoning (Appendix A), but no measurement of fallback token lengths is reported; if the fallback permits uncounted reasoning, REG overstates efficiency for all methods, and REO-RL—being trained on exactly this reward—would be the most likely to exploit it. Second, AReaL [35] is a self-cited framework and starting checkpoint, but it is an implementation detail and not load-bearing for the derivation. Neither issue makes Eq. 10 equal to Eq. 6 by construction; the central empirical claims could in principle fail. Thus no formal circular step meets the required evidentiary bar.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central empirical claims rest on a small number of modeling choices: the truncated-trace fallback, the finite-model empirical frontier, the trapezoidal approximation, and the hand-selected budget grids. No free physical parameters are fit; the free numerical hyperparameters (N, budget endpoints, REG grid) are implementation choices that influence the reported REG values.

free parameters (3)
  • Number of token budgets N = 5
    Set to 5 because Fig. 3 shows approximation error below 1% for N>=5; this choice controls training cost and the fidelity of the REO-RL objective to the full efficiency objective.
  • REG numerical integration grid = {64i | 0<=i<16} union {1024i | 1<=i<=16} with Lmax=16K
    The REG values in the experiments are computed by numerical integration over this hand-picked grid rather than the exact sum in Eq. 6; changing the grid changes REG values.
  • Exponential budget schedule endpoints (Lmin, Lmax) = not fully stated; Lmax=24376 for 1.5B and 32768 for others in training, Lmax=16K for REG
    REO-RL (Exp) selects L_i = Lmin * (Lmax/Lmin)^((i-1)/N); the endpoints are implementation choices not derived from theory.
assumptions (4)
  • standard math The trapezoidal rule gives a close approximation to the discrete sum of length-constrained rewards (Eq. 7 vs Eq. 8).
    The REO-RL objective replaces the sum over all L=1..Lmax with weighted rewards at N selected budgets; the approximation error is measured empirically in Fig. 3(b), not proven.
  • domain assumption Truncated-trace answer generation (Answer(pi, x, y:L)) faithfully measures accuracy under a token budget L.
    Sec. 4.1 Eq. 2 defines the fallback; if prompting for an answer from a truncated trace lets the model do extra budget-aware reasoning, the frontier and REG are biased. The paper contrasts this with MRT/BRPO's looser summarization strategy in Appendix A.
  • domain assumption The finite set of fine-tuned models hat-Theta yields an empirical frontier that is a useful proxy for the true optimal frontier.
    Definition 4.1 explicitly notes the empirical frontier is a lower bound of the optimal frontier; the strength of all 'gap' claims depends on how representative hat-Theta is.
  • ad hoc to paper Oracle-selected token budgets approximately follow an exponential pattern, justifying REO-RL (Exp).
    Sec. 5.2 and Fig. 3(a) report this as an empirical observation; REO-RL (Exp) relies on it when the frontier is unknown.

how reviews work

0 comments
Cite this review

Pith. "Pith review of How Far Are We from Optimal Reasoning Efficiency?." pith.science (2026). https://pith.science/paper/IDZ7AHMZ

@misc{pith2026250607104,
  author       = {Pith},
  title        = {Pith review of: How Far Are We from Optimal Reasoning Efficiency?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IDZ7AHMZ}},
  note         = {Machine review of arXiv:2506.07104}
}
read the original abstract

Large Reasoning Models (LRMs) demonstrate remarkable problem-solving capabilities through extended Chain-of-Thought (CoT) reasoning but often produce excessively verbose and redundant reasoning traces. This inefficiency incurs high inference costs and limits practical deployment. While existing fine-tuning methods aim to improve reasoning efficiency, assessing their efficiency gains remains challenging due to inconsistent evaluations. In this work, we introduce the reasoning efficiency frontiers, empirical upper bounds derived from fine-tuning base LRMs across diverse approaches and training configurations. Based on these frontiers, we propose the Reasoning Efficiency Gap (REG), a unified metric quantifying deviations of any fine-tuned LRMs from these frontiers. Systematic evaluation on challenging mathematical benchmarks reveals significant gaps in current methods: they either sacrifice accuracy for short length or still remain inefficient under tight token budgets. To reduce the efficiency gap, we propose REO-RL, a class of Reinforcement Learning algorithms that minimizes REG by targeting a sparse set of token budgets. Leveraging numerical integration over strategically selected budgets, REO-RL approximates the full efficiency objective with low error using a small set of token budgets. Through systematic benchmarking, we demonstrate that our efficiency metric, REG, effectively captures the accuracy-length trade-off, with low-REG methods reducing length while maintaining accuracy. Our approach, REO-RL, consistently reduces REG by >=50 across all evaluated LRMs and matching Qwen3-4B/8B efficiency frontiers under a 16K token budget with minimal accuracy loss. Ablation studies confirm the effectiveness of our exponential token budget strategy. Finally, our findings highlight that fine-tuning LRMs to perfectly align with the efficiency frontiers remains an open challenge.

Figures

Figures reproduced from arXiv: 2506.07104 by the authors.

Figure 1
Figure 1. Reasoning Efficiency Frontiers & Reasoning Efficiency Gap. (a) Starting from a base LRM πθbase , we apply diverse fine-tuning strategies to obtain a large amount of LRMs. (b) We then compute the best achievable accuracy across varying token budgets to obtain the reasoning efficiency frontiers (Eq. 5). Reasoning Efficiency Gap (REG) is a unified metric that captures both accuracy and length by measuring the area betw… view at source ↗
Figure 2
Figure 2. Reasoning efficiency frontiers for DeepSeek-R1-Distill-Qwen-1.5B/7B and Qwen3-4B/8B. Comparing the two classes of models, Qwen3 models have higher accuracy and are closer to the efficiency frontiers than DeepSeek-R1-Distill-Qwen models. we start from RL-fine-tuned versions and further fine-tune them using the strategies above. The evaluation is carried out on a set of challenging mathematical reasoning benchmarks. M… view at source ↗
Figure 3
Figure 3. Selection of Token Budgets in REO-RL. (a) Token budgets selected roughly follow an [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Performance comparison of REO-RL (Exp) with representative baseline methods [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Reasoning efficiency frontiers of DeepSeek-R1-Distill-Qwen-1.5B/7B and Qwen3-4B/8B. Qwen3 models obtain higher accuracy. Interestingly, DeepSeek-R1-Distill-Qwen-7B could reason more efficiently than Qwen3 models under the 4K token budget [PITH_FULL_IMAGE:figures/full_…
Figure 6
Figure 6. Figure 6: REG effectively captures the trade-off between accuracy and response length. Achiev￾ing a low REG require both competitive accuracy and short response length. By minimizing the efficiency gap, REO-RL outperforms baselines in terms of reasoning efficiency. Training Deta…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey

    cs.AI 2025-07 conditional novelty 3.0 of 10

    A comprehensive review that categorizes methods for shortening and adaptively triggering chain-of-thought reasoning in large language models.

Reference graph

Works this paper leans on

60 extracted references · 5 canonical work pages · cited by 1 Pith paper

  1. [1]

    L1: Controlling how long a reasoning model thinks with reinforcement learning.arXiv preprint arXiv:2503.04697, 2025

    Pranjal Aggarwal and Sean Welleck. L1: Controlling how long a reasoning model thinks with reinforcement learning.arXiv preprint arXiv:2503.04697, 2025

    arXiv 2025

  2. [2]

    Training language models to reason efficiently.arXiv preprint arXiv:2502.04463, 2025

    Daman Arora and Andrea Zanette. Training language models to reason efficiently.arXiv preprint arXiv:2502.04463, 2025

    arXiv 2025

  3. [3]

    Huatuogpt-o1, towards medical complex reasoning with llms, 2024

    Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang, Wanlong Liu, Rongsheng Wang, Jianye Hou, and Benyou Wang. Huatuogpt-o1, towards medical complex reasoning with llms, 2024. URLhttps://arxiv.org/abs/2412.18925

    arXiv 2024

  4. [5]

    Do not think that much for 2+3=? on the overthinking of o1-like llms, 2025

    Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. Do not think that much for 2+3=? on the overthinking of o1-like llms, 2025. URL https://arxiv.org/abs/2412.21187

    arXiv 2025

  5. [6]

    Thinkless: Llm learns when to think, 2025

    Gongfan Fang, Xinyin Ma, and Xinchao Wang. Thinkless: Llm learns when to think, 2025. URLhttps://arxiv.org/abs/2505.13379

    arXiv 2025

  6. [7]

    Efficiently serving llm reasoning programs with certaindex.arXiv preprint arXiv:2412.20993, 2024

    Yichao Fu, Junda Chen, Siqi Zhu, Zheyu Fu, Zhongdongming Dai, Aurick Qiao, and Hao Zhang. Efficiently serving llm reasoning programs with certaindex.arXiv preprint arXiv:2412.20993, 2024

    arXiv 2024

  7. [8]

    Reasoning without self-doubt: More efficient chain-of-thought through certainty probing

    Yichao Fu, Junda Chen, Yonghao Zhuang, Zheyu Fu, Ion Stoica, and Hao Zhang. Reasoning without self-doubt: More efficient chain-of-thought through certainty probing. InICLR 2025 Workshop on Foundation Models in the Wild, 2025. URLhttps://openreview.net/forum? id=wpK4IMJfdX

    2025

  8. [9]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    arXiv 2025

Show all 60 references
  1. [10]

    Token-budget-aware llm reasoning.arXiv preprint arXiv:2412.18547, 2024

    Tingxu Han, Chunrong Fang, Shiyu Zhao, Shiqing Ma, Zhenyu Chen, and Zhenting Wang. Token-budget-aware llm reasoning.arXiv preprint arXiv:2412.18547, 2024

    2024 arXiv

  2. [11]

    Skywork open reasoner series

    Jujie He, Jiacai Liu, Chris Yuhao Liu, Rui Yan, Chaojie Wang, Peng Cheng, Xiaoyu Zhang, Fuxiang Zhang, Jiacheng Xu, Wei Shen, Siyuan Li, Liang Zeng, Tianwen Wei, Cheng Cheng, Yang Liu, and Yahui Zhou. Skywork open reasoner series. https://capricious-hydrogen-41c.notion.site/ S...

    2025

  3. [12]

    Thinkprune: Pruning long chain-of-thought of llms via reinforcement learning.arXiv preprint arXiv:2504.01296, 2025

    Bairu Hou, Yang Zhang, Jiabao Ji, Yujian Liu, Kaizhi Qian, Jacob Andreas, and Shiyu Chang. Thinkprune: Pruning long chain-of-thought of llms via reinforcement learning.arXiv preprint arXiv:2504.01296, 2025

    2025 arXiv

  4. [13]

    Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model, 2025

    Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum. Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model, 2025. URLhttps://arxiv.org/abs/2503.24290

    2025 arXiv

  5. [14]

    Efficient test-time scaling via self-calibration.arXiv preprint arXiv:2503.00031, 2025

    Chengsong Huang, Langlin Huang, Jixuan Leng, Jiacheng Liu, and Jiaxin Huang. Efficient test-time scaling via self-calibration.arXiv preprint arXiv:2503.00031, 2025

    2025 arXiv

  6. [15]

    Think only when you need with large hybrid-reasoning models, 2025

    Lingjie Jiang, Xun Wu, Shaohan Huang, Qingxiu Dong, Zewen Chi, Li Dong, Xingxing Zhang, Tengchao Lv, Lei Cui, and Furu Wei. Think only when you need with large hybrid-reasoning models, 2025. URLhttps://arxiv.org/abs/2505.14631

    2025 arXiv

  7. [16]

    C3ot: Generating shorter chain-of- thought without compromising effectiveness.arXiv preprint arXiv:2412.11664, 2024

    Yu Kang, Xianghui Sun, Liangyu Chen, and Wei Zou. C3ot: Generating shorter chain-of- thought without compromising effectiveness.arXiv preprint arXiv:2412.11664, 2024. 13

    2024 arXiv

  8. [17]

    Solving quantitative reasoning problems with language models, 2022

    Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra. Solving quantitative reasoning problems with language models, ...

    2022 arXiv

  9. [18]

    Limr: Less is more for rl scaling, 2025

    Xuefeng Li, Haoyang Zou, and Pengfei Liu. Limr: Less is more for rl scaling, 2025. URL https://arxiv.org/abs/2502.11886

    2025 arXiv

  10. [19]

    Thinkswitcher: When to think hard, when to think fast, 2025

    Guosheng Liang, Longguang Zhong, Ziyi Yang, and Xiaojun Quan. Thinkswitcher: When to think hard, when to think fast, 2025. URLhttps://arxiv.org/abs/2505.14183

    2025 arXiv

  11. [20]

    Reward-guided speculative decoding for efficient llm reasoning.arXiv preprint arXiv:2501.19324, 2025

    Baohao Liao, Yuhui Xu, Hanze Dong, Junnan Li, Christof Monz, Silvio Savarese, Doyen Sahoo, and Caiming Xiong. Reward-guided speculative decoding for efficient llm reasoning.arXiv preprint arXiv:2501.19324, 2025

    2025 arXiv

  12. [21]

    Can language models learn to skip steps?arXiv preprint arXiv:2411.01855, 2024

    Tengxiao Liu, Qipeng Guo, Xiangkun Hu, Cheng Jiayang, Yue Zhang, Xipeng Qiu, and Zheng Zhang. Can language models learn to skip steps?arXiv preprint arXiv:2411.01855, 2024

    2024 arXiv

  13. [22]

    Fin-r1: A large language model for financial reasoning through reinforcement learning, 2025

    Zhaowei Liu, Xin Guo, Fangqi Lou, Lingfeng Zeng, Jinyi Niu, Zixuan Wang, Jiajie Xu, Weige Cai, Ziwei Yang, Xueqian Zhao, Chao Li, Sheng Xu, Dezhi Chen, Yun Chen, Zuo Bai, and Liwen Zhang. Fin-r1: A large language model for financial reasoning through reinforcement learning, 20...

    2025

  14. [23]

    Adacot: Pareto-optimal adaptive chain-of-thought triggering via reinforcement learning, 2025

    Chenwei Lou, Zewei Sun, Xinnian Liang, Meng Qu, Wei Shen, Wenqi Wang, Yuntao Li, Qing- ping Yang, and Shuangzhi Wu. Adacot: Pareto-optimal adaptive chain-of-thought triggering via reinforcement learning, 2025. URLhttps://arxiv.org/abs/2505.11896

    2025 arXiv

  15. [24]

    O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning.arXiv preprint arXiv:2501.12570, 2025

    Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning.arXiv preprint arXiv:2501.12570, 2025

    2025 arXiv

  16. [25]

    Deepcoder: A fully open-source 14b coder at o3-mini level

    Michael Luo, Sijun Tan, Roy Huang, Ameen Patel, Alpay Ariyak, Qingyang Wu, Xiaox- iang Shi, Rachel Xin, Colin Cai, Maurice Weber, Ce Zhang, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepcoder: A fully open-source 14b coder at o3-mini level. https://pretty-radio-b75.notion....

    2025

  17. [26]

    Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica

    Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y . Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl. https://pretty-radio-b75.notion.site/ DeepScaleR-Surpas...

  18. [27]

    Cot-valve: Length-compressible chain-of-thought tuning.arXiv preprint arXiv:2502.09601, 2025

    Xinyin Ma, Guangnian Wan, Runpeng Yu, Gongfan Fang, and Xinchao Wang. Cot-valve: Length-compressible chain-of-thought tuning.arXiv preprint arXiv:2502.09601, 2025

    2025 arXiv

  19. [28]

    Simpo: Simple preference optimization with a reference-free reward.Advances in Neural Information Processing Systems, 37:124198–124235, 2024

    Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with a reference-free reward.Advances in Neural Information Processing Systems, 37:124198–124235, 2024

    2024

  20. [29]

    s1: Simple test-time scaling, 2025

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling, 2025. URLhttps://arxiv.org/abs/2501.19393

    2025 arXiv

  21. [30]

    Self-training elicits concise reasoning in large language models.arXiv preprint arXiv:2502.20122, 2025

    Tergel Munkhbat, Namgyu Ho, Seohyun Kim, Yongjin Yang, Yujin Kim, and Se-Young Yun. Self-training elicits concise reasoning in large language models.arXiv preprint arXiv:2502.20122, 2025

    2025 arXiv

  22. [31]

    Learning to reason with llms

    OpenAI. Learning to reason with llms. urlhttps://openai.com/index/learning-to-reason-with-llms/. Accessed: 15 March 2025

    2025

  23. [32]

    Thoughtterminator: Benchmarking, calibrating, and mitigating overthinking in reasoning models, 2025

    Xiao Pu, Michael Saxon, Wenyue Hua, and William Yang Wang. Thoughtterminator: Benchmarking, calibrating, and mitigating overthinking in reasoning models, 2025. URL https://arxiv.org/abs/2504.13367. 14

    2025 arXiv

  24. [33]

    Optimizing anytime reasoning via budget relative policy optimization, 2025

    Penghui Qi, Zichen Liu, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Optimizing anytime reasoning via budget relative policy optimization, 2025. URL https://arxiv.org/ abs/2505.13438

    2025

  25. [34]

    Optimizing test-time compute via meta reinforcement fine-tuning.arXiv preprint arXiv:2503.07572, 2025

    Yuxiao Qu, Matthew YR Yang, Amrith Setlur, Lewis Tunstall, Edward Emanuel Beeching, Ruslan Salakhutdinov, and Aviral Kumar. Optimizing test-time compute via meta reinforcement fine-tuning.arXiv preprint arXiv:2503.07572, 2025

    2025 arXiv

  26. [35]

    Areal: Ant reasoning rl

    Ant Research RL Lab. Areal: Ant reasoning rl. https://github.com/inclusionAI/AReaL, 2025

    2025

  27. [36]

    Hawkeye:efficient reasoning with model collaboration, 2025

    Jianshu She, Zhuohao Li, Zhemin Huang, Qi Li, Peiran Xu, Haonan Li, and Qirong Ho. Hawkeye:efficient reasoning with model collaboration, 2025. URL https://arxiv.org/ abs/2504.00424

    2025 arXiv

  28. [37]

    Vlm-r1: A stable and generalizable r1-style large vision-language model.arXiv preprint arXiv:2504.07615, 2025

    Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, Ruochen Xu, and Tiancheng Zhao. Vlm-r1: A stable and generalizable r1-style large vision-language model.arXiv preprint arXiv:2504.07615, 2025

    2025 arXiv

  29. [38]

    Dast: Difficulty-adaptive slow-thinking for large reasoning models

    Yi Shen, Jian Zhang, Jieyun Huang, Shuming Shi, Wenjing Zhang, Jiangze Yan, Ning Wang, Kai Wang, and Shiguo Lian. Dast: Difficulty-adaptive slow-thinking for large reasoning models. arXiv preprint arXiv:2503.04472, 2025

    2025

  30. [39]

    Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256, 2024

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256, 2024

    2024 arXiv

  31. [40]

    Stop overthinking: A survey on efficient reasoning for large language models.arXiv preprint arXiv:2503.16419, 2025

    Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Hanjie Chen, Xia Hu, et al. Stop overthinking: A survey on efficient reasoning for large language models.arXiv preprint arXiv:2503.16419, 2025

    2025 arXiv

  32. [41]

    Fast best-of-n decoding via speculative rejection.arXiv preprint arXiv:2410.20290, 2024

    Hanshi Sun, Momin Haider, Ruiqi Zhang, Huitao Yang, Jiahao Qiu, Ming Yin, Mengdi Wang, Peter Bartlett, and Andrea Zanette. Fast best-of-n decoding via speculative rejection.arXiv preprint arXiv:2410.20290, 2024

    2024 arXiv

  33. [42]

    Confidence improves self-consistency in llms.arXiv preprint arXiv:2502.06233, 2025

    Amir Taubenfeld, Tom Sheffer, Eran Ofek, Amir Feder, Ariel Goldstein, Zorik Gekhman, and Gal Yona. Confidence improves self-consistency in llms.arXiv preprint arXiv:2502.06233, 2025

    2025

  34. [43]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms.arXiv preprint arXiv:2501.12599, 2025

    2025 arXiv

  35. [44]

    Learning when to think: Shaping adaptive reasoning in r1-style models via multi-stage rl,

    Songjun Tu, Jiahao Lin, Qichao Zhang, Xiangyu Tian, Linjing Li, Xiangyuan Lan, and Dongbin Zhao. Learning when to think: Shaping adaptive reasoning in r1-style models via multi-stage rl,

  36. [45]

    Self-consistency improves chain of thought reasoning in language models

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. InThe Eleventh International Conference on Learning Representations, 2023

    2023

  37. [46]

    Reinforcement learning for reasoning in large language models with one training example, 2025

    Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Lucas Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, Weizhu Chen, Shuohang Wang, Simon Shaolei Du, and Yelong Shen. Reinforcement learning for reasoning in large language models with one training example, 202...

    2025 arXiv

  38. [47]

    Tokenskip: Controllable chain-of-thought compression in llms.arXiv preprint arXiv:2502.12067, 2025

    Heming Xia, Yongqi Li, Chak Tou Leong, Wenjie Wang, and Wenjie Li. Tokenskip: Controllable chain-of-thought compression in llms.arXiv preprint arXiv:2502.12067, 2025

    2025

  39. [48]

    Scalable chain of thoughts via elastic reasoning, 2025

    Yuhui Xu, Hanze Dong, Lei Wang, Doyen Sahoo, Junnan Li, and Caiming Xiong. Scalable chain of thoughts via elastic reasoning, 2025. URLhttps://arxiv.org/abs/2505.05315. 15

    2025 arXiv

  40. [49]

    Qwen3 technical report, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jia...

    2025 arXiv

  41. [50]

    Think when you need: Self-adaptive chain-of-thought learning, 2025

    Junjie Yang, Ke Lin, and Xing Yu. Think when you need: Self-adaptive chain-of-thought learning, 2025. URLhttps://arxiv.org/abs/2504.03234

    2025 arXiv

  42. [51]

    Towards thinking-optimal scaling of test-time compute for llm reasoning, 2025

    Wenkai Yang, Shuming Ma, Yankai Lin, and Furu Wei. Towards thinking-optimal scaling of test-time compute for llm reasoning, 2025. URLhttps://arxiv.org/abs/2502.18080

    2025

  43. [52]

    Demystifying long chain-of-thought reasoning in llms.arXiv preprint arXiv:2502.03373, 2025

    Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying long chain-of-thought reasoning in llms.arXiv preprint arXiv:2502.03373, 2025

    2025 arXiv

  44. [53]

    Finemedlm-o1: Enhancing the medical reasoning ability of llm from supervised fine-tuning to test-time training, 2025

    Hongzhou Yu, Tianhao Cheng, Ying Cheng, and Rui Feng. Finemedlm-o1: Enhancing the medical reasoning ability of llm from supervised fine-tuning to test-time training, 2025. URL https://arxiv.org/abs/2501.09213

    2025 arXiv

  45. [54]

    Distilling system 2 into system 1.arXiv preprint arXiv:2407.06023, 2024

    Ping Yu, Jing Xu, Jason Weston, and Ilia Kulikov. Distilling system 2 into system 1.arXiv preprint arXiv:2407.06023, 2024

    2024 arXiv

  46. [55]

    Dapo: An open-source llm reinforcement learning system at scale, 2025

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wa...

    2025 arXiv

  47. [56]

    Z1: Efficient test-time scaling with code, 2025

    Zhaojian Yu, Yinghao Wu, Yilun Zhao, Arman Cohan, and Xiao-Ping Zhang. Z1: Efficient test-time scaling with code, 2025. URLhttps://arxiv.org/abs/2504.00810

    2025 arXiv

  48. [57]

    Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks, 2025

    Yu Yue, Yufeng Yuan, Qiying Yu, Xiaochen Zuo, Ruofei Zhu, Wenyuan Xu, Jiaze Chen, Chengyi Wang, TianTian Fan, Zhengyin Du, Xiangpeng Wei, Xiangyu Yu, Gaohong Liu, Juncai Liu, Lingjun Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Ru Zhang, X...

    2025 arXiv

  49. [58]

    Adaptthink: Reasoning models can learn when to think, 2025

    Jiajie Zhang, Nianyi Lin, Lei Hou, Ling Feng, and Juanzi Li. Adaptthink: Reasoning models can learn when to think, 2025. URLhttps://arxiv.org/abs/2505.13417

    2025 arXiv

  50. [59]

    R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization.arXiv preprint arXiv:2503.12937, 2025

    Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization.arXiv preprint arXiv:2503.12937, 2025

    2025 arXiv

  51. [60]

    The Final Answer is \boxed{

    Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, and Ying Sheng. Sglang: Efficient execution of structured language model programs, 2024. URL https://arxiv.org/abs/23...

    2024 arXiv

  52. [2025]

    URLhttps://arxiv.org/abs/2505.10832

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.