Pith. sign in

REVIEW 3 major objections 4 minor 45 references

SmartThinker: Learning to Compress and Preserve Reasoning by Step-Level Length Control

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SmartThinker claims that judging each reasoning step by how much deleting it lowers the probability of the correct answer—rather than penalizing total output length—lets large reasoning models reason just as well while writing far fewer…

desk verdict A concrete step-level length-control recipe with real token savings, held back by an under-validated importance estimator and noisy small-test-set numbers. read the letter →

arxiv 2507.04348 v2 pith:XKQ4OSRH submitted 2025-07-06 cs.AI cs.CL

Xingyang He , Xiao Ling , Jie Liu This is my paper
classification cs.AIcs.CL
keywords efficientreasoningstep-levellengthcontrolchain-of-thoughtcompressionoverthinkingreinforcementlearningGRPOpolicyoptimizationtokenefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large reasoning models solve hard problems by thinking at length, but they also overthink easy ones, burning compute on redundant steps. Penalizing the total output length during reinforcement learning is the obvious fix, but the paper argues that this global penalty misfires: the model saves tokens by crushing the steps that matter while leaving padding in the steps that do not. SmartThinker instead measures how much each reasoning step contributes—by the drop in the model's own probability of the right answer when that step is deleted—and trains with a step-level reward that compresses unimportant steps hard and leaves critical steps room to breathe. On six math benchmarks this yields comparable or better Pass@k and Maj@k scores while producing far fewer tokens: 3,283 per answer for the 1.5B model versus 5,763 for the original. A reader should care because this attacks the deployment cost of reasoning models directly, without giving up the long thinking that makes them useful.

What carries the argument

The load-bearing object is the step-level importance score: for each reasoning step, the drop in the model's probability of producing the correct answer when that step is removed, normalized by the step's length, plus a keyword bonus for reasoning-transition words such as 'but' and 'however' that grows with problem difficulty. This score sets a per-step length penalty coefficient $k_1 = k_0(1 - ed'_{i,j})$, so important steps are barely penalized while redundant steps are strongly compressed, and a step-number coefficient $k_2 = k_0(1 - \rho)$ uses the question's difficulty (the fraction of incorrect samples) to set how many steps the whole chain may take. The per-step rewards then flow through S-GAE, a discounted advantage estimator with $\gamma = 0.95$ that replaces GRPO's summed future rewards, so longer chains no longer accumulate larger advantages by construction. Difficulty-adaptive clipping widens the policy-update bounds on hard problems and narrows them on easy ones. Everything hangs on the importance estimator being honest: if the model's probability drop misjudges which steps matter, the reward shortens the wrong tokens.

What would settle it

Run the identical SCPO pipeline twice, once with true importance scores and once with scores randomly permuted across steps while keeping the SFT warm-up, token budget, discount factor, and clipping schedule fixed: the paper's mechanism predicts a clear accuracy drop for the shuffled control at matched token budgets, and similar results would mean the importance estimator is not the active ingredient. A second check: on the case-study questions of Appendix A.7, compute the estimator's step labels and measure agreement with the human-identified critical steps, since low agreement with preserved accuracy would contradict the estimator's causal role.

Watch

Extended reading notes

Core claim

The paper's central claim is that step-level length control is strictly better than global length control for making reasoning models concise without losing accuracy. The authors identify the failure mode of global length penalties: because the reward is sparse (one score per response), the model discovers that the cheapest way to satisfy the length penalty is to erase long derivation steps, while keeping short decorative verbiage that costs little, so the hard-won reasoning capacity is compressed exactly where it is needed most. SmartThinker replaces the sparse reward with a dense step-level signal: each step's importance is estimated online as the change in the probability of the correct answer when that step is removed, the length penalty coefficient is lowered for important steps and raised for unimportant ones, and the overall number of steps is tightened or relaxed according to problem difficulty. A discounted step-level advantage estimator ($\gamma = 0.95$) removes the length bias of GRPO's summed advantages, and a difficulty-adaptive clipping range lets hard problems explore while easy problems stay stable. The result, on the paper's own numbers, is the highest average Pass@k and Maj@k among the compared methods on six math benchmarks at the lowest token count for the 1.5B model and near-lowest for the 7B model.

Load-bearing premise

The load-bearing premise is that the drop in the model's own probability of the correct answer, when a reasoning step is removed, reliably identifies which steps matter: if that estimate is miscalibrated, or a step is critical only in combination with others, the step-level reward will compress the wrong tokens and the claimed accuracy-efficiency trade-off collapses.

Editorial extensions

If this is right

  • Replacing global length penalties with step-level penalties improves both halves of the trade-off: on the 1.5B backbone SmartThinker beats the global-penalty baselines Alpha-0.1 and L1-Max in average Pass@k (66.0 vs 65.1 and 63.2) while using fewer tokens (3,283 vs 4,318 and 3,349).
  • The reallocation is visible in the output distribution: the share of steps classified as important rises by 15.5% on easy questions and 9.3% on hard ones, and about 10% more of the total length is spent on those steps.
  • The SFT warm-up pays for itself in wall-clock time: rejection sampling plus SFT lets 1,300 training steps finish in 68 hours, versus 122 hours for 1,000 steps without it, at comparable accuracy.
  • Compression generalizes beyond math: on TruthfulQA, RACE, and LiveCodeBench, the trained model matches or slightly exceeds the original, indicating the step-level policy did not just fit math-specific shortcuts.
  • Difficulty adaptation is load-bearing: harder problems keep longer chains and wider exploration ranges while easy problems collapse to near-direct answers, so the method does not impose a single reasoning budget on all questions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same step-importance estimator could work as an anytime compressor at inference time: score the steps of a sampled chain, then truncate or rewrite the lowest-scoring steps until the chain fits the compute budget, which would remove the RL training stage entirely.
  • A successor to the keyword bonus would detect verification and self-correction steps by their causal effect on later steps (whether deleting them shifts the final answer distribution), which should transfer to code and tool-use reasoning where transition-word lists are less reliable.
  • The step-level-versus-global claim could be tested cheaply outside RL: use each reward scheme to re-rank a fixed pool of sampled trajectories by accuracy and length, and check whether step-level rewards pick out the short-correct frontier that the trained policy eventually produces.
  • Because the 7B model was tuned with low-rank (LoRA) adapters while the comparison baselines were fully fine-tuned—a comparison asymmetry the paper itself states—the 7B margins should be read with that imbalance in mind, and a full-fine-tune run would settle how much of the gap is the method versus the tuning budget.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes SmartThinker, a two-stage framework for reducing the token length of reasoning chains while preserving accuracy. Stage 1 adapts a reasoning model to a short-reasoning mode via rejection sampling and SFT on the shortest correct responses. Stage 2 applies Step-Level Length Control Policy Optimization (SCPO), which combines an online step-importance estimator (Eq. 4), a step-level length-control reward (Eq. 8), a discounted step-level advantage S-GAE (Eq. 16), and a difficulty-adaptive clipping strategy (Eqs. 17-18). Experiments on six math benchmarks with DeepSeek-R1-Distill-Qwen-1.5B and 7B report that SmartThinker achieves the lowest average token usage while matching or improving Pass@k and Maj@k relative to global-length-penalty and hybrid baselines. An appendix (A.8) presents a theoretical comparison of step-level versus global length control.

Significance. If the central mechanism is reliable, the paper addresses a practical problem in efficient reasoning: global length penalties can indiscriminately compress critical steps. The work is clearly described, compares against relevant baselines, includes ablation studies for SFT and the keyword-based supplementary score, and reports token reductions that are substantial and consistent across benchmarks. The main empirical story, that SmartThinker reduces AvgLen by roughly 40-60% with approximately preserved accuracy, is interesting and within the scope of the journal. The principal weakness is that the paper's load-bearing component, the step-importance estimator in Eq. 4, is not validated against any oracle or alternative importance measure, and the theoretical appendix assumes the very design choice it is meant to justify.

major comments (3)
  1. [SCPO, Online Importance Evaluator (Eq. 4)] The central mechanism of SCPO is the step-importance score d_{i,j} in Eq. (4), but the manuscript provides no validation of this estimator against any ground truth or alternative importance measure. The ablation in Fig. 4 tests only the keyword supplement in Eqs. (5)-(6), not the core probability-drop estimator, so one cannot tell whether the observed token reduction comes from correctly identifying redundant steps or from an indiscriminate penalty that happens to preserve accuracy. Because p_1 and p_2 are computed under the current policy, the score is also a moving target during training and is potentially gameable by shortening a step until its removal becomes cheap; no analysis of this reward-hacking channel is given. I would like to see (i) a validation study for Eq. (4), e.g., agreement with leave-one-out causal importance or human-annotated critical steps on a sample of trajectories, and (ii) an ablation that replaces ed_{i,j} with a uniform per-step importance while keeping all other components, to isolate the contribution of the estimator.
  2. [Appendix A.8 (Eqs. 19-36)] The theoretical analysis assumes lambda_i = 1 - v_i in Eq. (34) and equal post-update total token budgets in Eqs. (32)-(33). The first assumption is precisely the step-level design choice that SCPO is meant to realize, not a property derived from the estimator in Eq. (4); the second assumption removes the token-savings effect, which is the paper's main empirical contribution. Consequently, Eq. (36) is close to self-fulfilling and does not establish that the proposed importance scores reliably recover v_i. I recommend either deriving the result under the actual estimator and reward of Eqs. (4)-(10), or explicitly reclassifying A.8 as an intuition-level illustration of step-level penalties rather than a proof of SCPO's correctness.
  3. [Main Results, Tables 1 and 2] AIME24 and AIME25 each contain only 30 problems, so the reported Pass@10/Maj@10 differences of 3.3 percentage points correspond to exactly one question. Table 1's headline AIME24 Pass@10 advantage of SmartThinker (63.3) over Alpha-0.1 (60.0) is therefore within one question, and other comparisons (e.g., AIME25 Pass@10, MinervaMATH Pass@k) are similarly within the resolution of the test set. No standard errors or multiple-seed results are reported anywhere, and the 7B results in Table 2 include a 3.7-point drop on MinervaMATH that is not discussed. Please provide bootstrap confidence intervals or multiple-seed results, and temper claims of consistent superiority to the resolution supported by the data.
minor comments (4)
  1. [Table 2] In the AvgLen block, the first row is labeled 'DeepSeek-R1-1.5B' but reports the 7B model results; it should read 'DeepSeek-R1-7B'.
  2. [Eq. (7) and surrounding text] The sentence 'rho = 1 - Nc/N denotes the indicator function and problem difficulty, respectively' conflates rho with I(s); please rewrite to define rho as the difficulty measure and I(s) as the keyword indicator.
  3. [Appendix A.6] The final sentence contains a typo: 'rasoning' should be 'reasoning'.
  4. [Abstract] The phrase 'a more concise chains of thought' has a subject-verb agreement error; consider 'more concise chains of thought'.

Circularity Check

2 steps flagged · score 4.0 of 10

External Pass@k/AvgLen results are not circular, but the 'critical step' evaluation and the Appendix A.8 proof reuse the paper's own definitions as evidence.

  1. self definitional [Method: Online Importance Evaluator (Eq. 4); Appendix A.3; Experiments: Output Distribution Analysis]
    "Step importance is calculated using Eq. 4 and subsequently normalized via min-max scaling. Steps with normalized importance scores greater than 0.01 are defined as effective steps. ... The proportion of effective steps increases by 15.5% for low-difficulty questions and by 9.3% for high-difficulty questions, indicating a reduction in redundant steps and a greater focus on critical reasoning."

    Eq. 4 defines step importance as the change in the model's probability of the correct answer when the step is removed, and the SCPO reward (Eq. 8, with k1 = k0(1 - ed') and the -exp(-rho*ed'/k0) penalty) is constructed to lengthen high-ed' steps and shorten low-ed' steps. The Fig. 1 evaluation then labels 'effective steps' using the same Eq. 4 scores. Therefore the reported post-training increase in 'effective steps' is not an independent measurement of criticality; it is a measure of how well the model moved along the very objective that was optimized. This part of the claimed 'greater focus on critical reasoning' is true by construction, although the separate Pass@k and AvgLen comparisons are external and non-circular.

  2. self definitional [Appendix A.8, Eqs. (33)-(36)]
    "In the step-level length control algorithm, λi should be inversely proportional to the contribution density of the step. Thus, we define λi = 1 − vi, yielding: ... By the Cauchy–Schwarz inequality ... Therefore, we finally obtain: Istep ≥ Iglobal."

    The proof's conclusion is packed into its definition of λi. Once the paper sets λi = 1 − vi, the Cauchy-Schwarz inequality guarantees Istep ≥ Iglobal under equal token budgets; the 'theorem' is a restatement of the design choice rather than an independent derivation that step-level control beats global control. In addition, vi is a hypothetical 'contribution density' that is never shown to be equal to, or reliably estimated by, the Eq. 4 importance scores used in SCPO. Thus the appendix justifies an oracle penalty scheme, not the actual estimator-based method, and cannot serve as a non-circular proof of the empirical trade-off claim.

full rationale

The paper is not globally circular: Table 1 and Table 2 report Pass@k, Maj@k, and AvgLen on held-out math and out-of-domain benchmarks, and the SCPO reward is not fitted to those test numbers. There is also no load-bearing self-citation chain. The circularity is localized in two places. First, the qualitative claim that SmartThinker 'focuses on critical steps' is evaluated with the same Eq. 4 importance definition that the reward optimizes, so the measured improvement in 'effective steps' is a self-consistency check rather than independent evidence. Second, the Appendix A.8 proof assumes λi = 1 − vi before deriving Istep ≥ Iglobal, making the inequality a direct consequence of the assumption; it also never connects vi to the estimable Eq. 4 scores. These issues do not invalidate the external accuracy-efficiency comparisons, but they mean the step-importance-specific claims and their 'theoretical proof' should not be treated as independently established. The unvalidated Eq. 4 estimator is better classified as a correctness risk than as circularity, since a wrong estimator would break the method without making the derivation tautological.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The method rests on four principal premises: the leave-one-out importance estimator faithfully ranks step importance; keyword-based reflection bonuses help hard problems; discounted advantage estimation removes length bias; and the KL penalty can be dropped. The first two are heuristics validated only indirectly by ablations, and the theoretical appendix assumes rather than proves the core design choice.

free parameters (4)
  • k0 (base penalty coefficient) = 0.6
    Selected via hyperparameter experiments (Fig. 6c); larger k0 (0.8) cuts tokens but drops accuracy, smaller values narrow the penalty range.
  • gamma (S-GAE discount) = 0.95
    Chosen in Fig. 6b to balance length bias and advantage accuracy; gamma=1 yields overly long sequences, too-low values degrade performance.
  • delta1 (lower clip adjustment) = 0.03
    Set based on entropy behavior (Fig. 6a); larger delta1 causes entropy collapse.
  • delta2 (upper clip adjustment) = 0.08
    Taken from DAPO and kept fixed in the hyperparameter sweep.
assumptions (5)
  • domain assumption The importance of a step is measured by the change in the probability of the correct answer when the step is removed (Eq. 4).
    Invoked in the Online Importance Estimator; if the model's probability is not a faithful proxy for real step contribution, the step-level penalties are misdirected.
  • ad hoc to paper Steps containing keywords such as 'but' or 'however' are markers of useful self-reflection and should receive extra importance (Eqs. 5-6).
    This heuristic is added to prevent suppression of reflection; it is not derived from the importance estimator and is tuned only through the qualitative ablation in Fig. 4.
  • domain assumption Discounting future step rewards with gamma=0.95 improves advantage estimation under GRPO (Eq. 16).
    The paper states GRPO's summed future rewards introduce length bias and that a discount mitigates it; no derivation is given, only hyperparameter plots.
  • domain assumption The KL penalty can be omitted in the RL objective.
    A.4 omits the KL penalty citing DAPO and Liu et al.; the paper does not ablate this choice.
  • ad hoc to paper In the theoretical proof, lambda_i = 1 - v_i and equal total token budget are assumed (Appendix A.8).
    The inequality I_step >= I_global follows from Cauchy-Schwarz only once the step penalty is defined as one minus contribution density; this assumption is the design principle itself.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SmartThinker: Learning to Compress and Preserve Reasoning by Step-Level Length Control." pith.science (2026). https://pith.science/paper/XKQ4OSRH

@misc{pith2026250704348,
  author       = {Pith},
  title        = {Pith review of: SmartThinker: Learning to Compress and Preserve Reasoning by Step-Level Length Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XKQ4OSRH}},
  note         = {Machine review of arXiv:2507.04348}
}
read the original abstract

Large reasoning models (LRMs) have exhibited remarkable reasoning capabilities through inference-time scaling, but this progress has also introduced considerable redundancy and inefficiency into their reasoning processes, resulting in substantial computational waste. Previous work has attempted to mitigate this issue by penalizing the overall length of generated samples during reinforcement learning (RL), with the goal of encouraging a more concise chains of thought. However, we observe that such global length penalty often lead to excessive compression of critical reasoning steps while preserving unnecessary details in simpler ones, yielding a suboptimal trade-off between accuracy and efficiency. To address this issue, we propose SmartThinker, a two-stage learnable framework designed to enable fine-grained control over the length of reasoning chains based on the importance of each individual step. In the first stage, SmartThinker adapts a reasoning model to a short-form reasoning mode through rejection sampling combined with supervised fine-tuning (SFT). In the second stage, SmartThinker applies Step-Level Length Control Policy Optimization (SCPO) to refine the model output distribution, which increases the proportion of length allocated to critical steps while reducing redundancy in less important ones. SCPO consists of four core components: an online importance estimator, a step-level length control reward function, a step-level generalized advantage estimation (S-GAE) and a difficulty-adaptive clipping strategy. Working in concert, these components enable SCPO to implement differentiated length control across reasoning steps. Empirical results across multiple reasoning benchmarks and various backbone models demonstrate that SmartThinker significantly reduces redundant reasoning while achieving comparable or even superior performance to existing methods.

Figures

Figures reproduced from arXiv: 2507.04348 by the authors.

Figure 1
Figure 1. Comparison of the output distributions between [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed SmartThinker. remain concise to avoid redundancy. Second, the number of steps should adapt dynamically to problem difficulty: complex problems demand more steps for solution quality, whereas simpler ones benefit from brevity. Thus, the reward function must capture two key aspects: aligning step length with each step’s importance and adjusting the overall num￾ber of steps based on problem dif… view at source ↗
Figure 3
Figure 3. Pass@5 accuracy and AvgLen at different training [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Effect of additional scores on reasoning behavior [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 6
Figure 6. Figure 6: Hyper Parameter Selection stage. For example, without SFT, the model tends to gener￾ate lengthy responses, requiring 122 hours to complete 1,000 training steps. In contrast, with SFT, the model completes 1,300 steps in just 68 hours, achieving comparable accuracy while…
Figure 7
Figure 7. Figure 7: Out-of-domain generalization of SmartThinker. [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Examples of model responses to questions with straightforward solutions across different baselines. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Examples of model responses to challenging questions across different baselines. [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 3 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Aggarwal, P.; and Welleck, S. 2025. L1: Controlling how long a reasoning model thinks with reinforcement learning. arXiv preprint arXiv:2503.04697

    arXiv 2025

  4. [4]

    Arora, D.; and Zanette, A. 2025. Training Language Models to Reason Efficiently. arXiv preprint arXiv:2502.04463

    arXiv 2025

  5. [5]

    Z.; Zhang, W.; Chen, H.; Yang, F.; et al

    Chen, M.; Li, T.; Sun, H.; Zhou, Y.; Zhu, C.; Wang, H.; Pan, J. Z.; Zhang, W.; Chen, H.; Yang, F.; et al. 2025 a . Learning to reason with search for llms via reinforcement learning. arXiv preprint arXiv:2503.19470

    arXiv 2025

  6. [6]

    Chen, Q.; Qin, L.; Liu, J.; Peng, D.; Guan, J.; Wang, P.; Hu, M.; Zhou, Y.; Gao, T.; and Che, W. 2025 b . Towards reasoning era: A survey of long chain-of-thought for reasoning large language models. arXiv preprint arXiv:2503.09567

    arXiv 2025

  7. [7]

    Chen, X.; Xu, J.; Liang, T.; He, Z.; Pang, J.; Yu, D.; Song, L.; Liu, Q.; Zhou, M.; Zhang, Z.; et al. 2024. Do not think that much for 2+ 3=? on the overthinking of o1-like llms. arXiv preprint arXiv:2412.21187

    arXiv 2024

  8. [8]

    G.; Xia, T.; Mao, H.; et al

    Cuadron, A.; Li, D.; Ma, W.; Wang, X.; Wang, Y.; Zhuang, S.; Liu, S.; Schroeder, L. G.; Xia, T.; Mao, H.; et al. 2025. The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Tasks. arXiv preprint arXiv:2502.08235

    arXiv 2025

Show all 45 references
  1. [9]

    Fang, G.; Ma, X.; and Wang, X. 2025. Thinkless: Llm learns when to think. arXiv preprint arXiv:2505.13379

    2025 arXiv

  2. [10]

    Feng, S.; Fang, G.; Ma, X.; and Wang, X. 2025. Efficient reasoning models: A survey. arXiv preprint arXiv:2504.10903

    2025

  3. [11]

    Fu, Y.; Chen, J.; Zhu, S.; Fu, Z.; Dai, Z.; Qiao, A.; and Zhang, H. 2024. Efficiently Serving LLM Reasoning Programs with Certaindex. arXiv preprint arXiv:2412.20993

    2024 arXiv

  4. [12]

    Guo, D.; Yang, D.; Zhang, H.; Song, J.; Zhang, R.; Xu, R.; Zhu, Q.; Ma, S.; Wang, P.; Bi, X.; et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    2025 arXiv

  5. [13]

    Han, T.; Wang, Z.; Fang, C.; Zhao, S.; Ma, S.; and Chen, Z. 2024. Token-budget-aware llm reasoning. arXiv preprint arXiv:2412.18547

    2024 arXiv

  6. [14]

    L.; Shen, J.; Hu, J.; Han, X.; Huang, Y.; Zhang, Y.; et al

    He, C.; Luo, R.; Bai, Y.; Hu, S.; Thai, Z. L.; Shen, J.; Hu, J.; Han, X.; Huang, Y.; Zhang, Y.; et al. 2024. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. arXiv preprint arXiv:2402.14008

    2024 arXiv

  7. [15]

    Hendrycks, D.; Burns, C.; Kadavath, S.; Arora, A.; Basart, S.; Tang, E.; Song, D.; and Steinhardt, J. 2021. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874

    2021 arXiv

  8. [16]

    Hou, B.; Zhang, Y.; Ji, J.; Liu, Y.; Qian, K.; Andreas, J.; and Chang, S. 2025. Thinkprune: Pruning long chain-of-thought of llms via reinforcement learning. arXiv preprint arXiv:2504.01296

    2025 arXiv

  9. [17]

    J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; Chen, W.; et al

    Hu, E. J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; Chen, W.; et al. 2022. Lora: Low-rank adaptation of large language models. ICLR, 1(2): 3

    2022

  10. [18]

    Jaech, A.; Kalai, A.; Lerer, A.; Richardson, A.; El-Kishky, A.; Low, A.; Helyar, A.; Madry, A.; Beutel, A.; Carney, A.; et al. 2024. Openai o1 system card. arXiv preprint arXiv:2412.16720

    2024 arXiv

  11. [19]

    Jain, N.; Han, K.; Gu, A.; Li, W.-D.; Yan, F.; Zhang, T.; Wang, S.; Solar-Lezama, A.; Sen, K.; and Stoica, I. 2024. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974

    2024 arXiv

  12. [20]

    Kumar, A.; Roh, J.; Naseh, A.; Karpinska, M.; Iyyer, M.; Houmansadr, A.; and Bagdasarian, E. 2025. Overthink: Slowdown attacks on reasoning llms. arXiv e-prints, pages arXiv--2502

    2025

  13. [21]

    Lai, G.; Xie, Q.; Liu, H.; Yang, Y.; and Hovy, E. 2017. Race: Large-scale reading comprehension dataset from examinations. arXiv preprint arXiv:1704.04683

    2017 arXiv

  14. [22]

    Lewkowycz, A.; Andreassen, A.; Dohan, D.; Dyer, E.; Michalewski, H.; Ramasesh, V.; Slone, A.; Anil, C.; Schlag, I.; Gutman-Solo, T.; et al. 2022. Solving quantitative reasoning problems with language models. Advances in Neural Information Processing Systems, 35: 3843--3857

    2022

  15. [23]

    Lin, S.; Hilton, J.; and Evans, O. 2021. Truthfulqa: Measuring how models mimic human falsehoods, 2022. URL https://arxiv. org/abs/2109.07958, 1

    2021 arXiv

  16. [24]

    S.; and Lin, M

    Liu, Z.; Chen, C.; Li, W.; Qi, P.; Pang, T.; Du, C.; Lee, W. S.; and Lin, M. 2025. Understanding r1-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783

    2025 arXiv

  17. [25]

    Loshchilov, I.; and Hutter, F. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101

    2017 arXiv

  18. [26]

    Lou, C.; Sun, Z.; Liang, X.; Qu, M.; Shen, W.; Wang, W.; Li, Y.; Yang, Q.; and Wu, S. 2025. AdaCoT: Pareto-Optimal Adaptive Chain-of-Thought Triggering via Reinforcement Learning. arXiv preprint arXiv:2505.11896

    2025 arXiv

  19. [27]

    Luo, H.; He, H.; Wang, Y.; Yang, J.; Liu, R.; Tan, N.; Cao, X.; Tao, D.; and Shen, L. 2025 a . Ada-R1: Hybrid-CoT via Bi-Level Adaptive Reasoning Optimization. arXiv preprint arXiv:2504.21659

    2025 arXiv

  20. [28]

    Luo, L.; Liu, Y.; Liu, R.; Phatale, S.; Guo, M.; Lara, H.; Li, Y.; Shu, L.; Zhu, Y.; Meng, L.; et al. 2024. Improve mathematical reasoning in language models by automated process supervision. arXiv preprint arXiv:2406.06592

    2024 arXiv

  21. [29]

    Y.; Roongta, M.; Cai, C.; Luo, J.; Zhang, T.; Li, L

    Luo, M.; Tan, S.; Wong, J.; Shi, X.; Tang, W. Y.; Roongta, M.; Cai, C.; Luo, J.; Zhang, T.; Li, L. E.; et al. 2025 b . Deepscaler: Surpassing o1-preview with a 1.5 b model by scaling rl. Notion Blog

    2025

  22. [30]

    Qu, X.; Li, Y.; Su, Z.; Sun, W.; Yan, J.; Liu, D.; Cui, G.; Liu, D.; Liang, S.; He, J.; et al. 2025. A survey of efficient reasoning for large reasoning models: Language, multimodality, and beyond. arXiv preprint arXiv:2503.21614

    2025

  23. [31]

    Rajbhandari, S.; Rasley, J.; Ruwase, O.; and He, Y. 2020. Zero: Memory optimizations toward training trillion parameter models. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, 1--16. IEEE

    2020

  24. [32]

    Shao, Z.; Wang, P.; Zhu, Q.; Xu, R.; Song, J.; Bi, X.; Zhang, H.; Zhang, M.; Li, Y.; Wu, Y.; et al. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300

    2024 arXiv

  25. [33]

    Sui, Y.; Chuang, Y.-N.; Wang, G.; Zhang, J.; Zhang, T.; Yuan, J.; Liu, H.; Wen, A.; Zhong, S.; Chen, H.; et al. 2025. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503.16419

    2025 arXiv

  26. [34]

    Veeraboina, H. 2023. Aime problem set 1983-2024, 2023. URL https://www. kaggle. com/datasets/hemishveeraboina/aime-problem-set-1983-2024

    2023

  27. [35]

    Wu, J.; Zhu, J.; and Liu, Y. 2025. Agentic Reasoning: Reasoning LLMs with Tools for the Deep Research. arXiv preprint arXiv:2502.04644

    2025 arXiv

  28. [36]

    T.; Wang, W.; and Li, W

    Xia, H.; Li, Y.; Leong, C. T.; Wang, W.; and Li, W. 2025. Tokenskip: Controllable chain-of-thought compression in llms. arXiv preprint arXiv:2502.12067

    2025

  29. [37]

    Xu, S.; Xie, W.; Zhao, L.; and He, P. 2025. Chain of draft: Thinking faster by writing less. arXiv preprint arXiv:2502.18600

    2025 arXiv

  30. [38]

    Yan, Y.; Shen, Y.; Liu, Y.; Jiang, J.; Zhang, M.; Shao, J.; and Zhuang, Y. 2025. Inftythink: Breaking the length limits of long-context reasoning in large language models. arXiv preprint arXiv:2503.06692

    2025

  31. [39]

    Yang, A.; Zhang, B.; Hui, B.; Gao, B.; Yu, B.; Li, C.; Liu, D.; Tu, J.; Zhou, J.; Lin, J.; et al. 2024. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.12122

    2024 arXiv

  32. [40]

    Ye, Y.; Huang, Z.; Xiao, Y.; Chern, E.; Xia, S.; and Liu, P. 2025. LIMO: Less is More for Reasoning. arXiv preprint arXiv:2502.03387

    2025 arXiv

  33. [41]

    Yu, Q.; Zhang, Z.; Zhu, R.; Yuan, Y.; Zuo, X.; Yue, Y.; Dai, W.; Fan, T.; Liu, G.; Liu, L.; et al. 2025. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476

    2025 arXiv

  34. [42]

    Zhang, J.; Lin, N.; Hou, L.; Feng, L.; and Li, J. 2025 a . Adaptthink: Reasoning models can learn when to think. arXiv preprint arXiv:2505.13417

    2025 arXiv

  35. [43]

    Zhang, J.; Zhu, Y.; Sun, M.; Luo, Y.; Qiao, S.; Du, L.; Zheng, D.; Chen, H.; and Zhang, N. 2025 b . Lightthinker: Thinking step-by-step compression. arXiv preprint arXiv:2502.15589

    2025

  36. [44]

    Zhang, J.; and Zuo, C. 2025. Grpo-lead: A difficulty-aware reinforcement learning approach for concise mathematical reasoning in language models. arXiv preprint arXiv:2504.09696

    2025

  37. [45]

    Zhu, X.; Xia, M.; Wei, Z.; Chen, W.-L.; Chen, D.; and Meng, Y. 2025. The surprising effectiveness of negative reinforcement in LLM reasoning. arXiv preprint arXiv:2506.01347

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.