Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

AALC: Large Language Model Efficient Reasoning via Adaptive Accuracy-Length Control

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A training reward that suppresses overthinking can cut a math model's chain-of-thought length by more than half while preserving accuracy.

desk verdict A sensible RL reward tweak that gets real length reductions; the paper is worth reviewing but needs baseline comparisons and variance reporting. read the letter →

arxiv 2506.20160 v2 pith:TEMDNXY5 submitted 2025-06-25 cs.CL

Ruosen Li , Ziming Luo , Quan Zhang , Ruochen Li , Ben Zhou , Ali Payani , Xinya Du This is my paper
classification cs.CL
keywords reinforcementlearninglengthrewardoverthinkingreasoningefficiencychain-of-thoughtmathGRPO
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that overlong chain-of-thought reasoning in large language models is not a fixed cost: a training-time reward that ties brevity to measured accuracy can remove more than half of the output tokens without losing correctness. The proposed reward, AALC, multiplies the usual correctness reward with an "accuracy attention" weight and adds a length penalty that switches on only once validation accuracy nears a scheduled target. The authors report length reductions over 50% on standard math benchmarks, with accuracy held or slightly improved, and similar behavior on out-of-distribution math benchmarks and across model scales. If the claim holds, latency and inference cost for math reasoning can be cut substantially by changing how the model is rewarded rather than by post-hoc truncation.

What carries the argument

The load-bearing object is the pair of reward equations that define AALC. For each prediction, $r_{\mathrm{len}} = \min(1, L_{\mathrm{pred}}/L_{\max})$ normalizes length, $r_{\mathrm{acc}} = A_{\mathrm{val}}/A_{\mathrm{target}}$ normalizes current validation accuracy against a scheduled target, and the length reward $R_{\mathrm{len}} = 1-\min(r_{\mathrm{acc}}^\beta, r_{\mathrm{len}})$ produces a smooth penalty that is effectively off until accuracy is high. The accuracy attention term $\mathrm{Att}_{\mathrm{acc}} = \gamma + (1-\gamma)(1-r_{\mathrm{acc}})$ interpolates between the raw correctness reward and this length pressure, and the total reward $\mathrm{AALC} = \mathrm{Att}_{\mathrm{acc}}R_{\mathrm{raw}} + \alpha R_{\mathrm{len}}$ is what the policy optimises. Two schedulers—an exponential moving average and a "potential scheduling" rule—set $A_{\mathrm{target}}$ online, and it is this gating schedule, not a fixed token budget, that is meant to prevent the length penalty from activating before the model can answer correctly.

What would settle it

Run AALC training on the same benchmark with several different random 5% validation splits, or with validation accuracy deliberately computed on 1% of the data; if final accuracy varies by more than a few points or the length reduction falls well below 50% on any split, the accuracy gate is too fragile to support the claimed trade-off.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that overthinking in reasoning models is reward-driven: when correctness is rewarded with a 0/1 signal and length is ignored, models keep generating redundant verification and subgoal scaffolding long after it stops helping. AALC makes the reward accuracy-aware by computing a normalized accuracy ratio $r_{\mathrm{acc}}=A_{\mathrm{val}}/A_{\mathrm{target}}$, then applying a smooth length penalty $1-\min(r_{\mathrm{acc}}^\beta, r_{\mathrm{len}})$ that stays near zero until the model has reached a dynamic target accuracy. The combined reward $\mathrm{AALC} = \mathrm{Att}_{\mathrm{acc}}R_{\mathrm{raw}} + \alpha R_{\mathrm{len}}$ shifts weight from raw correctness to brevity exactly as validation accuracy rises, so the model first learns to solve problems and only then learns to solve them briefly. The reported result is that this preserves or improves exact-match accuracy while reducing average response length by over 50%, with compression appearing as fewer subgoal-setting and verification behaviors rather than truncated answers.

Load-bearing premise

The load-bearing premise is that the validation accuracy computed on a small 5% split (about 375 examples per benchmark) is stable and informative enough to decide when to start penalizing length.

Editorial extensions

If this is right

  • On the two in-distribution benchmarks, models trained with AALC generate roughly a quarter and less than half of the original average token length, respectively, while exact-match accuracy is maintained or slightly improved.
  • On out-of-distribution math benchmarks, the same trained models both use fewer tokens and match or beat the raw-score fine-tuned baselines in accuracy.
  • Behavioral analysis attributes the token savings to reduced frequencies of subgoal setting and verification, not to answers being cut off, so the compression is a learned reasoning style.
  • Across model scales and for both a base math model and its distilled variant, AALC-trained runs converge to similarly short response lengths, and one distilled model's accuracy improves.
  • A side effect is reduced interpretability: outputs drop narrative framing, explanatory context, and some derivation steps even though the final answer remains correct.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reliance on a roughly 375-example validation split means the method's promise depends on that accuracy estimate; a natural extension is to replace the fixed split with an online or confidence-weighted accuracy estimate, which the paper does not test.
  • If base and distilled models converge to the same concise length because the reward selects a common compact proof style, that style may transfer to other tasks with verifiable outputs; applying the same training to non-math domains would test this.
  • The reported loss of interpretability suggests a trade-off curve: users who need explanations may want an explicit reward term that penalizes removal of explanatory steps, something the current AALC reward does not provide.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes AALC, a reward-shaping method for RL training of large reasoning models. The AALC reward combines a correctness reward with a length penalty that is gated by a ratio of validation accuracy to a dynamically scheduled target accuracy; two schedules (EMA and Potential Scheduling) are introduced. Experiments on Qwen2.5-Math-7B and DeepSeek-R1-Distill-Qwen-7B, trained on GSM8k and MATH, report dramatic reductions in output length with accuracy roughly maintained or slightly improved. The paper also evaluates out-of-distribution generalization, ablates hyperparameters, reports win-rate-based semantic quality comparisons, and analyzes reasoning behaviors such as verification and subgoal setting. The central claim is that AALC reduces response length by over 50% while preserving or improving accuracy, through suppression of redundant reasoning rather than truncation.

Significance. If the central claim is robust, AALC is a simple and inexpensive training-time mechanism for controlling chain-of-thought length, with potential practical value for latency and cost reduction in deployed reasoning models. Strengths of the paper include the release of code, the use of two model families and several standard and out-of-distribution benchmarks, the inclusion of scaling experiments, and the qualitative behavior analysis that distinguishes compression of redundant subgoal-setting/verification from naive truncation. The paper also honestly reports that efficiency gains come with reduced interpretability. However, the empirical support is currently incomplete in three important respects: there is no direct comparison with existing length-reward methods, the validation-accuracy estimator that controls the entire schedule is not stress-tested, and some textual claims go beyond what the reported tables show. These issues are fixable and do not invalidate the core idea, but they do mean the paper's claims are stronger than the current evidence.

major comments (4)
  1. [§2, §4.2, Tables 1–2] The central efficiency claim is not situated against existing length-control methods. The related-work section describes several RL length-reward approaches (e.g., Arora and Zanette, 2025; Yeo et al., 2025; Aggarwal and Welleck, 2025; Shen et al., 2025; Kimi k1.5), but the experiments compare AALC only with the original model and with raw-score fine-tuning. Since raw-score fine-tuning is not a length-reward baseline, the current design cannot establish that AALC improves over prior length-control methods. Please add at least one existing length-reward or length-control baseline under a matched training budget, or explicitly scope the claim to improvement over raw-score fine-tuning.
  2. [§3.2–3.3, §4.1, §4.3] The reward schedule is driven by validation accuracy Aval measured on a 5% split of roughly 7.5k training problems (about 375 examples). With binary correctness, the standard error of Aval is about two percentage points at typical accuracies, and the running-max construction of Atarget is upward-biased; combined with beta=128, activation of the length penalty depends on whether a noisy validation batch happens to approach the inflated target. The paper ablates the validation interval (5/10/30 steps) but never varies the validation split size or reports seed-to-seed variance, despite stating that metrics are averaged over 3 runs. Please report per-seed accuracy and length (or standard deviations) and ablate the split fraction (e.g., 1%, 5%, 10%) to show that the accuracy-preservation claim is robust to the estimator that controls the whole schedule.
  3. [§4.2, Table 2] The text states that AALC-trained models 'consistently outperform both the original models and those trained with raw scores across all datasets,' but Table 2 does not support this for the Qwen2.5-Math model: on GPQA the AALC accuracy (30.0) is below the raw-score accuracy (32.3), and on AIME24 and CNMO24 the two are tied. In addition, the OOD comparisons are affected by the 1k-token generation cap, which truncates the original models more heavily than the AALC models. Please correct the overstatement and qualify the OOD accuracy comparisons accordingly.
  4. [§5.1, Table 4] The semantic-quality win-rate evaluation explicitly instructs evaluators to consider 'conciseness' as a criterion. For pairs in which both responses are correct, this rewards the shorter response by construction, so the win rates above 50% in Table 4 cannot be interpreted as evidence of superior semantic quality independent of length. The paper acknowledges GPT-4o's length bias but then includes a length-based criterion in the prompt. Please report win rates separately for correctness-conditional pairs, remove or analyze the effect of the conciseness criterion, and provide details of the human evaluation (number of annotators, instructions, and inter-annotator agreement).
minor comments (5)
  1. [§3.1] The sentence 'A value close to 1 implies that the rear is inactive' appears to contain a typo: 'rear' should be 'reward'.
  2. [§4.1 and §4.3] The main results are said to use the EMA schedule by default, but the ablation study says all its runs use the second dynamic schedule (PS); please state explicitly which schedule is used in each table and figure.
  3. [§5.1] The text says the win-rate evaluation is presented 'in Table 1,' but the win rates appear in Table 4; the cross-reference should be corrected.
  4. [General reporting] The paper says 'All metrics are averaged over 3 runs,' but no table or figure reports error bars, standard deviations, or per-seed values; adding this information would substantially strengthen the accuracy comparisons, especially where differences are small.
  5. [Appendix A] The appendix headings 'Qwen3-235B-A22B Output' and 'ChatGPT-4o Output' appear to be leftover labels from a model comparison, and the appendix does not explain how these outputs relate to the AALC analysis; please clarify or remove these labels.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity identified: AALC is an empirical reward-shaping method whose evaluation is external to its reward definitions.

full rationale

No circularity identified. AALC is an empirical RL reward-shaping method; its central result is not derived from a self-referential equation. The reward Rlen = 1 - min(r_acc^β, r_len) uses validation accuracy Aval measured on a 5% held-out split of the training set, and Atarget is updated from that same Aval; this serves as a control or gating signal during training, not as a fitted predictor of test-set accuracy. The claimed >50% length reduction and accuracy preservation are evaluated on external benchmarks (GSM8k, MATH, AIME24, AMC24, CNMO24, GPQA) against original models and raw-score fine-tuning, so the conclusion is not forced by construction. There are no load-bearing self-citations, no imported uniqueness theorems, and no ansatz smuggled in via citation: the Gandhi et al. behavior taxonomy, the Nayab et al. CCA metric, and Kimi1.5 are used as external tools or related work, not as premises that entail the paper's claims. The reviewer concern about the ~375-example validation split and potential upward bias in the running-max Atarget is a real statistical robustness question about when the length penalty activates, but it does not make the derivation circular. The paper is self-contained relative to its empirical benchmarks, so the honest finding is 'no significant circularity' with score 0.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the reward design, which introduces several hand-tuned hyperparameters (alpha, beta, gamma, epsilon, Lmax) that are shown to affect the accuracy-length trade-off. The method also assumes that a small validation split provides reliable accuracy signals for scheduling. No invented entities are introduced.

free parameters (5)
  • alpha (length reward weight) = 1e-6
    Weight in AALC = Att_acc * Rraw + alpha * Rlen. Ablation shows alpha=1e-6 gives the best accuracy-length trade-off; larger alpha drops accuracy, smaller fails to curb verbosity.
  • beta (penalty activation exponent) = 128
    Exponent in min(r_acc^beta, r_len). Ablation shows beta=8 reduces length more but drops accuracy; beta=128 preserves accuracy while still shortening.
  • gamma (minimum accuracy attention) = 0.9
    Minimum attention to accuracy in Att_acc = gamma + (1-gamma)(1-racc). Chosen as default; no ablation reported for gamma.
  • epsilon (scheduling inertia/decay) = 0.9
    Controls inertia in EMA and decay in Potential Scheduling for Atarget. Chosen as default; no ablations shown.
  • Lmax (maximum output length) = 1000 tokens
    Preset maximum length used in r_len = min(1, Lpred/Lmax). It also caps generation, which affects OOD results by truncating baselines.
assumptions (3)
  • domain assumption Validation accuracy on a 5% held-out split (about 375 examples per benchmark) is a stable estimate of the model's reasoning performance.
    All reward computations and schedules in Sections 3.2 and 3.3 use Aval; if it is noisy, the dynamic attention and target scheduling can misfire. Section 4.1 describes the 95:5 split.
  • domain assumption GRPO training with the AALC reward converges to the reported policy within about 300 steps without reward hacking, such as producing empty outputs to exploit the length penalty.
    The paper reports convergence, but the reward could in principle be gamed by minimal text. The qualitative analysis suggests outputs are not empty, but this is an assumption without formal guarantee.
  • domain assumption The evaluation benchmarks and answer-extraction methods are appropriate for measuring math reasoning accuracy.
    The paper does not describe the answer-extraction procedure in detail, only that 'raw score' uses extracted answers. Errors in extraction could bias the accuracy numbers.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AALC: Large Language Model Efficient Reasoning via Adaptive Accuracy-Length Control." pith.science (2026). https://pith.science/paper/TEMDNXY5

@misc{pith2026250620160,
  author       = {Pith},
  title        = {Pith review of: AALC: Large Language Model Efficient Reasoning via Adaptive Accuracy-Length Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TEMDNXY5}},
  note         = {Machine review of arXiv:2506.20160}
}
read the original abstract

Large reasoning models (LRMs) achieve impressive reasoning capabilities by generating lengthy chain-of-thoughts, but this "overthinking" incurs high latency and cost without commensurate accuracy gains. In this work, we introduce AALC, a lightweight, accuracy-aware length reward integrated into reinforcement learning that dynamically balances correctness and brevity during training. By incorporating validation accuracy into the reward and employing a smooth, dynamically scheduled length penalty, AALC delays length penalty until target performance is met. Through extensive experiments across standard and out-of-distribution math benchmarks, we show that our approach reduces response length by over 50% while maintaining or even improving the original accuracy. Furthermore, qualitative analysis reveals that our method curbs redundant reasoning patterns such as excessive subgoal setting and verification, leading to structurally refined outputs rather than naive truncation. We also identify that efficiency gains are accompanied by reduced interpretability: models trained with AALC omit some narrative framing and explanatory context. These findings highlight the potential of reward-based strategies to guide LRMs toward more efficient, generalizable reasoning paths.

Figures

Figures reproduced from arXiv: 2506.20160 by the authors.

Figure 2
Figure 2. Training Progress of Qwen2.5 Models Across Scales. Bold lines indicate training runs that incorporate the AALC. Models of the same scale are represented with similar colors. Models Acc (RS) Acc (AALC) Qwen2.5-0.5b 34.0 33.9 Qwen2.5-Math-1.5b 67.8 68.4 Qwen2.5-Math-7b 73.0 73.9 [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 1
Figure 1. # of inference tokens of models during train [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 3
Figure 3. Ablation analysis of token usage and accuracy under different training configurations. Top row (a–c): [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Response length and frequency of reason [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Response length and frequency of reason [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: The output length at different β values [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 8
Figure 8. Figure 8: (a) Comparison of response lengths over training steps between DeepSeek-Distill-Qwen and Qwen2.5 [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Performative Thinking? The Brittle Correlation Between CoT Length and Problem Complexity

    cs.AI 2025-09 conditional novelty 6.0 of 10

    A transformer trained to imitate A* produces traces whose length reflects similarity to training data, not true problem complexity, so long chain-of-thought should not be read as more 'thinking'.

Reference graph

Works this paper leans on

36 extracted references · 2 canonical work pages · cited by 1 Pith paper

  1. [1]

    Pranjal Aggarwal and Sean Welleck. 2025. L1: Controlling how long a reasoning model thinks with reinforcement learning. arXiv preprint arXiv:2503.04697

    arXiv 2025

  2. [2]

    Daman Arora and Andrea Zanette. 2025. Training language models to reason efficiently. arXiv preprint arXiv:2502.04463

    arXiv 2025

  3. [3]

    Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, et al. 2024. Do not think that much for 2+ 3=? on the overthinking of o1-like llms. arXiv preprint arXiv:2412.21187

    arXiv 2024

  4. [4]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    arXiv 2021

  5. [6]

    Yifu Ding, Wentao Jiang, Shunyu Liu, Yongcheng Jing, Jinyang Guo, Yingjie Wang, Jing Zhang, Zengmao Wang, Ziwei Liu, Bo Du, et al. 2025. Dynamic parallel tree search for efficient llm reasoning. arXiv preprint arXiv:2502.16235

    arXiv 2025

  6. [7]

    Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. 2025. Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars. arXiv preprint arXiv:2503.01307

    arXiv 2025

  7. [8]

    Kanishk Gandhi, Denise Lee, Gabriel Grand, Muxin Liu, Winson Cheng, Archit Sharma, and Noah D Goodman. 2024. Stream of search (sos): Learning to search in language. arXiv preprint arXiv:2404.03683

    arXiv 2024

  8. [9]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    arXiv 2025

Show all 36 references
  1. [10]

    Tingxu Han, Zhenting Wang, Chunrong Fang, Shiyu Zhao, Shiqing Ma, and Zhenyu Chen. 2024. Token-budget-aware llm reasoning. arXiv preprint arXiv:2412.18547

    2024 arXiv

  2. [11]

    Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. 2024. Training large language models to reason in a continuous latent space. arXiv preprint arXiv:2412.06769

    2024 arXiv

  3. [12]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874

    2021 arXiv

  4. [13]

    Chengsong Huang, Langlin Huang, Jixuan Leng, Jiacheng Liu, and Jiaxin Huang. 2025. Efficient test-time scaling via self-calibration. arXiv preprint arXiv:2503.00031

    2025 arXiv

  5. [14]

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276

    2024 arXiv

  6. [15]

    Baohao Liao, Yuhui Xu, Hanze Dong, Junnan Li, Christof Monz, Silvio Savarese, Doyen Sahoo, and Caiming Xiong. 2025. Reward-guided speculative decoding for efficient llm reasoning. arXiv preprint arXiv:2501.19324

    2025 arXiv

  7. [16]

    Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. 2025. O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning. arXiv preprint arXiv:2501.12570

    2025 arXiv

  8. [17]

    Xinyin Ma, Guangnian Wan, Runpeng Yu, Gongfan Fang, and Xinchao Wang. 2025. Cot-valve: Length-compressible chain-of-thought tuning. arXiv preprint arXiv:2502.09601

    2025 arXiv

  9. [18]

    Sania Nayab, Giulio Rossolini, Marco Simoni, Andrea Saracino, Giorgio Buttazzo, Nicolamaria Manes, and Fabrizio Giacomelli. 2024. Concise thoughts: Impact of output length on llm reasoning and cost. arXiv preprint arXiv:2407.19825

    2024 arXiv

  10. [19]

    OpenAI. 2024. https://openai.com/index/learning-to-reason-with-llms/ Learning to reason with llms . Accessed: 2025-04-30

    2024

  11. [20]

    Ning Qian. 1999. https://doi.org/https://doi.org/10.1016/S0893-6080(98)00116-6 On the momentum term in gradient descent learning algorithms . Neural Networks, 12(1):145--151

    1999 doi

  12. [21]

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2024. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling

    2024

  13. [22]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347

    2017 arXiv

  14. [23]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300

    2024 arXiv

  15. [24]

    Yi Shen, Jian Zhang, Jieyun Huang, Shuming Shi, Wenjing Zhang, Jiangze Yan, Ning Wang, Kai Wang, and Shiguo Lian. 2025. Dast: Difficulty-adaptive slow-thinking for large reasoning models. arXiv preprint arXiv:2503.04472

    2025

  16. [25]

    Prasann Singhal, Tanya Goyal, Jiacheng Xu, and Greg Durrett. 2023. A long way to go: Investigating length correlations in rlhf. arXiv preprint arXiv:2310.03716

    2023 arXiv

  17. [26]

    Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Hanjie Chen, Xia Hu, et al. 2025. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503.16419

    2025 arXiv

  18. [27]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. 2025. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599

    2025 arXiv

  19. [28]

    Qwen Team. 2024. https://qwenlm.github.io/blog/qwq-32b-preview/ Qwq-32b-preview . Accessed: 2025-04-30

    2024

  20. [29]

    Heming Xia, Yongqi Li, Chak Tou Leong, Wenjie Wang, and Wenjie Li. 2025. Tokenskip: Controllable chain-of-thought compression in llms. arXiv preprint arXiv:2502.12067

    2025

  21. [30]

    Tian Xie, Zitian Gao, Qingnan Ren, Haoming Luo, Yuqian Hong, Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, and Chong Luo. 2025. Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning. arXiv preprint arXiv:2502.14768

    2025 arXiv

  22. [31]

    Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He. 2025 a . Chain of draft: Thinking faster by writing less. arXiv preprint arXiv:2502.18600

    2025 arXiv

  23. [32]

    Yige Xu, Xu Guo, Zhiwei Zeng, and Chunyan Miao. 2025 b . Softcot: Soft chain-of-thought for efficient reasoning with llms. arXiv preprint arXiv:2502.12134

    2025 arXiv

  24. [33]

    An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang. 2024. Qwen2.5-math technical report: Toward mathematical expert model ...

    2024 arXiv

  25. [34]

    Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. 2025. Demystifying long chain-of-thought reasoning in llms. arXiv preprint arXiv:2502.03373

    2025 arXiv

  26. [35]

    Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. 2025. Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild. arXiv preprint arXiv:2503.18892

    2025 arXiv

  27. [36]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  28. [37]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.