Pith. sign in

REVIEW 4 major objections 5 minor 3 cited by

Making Small Language Models Efficient Reasoners: Intervention, Supervision, Reinforcement

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Small reasoning models waste tokens because they cannot choose a stopping point; EOS temperature scaling and a length-penalized RL objective cut length by half at little accuracy cost.

desk verdict Plausible recipe for cutting small-model reasoning tokens in half, but the headline numbers need an overlap audit and error bars before I'd trust them. read the letter →

arxiv 2505.07961 v3 pith:5JP5U7OC submitted 2025-05-12 cs.LG

Xuechen Zhang , Zijian Huang , Chenshun Ni , Ziyang Xiong , Jiasi Chen , Samet Oymak This is my paper · ORCID
classification cs.LG
keywords token-efficientreasoningsmalllanguagemodelschain-of-thoughtGRPOlengthpenaltytemperaturescalingstopping-pointcontrolsupervisedfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that small language models distilled by supervised fine-tuning reason wastefully: they cannot choose a good moment to stop, so they emit long, repetitive chains of thought, and wrong answers tend to be the most verbose. To fix this, the authors propose two controls. The first, temperature scaling, raises the sampling probability of the end-of-sequence token at inference time, letting the model stop earlier without retraining. The second, TLDR, adds a length penalty to GRPO reinforcement learning, with different penalty levels bound to prompt phrases such as SHORT, MODERATE, and LONG. On four math reasoning benchmarks, TLDR roughly halves response length while keeping accuracy at or near the supervised baseline, and TS tracks the accuracy-length Pareto front more cleanly than budget forcing.

What carries the argument

The load-bearing mechanism is the response-length penalty $\zeta(L)$ added to the GRPO reward, together with the EOS-logit temperature scaling $z'_{i_{\mathrm{eos}}} = z_{i_{\mathrm{eos}}}/T$ for training-free control. The penalty is a piecewise function of response length $L$ with thresholds set by the chosen level ($L_{\max}/4$ for SHORT, $L_{\max}/2$ for MODERATE, $L_{\max}$ for LONG), and it is paired with a prompt token such as "[Response Length: SHORT]", letting the same model switch length modes at inference time. The temperature-scaling mechanism performs stopping control at sampling time by making the EOS token more likely when $T<1$.

What would settle it

Normalize and compare the question text of every training example used for the TLDR reinforcement run against the test questions of MATH500, AMC, AIME24, and OlympiadBench; if any test question appears in training, re-run the four benchmarks after removing overlapping questions and check whether the roughly 50% token reduction and accuracy parity persist.

Watch

Extended reading notes

Core claim

The central discovery is that stopping time, not reasoning content, is the main wasted resource in distilled small reasoners. The authors find that the end-of-sequence token is treated by supervised fine-tuning like any ordinary token, so the model never learns an efficient stopping policy; in repeated failure cases the EOS token is among the top-five likely next tokens yet is not selected. Reinforcement learning, by contrast, treats stopping as a decision that affects reward. Their TLDR reward subtracts a length penalty $\eta(L)=\alpha L/L_{\max}$ from the accuracy reward, and a multi-level variant uses thresholds at $L_{\max}/4$ and $L_{\max}/2$ so that the prompt can request short, moderate, or long reasoning. With mild penalties the trained models produce responses about 50% shorter than the SFT baseline with minimal-to-no accuracy loss, and the wrong-answer repetition rate drops sharply. A training-free alternative, temperature scaling, modifies only the EOS logit by dividing it by $T<1$, which the authors show achieves a better accuracy-efficiency Pareto front than budget-forcing.

Load-bearing premise

The accuracy and token-efficiency results transfer to new problems only if the four evaluation benchmarks were not part of the distillation or reinforcement training data; the paper does not show that the sets are disjoint.

Editorial extensions

If this is right

  • A single small model trained once with TLDR can serve short, moderate, and long reasoning modes selected by prompt, so deployment can adjust compute per query without retraining.
  • Token cost at inference can drop by about half at roughly constant accuracy, which lowers serving cost and latency for math-style reasoning workloads.
  • Budget-forcing and prompt-based test-time controls are shown to miss exact length targets; TS gives a finer, model-agnostic dial, so training-free systems can use it.
  • RL-based length penalties can reduce both correct and wrong response lengths, implying that the model learns to stop early on problems it cannot solve rather than burning context.
  • SFT distillation from long teacher traces can hurt small students by inducing repetition; length-aware reward shaping is a more direct way to obtain efficient reasoning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the paper reports that wrong answers are systematically longer, an untested corollary is that an early-stopping or confidence-gated decoder could allocate tokens adaptively and save additional compute beyond the uniform length penalty.
  • The same prompt-conditioned reward shaping should transfer to non-mathematical reasoning tasks such as planning or code, where trace length is not tied to difficulty; testing TLDR there would show whether stopping-time control is a general property of RL rather than a math-benchmark artifact.
  • Without a data-contamination audit, the headline roughly 50% token saving is established primarily on benchmarks that may overlap the training mixture, so a clean evaluation on newly written problems is needed before treating the saving as a property of the method rather than of the data.
  • Combining TS with TLDR in one pipeline could give both a trained stopping policy and a fine-grained inference-time dial, allowing per-query length targets without additional gradient steps.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper addresses token-efficient reasoning for small language models (SLMs). It documents that SFT-distilled models produce verbose and repetitive chain-of-thought outputs, and that this verbosity is concentrated in incorrect answers. It then proposes two methods: TS, a training-free temperature scaling of the EOS logit, and TLDR, a length-penalized GRPO variant with prompt-conditioned multi-level penalties. Experiments on MATH500, AMC, AIME24, and OlympiadBench are used to argue that TS is more effective than budget forcing and that TLDR yields roughly 50% token reduction with minimal or no accuracy loss relative to SFT baselines, while allowing prompt-controlled short, moderate, and long responses.

Significance. If correct, the contribution is practically valuable: it offers simple, model-agnostic recipes for reducing inference cost of small reasoning models and highlights a real failure mode of SFT distillation. The paper also provides a useful comparison against contemporaneous length-control methods, including L1 and s1's budget forcing. The significance is conditional, however, because the evaluation infrastructure (single runs, no error bars, small AIME24 test set, and unresolved train/test overlap in Section 6.2) does not yet substantiate the strength of the claimed efficiency-accuracy improvements.

major comments (4)
  1. [§6.2 and Table 4] The RL training set is stated to be 'a combination of the training set of MATH, AIME, AMC, STILL, OlympiadBench, which is same as the training set of DeepScaleR-1.5B-Preview,' while evaluation uses MATH500 and the test sets of AMC, AIME24, and OlympiadBench. Since the train and eval sets come from the same benchmark families, the paper needs an exact-match and near-duplicate overlap audit, or an explicit exclusion statement, before the accuracy and accuracy-parity components of the token-efficiency claim can be interpreted as evidence of generalizable efficient reasoning; without it, memorization is a concrete alternative explanation.
  2. [Tables 2 and 4] All reported accuracies and lengths are single-run numbers with no seeds, error bars, or significance tests. On AIME24 this is especially problematic because the benchmark contains only about 30 problems, so a 3.33% difference is one question; claims such as 'TLDR significantly improves token efficiency by about 50% with minimal to no accuracy loss' need at least a few seeds and a variance-aware comparison.
  3. [Eq. (1), §6.2] The multi-level length-control result is substantially built into the objective: Eq. (1) penalizes correct answers exactly above Lmax/4 for SHORT and above Lmax/2 for MODERATE, and the prompts for those levels are paired with those penalties during training. To establish 'flexible control' as an empirical finding, the paper should report the achieved length distributions (e.g., calibration of mean and median lengths against the desired quartiles) rather than only the average lengths that the penalty directly shapes; otherwise the claim is partly a restatement of the reward design.
  4. [§4 and Table 1] The comparison between TS and budget forcing is not fully specified: BF is reported after sweeping budgets and selecting the configuration with the shortest output among those beating the base, while the TS result does not state how T was chosen or whether it was also selected with the same information. If T is tuned on the evaluation set, the headline comparison is an oracle-selected intervention; the paper should state the T values used and evaluate sensitivity to that choice.
minor comments (5)
  1. [Eq. (1) and §4] The text uses 'L > L/2' and 'L > L/4' where Lmax/2 and Lmax/4 are intended, and the logit notation switches from l to z in Section 4.
  2. [Figure 9] The y-axis is labeled 'Repeat rate' but the plotted values are token lengths (0 to 4000), which contradicts the caption and the surrounding text.
  3. [Throughout] There are several typos, including 'widly', 'thrshold', 'maitaining', 'natually', 'suitbale', 'preciously' in the Appendix D heading, and 'Additioned Experiment Results'.
  4. [Table 4] The layout is confusing: rows for the base models under 'No Length Control' are repeated across SHORT, MODERATE, and LONG, and the relationship between the 'No Length Control' and 'With Length Control' column groups should be clarified in the caption.
  5. [Reproducibility] No statement about code or checkpoint release is provided; adding one would improve reproducibility.

Circularity Check

1 steps flagged · score 4.0 of 10

Multi-level length control is built into the reward; main token-efficiency and accuracy claims remain empirical.

  1. fitted input called prediction [Section 5, Eq. (1), and Section 6.2 Results]
    "A unique aspect of our approach is that different parameter settings can be indirectly controlled by the end user through a special prompt, for example '[Response Length: LONG] Provide a detailed step-by-step solution.' ... The model is trained with the prompt and its corresponding penalty function. The model thus learns to pair the special prompt with the trajectories associated with that length penalty. Then during inference, the model should automatically produce responses that match that length penalty."

    The SHORT/MODERATE/LONG response-length ordering is the optimization target itself, not an independent prediction. Equation (1) assigns zero penalty for correct answers with L <= Lmax/4 under SHORT and L <= Lmax/2 under MODERATE, and penalty beta above those thresholds, while each prompt is paired with its corresponding penalty level during GRPO training. Therefore the observed result that 'SHORT', 'MODERATE', and 'LONG' prompts yield increasing trace lengths is a check that the model optimized the specified reward, i.e., the behavior is forced by construction of the objective. The accuracy-preservation and ~50% token-efficiency claims over SFT are separate empirical comparisons and are not circular; only the multi-level length-control capability reduces to the reward design.

full rationale

The paper's central claim, that TLDR improves token efficiency by about 50% with minimal accuracy loss relative to SFT, is evaluated against external baselines (SFT, s1's Budget Forcing, L1, DeepScaleR) and is not derivable from the reward function alone: nothing in the length penalty guarantees that accuracy is preserved, so the accuracy/efficiency Pareto comparison has independent empirical content. The one construction-level circularity is the multi-level SHORT/MODERATE/LONG control, which is directly encoded in Eq. (1) through the Lmax/4 and Lmax/2 thresholds and the paired prompts; reporting that these levels produce the corresponding lengths is an optimization check rather than a prediction. Self-citations in the paper ([13], [30]-[32]) are used as background for temperature scaling, loss design, and cascades, and are not load-bearing for the central results. The Section 6.2 concern that RL training data is drawn from the same benchmark families as the evaluation (MATH, AIME, AMC, OlympiadBench) is a legitimate contamination/memorization risk for validity, but it is not a circularity of the derivation chain and should be handled in a correctness review.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The empirical claims rest on a handful of hand-set hyperparameters, on released SFT checkpoints, and on an unverified assumption that evaluation sets are disjoint from RL training data. No code, checkpoints, or versioned data splits are shipped.

free parameters (5)
  • Length penalty strength alpha/gamma = 0.1
    Reward is r = r_hat - alpha * L / Lmax in the sweet-spot experiments and gamma = 0.1 for multi-level training; no sensitivity analysis is reported.
  • Correct-long penalty beta = 0.3
    Penalty for correct answers above the SHORT or MODERATE threshold in Equation (1); hand-set and not swept.
  • EOS temperature T = Not reported (T < 1)
    The temperature scaling results in Table 1 require choosing T per model, but exact values and selection rule are omitted.
  • Max response length Lmax = 2048 or 4096
    Length thresholds Lmax/4 and Lmax/2 are defined relative to Lmax, which is a compute-resource choice; results depend on it.
  • Repetition count threshold in repeat-rate judge = 10 occurrences
    Appendix C asks the judge to flag repetition only if the same content appears more than 10 times; this threshold is arbitrary and not validated.
assumptions (5)
  • domain assumption Evaluation benchmarks are disjoint from the RL training distribution.
    Section 6.2 states training data is the training set of MATH, AIME, AMC, STILL, and OlympiadBench, while tests are MATH500, AMC, AIME24, and OlympiadBench. No split audit is provided, so benchmark gains could reflect memorization.
  • domain assumption Accuracy plus format reward in GRPO is a reliable proxy for reasoning quality and does not induce reward hacking.
    The reward is r = 0.9 * accuracy + 0.1 * format, and wrong answers receive a length penalty. The paper does not analyze whether models learn to game format or to stop early without solving problems.
  • domain assumption GPT-4o-mini repetition annotations are accurate enough to measure repeat rate.
    Repeat rates in Figures 3, 9 and Table 6 rely on a proprietary model with a custom prompt in Appendix C; no validation or agreement metric is reported.
  • domain assumption Released SFT checkpoints (DeepSeek-R1-Distill, s1) are valid bases for the RL comparisons.
    The paper treats these checkpoints as given and does not control their training data; comparisons inherit any quirks of those releases.
  • standard math Modifying only the EOS logit by dividing by T is a valid softmax reweighting that preserves the relative ordering of other logits.
    Section 4 defines l_ieos' = z_ieos / T with T < 1; this changes only one logit's sampling probability and does not affect the ranking among other tokens.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Making Small Language Models Efficient Reasoners: Intervention, Supervision, Reinforcement." pith.science (2026). https://pith.science/paper/5JP5U7OC

@misc{pith2026250507961,
  author       = {Pith},
  title        = {Pith review of: Making Small Language Models Efficient Reasoners: Intervention, Supervision, Reinforcement},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5JP5U7OC}},
  note         = {Machine review of arXiv:2505.07961}
}
read the original abstract

Recent research enhances language model reasoning by scaling test-time compute via longer chain-of-thought traces. This often improves accuracy but also introduces redundancy and high computational cost, especially for small language models distilled with supervised fine-tuning (SFT). In this work, we propose new algorithms to improve token-efficient reasoning with small-scale models by effectively trading off accuracy and computation. We first show that the post-SFT model fails to determine the optimal stopping point of the reasoning process, resulting in verbose and repetitive outputs. Verbosity also significantly varies across wrong vs correct responses. To address these issues, we propose two solutions: (1) Temperature scaling (TS) to control the stopping point for the thinking phase and thereby trace length, and (2) TLDR: a length-regularized reinforcement learning method based on GRPO that facilitates multi-level trace length control (e.g. short, medium, long reasoning). Experiments on four reasoning benchmarks, MATH500, AMC, AIME24 and OlympiadBench, demonstrate that TS is highly effective compared to s1's budget forcing approach and TLDR significantly improves token efficiency by about 50% with minimal to no accuracy loss over the SFT baseline. Moreover, TLDR also facilitates flexible control over the response length, offering a practical and effective solution for token-efficient reasoning in small models. Ultimately, our work reveals the importance of stopping time control, highlights shortcomings of pure SFT, and provides effective algorithmic recipes.

Figures

Figures reproduced from arXiv: 2505.07961 by the authors.

Figure 1
Figure 1. The average performances across four reasoning benchmarks for 7B models. Our RL￾based length-control method TLDR enhances to￾ken efficiency by over 50% compared to SFT. Our temperature scaling (TS) method outperforms other test-time intervention techniques that avoid the need for training, such as Budget Forcing (BF) [19] and Prompting. The detailed explanation of baselines and our methods are provided in Sec. 5. Re… view at source ↗
Figure 2
Figure 2. We explore training strategies for small language models (SLM), focusing on the [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. SFT models have longer and more repetitive answers when wrong, especially for [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: Example of wrong and lengthy response, generated by the Deepseek-R1-Distill [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Test time compute strategies can trade off performance for efficiency, but cannot [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Changes in response length during reinforcement training without length penalty. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Length Penalty function. “Sweet Spot” of Length Penalty. To find the “sweet spot” of response efficiency without hurting the rea￾soning performance, we design the penalty function as η(L) = α L Lmax , where α is a hyperparameter for the strength of the length penalty, …
Figure 8
Figure 8. Figure 8: Performance of TLDR under varying model sizes and token budget. TLDR outperforms length-controlled baselines (S1) both in performance and token efficiency. is max length that can be trained with our resource, and 4096 is generally long enough to solve problems in MATH …
Figure 9
Figure 9. Figure 9: The response length of models before and after trained with [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 10
Figure 10. Figure 10: The length distribution of wrong and correct responses of different models. The [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: The prompt used for detecting answer repetition with GPT-4o-mini. [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]
Figure 12
Figure 12. Figure 12: Each subplot shows the performance of different models (1.5B, 7B, and 14B) on three benchmarks: MATH-500, AIME24, and GPQA-Diamond. The x-axis represents the average number of tokens in the generated response (including both thinking trace and final answer), while the…
Figure 13
Figure 13. Figure 13: Length distribution of thinking/solution trajectories under different test-time strategies (Budget Forcing, Exact Control, Prompt Control), model sizes (1.5B vs 7B), and benchmarks (AIME24, MATH-500, GPQA-Diamond). 0 5000 10000 15000 20000 25000 Response Length 0 10 2…
Figure 14
Figure 14. Figure 14: Histograms of response lengths by Gemini and DeepSeek on s1K dataset. [PITH_FULL_IMAGE:figures/full_fig_p015_14.png]
Figure 15
Figure 15. Figure 15: The response generated by Qwen-Instruct-1.5b finetuned by S1K-1.1 on AIME24 [PITH_FULL_IMAGE:figures/full_fig_p016_15.png]
Figure 16
Figure 16. Figure 16: TLDR compares on par with L1 under an max trace length constraint. Moreover, [PITH_FULL_IMAGE:figures/full_fig_p016_16.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. OS-Pruner: Pruning Chains-of-Thought of Reasoning Models via Optimal Stopping

    cs.AI 2026-07 conditional novelty 6.0 of 10

    A lightweight optimal-stopping policy on frozen reasoning LLMs cuts CoT length 20–60% with minimal accuracy loss by trading answer correctness against token cost via a tunable λ.

  2. From Trajectories to Prefixes: Reusing Teacher Trajectories via Replayed Prefixes and Online Continuation

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Replaying teacher trajectory prefixes and explicitly optimizing the historical prefix tokens improves small-model agent success rates over distillation and response-only GRPO baselines in TextCraft, BabyAI, and ALFWorld.

  3. BREAD: Branched Rollouts from Expert Anchors Bridge SFT & RL for Reasoning

    cs.LG 2025-06 conditional novelty 6.0 of 10

    BREAD, a GRPO variant with branched rollouts from expert anchors, densifies reward and outperforms SFT and GRPO for small language models on math reasoning, with a theory based on a Markov chain model.

Reference graph

Works this paper leans on

36 extracted references · 12 canonical work pages · cited by 3 Pith papers

  1. [1]

    L1: Controlling how long a reasoning model thinks with reinforcement learning

    Pranjal Aggarwal and Sean Welleck. L1: Controlling how long a reasoning model thinks with reinforcement learning. arXiv preprint arXiv:2503.04697, 2025

    arXiv 2025

  2. [2]

    The surprising effectiveness of test-time training for few-shot learning

    Ekin Aky ¨urek, Mehul Damani, Adam Zweiger, Linlu Qiu, Han Guo, Jyothish Pari, Yoon Kim, and Jacob Andreas. The surprising effectiveness of test-time training for few-shot learning. arXiv preprint arXiv:2411.07279, 2024

    arXiv 2024

  3. [3]

    Precise length control in large language models

    Bradley Butcher, Michael O’Keefe, and James Titchener. Precise length control in large language models. arXiv preprint arXiv:2412.11937, 2024

    arXiv 2024

  4. [4]

    Frugalgpt: How to use large lan- guage models while reducing cost and improving performance

    Lingjiao Chen, Matei Zaharia, and James Zou. Frugalgpt: How to use large lan- guage models while reducing cost and improving performance. arXiv preprint arXiv:2305.05176, 2023

    arXiv 2023

  5. [5]

    Gptq: Accurate post-training quantization for generative pre-trained transformers

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323, 2022

    arXiv 2022

  6. [6]

    Gemini 2.0 flash thinking mode (gemini-2.0f lash-thinking-exp-1219), 2024

    Google. Gemini 2.0 flash thinking mode (gemini-2.0f lash-thinking-exp-1219), 2024. https://cloud.google.com/vertex-ai/generative-ai/docs/thinking

    work page 2024

  7. [7]

    Test-time training provably improves transformers as in-context learners

    Halil Alperen Gozeten, M Emrullah Ildiz, Xuechen Zhang, Mahdi Soltanolkotabi, Marco Mondelli, and Samet Oymak. Test-time training provably improves transformers as in-context learners. arXiv preprint arXiv:2503.11842, 2025

    arXiv 2025

  8. [8]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    arXiv 2025

Show all 36 references
  1. [9]

    Language model cascades: Token-level uncertainty and beyond

    Neha Gupta, Harikrishna Narasimhan, Wittawat Jitkrittum, Ankit Singh Rawat, Aditya Krishna Menon, and Sanjiv Kumar. Language model cascades: Token-level uncertainty and beyond. arXiv preprint arXiv:2404.10136, 2024

    2024 arXiv

  2. [10]

    Thinkprune: Pruning long chain-of-thought of llms via reinforcement learning

    Bairu Hou, Yang Zhang, Jiabao Ji, Yujian Liu, Kaizhi Qian, Jacob Andreas, and Shiyu Chang. Thinkprune: Pruning long chain-of-thought of llms via reinforcement learning. arXiv preprint arXiv:2504.01296, 2025

    2025 arXiv

  3. [11]

    Openai o1 system card

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720, 2024

    2024 arXiv

  4. [12]

    Fast inference from transformers via speculative decoding

    Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, pages 19274– 19286. PMLR, 2023

    2023

  5. [13]

    Autobalance: Optimized loss functions for imbalanced data

    Mingchen Li, Xuechen Zhang, Christos Thrampoulidis, Jiasi Chen, and Samet Oy- mak. Autobalance: Optimized loss functions for imbalanced data. Advances in Neural Information Processing Systems, 34:3163–3177, 2021

    2021

  6. [14]

    Small models struggle to learn from strong reasoners

    Yuetai Li, Xiang Yue, Zhangchen Xu, Fengqing Jiang, Luyao Niu, Bill Yuchen Lin, Bhaskar Ramasubramanian, and Radha Poovendran. Small models struggle to learn from strong reasoners. arXiv preprint arXiv:2502.12143, 2025

    2025

  7. [15]

    Let’s verify step by step

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In The Twelfth International Conference on Learning Representations, 2023

    2023

  8. [16]

    Awq: Activation-aware weight quantization for on-device llm compression and acceleration

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. Awq: Activation-aware weight quantization for on-device llm compression and acceleration. Proceedings of Machine Learning and Systems, 6:87–100, 2024

    2024

  9. [17]

    Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Li Erran Li, Raluca Ada Popa, and Ion Stoica

    Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl. https://pretty-radio-b75.notion.site/ Dee...

  10. [18]

    Long-tail learning via logit adjustment

    Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, and Sanjiv Kumar. Long-tail learning via logit adjustment. ICLR, 2021

    2021

  11. [19]

    s1: Simple test-time scaling

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Ha- jishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Cand`es, and Tatsunori Hashimoto. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393, 2025

    2025 arXiv

  12. [20]

    Gpt-4o mini: advancing cost-efficient intelligence, 2024

    OpenAI. Gpt-4o mini: advancing cost-efficient intelligence, 2024. https://openai. com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/∼

    2024

  13. [21]

    Gpqa: A graduate-level google-proof q&a benchmark

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, 2024

    2024

  14. [22]

    Scaling test-time compute without verification or rl is suboptimal

    Amrith Setlur, Nived Rajaraman, Sergey Levine, and Aviral Kumar. Scaling test-time compute without verification or rl is suboptimal. arXiv preprint arXiv:2502.12118, 2025

    2025 arXiv

  15. [23]

    Outrageously large neural networks: The sparsely-gated mixture-of-experts layer

    Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. arXiv preprint arXiv:1701.06538, 2017

    2017 arXiv

  16. [24]

    Scaling llm test-time com- pute optimally can be more effective than scaling model parameters

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time com- pute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314, 2024

    2024 arXiv

  17. [25]

    Scalable chain of thoughts via elastic reasoning

    Yuhui Xu, Hanze Dong, Lei Wang, Doyen Sahoo, Junnan Li, and Caiming Xiong. Scalable chain of thoughts via elastic reasoning. arXiv preprint arXiv:2505.05315, 2025

    2025 arXiv

  18. [26]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  19. [27]

    An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, et al. Qwen2. 5-math technical report: To- ward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.12122, 2024

    2024 arXiv

  20. [28]

    Towards thinking-optimal scaling of test-time compute for llm reasoning

    Wenkai Yang, Shuming Ma, Yankai Lin, and Furu Wei. Towards thinking-optimal scaling of test-time compute for llm reasoning. arXiv preprint arXiv:2502.18080, 2025

    2025

  21. [29]

    Following length constraints in instructions

    Weizhe Yuan, Ilia Kulikov, Ping Yu, Kyunghyun Cho, Sainbayar Sukhbaatar, Jason Weston, and Jing Xu. Following length constraints in instructions. arXiv preprint arXiv:2406.17744, 2024

    2024 arXiv

  22. [30]

    Selective attention: Enhancing transformer through principled context control

    Xuechen Zhang, Xiangyu Chang, Mingchen Li, Amit Roy-Chowdhury, Jiasi Chen, and Samet Oymak. Selective attention: Enhancing transformer through principled context control. Advances in Neural Information Processing Systems, 37:11061–11086, 2024

    2024

  23. [31]

    Efficient contextual LLM cascades through budget-constrained policy learning

    Xuechen Zhang, Zijian Huang, Ege Onur Taga, Carlee Joe-Wong, Samet Oymak, and Jiasi Chen. Efficient contextual LLM cascades through budget-constrained policy learning. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  24. [32]

    Class-attribute priors: adapting optimization to heterogeneity and fairness objective

    Xuechen Zhang, Mingchen Li, Jiasi Chen, Christos Thrampoulidis, and Samet Oymak. Class-attribute priors: adapting optimization to heterogeneity and fairness objective. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 16890–16898, 2024. 12 0 50...

    2024

  25. [33]

    Pleaseanalyze the following text and determine if there are any meaningless repetitions of identical sentences

    Budget Forcing (BF): A maximum token budget is imposed on the thinking trajec- tory. Within this limit, the model is free to decide when to generate a special token 13 prompt = (f"Pleaseanalyze the following text and determine if there are any meaningless repetitions of identi...

  26. [34]

    If the model prematurely generates the end-of-thinking token before reaching the desired length, the token is removed, and generation continues until the target length is reached

    Exact Control (EC): The thinking trajectory is forced to be of a fixed length. If the model prematurely generates the end-of-thinking token before reaching the desired length, the token is removed, and generation continues until the target length is reached. At that point, the...

  27. [35]

    Prompt Control (PC): A soft constraint is applied by including an instruction in the prompt that explicitly tells the model not to exceed a specified number of tokens for the thinking trajectory

  28. [36]

    2k” and “4k

    Auto: The model is left unrestricted, allowing it to autonomously decide when to terminate the thinking trajectory and begin generating the final answer. In Figure 12, we can see that none of the test time strategies can exactly control the response under the length limitation...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.