Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Thinking Short and Right Over Thinking Long: Serving LLM Reasoning Efficiently and Accurately

T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SART claims that multi-branch LLM reasoning can be served up to 28.2x faster at the same accuracy by stopping early and pruning weak branches.

desk verdict The integrated serving scheduler is a real engineering contribution, but the headline speedup is measured at equal N, not matched accuracy, so the 15.7x/28.2x claim is inflated. read the letter →

arxiv 2505.13326 v1 pith:ASKXZCX5 submitted 2025-05-19 cs.LG

Yuhang Wang , Youhe Jiang , Bin Cui , Fangcheng Fu This is my paper · ORCID
classification cs.LG
keywords LLMservingtest-timescalingchain-of-thoughtreasoningbranchsamplingearlystoppingprocessrewardmodelcontinuousbatchinglatencyoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that multi-branch test-time scaling—generating several reasoning chains and ensembling them—need not cost the full latency and memory of running every branch to completion. It presents SART, a serving framework that samples more branches than needed, stops the batch once enough branches have finished, and prunes low-quality branches during decoding using a process reward model. The claim is that this 'short and right' scheduling makes reasoning LLMs both faster and slightly more accurate than single-chain serving, and far faster than existing branch-sampling approaches at the same accuracy. A sympathetic reader would care because accuracy gains from test-time scaling are currently bought with severe serving slowdowns; SART tries to make those gains serviceable under real request queues.

What carries the argument

The load-bearing objects are two scheduling mechanisms. Redundant sampling with early stopping draws on the order-statistic distribution of the M-th smallest response length (Lemma 1): sampling N > M branches and keeping the first M to finish replaces the tail latency of the longest branch with the M-th order statistic, which shrinks as N grows. Two-phase dynamic pruning maintains, per request, an exploration phase with a low fixed threshold α with a cap β on pruned branches, then switches to an exploitation phase whose threshold is the reward of the first completed branch, pruning any branch whose PRM score falls below it. A process reward model (Qwen2.5-Math-PRM-7B) supplies the per-branch quality scores, and the framework integrates with continuous batching, treating each branch as a batch unit and releasing KV cache immediately on prune, stop, or completion. Together these convert branch sampling from a latency-multiplying workload into an accuracy-boosting one whose cost tracks the short, correct branches.

What would settle it

Take a benchmark where correct branches systematically receive lower process-reward scores than incorrect branches at early decoding steps (for example, by corrupting or inverting the PRM's per-step scores) and check whether SART's accuracy drops below Self-Consistency at the same N. If it does, the dependence on PRM calibration is confirmed as the mechanism behind the accuracy results; if accuracy holds, the pruning is more robust than the paper assumes.

Watch

Extended reading notes

Core claim

SART's central claim is that end-to-end latency for multi-branch reasoning can be made robust to the number of sampled branches N. By redundant sampling with early stopping—start N branches, stop when M complete—the decoding time depends on the M-th completed response, not the longest branch, and the paper proves via order-statistic theory that this probability improves as N grows. Two-phase dynamic pruning then releases resources of low-scoring branches, cutting queuing delay. In evaluation on GPQA and GAOKAO with DeepSeek-R1-Distill models, SART beat Vanilla, Self-Consistency, and Rebase by up to 3.1x, 28.2x, and 14.4x (averages 2.0x, 15.7x, 8.0x) at comparable or better accuracy, with accuracy gaps to Self-Consistency within 1.6%.

Load-bearing premise

The whole scheme depends on the per-step quality scores (from a process reward model) actually tracking whether a partial reasoning chain is heading toward the correct answer, since SART prunes any chain that falls below a threshold and chooses the top-scoring chain at the end.

Editorial extensions

If this is right

  • Multi-branch reasoning can be served with end-to-end latency close to or below single-chain serving, so accuracy gains from test-time scaling no longer imply proportional slowdown.
  • Systems can tune N and M independently: larger N raises accuracy while the scheduler keeps tail latency in check, trading a modest increase in queuing delay for much lower tail latency.
  • Process-reward-model-guided pruning can remove a large fraction of branches before completion without measurable accuracy loss, freeing GPU memory for more concurrent requests.
  • The approach generalizes across model sizes (14B and 70B) and request arrival rates (1 and 4 requests per second), with standard deviations under 10% over five trials.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the same early-stop-plus-prune schedule could plausibly be applied with a verifier other than a PRM—for example, a learned verifier, a code executor, or a cheaper proxy model—whenever per-step quality signals are available; the paper only demonstrates the PRM case.
  • The order-statistic argument suggests that as branch lengths become more heavy-tailed (more over-thinking), the advantage of redundant sampling grows; a testable prediction is that SART's speedup over Self-Consistency should increase on distributions with fatter length tails.
  • The paper leaves open whether weak length-correctness correlation holds outside STEM QA; if it fails on a dataset where long chains are systematically more accurate, early stopping would trade away accuracy, so that boundary is worth probing.
  • Because the framework treats each branch as a schedulable unit, it could combine with preemption or priority policies for SLO-aware serving, though SART itself processes requests first-come-first-served.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes SART, a serving framework for LLM reasoning that combines two techniques: redundant sampling with early stopping (sample N branches, stop once M complete) and two-phase dynamic branch pruning driven by a process reward model (PRM), integrated with continuous batching in vLLM. The authors report that SART improves accuracy over vanilla single-branch reasoning and improves serving efficiency over Self-Consistency and Rebase, claiming up to 28.2x and on average 15.7x efficiency gains at the same level of accuracy. The paper includes an order-statistic analysis to justify early stopping, a system implementation, and experiments on GPQA and GAOKAO with two models.

Significance. If the empirical claims hold, SART addresses a real and timely problem: the tail latency and memory overhead of multi-branch test-time scaling for reasoning models. The system is built on a mainstream serving stack (vLLM), the design is clearly motivated, and the evaluation covers two datasets and two model scales. The order-statistic observation is correct as a statement about early-completion probabilities. The paper also ships concrete ablation results that help attribute gains to the two proposed mechanisms. The main concern is that the headline efficiency claim rests on a comparison at equal N rather than at matched accuracy, which materially weakens the 'same level of accuracy' claim; this is fixable but requires re-analysis or re-benchmarking.

major comments (4)
  1. [Abstract and §5.2] The headline claim that SART achieves 'up to 28.2x and on average 15.7x' speedup 'when achieving the same level of accuracy' is not supported by the reported experiments. The speedup ratios in Section 5.2 are computed at N=8 for all methods, while the text immediately concedes that SART accuracy is lower than Self-Consistency by up to 1.6% in several cases. Because Self-Consistency accuracy increases with N, the correct matched-accuracy baseline is the smallest N whose accuracy reaches SART's N=8 accuracy; that baseline has lower latency than Self-Consistency at N=8, so the reported ratios overstate the efficiency gain. The authors should either re-compute speedups at accuracy-matched operating points or explicitly reframe the claim as 'at equal N, with a small accuracy gap.'
  2. [§3 (Observation 1, Figure 2)] The empirical premise that response correctness is weakly related to response length is inferred from only 64 responses across 3 questions. This is too small a sample to support a general claim about length-correctness correlation across models and datasets, and it is a load-bearing premise for early stopping and for the claim that pruning short branches does not harm quality. The authors should provide a substantially larger study or explicitly label this as a preliminary motivating observation rather than a general empirical law.
  3. [§3 (Lemma 1 and following)] Lemma 1 and the surrounding argument only show that the probability of completing M of N branches within L decoding steps increases with N. The text concludes that 'by sampling more branches ... we can improve the efficiency of LLM reasoning,' but this ignores the extra compute and memory cost of launching N branches and the effect of PRM-based pruning on the length distribution. The theoretical result is not sufficient to establish a system-level efficiency improvement; that conclusion must rest on the end-to-end experiments. Please revise the wording to separate the probabilistic tail-length statement from the system-level efficiency claim.
  4. [§4 (Algorithm 1) and §5.3] The accuracy of SART depends critically on the PRM scores being a reliable proxy for branch correctness, since Algorithm 1 prunes branches with scores below the threshold and selects the highest-scoring remaining branch as the answer. The paper does not validate PRM calibration for the tested workloads, and the ablation in Section 5.3 covers only one dataset/model combination. If the PRM is miscalibrated on a given request, SART could prune the only correct branch or return an incorrect one. Please add a sensitivity analysis of the pruning threshold and, if possible, an evaluation of PRM calibration or an ablation with an alternative PRM.
minor comments (4)
  1. [Algorithm 1, line 18] The inline comment says 'Add M branches for decoding,' but the loop pushes N branches; the comment should say 'Add N branches.'
  2. [Algorithm 1, Decode function] Line 12 calls 'Decode(current batch, T)', but the function definition at line 21 takes only 'current batch' as an argument. The parameter T is referenced inside the function body but not declared in the signature; this is confusing.
  3. [§3, paragraph after Lemma 1] The phrase 'to complete M response over N branches' should be 'to complete M responses over N branches' (plural agreement).
  4. [§5.2, Figure 5] The figure contains many panels and it is difficult to map the latency and accuracy sub-panels to the specific model/dataset/rate configurations; please consider labeling each panel explicitly or using a table for the exact numbers behind the speedup claims.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SART's accuracy and efficiency claims are evaluated against external benchmarks with hand-set hyperparameters, and the order-statistic argument is a standard mathematical result rather than a self-referential construction.

full rationale

The paper's central claims are empirical systems results. SART is evaluated on external datasets (GPQA, GAOKAO) against continuous-batching baselines, and its accuracy is measured by ground-truth labels, not by any quantity fitted into the method. No parameter is fit to the target result: N, M, alpha, beta, and T are explicitly listed as defaults, and the PRM is an external model (Qwen2.5-Math-PRM-7B). The use of Lemma 1 is a standard order-statistics identity with an external citation; it establishes a stochastic-ordering statement about the M-th completion length and is not used to derive accuracy improvements. Observation 1, even if based on only 64 responses and 3 questions, is motivation rather than a load-bearing derivation; its weakness is an empirical-evidence concern, not circularity. The paper's speedup claim is reported at equal N, while the text concedes an accuracy gap of within 1.6%; that is a fairness or comparability issue, not a case where a prediction reduces to its inputs by construction. No load-bearing self-citation chain appears: references are to external prior work and do not substitute for the measured results. Therefore the derivation chain is self-contained with respect to the paper's own equations and claims, and no circular step can be exhibited.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The paper's contribution is algorithmic and architectural, not a fitted physical model. The main load-bearing premises are length-quality independence, PRM calibration, and the application of i.i.d. order statistics to a batched serving schedule. Several hyperparameters are hand-set and swept rather than derived.

free parameters (6)
  • N = 2, 4, 8 (varied in experiments)
    Number of sampled branches; controls redundancy. Chosen by hand and swept, not derived.
  • M = N/2 (default)
    Number of completed branches triggering early stopping; sets the length versus accuracy tradeoff.
  • alpha = 0.5
    First-phase pruning threshold; hand-chosen to keep exploration and avoid aggressive early pruning.
  • beta = N/2
    Maximum number of pruned branches in the exploration phase; hand-chosen to avoid pruning all branches.
  • T = 400
    Number of decoding steps between PRM checks; hand-chosen to balance reward-calculation overhead against pruning responsiveness.
  • B = configured per workload
    Decoding batch size; hardware-dependent serving knob that affects queuing and efficiency but is not part of the accuracy claim.
assumptions (4)
  • domain assumption Response lengths of branches are i.i.d. draws from a common CDF, and completion order follows length order.
    Used in Section 3 Lemma 1 to argue that increasing N makes the M-th completion happen in fewer steps. The paper later notes branches may not start simultaneously, weakening the direct application.
  • domain assumption PRM scores on partial decoding prefixes are valid predictors of final branch correctness.
    Algorithm 1 lines 33-37 prune low-score branches, and Section 5.1 selects the highest-reward branch as the final answer. No calibration analysis is provided.
  • domain assumption Correctness is weakly correlated with response length for reasoning models.
    Observation 1 (Figure 2) uses 64 responses on 3 questions; the early-stopping design relies on this generalizing to the tested workloads.
  • standard math The order-statistics lemma from [5] applies as stated.
    Lemma 1 is a standard textbook result and is correctly quoted, but its application to batched serving assumes synchronized branch starts.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Thinking Short and Right Over Thinking Long: Serving LLM Reasoning Efficiently and Accurately." pith.science (2026). https://pith.science/paper/ASKXZCX5

@misc{pith2026250513326,
  author       = {Pith},
  title        = {Pith review of: Thinking Short and Right Over Thinking Long: Serving LLM Reasoning Efficiently and Accurately},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ASKXZCX5}},
  note         = {Machine review of arXiv:2505.13326}
}
read the original abstract

Recent advances in test-time scaling suggest that Large Language Models (LLMs) can gain better capabilities by generating Chain-of-Thought reasoning (analogous to human thinking) to respond a given request, and meanwhile exploring more reasoning branches (i.e., generating multiple responses and ensembling them) can improve the final output quality. However, when incorporating the two scaling dimensions, we find that the system efficiency is dampened significantly for two reasons. Firstly, the time cost to generate the final output increases substantially as many reasoning branches would be trapped in the over-thinking dilemma, producing excessively long responses. Secondly, generating multiple reasoning branches for each request increases memory consumption, which is unsuitable for LLM serving since we can only batch a limited number of requests to process simultaneously. To address this, we present SART, a serving framework for efficient and accurate LLM reasoning. The essential idea is to manage the thinking to be short and right, rather than long. For one thing, we devise a redundant sampling with early stopping approach based on empirical observations and theoretic analysis, which increases the likelihood of obtaining short-thinking responses when sampling reasoning branches. For another, we propose to dynamically prune low-quality branches so that only right-thinking branches are maintained, reducing the memory consumption and allowing us to batch more requests. Experimental results demonstrate that SART not only improves the accuracy of LLM reasoning but also enhances the serving efficiency, outperforming existing methods by up to 28.2 times and on average 15.7 times in terms of efficiency when achieving the same level of accuracy.

Figures

Figures reproduced from arXiv: 2505.13326 by the authors.

Figure 1
Figure 1. Illustration of the two key challenges and our solutions. Although both scaling dimensions have shown remarkable successes in enhancing the quality of LLM responses, integrating them poses significant challenges to the system efficiency. We illustrate two key challenges in [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The numbers of correct and wrong responses in each length range, testing with three questions respectively. Each i-j on x-axis denotes a range of response lengths (e.g., “7-8” indicates 7K-8K tokens). branches. Specifically, we conduct a small testbed by generating 64 responses given three requests and examining the length and correctness. As shown in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. An example of our two-phase prun￾Threshold &=0.5 ing. Each value represents the reward of each branch at the corresponding decoding step. correctness, it essentially requires maintaining right-thinking branches and pruning the others so that the final response quality would remain intact. Undoubtedly, it is non-trivial to judge whether a reasoning branch is correct or not, yet fortunately, many works have developed … view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: In general, SART consistently achieves both better efficiency and higher accuracy compared to Vanilla, demonstrating its superior performance. 2We say “up to” since some branches may complete with fewer steps. 7 [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 5
Figure 5. Figure 5: End-to-end (E2E) latency (lower is better) and accuracy (higher is better) of each method with different N. We plot a horizontal line for Vanilla (which corresponds to N = 1) to indicate the baseline performance of reasoning without branch sampling. Vanilla (N=1) Self-…
Figure 6
Figure 6. Figure 6: Ablation studies (the 70B model on GAOKAO). The two plots on the left present the distributions of response length and queuing time, respectively (N = 4 for Self-Consistency, N = 8, M = 4 for SART). The three plots on the right present the end-to-end (E2E) latency and …
Figure 7
Figure 7. Figure 7: End-to-end (E2E) latency and inference latency (i.e., E2E latency excluding queuing delay) under various N, evaluated with the 14B model. It can be seen that our approach of redundant sampling with early stopping effectively reduces the overall response lengths compare…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey

    cs.AI 2025-07 conditional novelty 3.0 of 10

    A comprehensive review that categorizes methods for shortening and adaptively triggering chain-of-thought reasoning in large language models.

Reference graph

Works this paper leans on

36 extracted references · 10 canonical work pages · cited by 1 Pith paper

  1. [1]

    Let’s sample step by step: Adaptive- consistency for efficient reasoning and coding with llms

    Pranjal Aggarwal, Aman Madaan, Yiming Yang, et al. Let’s sample step by step: Adaptive- consistency for efficient reasoning and coding with llms. In Proceedings of the 2023 Confer- ence on Empirical Methods in Natural Language Processing (EMNLP 2023) , pages 12375– 12396, 2023

    work page 2023

  2. [2]

    A survey of monte carlo tree search methods

    Cameron B Browne, Edward Powley, Daniel Whitehouse, Simon M Lucas, Peter I Cowl- ing, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, and Simon Colton. A survey of monte carlo tree search methods. IEEE Transactions on Computational Intelligence and AI in games, 4(1):1–43, 2012

    2012

  3. [3]

    Are more llm calls all you need? towards the scaling properties of compound ai systems

    Lingjiao Chen, Jared Quincy Davis, Boris Hanin, Peter Bailis, Ion Stoica, Matei A Zaharia, and James Y Zou. Are more llm calls all you need? towards the scaling properties of compound ai systems. Advances in Neural Information Processing Systems, 37:45767–45790, 2024

    2024

  4. [4]

    Towards reasoning era: A survey of long chain- of-thought for reasoning large language models

    Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. Towards reasoning era: A survey of long chain- of-thought for reasoning large language models. arXiv preprint arXiv:2503.09567, 2025

    arXiv 2025

  5. [5]

    Order statistics

    Herbert A David and Haikady N Nagaraja. Order statistics. John Wiley & Sons, 2004

    work page 2004

  6. [6]

    Reasoning without self-doubt: More efficient chain-of-thought through certainty probing

    Yichao Fu, Junda Chen, Yonghao Zhuang, Zheyu Fu, Ion Stoica, and Hao Zhang. Reasoning without self-doubt: More efficient chain-of-thought through certainty probing. In ICLR 2025 Workshop on Foundation Models in the Wild, 2025

    work page 2025

  7. [7]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    arXiv 2025

  8. [8]

    Rewarding chatbots for real-world engagement with millions of users

    Robert Irvine, Douglas Boubert, Vyas Raina, Adian Liusie, Ziyi Zhu, Vineet Mudupalli, Ali- aksei Korshuk, Zongyi Liu, Fritz Cremer, Valentin Assassi, et al. Rewarding chatbots for real-world engagement with millions of users. arXiv preprint arXiv:2303.06135, 2023

    arXiv 2023

Show all 36 references
  1. [9]

    Towards effective disambiguation for machine translation with large language models

    Vivek Iyer, Pinzhen Chen, and Alexandra Birch. Towards effective disambiguation for machine translation with large language models. arXiv preprint arXiv:2309.11668, 2023

    2023 arXiv

  2. [10]

    Openai o1 system card

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720, 2024

    2024 arXiv

  3. [11]

    Large language models are state-of-the-art evaluators of translation quality

    Tom Kocmi and Christian Federmann. Large language models are state-of-the-art evaluators of translation quality. arXiv preprint arXiv:2302.14520, 2023

    2023 arXiv

  4. [12]

    Efficient memory management for large lan- guage model serving with pagedattention

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large lan- guage model serving with pagedattention. InProceedings of the 29th Symposium on Operating Systems Principles, ...

    2023

  5. [13]

    Leveraging constrained monte carlo tree search to generate reliable long chain-of-thought for mathematical reasoning

    Qingwen Lin, Boyan Xu, Zijian Li, Zhifeng Hao, Keli Zhang, and Ruichu Cai. Leveraging constrained monte carlo tree search to generate reliable long chain-of-thought for mathematical reasoning. arXiv preprint arXiv:2502.11169, 2025

    2025 arXiv

  6. [14]

    s1: Simple test-time scaling

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Cand `es, and Tatsunori Hashimoto. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393, 2025

    2025 arXiv

  7. [15]

    ChatGPT: Optimizing Language Models for Dialogue, 2022

    OpenAI. ChatGPT: Optimizing Language Models for Dialogue, 2022. https://openai. com/blog/chatgpt

    2022

  8. [16]

    GPT-4 technical report

    OpenAI. GPT-4 technical report. CoRR, abs/2303.08774, 2023. 10

    2023 arXiv

  9. [17]

    Splitwise: Efficient generative llm inference using phase splitting

    Pratyush Patel, Esha Choukse, Chaojie Zhang, Aashaka Shah, ´I˜nigo Goiri, Saeed Maleki, and Ricardo Bianchini. Splitwise: Efficient generative llm inference using phase splitting. In 2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA 2024), pages ...

    2024

  10. [18]

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. Gpqa: A graduate-level google-proof q&a benchmark, 2023

    2023

  11. [19]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  12. [20]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  13. [21]

    Scaling llm test-time compute opti- mally can be more effective than scaling model parameters

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute opti- mally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314, 2024

    2024 arXiv

  14. [22]

    Between underthinking and overthinking: An empirical study of reasoning length and correctness in llms

    Jinyan Su, Jennifer Healey, Preslav Nakov, and Claire Cardie. Between underthinking and overthinking: An empirical study of reasoning length and correctness in llms. arXiv preprint arXiv:2505.00127, 2025

    2025 arXiv

  15. [23]

    Stop overthinking: A survey on efficient reasoning for large language models

    Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Hanjie Chen, et al. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503.16419, 2025

    2025 arXiv

  16. [24]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599, 2025

    2025 arXiv

  17. [25]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. InAnnual Conference on Neural Information Processing Systems 2017 (NeurIPS 2017), 2017

    2017

  18. [26]

    Self-consistency improves chain of thought reasoning in lan- guage models

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in lan- guage models. In The Eleventh International Conference on Learning Representations (ICLR 2023)

    2023

  19. [27]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  20. [28]

    Inference scaling laws: An empirical analysis of compute-optimal inference for problem-solving with language models

    Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. Inference scaling laws: An empirical analysis of compute-optimal inference for problem-solving with language models. arXiv preprint arXiv:2408.00724, 2024

    2024 arXiv

  21. [29]

    Dynamic early exit in reasoning models

    Chenxu Yang, Qingyi Si, Yongjie Duan, Zheliang Zhu, Chenyu Zhu, Zheng Lin, Li Cao, and Weiping Wang. Dynamic early exit in reasoning models. arXiv preprint arXiv:2504.15895, 2025

    2025

  22. [30]

    Tree of thoughts: Deliberate problem solving with large language models

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Ad- vances in neural information processing systems (NeurIPS 2023), 36:11809–11822, 2023

    2023

  23. [31]

    Shorterbetter: Guiding reasoning models to find optimal inference length for efficient reasoning

    Jingyang Yi and Jiazheng Wang. Shorterbetter: Guiding reasoning models to find optimal inference length for efficient reasoning. arXiv preprint arXiv:2504.21370, 2025

    2025

  24. [32]

    Orca: A distributed serving system for transformer-based generative models

    Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soojeong Kim, and Byung-Gon Chun. Orca: A distributed serving system for transformer-based generative models. In 16th USENIX Sym- posium on Operating Systems Design and Implementation (OSDI 22), pages 521–538, 2022. 11

    2022

  25. [33]

    What, how, where, and how well? a survey on test-time scaling in large language models

    Qiyuan Zhang, Fuyuan Lyu, Zexu Sun, Lei Wang, Weixu Zhang, Zhihan Guo, Yufei Wang, Irwin King, Xue Liu, and Chen Ma. What, how, where, and how well? a survey on test-time scaling in large language models. arXiv preprint arXiv:2503.24235, 2025

    2025 arXiv

  26. [34]

    Evaluating the performance of large language models on gaokao benchmark, 2024

    Xiaotian Zhang, Chunyang Li, Yi Zong, Zhengyu Ying, Liang He, and Xipeng Qiu. Evaluating the performance of large language models on gaokao benchmark, 2024

    2024

  27. [35]

    The lessons of developing process reward models in mathematical reasoning

    Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. The lessons of developing process reward models in mathematical reasoning. arXiv preprint arXiv:2501.07301, 2025

    2025 arXiv

  28. [36]

    Multilingual machine translation with large language models: Empirical results and analysis

    Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, and Lei Li. Multilingual machine translation with large language models: Empirical results and analysis. arXiv preprint arXiv:2304.04675, 2023. 12

    2023 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.