pith. sign in

arxiv: 2605.23926 · v1 · pith:WG5BZA2Cnew · submitted 2026-04-21 · 💻 cs.AI · cs.LG

How Much Thinking is Enough? Quantifying and Understanding Redundancy in LLM Reasoning

Zhiyuan Zhai , Xinkai You , Wenjing Yan , Xin Wang This is my paper

Pith reviewed 2026-07-05 09:54 UTC · model glm-5.2

classification 💻 cs.AI cs.LG
keywords modelsreasoningredundancymodelacrossanswerconditionscorrect
0
0 comments X

The pith

Over-thinking is structural: rewards that ignore length make it optimal

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks three questions about reasoning-capable language models: how do you define redundant reasoning, how much of it is there, and why does it happen? The authors define redundancy as the largest fraction of trailing steps in a correct reasoning trace that can be truncated while the model, forced to answer immediately, still produces the correct answer. Across four frontier reasoning models and two math benchmarks, they find that 61-93% of reasoning steps are redundant by this measure, with the median trace needing just one step to recover the answer in six of eight conditions. They then prove that this over-thinking is not a bug of any particular model or training recipe but a structural consequence of length-agnostic outcome rewards: when the training signal rewards only answer correctness and ignores trace length, no policy with finite expected reasoning time is optimal, because a policy that keeps trying indefinitely can always improve its success probability. The fix, they argue, must come from training-time interventions that explicitly penalize length.

Core claim

The central object is the redundancy ratio rho, defined as one minus the critical point k* divided by total steps N, where k* is the smallest prefix length at which the model still produces the correct answer under forced termination. The authors measure rho across 1,880 correct traces from four models (DeepSeek-R1, QwQ-32B, R1-Distill-7B, Qwen3-30B-Thinking) on GSM8K and MATH-500, finding rho exceeds 60% in all eight model-benchmark conditions. They then model reasoning as a sequential decision process with three actions (ADVANCE, IDLE, STOP) and prove Theorem 1: under the length-agnostic objective J_0(pi) = P[n_A >= K], no policy with finite expected stopping time is optimal, because any政策

What carries the argument

The redundancy ratio rho = 1 - k*/N, where k* is the smallest prefix at which forced termination still yields the correct answer. Theorem 1 proves that under any reward depending only on outcome correctness (not trace length), the optimal policy has infinite expected stopping time, because a policy that never stops can always accumulate more successful advances and thus strictly dominates any finite-stopping policy.

If this is right

  • Any reasoning model trained with pure outcome-only rewards will exhibit over-thinking regardless of architecture, scale, or training algorithm; length penalties are not optional refinements but structural necessities.
  • The median critical prefix being a single step in most conditions suggests that reasoning models often solve the problem early and spend the remaining tokens on verification and self-reflection that does not change the answer, pointing to a large gap between computational cost and informational value.
  • The finding that redundancy decreases with problem difficulty but remains above 46% even on the hardest problems implies that over-thinking is not confined to easy problems but is a persistent feature across the difficulty spectrum.
  • The self-versus-external-judge gap varying by training recipe suggests that some models produce traces that are only readable by themselves, which has implications for interpretability and knowledge distillation.
  • The theorem's independence from specific RL algorithms means that switching from PPO to other methods, or from RL to distillation, will not cure over-thinking unless the reward shape itself changes.

Load-bearing premise

The theoretical model assumes reasoning decomposes into discrete steps that independently succeed with fixed probability, like coin flips toward a target count of successes. Real reasoning steps are not independent: a step's contribution depends on what came before it, and the notion of counting a fixed number of successful advances as the difficulty threshold does not map cleanly onto how actual chain-of-thought reasoning works.

What would settle it

If a model trained with purely outcome-based rewards were found to have bounded, non-redundant reasoning traces as a stable equilibrium rather than a context-window artifact, Theorem 1's prediction would be challenged. More directly: if advances in real reasoning traces are strongly correlated rather than independent, the Borel-Cantelli argument underlying the proof breaks down.

Figures

Figures reproduced from arXiv: 2605.23926 by Wenjing Yan, Xinkai You, Xin Wang, Zhiyuan Zhai.

Figure 1
Figure 1. Figure 1: Difficulty–redundancy curves, four models. Left: ρ(d) with 95% bootstrap CI bands. All four models decrease monotonically with difficulty. Right: average trace length N¯ (dashed) and average critical prefix ¯k ⋆ (solid), log scale. ¯k ⋆ scales super-linearly with difficulty across all four models [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Step-level vs. word-level redundancy. Each point is one (model, benchmark) cell from [PITH_FULL_IMAGE:figures/full_fig_p015_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Prefix-position ablation. For each relative prefix length k/N ∈ {0.05, 0.15, . . . , 0.95} we plot judge accuracy under four strategies: first-k (blue), last-k (red), middle-k (green), random-k (orange), on 120 correct MATH-500 traces per model. last-k reaches 100% accuracy at every k for QwQ-32B and ≥ 96% for DeepSeek-R1; first-k requires k ≥ 0.5N to reach 75–85%. The prefix-position ablation complements … view at source ↗
Figure 4
Figure 4. Figure 4: Empirical cumulative distribution function (ECDF) of k ⋆/N, four models × two benchmarks. An ECDF of a quantity X reads as follows: the value at horizontal coordinate x is the fraction of traces with X ≤ x. So each curve at x = 0.1 gives the fraction of traces whose critical prefix lies within the first 10% of the trace. P50 and P90 markers (the 50th and 90th percentiles of the distribution, i.e. the k ⋆/N… view at source ↗
Figure 5
Figure 5. Figure 5: Redundancy by MATH-500 subject, four models, bar view. Per-subject ρ with 95% bootstrap CI error bars. Intermediate Algebra (the longest traces) is consistently among the lowest-ρ subjects for every model. B.7 Positional distribution of redundancy, stratified by difficulty B.8 Leave-one-out critical-step fractions Leave-one-out is a stricter test than truncation: it asks whether a specific individual step … view at source ↗
Figure 6
Figure 6. Figure 6: Length–accuracy decile curves, four models, stratified by difficulty level. Each panel shows one model; coloured lines are per-level decile curves, thick black is the aggregate. DS-R1 and R1-Distill show the strongest inverted-U shape; QwQ is weakly monotone; Qwen3-30B-Thinking is essentially flat. 0.0 0.2 0.4 0.6 0.8 1.0 Relative position in trace p=i/N 0 20 40 60 80 100 P(ste p i is re du n d a nt) (\%) … view at source ↗
Figure 7
Figure 7. Figure 7: Positional redundancy stratified by difficulty level. Each panel = one model; coloured lines = per-level curves; thick black = aggregate. Early steps (p ≤ 0.1) are redundant 30–50% of the time; late steps (p ≥ 0.9) exceed 90–98%. B.9 External-judge replication To validate that our main-text numbers do not depend on π serving as its own decoder (Definition 1), we replicate every MATH-500 and GSM8K measureme… view at source ↗
Figure 8
Figure 8. Figure 8: Step taxonomy, four models × five MATH-500 difficulty levels. Red = critical (removal flips the judge’s answer); green = redundant. Green dominates every panel: > 90% of individual segmented steps are dispensable at every difficulty level and every model. The small red slivers at a handful of (model, level) combinations mark where the few individually-critical insight steps concentrate. B.15 Qualitative ca… view at source ↗
read the original abstract

Reasoning-capable large language models solve hard problems by emitting long chains of thought, paying heavily in latency, GPU time, and energy. Casual inspection of their traces reveals extensive reformulation, verification, and circular self-reflection, yet how much of this deliberation is actually necessary has never been measured at scale or explained from first principles. This paper closes both gaps. We formalise reasoning redundancy directly in terms of the reasoning model itself: the redundancy of a correct trace is the largest fraction of its trailing segmented steps that can be truncated while $\pi$, forced to terminate thinking and emit a final answer, still produces the correct answer. A large-scale quantification across four frontier reasoning models and two mathematical benchmarks shows that step-level redundancy is consistently high -- between 61% and 93% across the 8 (model, benchmark) conditions we study, with the median critical prefix equal to a single segmented step in six of the eight conditions -- that the finding is robust to the choice of judge family, and that although $\rho$ decreases with problem difficulty on MATH-500, all four models remain substantially redundant ($\rho \in [46\%, 85\%]$) even on the hardest Level-5 problems. We then prove that this redundancy is a structural consequence of length-agnostic outcome rewards, not a model-specific artefact: under any such reward, no finite expected stopping time is optimal. The result holds regardless of RL algorithm, base model, data distribution, or whether the policy is obtained via RL or distillation; over-thinking is therefore not a bug to be patched in individual models but a structural property of how current reasoning models are trained. Code: https://github.com/zhiyuanZhai20/how-much-thinking-is-enough

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 10 minor

Summary. This paper addresses three questions about reasoning redundancy in LLMs: (1) how to define it, (2) how much exists in frontier models, and (3) why it occurs. The authors define redundancy (ρ) as the largest fraction of trailing steps in a correct trace that can be truncated while the model still produces the correct answer under forced termination. They measure ρ across four models (DeepSeek-R1, QwQ-32B, R1-Distill-7B, Qwen3-30B-Thinking) and two benchmarks (GSM8K, MATH-500), finding ρ > 60% in all 8 conditions, with median critical prefix of 1 step in 6 of 8 conditions. They provide robustness checks including external-judge replication, difficulty stratification, leave-one-out ablation, and within-problem variance analysis. They then prove Theorem 1: under a length-agnostic outcome reward J₀(π) = P[n_A ≥ K], no policy with finite expected stopping time is optimal. The theorem is intended to explain the empirical findings as a structural consequence of training with outcome-only rewards.

Significance. The empirical contribution is substantial and well-executed. The definition of redundancy (Definition 1) is clean and operationally measurable. The large-scale quantification across four models, two benchmarks, and two judges — with bootstrap CIs at the problem level, segmentation robustness checks, leave-one-out ablation, and within-problem variance analysis — represents the most thorough empirical characterization of over-thinking to date. The release of code and data is a strength. The theoretical contribution (Theorem 1) aims to provide a first-principles explanation, but its connection to the empirical findings has a significant gap that must be addressed (see Major Comment 1).

major comments (1)
  1. Appendix A, proof of Theorem 1: The proof contains an internal inconsistency between the observability assumptions used in the two halves of the argument. In the 'π does not succeed almost surely' part, the proof writes: 'Let M := Σ 1[a_t = ADVANCE] denote the total number of ADVANCE actions π emits before stopping... Conditional on M, n_A ~ Bin(M, p).' This conditional distribution is valid only if M is independent of the success variables ξ_t — i.e., if the policy cannot observe whether its ADVANCE actions succeeded. But in the 'Construction of π′' part, π′ is defined as a policy that 'continues indefinitely until n_A ≥ K is reached,' which requires π′ to observe n_A (and hence the ξ_t's). These two requirements are contradictory. If the policy CAN observe ξ_t (the natural assumption for LLM reasoning, where the model reads its own chain of thought), then the policy π* = 'always ADVACE
minor comments (10)
  1. §4: The forced-termination prompt (Appendix C.2) uses T=0 with max_tokens=64 for the decoder. It would be useful to report how often the 64-token cap was binding, as this could artificially inflate k* if the model cannot complete its answer.
  2. Table 1: The ρ_ext column for Qwen3-30B-Thinking shows a 57-60 point gap from ρ_π. The paper attributes this to 'stylistic affinity' but does not test this hypothesis directly. A simple control (e.g., using a different external judge) would strengthen the claim.
  3. §5.3, Finding 3: The claim that k* scales 'super-linearly' with difficulty is stated but not formally tested. A power-law fit or similar would make this quantitative.
  4. Appendix B.12: The segmentation robustness check varies only the merge threshold. It would be more convincing to also vary the discourse marker list, as this is the primary segmentation mechanism.
  5. §6: The definition of 'difficulty K' as 'the number of successful ADVANCE moves required to produce a correct answer' is not connected to any operational procedure for measuring K from actual traces. This makes the model's parameters untestable on real data.
  6. Table 4: The length-accuracy decile analysis uses 450 samples per model but the table header says 'n=450 per model' while the text says 'MATH-500, n=450 per model.' Clarify whether this is all MATH-500 problems or a subset.
  7. References: Several citations use 'arXiv preprint' format without complete author lists (e.g., 'et al.' appears in some entries). Ensure consistency with journal style.
  8. Figure 1 (right panel): The y-axis is labeled as log scale but the caption does not mention this. Add a note about the log scale to the caption.
  9. §3, Definition 1: The word-level redundancy ρ_L is defined but the relationship between ρ and ρ_L is only discussed in Appendix B.2. A brief note in the main text pointing to this agreement would help readers.
  10. Appendix B.10: The convergence-detection early stop is described as a 'negative result' but the specific stopping rule (two consecutive steps mentioning the same trailing number) is quite naive. The framing could acknowledge that more sophisticated syntactic stopping rules might fare better.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for a careful and constructive report. The referee correctly identifies an internal inconsistency in the proof of Theorem 1: the two halves of the argument use contradictory observability assumptions about the success variables ξ_t. We agree this must be fixed. We have revised the theorem statement and proof to work under a single, explicitly stated observability assumption (the observable setting, which is the natural one for LLM reasoning), and we clarify the connection between the theoretical result and the empirical findings. The empirical contributions are unchanged.

read point-by-point responses

  1. Referee: Major Comment 1: The proof of Theorem 1 contains an internal inconsistency between the observability assumptions used in the two halves of the argument. In the 'π does not succeed almost surely' part, the proof writes: 'Let M := Σ 1[a_t = ADVANCE] denote the total number of ADVANCE actions π emits before stopping... Conditional on M, n_A ~ Bin(M, p).' This conditional distribution is valid only if M is independent of the success variables ξ_t — i.e., if the policy cannot observe whether its ADVANCE actions succeeded. But in the 'Construction of π′' part, π′ is defined as a policy that 'continues indefinitely until n_A ≥ K is reached,' which requires π′ to observe n_A (and hence the ξ_t's). These two requirements are contradictory. If the policy CAN observe ξ_t (the natural assumption for LLM reasoning, where the model reads its own chain of thought), then the policy π* = 'always ADVANCE'

    Authors: The referee is correct that the proof as written contains an internal inconsistency. We acknowledge this without reservation and have revised the proof in the revised manuscript. Below we explain the issue, the fix, and the (minor) implications for the paper's claims. The referee's comment was truncated, but the core objection is clear: the two halves of the proof use contradictory observability assumptions about the success variables ξ_t. Specifically, the 'π does not succeed almost surely' part conditions on M (the number of ADVANCE actions) and writes n_A ~ Bin(M, p), which is valid only if M is independent of the ξ_t's — i.e., if the policy cannot observe whether its ADVANCE actions succeeded. But the 'Construction of π′' part defines π′ as a policy that 'continues indefinitely until n_A ≥ K is reached,' which requires π′ to observe n_A and hence the ξ_t's. These two requirements are contradictory. We agree. The fix is to state and prove the theorem under a single, consistent observability assumption. We choose the observable setting (the policy can observe ξ_t), because this is the natural assumption for LLM reasoning: the model reads its own chain of thought and can in principle tell whether a step advanced the solution. Under this assumption, the proof simplifies and the inconsistency disappears. Revised proof (observable setting): Assume the policy can observe the ξ_t's. Then the optimal policy under J_0 is π* = 'always ADVANCE until n_A ≥ K, then STOP.' This policy succeeds almost surely (by the second Borel–Cantelli lemma, since each ADVANCE succeeds independently with probability p > 0, and infinitely many attempts yield infinitely many successes almost surely). Its expected stopping time is E[T] = K/p, which is finite. Now consider any policy π with E_π[T] < revision: no

Circularity Check

0 steps flagged

No significant circularity found

full rationale

The paper has two components, both self-contained. (1) The empirical redundancy measure ρ (Definition 1) is computed directly via progressive forced-termination — no parameters are fitted to data and then 'predicted back.' The external-judge replication uses a completely different decoder. (2) Theorem 1 is proved from first principles in Appendix A using only the stated assumptions (i.i.d. Bernoulli advances, length-agnostic reward, Borel-Cantelli). No self-citations serve as load-bearing premises — all citations are to external work. The paper explicitly acknowledges that the theorem does not predict the specific ρ values measured empirically ('Theorem 1 does not, however, predict the functional form of ρ(d)'), which is an honest scope limitation rather than circularity. The skeptic's concern about the proof's internal consistency (conditional distribution requiring unobservability vs. π′ requiring observability of n_A) is a correctness issue, not a circularity issue — it does not involve the result being equivalent to its inputs by construction. The theorem is a formalization of a simple intuition ('no stopping cost → never stop is optimal'), but it is properly derived with explicit assumptions and a rigorous proof, not merely a renaming. Score 1 reflects the theorem's simplicity but confirms no circular derivation chain.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 2 invented entities

The paper introduces two invented entities (ADVANCE/IDLE/STOP taxonomy, difficulty parameter K) and one ad-hoc axiom (i.i.d. Bernoulli advances) to make the theorem tractable. These are not independently validated against real LLM reasoning. The empirical measurements (Definition 1, ρ) are parameter-free and do not depend on the theoretical model. The free parameters (p, K, λ) are abstract, not fitted to data.

free parameters (3)
  • p (ADVANCE success probability) = not fitted; stated as p ∈ (0,1)
    Used in the sequential-decision model of §6. Not fitted to data; treated as an abstract parameter. The theorem's conclusion holds for any p ∈ (0,1).
  • K (problem difficulty as count of required successful advances) = not fitted; stated as K ∈ ℕ
    Used in §6 to define the outcome reward R = 1[n_A ≥ K]. Not fitted to data; treated as an abstract parameter. No operational definition is given for real traces.
  • λ (length penalty coefficient) = not fitted; λ = 0 for length-agnostic case
    Introduced in J_λ(π) = E[R] - λE[T]. The theorem concerns λ = 0. The prescription is to set λ > 0, but no specific value is derived or fitted.
axioms (5)
  • ad hoc to paper Reasoning decomposes into discrete steps each classifiable as ADVANCE, IDLE, or STOP.
    Introduced in §6 to model reasoning as a sequential decision process. No empirical validation that real LLM traces admit this decomposition. The theorem's proof depends on this structure.
  • ad hoc to paper Each ADVANCE attempt succeeds independently with probability p ∈ (0,1).
    Stated in §6: 'ξ_t i.i.d. ~ Bernoulli(p)'. The i.i.d. assumption is required for the second Borel–Cantelli lemma in the proof (Appendix A). Real reasoning steps are sequentially dependent.
  • domain assumption The outcome reward depends only on whether n_A ≥ K (answer correctness), not on trace length or content.
    Stated in §6 as the definition of length-agnostic training. This is the paper's central modeling assumption and is empirically justified by surveying the training recipes of R1, QwQ, R1-Distill, and Qwen3.
  • domain assumption The critical point k*(r) is well-defined: there exists a smallest k at which π, forced to terminate, produces the correct answer.
    Definition 1 in §3. Assumes that forced termination after a prefix yields a deterministic answer (sampling at T=0). Validated operationally but assumes the forced-termination protocol is meaningful.
  • domain assumption Segmentation by paragraph boundaries and discourse markers produces meaningful reasoning steps.
    §4 and Appendix C.1. The three-step segmentation procedure (split on double-newline, cut before discourse markers, merge <12-word fragments) is deterministic but its alignment with actual reasoning steps is assumed, not validated. Robustness check in Appendix B.12 shows ρ varies by ≤0.3 points across merge thresholds.
invented entities (2)
  • ADVANCE/IDLE/STOP action taxonomy no independent evidence
    purpose: Models reasoning steps as either substantive progress attempts (ADVANCE), non-advancing content (IDLE), or termination (STOP) in the sequential-decision framework of §6.
    Introduced to make the theorem tractable. No empirical validation that real traces decompose into these three categories. The paper does not operationalize ADVANCE vs IDLE on actual traces.
  • Difficulty parameter K (count of required successful advances) no independent evidence
    purpose: Defines the outcome threshold in the sequential-decision model: reward R = 1[n_A ≥ K].
    Abstract parameter with no operational definition for real problems. The paper uses MATH-500 difficulty levels (1–5) empirically but never connects them to K.

pith-pipeline@v1.1.0-glm · 43189 in / 3990 out tokens · 255637 ms · 2026-07-05T09:54:14.232445+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

28 extracted references · 28 canonical work pages · 20 internal anchors

  1. [1]

    L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning

    Pranjal Aggarwal and Sean Welleck. L1: Controlling how long a reasoning model thinks with reinforcement learning.arXiv preprint arXiv:2503.04697,

    work page internal anchor Pith review Pith/arXiv arXiv

  2. [2]

    Training language models to reason efficiently.arXiv preprint arXiv:2502.04463,

    Daman Arora and Andrea Zanette. Training language models to reason efficiently.arXiv preprint arXiv:2502.04463,

    work page arXiv

  3. [3]

    Large Language Monkeys: Scaling Inference Compute with Repeated Sampling

    Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprint arXiv:2407.21787,

    work page internal anchor Pith review Pith/arXiv arXiv

  4. [4]

    Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs

    Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, et al. Do not think that much for 2+3=? on the overthinking of o1-like LLMs.arXiv preprint arXiv:2412.21187,

    work page internal anchor Pith review Pith/arXiv arXiv

  5. [5]

    Training Verifiers to Solve Math Word Problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, et al. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168,

    work page internal anchor Pith review Pith/arXiv arXiv

  6. [6]

    Complexity-Based Prompting for Multi-Step Reasoning

    Yao Fu, Hao Peng, Ashish Sabharwal, Peter Clark, and Tushar Khot. Complexity-based prompting for multi-step reasoning.arXiv preprint arXiv:2210.00720,

    work page internal anchor Pith review Pith/arXiv arXiv

  7. [7]

    Adaptive Computation Time for Recurrent Neural Networks

    Alex Graves. Adaptive computation time for recurrent neural networks.arXiv preprint arXiv:1603.08983,

    work page internal anchor Pith review Pith/arXiv arXiv

  8. [8]

    Reinforced Self-Training (ReST) for Language Modeling

    Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, et al. Reinforced self-training (ReST) for language modeling.arXiv preprint arXiv:2308.08998,

    work page internal anchor Pith review Pith/arXiv arXiv

  9. [9]

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, et al. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning.arXiv preprint arXiv:2501.12948,

    work page internal anchor Pith review Pith/arXiv arXiv

  10. [10]

    Token- budget-aware LLM reasoning.arXiv preprint arXiv:2412.18547,

    Tingxu Han, Zhenting Wang, Chunrong Fang, Shiyu Zhao, Shiqing Ma, and Zhenyu Chen. Token- budget-aware LLM reasoning.arXiv preprint arXiv:2412.18547,

    work page arXiv

  11. [11]

    Training Large Language Models to Reason in a Continuous Latent Space

    Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space.arXiv preprint arXiv:2412.06769,

    work page internal anchor Pith review Pith/arXiv arXiv

  12. [12]

    OpenAI o1 System Card

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, et al. OpenAI o1 system card.arXiv preprint arXiv:2412.16720,

    work page internal anchor Pith review Pith/arXiv arXiv

  13. [13]

    Kimi k1.5: Scaling Reinforcement Learning with LLMs

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, et al. Kimi k1.5: Scaling reinforcement learning with LLMs.arXiv preprint arXiv:2501.12599,

    work page internal anchor Pith review Pith/arXiv arXiv

  14. [14]

    Understanding R1-Zero-Like Training: A Critical Perspective

    11 Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding R1-zero-like training: A critical perspective.arXiv preprint arXiv:2503.20783,

    work page internal anchor Pith review Pith/arXiv arXiv

  15. [15]

    s1: Simple test-time scaling

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling.arXiv preprint arXiv:2501.19393,

    work page internal anchor Pith review Pith/arXiv arXiv

  16. [16]

    Magistral

    Abhinav Rastogi, Albert Q Jiang, Andy Lo, Gabrielle Berrada, Guillaume Lample, et al. Magistral. arXiv preprint arXiv:2506.10910,

    work page arXiv

  17. [17]

    Rethinking reflection in pre-training.arXiv preprint arXiv:2504.04022,

    Darsh J Shah, Peter Rushton, Somanshu Singla, Mohit Parmar, Kurt Smith, Yash Vanjani, et al. Rethinking reflection in pre-training.arXiv preprint arXiv:2504.04022,

    work page arXiv

  18. [18]

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, et al. DeepSeek- Math: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300,

    work page internal anchor Pith review Pith/arXiv arXiv

  19. [19]

    Solving math word problems with process- and outcome-based feedback

    Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. Solving math word problems with process- and outcome-based feedback.arXiv preprint arXiv:2211.14275,

    work page internal anchor Pith review Pith/arXiv arXiv

  20. [20]

    Qwen2.5 Technical Report

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, et al. Qwen2.5 technical report.arXiv preprint arXiv:2412.15115,

    work page internal anchor Pith review Pith/arXiv arXiv

  21. [21]

    Qwen3 Technical Report

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    work page internal anchor Pith review Pith/arXiv arXiv

  22. [22]

    DAPO: An Open-Source LLM Reinforcement Learning System at Scale

    12 Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, et al. DAPO: An open-source LLM reinforcement learning system at scale.arXiv preprint arXiv:2503.14476,

    work page internal anchor Pith review Pith/arXiv arXiv

  23. [23]

    SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild

    Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. SimpleRL- Zoo: Investigating and taming zero reinforcement learning for open base models in the wild.arXiv preprint arXiv:2503.18892,

    work page internal anchor Pith review Pith/arXiv arXiv

  24. [24]

    Echo chamber: RL post-training amplifies behaviors learned in pretraining.arXiv preprint arXiv:2504.07912,

    Rosie Zhao, Alexandru Meterez, Sham Kakade, Cengiz Pehlevan, Samy Jelassi, and Eran Malach. Echo chamber: RL post-training amplifies behaviors learned in pretraining.arXiv preprint arXiv:2504.07912,

    work page arXiv

  25. [25]

    Least-to-Most Prompting Enables Complex Reasoning in Large Language Models

    Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, et al. Least-to-most prompting enables complex reasoning in large language models.arXiv preprint arXiv:2205.10625,

    work page internal anchor Pith review Pith/arXiv arXiv

  26. [26]

    all M attempts fail

    π does not succeed almost surely.Let M:= PT t=1 1[at =ADVANCE] denote the total number of ADVANCE actions π emits before stopping. Because M≤T and Eπ[T]<∞ , M is almost surely finite. Conditional on M, nA ∼Bin(M, p) . The event “all M attempts fail” is contained in {nA < K} (since K≥1 ) and has conditional probability (1−p) M >0 whenever M <∞ and p <1. Ta...

    work page 1946

  27. [27]

    k⋆ = 1 segmented step

    Marker shape encodes benchmark; colour encodes model. All eight points lie on the y=x line up to±5points. Definition 1 provides two natural measures of redundancy: a step-level ratio ρ and a word-level ratio ρL. These could in principle diverge — if redundant steps were systematically shorter or longer 15 than informative steps, the two measures would pul...

    work page 2024

  28. [28]

    So”, “Therefore

    Accuracy on the retained set peaks at α= 1.0 (87.6% at 343 average tokens, up from 79.8% at 1106 average tokens unconstrained). This is apost-hocobservation of the data already generated, not an inference-time mitigation. Table 10:Difficulty-aware budget sweep, fine grid (DeepSeek-R1 on MATH-500).Each row reports the retained-set accuracy and average toke...

    work page 2024