Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

Walk Before You Run! Concise LLM Reasoning via Reinforcement Learning

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A two-stage reinforcement learning schedule that first trains reasoning ability and only then, when every sampled response in a group is correct, optimizes for shorter chain-of-thought produces more concise responses and beats zero-RL…

desk verdict Two-stage RL recipe with an all-correct-gated length reward is a sensible, concrete contribution, but the 'outperforms baselines' claim rests on unmatched evaluation protocols and no error bars. read the letter →

arxiv 2505.21178 v1 pith:JAQ7W3LZ submitted 2025-05-27 cs.CL

Mingyang Song , Mao Zheng This is my paper · ORCID
classification cs.CL
keywords concisereasoningchain-of-thoughtreinforcementlearningGRPOlengthrewardoverthinkingLLMzeroRLparadigm
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a two-stage reinforcement learning schedule can make a large language model reason more concisely without losing accuracy. The first stage strengthens the model's problem-solving ability through GRPO++ (GRPO with clip-higher, dynamic sampling, and an entropy bonus); the second stage applies a length-aware reward that only activates when every sampled response for a question is correct. Trained this way, ConciseR-Zero-7B produces noticeably shorter chain-of-thought responses and, in the paper's comparison, outperforms existing zero-RL reasoning baselines across five competition-level math benchmarks. The point matters because long chain-of-thought responses are expensive to generate and often contain redundant 'overthinking', so a method that cuts length while keeping accuracy has a direct computational payoff.

What carries the argument

The load-bearing object is the length-aware reward used in L-GRPO: for a group of $G$ responses to one question, the reshaped reward is $\hat r_i = r_i + \lambda \hat L_i$ with $\hat L_i = 1 - L_i/L_{\mathrm{Max}}$ when $\sum_i r_i = G$ (all rollouts correct) and $\hat L_i = 0$ otherwise. This keeps the standard binary accuracy reward for the group-relative advantage computation and adds a gentle pressure to finish within fewer tokens, but only after the group has already solved the problem. The 'walk before you run' gate is what distinguishes L-GRPO from naive length penalties: it prevents the policy from shortening responses by giving up on hard reasoning, the failure mode the paper documents in its ablation of direct length rewards. GRPO++ supplies the first-stage machinery—asymmetric clip-higher bounds, dynamic sampling, and an entropy bonus—that lets the model acquire the reasoning ability before compression begins.

What would settle it

Run ConciseR-Zero-7B and each zero-RL baseline in Table 3 through one identical evaluation script with the same 32-sample temperature-0.6 pass@1 protocol; if the average accuracy gap over Oat-Zero-7B (56.1 vs 50.4) closes or reverses, the central outperformance claim fails. Separately, retrain stage 2 with the length reward applied unconditionally; if accuracy falls, the all-correct gate is doing the work.

Watch

Extended reading notes

Core claim

The central claim is that concise reasoning can be trained without trading away accuracy. Starting from Qwen2.5-Math-7B, the paper first runs GRPO++ to incentivize correct problem solving, then switches to L-GRPO, which reshapes the group-relative reward to include a length term $\hat L_i = 1 - L_i/L_{\mathrm{Max}}$ only in groups where all rollouts are correct. This gating embodies the 'walk before you run' principle: the model is never rewarded for brevity until it has already achieved correctness on that example. On AIME 2024, MATH-500, AMC 2023, Minerva, and Olympiad, the resulting model averages 56.1% pass@1 (k=32), above the best zero-RL baseline's 50.4%, while response lengths fall roughly 20% to 23% across the five benchmarks. The paper also reports that the first stage alone already surpasses the zero-RL baselines, suggesting the compression stage adds efficiency on top of a stronger reasoning model.

Load-bearing premise

The central comparison assumes that the paper's pass@1 scores, computed with 32 sampled responses at temperature 0.6, are numerically comparable to baseline numbers in Table 3 that come from other papers using possibly different decoding protocols (greedy or different sample counts); if that comparability fails, the claim that ConciseR outperforms the baselines loses its support.

Editorial extensions

If this is right

  • Under the paper's evaluation protocol, ConciseR-Zero-7B scores 56.1 average pass@1 across AIME 2024, MATH-500, AMC 2023, Minerva, and Olympiad, above the best zero-RL baseline's 50.4.
  • Response lengths drop by about 20% to 23% across the five benchmarks after stage 2, while per-benchmark accuracy stays flat or improves.
  • The all-correct gate prevents the length reward from making the model abandon reasoning: in the paper's ablations, direct length rewards caused empty reasoning and answer guessing.
  • GRPO++ on its own already beats all zero-RL baselines, so the conciseness stage layers efficiency onto a model that has first been made stronger.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The gating idea should transfer to other verifiable efficiency targets—e.g., fewer tool calls, lower latency, or shorter proofs—whenever a group of rollouts can be certified correct.
  • The observed shift toward Python-based verification during stage 2 suggests that compression pressure may push models toward more reliable external checks; this is a pattern the paper displays, not a claim it establishes.
  • Because the comparison mixes evaluation protocols, the compression effect (a within-paper, matched-protocol result) is more robust than the ranking claim; a matched-protocol rerun could plausibly preserve the length reduction but reorder the baselines.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes ConciseR, a two-stage reinforcement learning framework for training math reasoning models (starting from Qwen2.5-Math-7B) to produce shorter chain-of-thought responses without sacrificing accuracy. The first stage trains reasoning ability via GRPO++ (GRPO with clip-higher, dynamic sampling, and an entropy bonus); the second stage applies L-GRPO, which adds a length reward that is active only when all sampled rollouts for a training question are correct. The authors report that ConciseR outperforms recent zero-RL reasoning models on AIME 2024, MATH-500, AMC 2023, Minerva, and OlympiadBench while reducing response length by roughly 20%.

Significance. If the claims hold, the work is practically valuable: a roughly 20% inference-cost reduction at approximately constant accuracy would meaningfully lower deployment costs for long-CoT reasoning models. The proposed gating heuristic ('walk before you run') is simple and interesting, and the paper honestly documents a failure mode of a direct length-reward variant. The significance is currently limited by the absence of same-protocol baseline comparisons and the lack of any uncertainty quantification, both of which are required to support the headline 'outperforms' claim.

major comments (4)
  1. [Sec. 4 'Evaluation Setup' and Table 3] The pass@1 numbers for ConciseR are computed with 32 samples at temperature 0.6, while the baseline numbers marked with a dagger are taken from Hochlehnert et al. [20] and were produced under heterogeneous decoding protocols (varying sample counts, temperatures, or greedy decoding). Pass@1 is sensitive to the sampling protocol, so the few-point gaps over Oat-Zero-7B (50.4 vs 56.1 average) do not by themselves establish that ConciseR outperforms these baselines. Please re-evaluate at least the strongest zero-RL baselines under the exact ConciseR protocol, or provide protocol-matched numbers from the original sources.
  2. [Sec. 4.1 and Figure 3] No error bars, confidence intervals, or multiple seeds are reported for any accuracy estimate. AIME 2024 has only 30 questions, so a 3-5 point difference can easily arise from sampling noise; the same concern applies to Minerva and OlympiadBench. The claim that accuracy remains stable across L-GRPO steps is not supported without quantifying the sampling variability. Please provide bootstrap confidence intervals for the pass@1 estimates or repeat the evaluation under additional random seeds.
  3. [Sec. 3.2 and Sec. 5.3] The all-correct gating condition is the paper's central design principle, but it is not ablated. The failure-experience experiments in Section 5.3 compare L-GRPO against two alternative length-reward formulations, but they do not test whether the gating on all-correct rollouts is necessary or superior to, for example, applying the length reward unconditionally or to groups with at least one correct answer. Without such an ablation, the claim that this gating is what preserves accuracy is unsupported.
  4. [Abstract and Sec. 4.1] The abstract states that ConciseR 'generates more concise CoT reasoning responses,' but no response-length numbers for any baseline model are reported. The length reductions of 21-23% cited in Section 4.1 are relative to ConciseR's own first-stage model. To substantiate the comparative conciseness claim, please report average output lengths for the baselines under the same evaluation protocol.
minor comments (6)
  1. [Sec. 4 'Evaluation Benchmarks'] The benchmark name 'OlympaidBench' should be spelled 'OlympiadBench'.
  2. [Sec. 4 'Evaluation Setup'] The sentence 'The used training template is shown in Figure 2' should refer to Table 2, which actually contains the template.
  3. [Sec. 5.1 and Figure captions] The text says 14,022 responses correspond to 1,558 questions times 11 test times, but 1,558 times 11 is 17,138; Figure 6 uses 15,580 for the same product. Please correct the arithmetic so the counts match.
  4. [Sec. 4.1] The claim of an 'average accuracy improvement of 55.2%' compared to the base model is unclear: Table 3 shows 37.5 to 56.1, which is a 49.6% relative improvement or an 18.6-point absolute gain. Please specify the formula used.
  5. [Sec. 7 Conclusion] The conclusion says ConciseR 'significantly outperforming existing efficient reasoning methods' without the zero-RL qualifier, which is too broad given that FastCuRL-1.5B-V3 and DeepScaleR-1.5B-Preview in Table 3 have higher or comparable averages. Please qualify the claim.
  6. [Eq. (8) vs. Eq. (6)] Equation (8) uses the constraint '0 < |{oi|is_equivalent(oi,a)}|' while Equation (6) uses '0 < ... < G'; please explain whether all-correct groups are intentionally included in L-GRPO updates and align the notation.

Circularity Check

1 steps flagged · score 2.0 of 10

No significant circularity: accuracy claims are externally benchmarked; only the length-reduction result is entailed by L-GRPO's own reward definition.

  1. self definitional [Section 3.2, Eq. (7); Section 4.1 (Main Results)]
    "When the model's rollout results for a question are all correct, we further optimize the model's reasoning length for that question by using the remaining maximum response length as a reward (under the specified context length, the more remaining context length, the higher the reward) ... the average response length on each benchmark consistently decreases, with reductions of 21%, 22%, 20%, 22%, and 23% observed on AIME 2024, MATH-500, AMC 2023, Minerva, and Olympiad benchmarks, respectively."

    In Eq. (7), for all-correct groups the auxiliary reward is L-hat_i = 1 - L_i / L_Max, so longer rollouts are assigned lower rewards and lower group-relative advantages. The 21-23% length reductions in Section 4.1 are therefore entailed by the reward definition; the paper is transparent that the second stage 'explicitly enforces conciseness.' This is a minor self-definitional observation, not a hidden prediction. It does not affect the accuracy claims, which are evaluated on external benchmarks and are not fitted inputs.

full rationale

The central derivation chain is otherwise self-contained: GRPO++ adopts DAPO's clip-higher and dynamic sampling with an added entropy bonus, and L-GRPO adds a length reward gated on all-correct groups. The 'walk before you run' accuracy-stability claim is empirical, not by construction. The authors' prior FastCuRL [11] is cited as a baseline and related work, but the head-to-head numbers are empirical and not used as a proof step; under the rules, this self-citation is not load-bearing and does not raise the score. The Eq. (7) length reward is the only by-construction element, and the score is low because the paper's headline accuracy comparison is externally grounded. The mixed-protocol pass@1 caveat in Table 3 is a benchmark-comparability risk, not a circularity. No uniqueness theorem, no ansatz smuggled via citation, and no renamed known result were found.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The method is a combination of known RL components with a new gating condition. The free parameters are all hand-tuned hyperparameters. No new physical or conceptual entities are introduced.

free parameters (6)
  • lambda (length reward weight) = 2e-6
    Controls trade-off between accuracy and length in L-GRPO (Eq. 7). Chosen by hand.
  • alpha (entropy bonus coefficient) = 0.001
    Added to GRPO++ objective (Eq. 6) to encourage exploration; hand-tuned.
  • beta (KL penalty coefficient) = 0.01
    Constrains deviation from reference policy in L-GRPO (Eq. 8).
  • epsilon_h and epsilon_l (clip bounds) = 0.28 and 0.2
    Clip-higher components from DAPO; set as in prior work.
  • L_max (max response length) = 3072 tokens
    Normalization factor in length reward and generation limit; chosen as evaluation cap.
  • Sampling temperature and top-p = 0.6 and 0.95
    Decoding hyperparameters used for pass@1 estimation.
assumptions (4)
  • domain assumption Correct CoT responses are statistically shorter than incorrect ones for the models under study
    Motivates the length reward design in Section 3.2 and Table 1; observed on DeepSeek-R1-Distill-Qwen-1.5B/7B, assumed to hold for the trained Qwen2.5-Math-7B policy.
  • ad hoc to paper Optimizing response length only when all rollouts for a sample are correct preserves accuracy
    Central gating rule in L-GRPO (Eq. 7); no quantitative ablation compares it to an unconditional length reward, only an anecdotal failed alternative in Section 5.3.
  • domain assumption MATH Level 3-5 alone is a sufficient stage-2 training distribution to transfer conciseness to all five evaluation benchmarks
    Section 3.4 states stage 2 uses only MATH (Level 3-5) data; the paper relies on this transferring to AIME, AMC, Minerva, Olympiad.
  • domain assumption Math-Verify based rule reward correctly identifies answer equivalence
    Rule-based reward in Eq. 9 assumes reliable answer extraction and matching from the generated text.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Walk Before You Run! Concise LLM Reasoning via Reinforcement Learning." pith.science (2026). https://pith.science/paper/JAQ7W3LZ

@misc{pith2026250521178,
  author       = {Pith},
  title        = {Pith review of: Walk Before You Run! Concise LLM Reasoning via Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JAQ7W3LZ}},
  note         = {Machine review of arXiv:2505.21178}
}
read the original abstract

As test-time scaling becomes a pivotal research frontier in Large Language Models (LLMs) development, contemporary and advanced post-training methodologies increasingly focus on extending the generation length of long Chain-of-Thought (CoT) responses to enhance reasoning capabilities toward DeepSeek R1-like performance. However, recent studies reveal a persistent overthinking phenomenon in state-of-the-art reasoning models, manifesting as excessive redundancy or repetitive thinking patterns in long CoT responses. To address this issue, in this paper, we propose a simple yet effective two-stage reinforcement learning framework for achieving concise reasoning in LLMs, named ConciseR. Specifically, the first stage, using more training steps, aims to incentivize the model's reasoning capabilities via Group Relative Policy Optimization with clip-higher and dynamic sampling components (GRPO++), and the second stage, using fewer training steps, explicitly enforces conciseness and improves efficiency via Length-aware Group Relative Policy Optimization (L-GRPO). Significantly, ConciseR only optimizes response length once all rollouts of a sample are correct, following the "walk before you run" principle. Extensive experimental results demonstrate that our ConciseR model, which generates more concise CoT reasoning responses, outperforms recent state-of-the-art reasoning models with zero RL paradigm across AIME 2024, MATH-500, AMC 2023, Minerva, and Olympiad benchmarks.

Figures

Figures reproduced from arXiv: 2505.21178 by the authors.

Figure 1
Figure 1. A detailed evaluation of accuracy and response length throughout the training steps. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Distribution of average length for correct and incorrect answers under the same question. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. A detailed evaluation of accuracy and response length throughout the training steps. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Comparison of responses of ConciseR-Zero-7B-Preview to the same question. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Count of keyword occurrences out of 14,022 responses (1558 questions [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Count of keyword occurrences out of 15,580 responses (1558 questions [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Illustration of cases. rˆi = Gri̸=0 − 1 Gri̸=0 ri + Lˆ i PG i=1 Lˆ i , Lˆ i = ( 0, if is_equivalent(oi , a) 1 − P Li G i=1 Li , if not is_equivalent(oi , a) . (11) However, we find that this direct rewarding easily causes the model to skip the reasoning process and imm…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Token-Operations-Oriented Inference Optimization Techniques for Large Models

    cs.SE 2026-06 unverdicted novelty 3.0 of 10

    The paper introduces a four-layer technical architecture for token-operations-oriented inference optimization in large models and reviews key technologies and industry status at each layer.

  2. Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey

    cs.AI 2025-07 conditional novelty 3.0 of 10

    A comprehensive review that categorizes methods for shortening and adaptively triggering chain-of-thought reasoning in large language models.

Reference graph

Works this paper leans on

29 extracted references · 4 canonical work pages · cited by 2 Pith papers

  1. [20]

    A sober look at progress in language model reasoning: Pitfalls and paths to reproducibility, 2025

    Andreas Hochlehnert, Hardik Bhatnagar, Vishaal Udandarao, Samuel Albanie, Ameya Prabhu, and Matthias Bethge. A sober look at progress in language model reasoning: Pitfalls and paths to reproducibility, 2025

    work page 2025

  2. [1]

    Scaling llm test-time compute optimally can be more effective than scaling model parameters, 2024

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters, 2024

    2024

  3. [2]

    s1: Simple test-time scaling

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393, 2025

    arXiv 2025

  4. [3]

    Chi, Quoc V Le, and Denny Zhou

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022

    2022

  5. [4]

    Openai o1 system card

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720, 2024

    arXiv 2024

  6. [5]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    arXiv 2025

  7. [6]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024

    2024

  8. [7]

    Demystifying long chain-of-thought reasoning in llms, 2025

    Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying long chain-of-thought reasoning in llms, 2025

    2025

Show all 29 references
  1. [8]

    Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Li Erran Li, Raluca Ada Popa, and Ion Stoica

    Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y . Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deep- scaler: Surpassing o1-preview with a 1.5b model by scaling rl. https://github.com/ agentica-project/d...

    2025

  2. [9]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599, 2025

    2025 arXiv

  3. [10]

    Gonzalez

    Alejandro Cuadron, Dacheng Li, Wenjie Ma, Xingyao Wang, Yichuan Wang, Siyuan Zhuang, Shu Liu, Luis Gaspar Schroeder, Tian Xia, Huanzhi Mao, Nicholas Thumiger, Aditya Desai, Ion Stoica, Ana Klimovic, Graham Neubig, and Joseph E. Gonzalez. The danger of overthinking: Examining t...

    2025

  4. [11]

    Fastcurl: Curriculum reinforcement learning with progressive context extension for efficient training r1-like reasoning models, 2025

    Mingyang Song, Mao Zheng, Zheng Li, Wenjie Yang, Xuan Luo, Yue Pan, and Feng Zhang. Fastcurl: Curriculum reinforcement learning with progressive context extension for efficient training r1-like reasoning models, 2025

    2025

  5. [12]

    Dapo: An open-source llm reinforcement learning system at scale, 2025

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wa...

    2025

  6. [13]

    Understanding r1-zero-like training: A critical perspective

    Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding r1-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783, 2025

    2025 arXiv

  7. [14]

    Concise reasoning via reinforcement learning

    Mehdi Fatemi, Banafsheh Rafiee, Mingjie Tang, and Kartik Talamadupula. Concise reasoning via reinforcement learning. arXiv preprint arXiv:2504.05185, 2025

    2025

  8. [15]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  9. [16]

    Measuring mathematical problem solving with the math dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021. 11

    2021 arXiv

  10. [17]

    Solving quantitative reasoning problems with language models

    Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al. Solving quantitative reasoning problems with language models. Advances in Neural Information Processing Systems, 35...

    2022

  11. [18]

    Omni-math: A universal olympiad level mathematic benchmark for large language models

    Bofei Gao, Feifan Song, Zhe Yang, Zefan Cai, Yibo Miao, Qingxiu Dong, Lei Li, Chenghao Ma, Liang Chen, Runxin Xu, Zhengyang Tang, Benyou Wang, Daoguang Zan, Shanghaoran Quan, Ge Zhang, Lei Sha, Yichang Zhang, Xuancheng Ren, Tianyu Liu, and Baobao Chang. Omni-math: A universal ...

    arXiv

  12. [19]

    Imitate, explore, and self-improve: A reproduction report on slow-thinking reasoning systems, 2024

    Yingqian Min, Zhipeng Chen, Jinhao Jiang, Jie Chen, Jia Deng, Yiwen Hu, Yiru Tang, Jiapeng Wang, Xiaoxue Cheng, Huatong Song, Wayne Xin Zhao, Zheng Liu, Zhongyuan Wang, and Ji-Rong Wen. Imitate, explore, and self-improve: A reproduction report on slow-thinking reasoning systems, 2024

    2024

  13. [21]

    Qwen2.5 technical report.arXiv preprint arXiv:2412.15115, 2024

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 arXiv

  14. [22]

    Qwen2.5-math technical report: Toward mathematical expert model via self-improvement, 2024

    An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang. Qwen2.5-math technical report: Toward mathematical expert model via se...

    2024

  15. [23]

    Process reinforcement through implicit rewards

    Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, et al. Process reinforcement through implicit rewards. arXiv preprint arXiv:2502.01456, 2025

    2025 arXiv

  16. [24]

    Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model, 2025

    Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum. Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model, 2025

    2025

  17. [25]

    Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild, 2025

    Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild, 2025

    2025

  18. [26]

    Hybridflow: A flexible and efficient rlhf framework

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. In Proceedings of the Twentieth European Conference on Computer Systems, page 1279–1297. ACM, March 2025

    2025

  19. [27]

    Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems

    Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, et al. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. In Proceedings of the 62nd Annu...

    2024

  20. [28]

    Open r1: A fully open reproduction of deepseek-r1, January 2025

    Hugging Face. Open r1: A fully open reproduction of deepseek-r1, January 2025

    2025

  21. [29]

    Light-r1: Curriculum sft, dpo and rl for long cot from scratch and beyond

    Liang Wen, Yunke Cai, Fenrui Xiao, Xin He, Qi An, Zhenyu Duan, Yimin Du, Junchen Liu, Lifu Tang, Xiaowei Lv, et al. Light-r1: Curriculum sft, dpo and rl for long cot from scratch and beyond. arXiv preprint arXiv:2503.10460, 2025. 12

    2025 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.