Pith. sign in

REVIEW 3 major objections 6 minor 6 cited by

One-shot Entropy Minimization

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that minimizing a language model's token-level entropy on a single unlabeled prompt for ten optimization steps improves reasoning performance as much as, or more than, rule-based reinforcement learning trained on…

desk verdict The 13,440-run study is real and the EM effect is plausible, but the 'unlabeled one-shot' claim is contradicted by the label-based data selection in Section 2.2. read the letter →

arxiv 2505.20282 v4 pith:PVUK3DSP submitted 2025-05-26 cs.CL

Zitian Gao , Lynx Chen , Haoming Luo , Joey Zhou , Bryan Dai This is my paper
classification cs.CL
keywords entropyminimizationpost-trainingreinforcementlearninglargelanguagemodelstokenrerankinglogitsshiftvariance-baseddataselectionmathreasoning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Minimizing the token-level entropy of a language model's own predictions on a single unlabeled prompt, for ten optimization steps, is claimed to improve reasoning performance as much as or more than rule-based reinforcement learning trained on thousands of labeled examples with hand-designed rewards. The authors trained 13,440 models to show the effect holds across seeds and report large accuracy jumps on math, logic, and code benchmarks. They argue that entropy minimization and RL both work by 'token reranking' that reshapes the model's output distribution, but EM shifts the logit distribution rightward (more confidence in high-probability tokens) while RL shifts it leftward. They conclude that EM is a distribution-shaping tool rather than a learning method, so it can act as a nearly free post-training boost. If this claim is right, much of the data collection and reward engineering behind RL post-training becomes unnecessary for improving reasoning.

What carries the argument

The central object is the one-shot entropy minimization loss, the average token-level entropy over generated positions: $\mathcal{L}_{EM}(x;\theta)=\frac{1}{|I|}\sum_{t\in I} H_t$, with $H_t=-\sum_{v\in V} p_\theta(v|y_{<t},x)\log p_\theta(v|y_{<t},x)$. This loss is fully differentiable and needs no reward model, labels, or value baseline. The second load-bearing piece is variance-based data selection: from an unlabeled pool, pick the single prompt $x^*$ that maximizes the sample variance of pass@k correctness across multiple sampled responses, on the grounds that such prompts sit at the model's decision boundary and produce the largest entropy gradients. Together these turn entropy minimization into a ten-step, one-example procedure that the paper claims rivals RL.

What would settle it

Take a prompt whose correct answer is a rare, low-probability token but whose incorrect answer is high-confidence, run one-shot EM on that prompt, and check whether held-out accuracy falls instead of rises; this would directly test whether low entropy is actually correlated with correctness. A simpler version is to correlate, across many prompts, the model's entropy on its own final answer with whether that answer is correct, and show whether low-entropy wrong answers occur often enough to break the method.

Watch

Extended reading notes

Core claim

The central claim is that entropy minimization is a fully unsupervised post-training method that rivals or surpasses rule-based RL. The loss is the average conditional entropy over the generated tokens, and minimizing it makes the model more confident in its own token predictions. With a single unlabeled prompt chosen for high variance in the model's pass@k correctness, ten steps at learning rate 2e-5 and temperature 0.5 raised a 7-billion-parameter math base model from 53.0 to 78.8 on MATH500 and produced comparable jumps on other math benchmarks, a logic benchmark, and a code benchmark. The paper also reports that EM's training loss keeps decreasing past step 10 while benchmark scores decline, an 'overconfidence' effect, and that EM-trained models have strongly right-skewed logits whereas RL-trained models have left-skewed logits; it frames EM as confidence shaping that concentrates probability mass on high-probability reasoning paths, and finds that EM before RL helps while EM after RL hurts.

Load-bearing premise

The method assumes that a model's low-entropy token choices tend to be correct answers, so making the model more confident in its own predictions generally improves accuracy; if a confidently wrong answer has low entropy, EM will lock in that error.

Editorial extensions

If this is right

  • One unlabeled prompt and ten optimization steps can substitute for thousands of labeled examples and hand-designed rewards when improving reasoning performance.
  • Continuing EM beyond roughly ten steps degrades performance even as the loss falls, so early stopping is a built-in requirement, not optional.
  • EM-trained models perform best at low sampling temperatures (greedy decoding), while RL-trained models improve at higher temperatures, so the two methods have opposite inference-time behavior.
  • Applying EM before RL gives consistent gains, but applying EM after RL degrades performance, which constrains how post-training methods can be sequenced.
  • The ceiling of EM's gains is set by the base model's intrinsic reasoning strength, so weak base models gain little.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not independently test whether low-entropy predictions tend to be correct; if confidently wrong answers are common, one-shot EM could reinforce errors, and the observed gains would depend on the luck of the chosen prompt.
  • Because EM shifts logits rightward and prefers greedy decoding, its gains may be partly reproducible by temperature scaling alone; comparing one-shot EM against inference-time temperature reduction would isolate what the parameter updates actually add.
  • The paper's claim that EM is distribution shaping, not learning, implies EM cannot add knowledge; it can only expose latent ability, so one-shot EM could serve as a cheap diagnostic of a model's untapped reasoning quality before investing in RL.
  • Seed-level variance in results (scores can differ by as much as a factor of two across identical settings) means the headline numbers are best-case or seed-lucky points; reporting the full seed distribution would be needed to know how robust the single-prompt result is.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes one-shot entropy minimization (EM) as a post-training method: minimize token-level entropy on a single prompt for only 10 optimization steps. The authors report that this achieves performance comparable to or better than rule-based RL baselines that use thousands of examples, on math, logic, and code benchmarks. The study is large-scale (13,440 trained models) and includes analyses of logit shifts, temperature sensitivity, and EM before/after RL.

Significance. If the claims held, this would be a striking result: a nearly free, fully unsupervised post-training boost that rivals RL. The paper's strengths are its scale, the release of code, and the attempt to characterize EM's behavior through logits and temperature analyses. However, the central claim is currently undermined by label-based data selection and by reporting best-of-seed results, so the headline contribution is not yet established.

major comments (3)
  1. [Section 2.2, Eq. (2)-(3)] The data-selection step is not label-free. Computing Var_pass@k(x) requires the indicator I[y^(i) is correct] via execution or string match, which requires ground-truth answers. This directly contradicts the abstract's "single unlabeled data" and Section 3.1's "we do not need any data labels." Because all reported results use the variance-selected prompt, the observed gains may come from supervised selection of a high-variance example rather than from entropy minimization per se. Please report results with a randomly chosen prompt, or explicitly reframe the contribution as one-shot EM with oracle example selection.
  2. [Section 3.5 and Section 3.8] The reported results are the peak over 16 seeds, and Figure 3 states that average scores can differ by as much as a factor of two across seeds. Tables 1 and 2 report single numbers without mean/median, standard deviation, or confidence intervals, so the headline improvements may be selection artifacts. Furthermore, Section 3.5 says temperature 0.5 was chosen because it creates "opportunities for higher peak performance," making the final result a best-of-temperature and best-of-seed selection.
  3. [Section 3.4, Figure 2] The choice of 10 training steps is made post hoc: the figure shows that benchmark performance peaks at step 10 while the EM loss continues to decrease. Thus "converges within 10 steps" is not a property of the method but an early-stopping rule selected using labeled evaluation data. Please report the full performance trajectory and either justify the stopping rule independently or acknowledge that 10 steps is a tuned hyperparameter.
minor comments (6)
  1. [Section 3.2] The text says "We present our experimental results in Table 3," but the corresponding table is Table 1.
  2. [Section 3.9] The phrase "as depicted in Figure 3.4" appears to refer to Figure 2; the reference is incorrect.
  3. [Abstract and Section 3.1] There are typos: "avaliable" in the abstract, "Acclerate" in Section 3.1, and "evalutating" in Section 3.6.
  4. [Section 3.8 vs Table 2] The text's LLaMA-3.1-8B numbers (23.6% to 24.3%) and Qwen2.5-7B-Instruct numbers (43.12% to 44.5%) do not match Table 2, which shows 32.0 to 33.1 and 50.3 to 49.8, respectively. This inconsistency makes the cross-model discussion unreliable.
  5. [Table 1] The claim of competitiveness with RL should be qualified: on KK and MBPP, the EM model scores 17.4 and 65.1, while SimpleRL-Zoo scores 27.2 and 78.3, so EM is not universally superior.
  6. [Table 1] The RL baselines may not be directly comparable because they use different base models and training data; please state the base model and data for each baseline clearly.

Circularity Check

1 steps flagged · score 4.0 of 10

Supervised prompt selection contradicts the 'single unlabeled data' claim, but the EM objective itself is label-free and benchmark results are external.

  1. fitted input called prediction [Section 2.2 (Data Selection), pass@k and x* definitions; Section 3.1 (Experimental Setting)]
    "We then compute the pass@k score as: pass@k(x) = 1/k Σ I[y(i) is correct], where I[·] is the indicator function for whether a sample is considered correct (via execution or string match). ... x∗ = arg max x∈D Varpass@k(x), where D denotes the unlabeled data pool. ... Since it is an unsupervised method, we do not need any data labels."

    The single prompt used in all headline EM runs is chosen by x* = argmax Varpass@k(x), and Varpass@k is defined from I[y(i) is correct], which requires ground-truth correctness labels. Therefore the 'unlabeled' status of the single data is false by construction: labels are used to select the one prompt, even though the EM loss itself does not consume labels. All reported results use this label-selected prompt, so the advertised 'single unlabeled data' result is partly an artifact of supervised selection and cannot be taken as evidence for label-free entropy minimization alone.

full rationale

The paper is primarily an empirical study rather than a formal derivation, and its benchmark evaluations on MATH500, MinervaMath, OlympiadBench, AMC23, KK, and MBPP are external, so the core results are not tautological. The EM objective is a standard token-level entropy loss and is itself label-free; no load-bearing self-citation chain or imported uniqueness theorem was found. The one significant circularity-adjacent defect is the data-selection step: the single prompt is selected by maximizing pass@k variance, which requires knowing which sampled responses are correct, directly contradicting the abstract's and Section 3.1's 'unlabeled' and 'no data labels' claims. This does not force the benchmark numbers—the gains could still be real—but it means the headline 'single unlabeled data' framing overstates how unsupervised the pipeline is. Score 4 reflects that partial circularity while acknowledging the independent empirical content.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The ledger shows the method's success depends on hand-chosen hyperparameters (temperature, learning rate, batch size, steps), a label-dependent data selection step that contradicts the 'unlabeled' claim, and an unproven assumption that correct answers have lower entropy. No new theoretical entities are introduced.

free parameters (5)
  • Training temperature = 0.5
    Swept in Section 3.5; selected because it yields highest peak scores, though the text also says higher temperatures give better averages.
  • Learning rate = 2e-5
    Swept in Section 3.10; chosen as baseline, affects convergence and overconfidence.
  • Batch size = 64
    Stated in Section 3.1; no sweep or justification. It is unclear how one prompt yields batch size 64.
  • Training steps = 10
    Peak performance at step 10 in Figure 2; used as early stopping. The paper admits loss continues to decrease past step 10 while performance falls.
  • Reported seed selection = best of 16
    Section 3.8 reports 'peak performance' over 16 seeds; the paper notes up to 2x variation across seeds, so the headline numbers are best-case.
assumptions (3)
  • domain assumption Correct answers generally have lower entropy than incorrect ones.
    Stated in Section 1 as the basis for entropy minimization; if false, EM can reinforce confidently wrong outputs. No independent evidence is provided in the paper.
  • domain assumption A single prompt selected by pass@k variance is sufficient to transfer distribution reshaping to broad benchmarks.
    The paper's one-shot transfer assumption; not proven, and the selection requires ground-truth labels.
  • domain assumption The model's pass@k variance is a computable and meaningful 'entropy-sensitivity' measure.
    Used to select the one prompt in Section 2.2; relies on the ability to determine sample correctness, which itself needs labels.

how reviews work

0 comments
Cite this review

Pith. "Pith review of One-shot Entropy Minimization." pith.science (2026). https://pith.science/paper/PVUK3DSP

@misc{pith2026250520282,
  author       = {Pith},
  title        = {Pith review of: One-shot Entropy Minimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PVUK3DSP}},
  note         = {Machine review of arXiv:2505.20282}
}
read the original abstract

We trained 13,440 large language models and found that entropy minimization requires only a single unlabeled data and 10 steps optimization to achieve performance improvements comparable to or even greater than those obtained using thousands of data and carefully designed rewards in rule-based reinforcement learning. This striking result may prompt a rethinking of post-training paradigms for large language models. Our code is avaliable at https://github.com/zitian-gao/one-shot-em.

Figures

Figures reproduced from arXiv: 2505.20282 by the authors.

Figure 1
Figure 1. Distribution plot of the flattened logits sampled from 20 sample data points for each of the [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. The left y-axis represents the EM training loss, while the right y-axis shows the average [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. The impact of generation temperature during EM training on the average performance of the [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The impact of generation temperature during evalutating on the average performance of the [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: The blue curve on the left represents the average performance across four mathematical [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: The impact of learning rate during EM training on the average performance of the model [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Relationship-Centered Care: Relatedness and Responsible Design for Human Connections in Mental-Health Care

    cs.HC 2026-03 accept novelty 7.0 of 10

    Length-aware GFlowNet matching of α-power base distributions elicits either stronger LLM reasoning (α>1) or restored creativity (α<1) without external supervision, matching or beating RLIF and GRPO baselines.

  2. Know When to Explore: Difficulty-Aware Certainty as a Guide for LLM Reinforcement Learning

    cs.AI 2025-08 conditional novelty 6.0 of 10

    A new RL reward-shaping method that rewards low confidence on hard problems and high confidence on easy ones improves LLM math reasoning over a GRPO baseline.

  3. THIRDEYE: Cue-Aware Monocular Depth Estimation via Brain-Inspired Multi-Stage Fusion

    cs.CV 2025-06 reject novelty 6.0 of 10

    A cue-aware monocular depth estimation architecture that fuses frozen specialist networks via a cortical-style hierarchy and key-value memory, with no experimental results provided yet.

  4. Harnessing Uncertainty: Entropy-Modulated Policy Gradients for Long-Horizon LLM Agents

    cs.LG 2025-09 conditional novelty 5.0 of 10

    EMPG re-weights policy-gradient updates by step-level token entropy, amplifying confident correct actions and muting uncertain ones, and adds a future-clarity bonus.

  5. EDGE-GRPO: Entropy-Driven GRPO with Guided Error Correction for Advantage Diversity

    cs.AI 2025-07 conditional novelty 5.0 of 10

    EDGE-GRPO reduces advantage collapse in GRPO by injecting reference solutions into response groups and scaling advantages by policy entropy, achieving competitive math reasoning with only 1K training samples.

  6. No Free Lunch: Rethinking Internal Feedback for LLM Reasoning

    cs.LG 2025-06 conditional novelty 5.0 of 10

    Internal feedback rewards (entropy and self-certainty) improve base LLM math reasoning only in early training and degrade later, with little benefit for instruct models.

Reference graph

Works this paper leans on

28 extracted references · 6 canonical work pages · cited by 6 Pith papers

  1. [1]

    The unreasonable effectiveness of entropy minimization in llm reasoning, 2025

    Shivam Agarwal, Zimin Zhang, Lifan Yuan, Jiawei Han, and Hao Peng. The unreasonable effectiveness of entropy minimization in llm reasoning, 2025

    2025

  2. [2]

    Program synthesis with large language models, 2021

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. Program synthesis with large language models, 2021

    2021

  3. [3]

    Michaud, Jacob Pfau, Dmitrii Krasheninnikov, Xin Chen, Lauro Langosco, Peter Hase, Erdem Bıyık, Anca Dragan, David Krueger, Dorsa Sadigh, and Dylan Hadfield-Menell

    Stephen Casper, Xander Davies, Claudia Shi, Thomas Krendl Gilbert, Jérémy Scheurer, Javier Rando, Rachel Freedman, Tomasz Korbak, David Lindner, Pedro Freire, Tony Wang, Samuel Marks, Charbel-Raphaël Segerie, Micah Carroll, Andi Peng, Phillip Christoffersen, Mehul Damani, Stewart Slocum, Usman Anwar, Anand Siththaranjan, Max Nadeau, Eric J. Michaud, Jacob...

    work page 2023

  4. [4]

    Process reinforcement through implicit rewards, 2025

    Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, Jiarui Yuan, Huayu Chen, Kaiyan Zhang, Xingtai Lv, Shuo Wang, Yuan Yao, Xu Han, Hao Peng, Yu Cheng, Zhiyuan Liu, Maosong Sun, Bowen Zhou, and Ning Ding. Process reinforcement through implicit rewards, 2025

    2025

  5. [5]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai D...

    work page 2025

  6. [6]

    Interpretable contrastive monte carlo tree search reasoning, 2024

    Zitian Gao, Boye Niu, Xuzheng He, Haotian Xu, Hongzhang Liu, Aiwei Liu, Xuming Hu, and Lijie Wen. Interpretable contrastive monte carlo tree search reasoning, 2024

    2024

  7. [7]

    Mixed preference optimization: Reinforcement learning with data selection and better reference model, 2025

    Qi Gou and Cam-Tu Nguyen. Mixed preference optimization: Reinforcement learning with data selection and better reference model, 2025

    work page 2025

  8. [8]

    Accelerate: Training and inference at scale made simple, efficient and adaptable

    Sylvain Gugger, Lysandre Debut, Thomas Wolf, Philipp Schmid, Zachary Mueller, Sourab Mangrulkar, Marc Sun, and Benjamin Bossan. Accelerate: Training and inference at scale made simple, efficient and adaptable. https://github.com/huggingface/accelerate, 2022

    2022

Show all 28 references
  1. [9]

    Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems, 2024

    Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scienti...

    2024

  2. [10]

    Reinforce++: A simple and efficient approach for aligning large language models, 2025

    Jian Hu. Reinforce++: A simple and efficient approach for aligning large language models, 2025

    2025

  3. [11]

    Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model, 2025

    Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum. Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model, 2025

    2025

  4. [12]

    Solving quantitative reasoning problems with language models, 2022

    Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra. Solving quantitative reasoning problems with language models, 2022

    2022

  5. [13]

    Numinamath

    Jia LI, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Costa Huang, Kashif Rasul, Longhui Yu, Albert Jiang, Ziju Shen, Zihan Qin, Bin Dong, Li Zhou, Yann Fleureau, Guillaume Lample, and Stanislas Polu. Numinamath. [https://huggingface.co/AI-MO/NuminaMath...

    2024

  6. [14]

    Let’s verify step by step

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. arXiv preprint arXiv:2305.20050, 2023

    2023 arXiv

  7. [15]

    Understanding r1-zero-like training: A critical perspective, 2025

    Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding r1-zero-like training: A critical perspective, 2025

    2025

  8. [16]

    Introducing openai o1

    OpenAI. Introducing openai o1. https://openai.com/o1/, 2024. Accessed: 2024-10-02

    2024

  9. [17]

    Introducing openai o3 and o4-mini, April 2025

    OpenAI. Introducing openai o3 and o4-mini, April 2025. Accessed: 2025-05-27

    2025

  10. [18]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in N...

    2023

  11. [19]

    Lee, and Sanjeev Arora

    Noam Razin, Zixuan Wang, Hubert Strauss, Stanley Wei, Jason D. Lee, and Sanjeev Arora. What makes a reward model a good teacher? an optimization perspective, 2025

    2025

  12. [20]

    Proximal policy optimization algorithms, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017

    2017

  13. [21]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. 13

    2024

  14. [22]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, Chuning Tang, Congcong Wang, Dehao Zhang, Enming Yuan, Enzhe Lu, Fengxiang Tang, Flood Sung, Guangda Wei, Guokun Lai, Haiqing Guo, Han Zhu, Hao Ding, ...

    2025

  15. [23]

    Reinforcement learning for reasoning in large language models with one training example, 2025

    Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Lucas Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, Weizhu Chen, Shuohang Wang, Simon Shaolei Du, and Yelong Shen. Reinforcement learning for reasoning in large language models with one training example, 2025

    2025

  16. [24]

    On memorization of large language models in logical reasoning

    Chulin Xie, Yangsibo Huang, Chiyuan Zhang, Da Yu, Xinyun Chen, Bill Yuchen Lin, Bo Li, Badih Ghazi, and Ravi Kumar. On memorization of large language models in logical reasoning. 2024

    2024

  17. [25]

    Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning, 2025

    Tian Xie, Zitian Gao, Qingnan Ren, Haoming Luo, Yuqian Hong, Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, and Chong Luo. Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning, 2025

    2025

  18. [26]

    Towards large reasoning models: A survey of reinforced reasoning with large language models, 2025

    Fengli Xu, Qianyue Hao, Zefang Zong, Jingwei Wang, Yunke Zhang, Jingyi Wang, Xiaochong Lan, Jiahui Gong, Tianjian Ouyang, Fanjin Meng, Chenyang Shao, Yuwei Yan, Qinglong Yang, Yiwen Song, Sijian Ren, Xinyuan Hu, Yu Li, Jie Feng, Chen Gao, and Yong Li. Towards large reasoning m...

    2025

  19. [27]

    Redstar: Does scaling long-cot data unlock better slow-reasoning systems?, 2025

    Haotian Xu, Xing Wu, Weinong Wang, Zhongzhi Li, Da Zheng, Boyuan Chen, Yi Hu, Shijia Kang, Jiaming Ji, Yingying Zhang, Zhijiang Guo, Yaodong Yang, Muhan Zhang, and Debing Zhang. Redstar: Does scaling long-cot data unlock better slow-reasoning systems?, 2025

    2025

  20. [28]

    Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild, 2025

    Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild, 2025. 14

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.