Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Fast Quiet-STaR: Thinking Without Thought Tokens

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fast Quiet-STaR claims that a curriculum shrinking Quiet-STaR thought traces from 16 to 8 tokens, then distilling the model to plain next-token prediction with reinforcement learning, preserves the reasoning gains while eliminating…

desk verdict Plausible efficiency idea for Quiet-STaR, but the headline NTP gains are missing the step-matched and plain-continued-pretraining controls needed to prove them. read the letter →

arxiv 2505.17746 v2 pith:6MQ7O56V submitted 2025-05-23 cs.CL

Wei Huang , Yizhe Xiong , Xin Ye , Zhijie Deng , Hui Chen , Zijia Lin , Guiguang Ding This is my paper
classification cs.CL
keywords quietreasoningtoken-levelthoughttracescurriculumlearningnexttokenpredictionreinforcementinferenceefficiencylargelanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Fast Quiet-STaR tries to show that token-level "thinking" does not have to be paid for at every inference step. Quiet-STaR improves reasoning by generating a short thought trace before each predicted token, but this makes time-to-first-token more than ten times slower than ordinary next-token prediction. The paper's proposal is a curriculum: train on long thought traces first, then progressively fewer thought tokens, and finally use reinforcement learning to distill the model into a plain next-token predictor. The result, Fast Quiet-STaR NTP, keeps or improves the accuracy gains while running at the same latency as the original pre-trained model, with average gains of 9.0% on Mistral 7B and 5.7% on Qwen2.5 7B.

What carries the argument

The central machinery is a curriculum over Quiet-STaR's "n-m" configuration, where n is the number of thought tokens and m the number of ahead tokens generated per position. The model first learns under the easy 16-8 setting (verbose thoughts), then is re-trained under 12-4 and 8-4, each stage initialized from the previous checkpoint, so the harder compressed-thought settings are reached gradually. A final REINFORCE stage converts the 8-4 checkpoint into an NTP model by rewarding it for matching the 8-4 model's negative log-likelihood after thinking; this removes explicit thought-token generation while retaining the predictive signal the thoughts carried.

What would settle it

Train Quiet-STaR 8-4 for 200 steps with the same data, learning rate, and batching as Fast Quiet-STaR 8-4, and compare the four-benchmark average accuracy; if the step-matched baseline matches or exceeds Fast Quiet-STaR 8-4, the curriculum ordering is not what carries the gain.

Watch

Extended reading notes

Core claim

The central claim is that a model trained in the Quiet-STaR token-level thinking paradigm can learn to reason in progressively shorter thought traces, and finally in no explicit trace at all, without losing the reasoning benefit. Starting from a 16-thought-token, 8-ahead-token Quiet-STaR configuration, Fast Quiet-STaR trains through 12-4 and 8-4 stages, then uses REINFORCE to make a standard next-token-prediction head match the 8-4 model's post-thought prediction quality. On PIQA, SIQA, CommonsenseQA, and GSM8K with Mistral 7B and Qwen2.5 7B, the resulting Fast Quiet-STaR NTP improves average accuracy by 9.0% and 5.7% over the pre-trained baselines while keeping the same time-to-first-token and generation latency as ordinary next-token prediction.

Load-bearing premise

The load-bearing premise is that Fast Quiet-STaR's accuracy gains over Quiet-STaR come from the easy-to-hard curriculum ordering rather than from the extra training steps, because the Quiet-STaR baselines are trained for 100 steps while the Fast Quiet-STaR 12-4 and 8-4 checkpoints receive additional stages of training.

Editorial extensions

If this is right

  • With the same number of thought tokens and therefore the same inference time, Fast Quiet-STaR 12-4 and 8-4 outperform Quiet-STaR at those settings on both base models and all four benchmarks.
  • Fast Quiet-STaR 8-4 on Mistral 7B beats Quiet-STaR 16-8 by 1.8% average accuracy while cutting time-to-first-token to about 41.3% of the 16-8 value.
  • Fast Quiet-STaR NTP keeps ordinary NTP latency and still improves average accuracy by 9.0% on Mistral 7B and 5.7% on Qwen2.5 7B; on GSM8K, it also lifts chain-of-thought majority-vote accuracy from 43.3% to 52.4% at maj@6.
  • The curriculum ordering is necessary: a reversed curriculum (8-4 to 12-4 to 16-8) yields no improvement, and jumping straight from Quiet-STaR 16-8 to NTP loses 5.2% average accuracy against the full curriculum.
  • The training cost is small (0.5M tokens, about 54 minutes on eight H800 GPUs), and the paper estimates the time saved over 67 end-to-end inference runs of Quiet-STaR 16-8 roughly offsets the entire training cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the Quiet-STaR baselines are trained for 100 steps while each Fast Quiet-STaR stage adds 50 more, the reported gains at matched thought-token counts mix the curriculum effect with extra training compute; a step-matched Quiet-STaR baseline would isolate the curriculum's contribution.
  • Editorial inference: the same easy-to-hard thought-compression recipe could be tested on other verbose-reasoning methods, such as chain-of-thought fine-tuning or latent-space reasoning, to see whether 'compress the reasoning, then drop it' is a general route to fast reasoning.
  • Editorial inference: the thought-token visualizations suggest Fast Quiet-STaR's compressed thoughts are more goal-directed than Quiet-STaR's, which would predict that the curriculum teaches a reusable abstraction skill that transfers to reasoning datasets beyond the four tested.
  • Editorial inference: the RL-to-NTP stage treats post-thought log-likelihood as a soft training target, so one testable extension is to regularize the NTP model against the full token distribution of the thinking model rather than only the next-token loss, and to ablate the reward baseline.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Fast Quiet-STaR, a curriculum-learning variant of Quiet-STaR that progressively reduces the number of thought tokens (16-8 → 12-4 → 8-4) during continued pretraining, and then applies a REINFORCE-style objective (Eq. 4) to fine-tune the final 8-4 checkpoint into a standard next-token-prediction (NTP) model that does not generate explicit thoughts at inference. The authors report experiments on Mistral 7B and Qwen2.5 7B over PIQA, SIQA, CommonsenseQA, and GSM8K, claiming that Fast Quiet-STaR outperforms Quiet-STaR at equal thought-token counts and that Fast Quiet-STaR NTP improves average accuracy by 9% (Mistral) and 5.7% (Qwen2.5) over the original pre-trained models at identical inference latency.

Significance. The practical goal of removing thought-token overhead while retaining reasoning gains is timely and, if the empirical claims hold, would make Quiet-STaR-style training substantially more deployable. The paper also reports a useful latency analysis (Section 4.3.4) showing that the added training cost can be amortized over very few inference runs. However, the current evidence does not isolate the proposed mechanism: the Fast-vs-Quiet comparisons are not compute-matched, the headline NTP gains have no plain continued-pretraining control, and the evaluation metric is not fully specified. The qualitative thought-token examples (Figures 6-7) are suggestive but not quantified.

major comments (4)
  1. [4.1, 4.2, Table 1] The central Fast Quiet-STaR vs. Quiet-STaR comparison is confounded by training steps. Implementation details state that Quiet-STaR is trained for 100 steps while Fast Quiet-STaR receives 100 steps on the 16-8 stage plus 50 steps for each subsequent stage (150 steps for 12-4 and 200 for 8-4). No step-matched Quiet-STaR baseline (e.g., Quiet-STaR 8-4 trained for 200 steps) is reported, so the improved average accuracy in Table 1 could reflect additional optimization on OpenWebMath rather than the proposed curriculum. A matched-compute baseline is needed to support the claim in Section 4.2 that Fast Quiet-STaR 'consistently outperforms Quiet-STaR' under equal thought-token counts.
  2. [4.2, Table 1, 4.3.4] The headline result in the abstract and Table 1 compares Fast Quiet-STaR NTP against the original pre-trained checkpoint, which has received zero OpenWebMath tokens, while the Fast Quiet-STaR pipeline consumes 0.5M tokens of continued pretraining (Section 4.3.4) plus RL fine-tuning. The paper does not include a plain NTP model trained on the same 0.5M OpenWebMath tokens (or the same number of gradient steps with a standard LM loss). Such a control is necessary to rule out that the reported 9%/5.7% average gains are due to domain-adaptive continued pretraining alone. The data-efficiency analysis in Figure 5 tracks only Quiet-STaR-style models and therefore does not address this confound.
  3. [4.1, Accuracy formula] The evaluation metric is not well defined. The accuracy formula in Section 4.1 is syntactically garbled: as typeset it reads as a ratio of a product of per-token answer likelihoods to a product over sums of candidate-answer likelihoods, which is not a standard accuracy measure and is not a faithful reproduction of the Quiet-STaR protocol (Zelikman et al., 2024) without additional definitions. All quantitative claims, including the abstract's 9% and 5.7%, rest on this metric. The authors should provide the exact computation, define the candidate set S_ans for each dataset (especially GSM8K, where no candidate set is given), and state how final answers are extracted and matched.
  4. [4.2, Table 1] The conclusion that Fast Quiet-STaR NTP 'performs on par with Quiet-STaR' is not supported on Qwen2.5, where Quiet-STaR 16-8 achieves an average accuracy of 57.5 vs. 54.4 for Fast Quiet-STaR NTP; on Mistral the NTP model (41.0) is actually above Quiet-STaR 16-8 (39.2), but both comparisons are within a few points and no error bars, seeds, or significance tests are reported. Differences of about 2-4 points in Table 1 (e.g., Fast Quiet-STaR 12-4 vs. Quiet-STaR 12-4 on Qwen) may be within run-to-run variability. The paper should report multiple seeds or otherwise quantify variance before drawing these comparative conclusions.
minor comments (6)
  1. [4.1 vs 4.3.2] Section 4.1 says Quiet-STaR is trained for 100 steps, while Section 4.3.2 says Quiet-STaR trains for 150 steps; these numbers should be reconciled.
  2. [4.1] The accuracy formula uses 'Ql' and 'Q' where product symbols are intended; please typeset with \prod and define all variables explicitly.
  3. [Table 2] The header 'A VG ACC' appears to be a typo for 'AVG ACC'.
  4. [Figure 1] The legend label 'Thought T okens Number' has a stray space, and the x-axis labels are not clearly tied to the curve types.
  5. [4.3.4] The section contains an unresolved cross-reference 'see Table (?)' that should be replaced with the actual table number.
  6. [References] The reference entries for 'OpenAI' and 'Qwen' begin with an empty author field ('OpenAI, :' and 'Qwen, :'), which should be cleaned up.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the reported gains are measured benchmark outcomes, and Eq. 4 is a fixed-teacher distillation objective; the missing plain-NTP control is an experimental confound, not a circular step.

full rationale

The paper's derivation chain is empirical rather than derivational. The curriculum procedure (16-8 → 12-4 → 8-4) is a training schedule, and the NTP transition uses the RL reward in Eq. 4, r_j = L_FastQuietSTaR − L_FastQuietSTaR−NTP, which compares the student's negative log-likelihood to that of a fixed Fast Quiet-STaR 8-4 teacher. This is a standard teacher–student distillation objective: the teacher is trained independently (on OpenWebMath with the Quiet-STaR-style think/talk/learn procedure), and no benchmark accuracy is fed back into the objective. The headline 9% (Mistral 7B) and 5.7% (Qwen2.5 7B) gains are measured on PIQA, SIQA, CommonsenseQA, and GSM8K after training, not quantities forced by the method's equations. The curriculum claim is supported by explicit controls: a reversed-curriculum variant (Rev Quiet-STaR, Section 4.3.1) and a no-curriculum variant (Quiet-STaR NTP, Table 3) both underperform the full curriculum, so the central comparison is not a self-fulfilling definition. The paper contains no load-bearing self-citations by the present authors, no imported uniqueness theorem, and no fitted constant that determines the reported accuracies. The most serious weakness is the absence of a plain NTP continued-pretraining control trained for the same 0.5M tokens, which leaves open the possibility that some of the gain comes from OpenWebMath domain adaptation rather than the proposed mechanism. That is a missing experimental baseline and a correctness risk, not circularity under the definitions used here. Overall circularity score: 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method has no new physical or conceptual entities. The main hidden inputs are hyperparameters and inherited assumptions about Quiet-STaR and the evaluation protocol.

free parameters (3)
  • curriculum stage token counts = 16-8 -> 12-4 -> 8-4
    Hand-chosen schedule; no sensitivity analysis shown.
  • stage training steps = 100, 50, 50
    Hand-chosen; the baseline Quiet-STaR is trained for 100 steps, creating a confound for Fast Quiet-STaR stages that inherit 100 plus 50.
  • learning rates = 1e-6 (Mistral), 8e-6 (Qwen2.5)
    Chosen per model.
assumptions (4)
  • domain assumption Quiet-STaR training framework and its reward (Eq. 2) are taken as given from Zelikman et al. 2024.
    The paper builds on Quiet-STaR without re-deriving its correctness.
  • ad hoc to paper The evaluation metric ACC (Section 4.1) is presumed to measure accuracy correctly despite the garbled formula.
    The formula is typeset incorrectly; the exact matching/score procedure, especially for free-form GSM8K answers, is not specified.
  • domain assumption OpenWebMath continue-pretraining is a valid proxy for general reasoning acquisition.
    Follows Quiet-STaR's setup.
  • domain assumption The reward in Eq. 4, defined as the difference between teacher and student negative log-likelihoods, yields a useful RL signal.
    No analysis of reward scale, variance, or credit assignment.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fast Quiet-STaR: Thinking Without Thought Tokens." pith.science (2026). https://pith.science/paper/6MQ7O56V

@misc{pith2026250517746,
  author       = {Pith},
  title        = {Pith review of: Fast Quiet-STaR: Thinking Without Thought Tokens},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6MQ7O56V}},
  note         = {Machine review of arXiv:2505.17746}
}
read the original abstract

Large Language Models (LLMs) have achieved impressive performance across a range of natural language processing tasks. However, recent advances demonstrate that further gains particularly in complex reasoning tasks require more than merely scaling up model sizes or training data. One promising direction is to enable models to think during the reasoning process. Recently, Quiet STaR significantly improves reasoning by generating token-level thought traces, but incurs substantial inference overhead. In this work, we propose Fast Quiet STaR, a more efficient reasoning framework that preserves the benefits of token-level reasoning while reducing computational cost. Our method introduces a curriculum learning based training strategy that gradually reduces the number of thought tokens, enabling the model to internalize more abstract and concise reasoning processes. We further extend this approach to the standard Next Token Prediction (NTP) setting through reinforcement learning-based fine-tuning, resulting in Fast Quiet-STaR NTP, which eliminates the need for explicit thought token generation during inference. Experiments on four benchmark datasets with Mistral 7B and Qwen2.5 7B demonstrate that Fast Quiet-STaR consistently outperforms Quiet-STaR in terms of average accuracy under the same inference time budget. Notably, Fast Quiet-STaR NTP achieves an average accuracy improvement of 9\% on Mistral 7B and 5.7\% on Qwen2.5 7B, while maintaining the same inference latency. Our code will be available at https://github.com/huangwei200012/Fast-Quiet-STaR.

Figures

Figures reproduced from arXiv: 2505.17746 by the authors.

Figure 1
Figure 1. Performance comparison between Fast Quiet [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Fast Quiet-STaR training pipeline and Curriculum Learning Examples. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Results show that Fast Quiet-STaR 8-4 yields the best performance, followed by Quiet￾STaR 16-8, and then the pre-trained model. We attribute this to Fast Quiet-STaR 8-4’s ability to generate a compact yet informative thought trace, which is conducive to further improving efficiency and expanding the reasoning paradigm to NTP. In PIQA SIQA CommonsenseQA GSM8K Tasks 40.0 42.5 45.0 47.5 50.0 52.5 55.0 57.5 60.0 PIQA,SI… view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Comparison of the accuracy of Quiet-STaR [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Examples of the text and its thought process [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: More examples of the text and its thought [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Adaptive Depth in Looped Transformers: Diagnosing Learned Halting Gates and Trajectory Readouts

    cs.LG 2026-07 conditional novelty 6.0 of 10

    In looped transformers, halting-gate failures come mainly from how gate training reshapes the trajectory; fixed-prior depth supervision plus simple confidence readouts yields better accuracy per unit of compute.

Reference graph

Works this paper leans on

35 extracted references · 4 canonical work pages · cited by 1 Pith paper

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  2. [2]

    Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, and 1 others. 2020. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432--7439

    2020

  3. [3]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    arXiv 2021

  4. [5]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024 b . The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    arXiv 2024

  5. [6]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    arXiv 2025

  6. [7]

    Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. 2024. Training large language models to reason in a continuous latent space. arXiv preprint arXiv:2412.06769

    arXiv 2024

  7. [8]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. NeurIPS

    2021

  8. [9]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023. https://arxiv.org/abs/2310.0...

    arXiv 2023

Show all 35 references
  1. [10]

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199--22213

    2022

  2. [11]

    Dongyuan Li, Ying Zhang, Zhen Wang, Shiyin Tan, Satoshi Kosugi, and Manabu Okumura. 2024. Active learning for abstractive text summarization via llm-determined curriculum and certainty gain maximization. In Findings of the Association for Computational Linguistics: EMNLP 2024,...

    2024

  3. [12]

    Tengxiao Liu, Qipeng Guo, Xiangkun Hu, Cheng Jiayang, Yue Zhang, Xipeng Qiu, and Zheng Zhang. 2024. Can language models learn to skip steps? arXiv preprint arXiv:2411.01855

    2024 arXiv

  4. [13]

    Marwa Na \" r, Kamel Yamani, Lynda Said Lhadj, and Riyadh Baghdadi. 2024. Curriculum learning for small code language models. arXiv preprint arXiv:2407.10194

    2024 arXiv

  5. [14]

    OpenAI, :, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, and 244 oth...

    2024 arXiv

  6. [15]

    Rui Pan, Yinwei Dai, Zhihao Zhang, Gabriele Oliaro, Zhihao Jia, and Ravi Netravali. 2025. Specreason: Fast and accurate inference-time compute via speculative reasoning. arXiv preprint arXiv:2504.07891

    2025 arXiv

  7. [16]

    Keiran Paster, Marco Dos Santos, Zhangir Azerbayev, and Jimmy Ba. 2023. Openwebmath: An open dataset of high-quality mathematical web text. arXiv preprint arXiv:2310.06786

    2023 arXiv

  8. [17]

    Du Phan, Matthew Douglas Hoffman, David Dohan, Sholto Douglas, Tuan Anh Le, Aaron Parisi, Pavel Sountsov, Charles Sutton, Sharad Vikram, and Rif A Saurous. 2023. Training chain-of-thought via latent-variable inference. Advances in Neural Information Processing Systems, 36:72819--72841

    2023

  9. [18]

    Zhenting Qi, Mingyuan Ma, Jiahang Xu, Li Lyna Zhang, Fan Yang, and Mao Yang. 2024. Mutual reasoning makes smaller llms stronger problem-solvers. arXiv preprint arXiv:2408.06195

    2024 arXiv

  10. [19]

    Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. https://arxiv.org/abs/2412.15115 Qwe...

    2025 arXiv

  11. [20]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728--53741

    2023

  12. [21]

    Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. 2019. Explain yourself! leveraging language models for commonsense reasoning. arXiv preprint arXiv:1906.02361

    2019 arXiv

  13. [22]

    Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, and Yejin Choi. 2019. Socialiqa: Commonsense reasoning about social interactions. arXiv preprint arXiv:1904.09728

    2019 arXiv

  14. [23]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347

    2017 arXiv

  15. [24]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300

    2024 arXiv

  16. [25]

    Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2018. Commonsenseqa: A question answering challenge targeting commonsense knowledge. arXiv preprint arXiv:1811.00937

    2018 arXiv

  17. [26]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, and 1 others. 2025. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599

    2025 arXiv

  18. [27]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171

    2022 arXiv

  19. [28]

    Benfeng Xu, Licheng Zhang, Zhendong Mao, Quan Wang, Hongtao Xie, and Yongdong Zhang. 2020. Curriculum learning for natural language understanding. In Proceedings of the 58th annual meeting of the association for computational linguistics, pages 6095--6104

    2020

  20. [29]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36:11809--11822

    2023

  21. [30]

    Yuanhao Yue, Chengyu Wang, Jun Huang, and Peng Wang. 2024. Distilling instruction-following abilities of large language models with task-aware curriculum planning. arXiv preprint arXiv:2405.13448

    2024 arXiv

  22. [31]

    Eric Zelikman, Georges Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, and Noah D Goodman. 2024. Quiet-star: Language models can teach themselves to think before speaking. arXiv preprint arXiv:2403.09629

    2024 arXiv

  23. [32]

    Jinghan Zhang, Xiting Wang, Fengran Mo, Yeyang Zhou, Wanfu Gao, and Kunpeng Liu. 2025. Entropy-based exploration conduction for multi-step reasoning. arXiv preprint arXiv:2503.15848

    2025 arXiv

  24. [33]

    Xuan Zhang, Chao Du, Tianyu Pang, Qian Liu, Wei Gao, and Min Lin. 2024. Chain of preference optimization: Improving chain-of-thought reasoning in llms. Advances in Neural Information Processing Systems, 37:333--356

    2024

  25. [34]

    Chujie Zheng, Zhenru Zhang, Beichen Zhang, Runji Lin, Keming Lu, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. 2024. Processbench: Identifying process errors in mathematical reasoning. arXiv preprint arXiv:2412.06559

    2024 arXiv

  26. [35]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  27. [36]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.