Pith. sign in

REVIEW 4 major objections 4 minor 3 cited by

Incentivizing Dual Process Thinking for Efficient Large Language Model Reasoning

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ACPO trains large reasoning models to switch between explicit fast and slow thinking, cutting token use by up to 60.5% on AIME 2024 with accuracy intact.

desk verdict ACPO is a plausible RL recipe for token reduction on math benchmarks, but the dual-process interpretability story is not supported by the annotation proxy. read the letter →

arxiv 2505.16315 v2 pith:EI23DEVY submitted 2025-05-22 cs.AI cs.CL

Xiaoxue Cheng , Junyi Li , Zhenduo Zhang , Xinyu Tang , Wayne Xin Zhao , Xinyu Kong , Zhiqiang Zhang This is my paper
classification cs.AIcs.CL
keywords dualprocesstheorylargereasoningmodelsreinforcementlearningtokenlengthbudgetsystem-awaretokensadaptivecognitionefficiencyGRPO
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that large reasoning models can be trained to switch between two explicit thinking modes—fast and slow—so that they spend fewer tokens on easy questions and more on hard ones. It introduces ACPO, a reinforcement learning method that adds special tokens marking fast and slow reasoning steps, estimates each question's difficulty online from sampling success, and rewards both correct answers and length that matches the difficulty. On AIME 2024, ACPO cuts token use by up to 60.5% (from 16,894 to 6,670 tokens on the 1.5B model) while keeping accuracy slightly above the baseline. If correct, this would make long-chain-of-thought models cheaper to run without giving up much accuracy, and would make their reasoning process visible through explicit cognitive-mode tokens.

What carries the argument

The load-bearing machinery is the combination of system-aware reasoning tokens and an online token length budget inside a GRPO reinforcement loop. The tokens make fast and slow thinking explicit in the generated text, so the reward can directly reward or penalize cognitive allocation. The online TLB uses the group sampling success rate $p$ as a difficulty estimate, sets a length budget $L_{\text{budget}} = p \cdot L_r + (1-p) \cdot L_{\max}$, and gives reward based on how much each response deviates from that budget; a system pattern reward then pushes the model toward fast thinking when $p$ is high and slow thinking when $p$ is low. The two-stage schedule—SFT on 745 GPT-4-annotated examples, then one epoch of GRPO with the composite reward—carries the training.

What would settle it

On a held-out set with independent human or model annotations of which reasoning steps require deliberate thought, ACPO's <fast_think> labels should match the independently labeled fast steps at a high rate; if they do not, or if removing the system pattern reward leaves the token savings unchanged, then the adaptive system-switch mechanism is not what is doing the work.

Watch

Extended reading notes

Core claim

The central discovery is that a reasoning model's 'thinking mode' can be made an explicit trainable output: by wrapping reasoning steps in <fast_think> and <slow_think> tokens and training with a reward that balances correctness, token budget, and fast/slow proportions, the model learns to shorten paths on easy problems and preserve longer deliberation on hard ones. The paper shows this on three DeepSeek-R1-Distill models, achieving large token reductions (for example, 60.5% fewer tokens on AIME 2024 with the 1.5B model) with essentially unchanged or slightly improved accuracy, and demonstrates that the fast/slow ratio shifts with problem difficulty.

Load-bearing premise

The annotation rule treats a step that appears in both the short and long correct answer as 'slow thinking' and a step that appears only in the long answer as 'fast thinking'; if length omission is not a faithful proxy for cognitive mode, the pattern reward trains the model to imitate a token-distribution artifact rather than genuine fast/slow reasoning.

Editorial extensions

If this is right

  • ACPO cuts average token usage by roughly half or more on hard math benchmarks while keeping accuracy within about two points, so long-reasoning models become cheaper to deploy.
  • On simpler tasks like GSM8K, ACPO keeps or slightly improves accuracy while using fewer tokens, so the method does not blindly compress.
  • The system-aware tokens make the model's reasoning mode observable, allowing users to see and audit when the model switches between fast and slow thinking.
  • Because the length budget is estimated online from the current policy's sampling success, ACPO does not rely on precomputed budgets or fixed preference pairs and can adapt as the policy improves.
  • Across difficulty levels in MATH 500, the model's fast-thinking ratio falls and its slow-thinking ratio rises as difficulty increases, matching the intended dual-process allocation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One testable extension is to treat the fast/slow tokens as an inference-time control channel, letting an external controller override the model's own system switch to dial compute per question without retraining.
  • The difficulty proxy $p$ measures the current model's success rate, so what ACPO adapts to is better described as model-relative difficulty than intrinsic task difficulty; a question's budget would change as the policy becomes stronger.
  • Since the reward needs verifiable correct answers, extending ACPO to open-domain tasks would require a learned reward or outcome model; the fast/slow annotation rule from Section 3.1.1 may not transfer to unverifiable reasoning.
  • An ablation that removes the SFT cold start would clarify how much of the compression comes from the RL rewards versus the 745 annotated examples; the paper does not isolate this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes Adaptive Cognition Policy Optimization (ACPO), a two-stage framework for making large reasoning models faster by explicitly tokenizing 'fast' and 'slow' thinking modes. In the first stage, 745 samples built from LIMO are annotated by GPT-4 with <fast_think>/<slow_think> tags according to whether a reasoning step appears in both the longest and shortest correct responses, and the model is SFT cold-started on these samples. In the second stage, GRPO is extended with three reward components: an accuracy reward, an online token length budget (TLB) reward computed from the group sampling success rate, and a system-pattern reward that encourages more fast thinking on easy queries and more slow thinking on hard queries. Experiments on DeepSeek-R1-Distill-Qwen-1.5B/7B and DeepSeek-R1-Distill-Llama-8B report substantial token reductions on MATH 500 and AIME 2024 (up to 60.5% on AIME 2024 for the 1.5B model) with small accuracy changes, and Figure 3 is presented as evidence that the fast/slow ratio adapts to problem difficulty.

Significance. If the efficiency results are robust, ACPO offers a practical mechanism for reducing inference cost in reasoning models while keeping accuracy competitive, and the idea of making thinking modes explicit through special tokens is useful for monitoring and controlling reasoning behavior. The online TLB reward is a clean extension of the offline DAST budget idea. However, the paper's stronger interpretability claim—that the model learns dual-process cognition and transparently switches between fast and slow thinking based on task difficulty—rests on a proxy that equates 'omitted from a length-prompted short response' with 'fast thinking', which is not independently validated. The ablation comparison against GRPO is also less favorable than the abstract suggests, with ACPO losing accuracy to GRPO in 5 of 6 model/dataset pairs. The manuscript gives a clear method specification and training details, but no code release, error bars, or significance tests, and several baseline numbers are taken from other papers rather than measured in a common harness.

major comments (4)
  1. [§3.1.1, Eq. (6), Figure 3] The annotation rule in §3.1.1 labels steps present in both the shortest and longest correct responses as <slow_think> and steps omitted or summarized in the short response as <fast_think>. This equates response-length omission with cognitive mode; a step can be omitted because of the length-prompting instruction (a compression artifact) rather than because it is intuitive or fast. Since the system-pattern reward in Eq. (6) then maximizes ρ_fast when p > p_thresh, the fast/slow ratio in Figure 3 is directly optimizing this compression-based proxy. The claim of 'transparent cognitive process' and the interpretation of Figure 3 as evidence of difficulty-adaptive dual-process cognition therefore need independent validation—for example, human judgments of whether tagged steps are actually deliberative, or an analysis of tag agreement with a different annotation protocol—before those claims can be accepted. The token-efficiency result does not depend on this proxy, but the interpretability/adaptivity half of the contribution does.
  2. [§3.2.1, Eqs. (3), (5), (6)] The TLB and the difficulty proxy p in Eq. (3) are computed from the same group of sampled responses whose rewards are being optimized. This makes the difficulty signal a moving target that is partly determined by the current policy's competence, rather than a property of the question alone. The accuracy reward is externally grounded, so the token-reduction result is not circular, but the claim that ACPO adapts to 'task difficulty' needs a test with fixed or oracle difficulty labels (e.g., human-annotated difficulty levels or success rates from an independent solver) to rule out the model fitting its own emergent token statistics. The limitation section acknowledges that online difficulty estimation relies on verifiable data, but it does not mention this self-referential aspect of the reward loop.
  3. [Table 3] In the ablation, ACPO has lower accuracy than GRPO on 5 of the 6 model/dataset pairs: MATH 500 accuracies are 81.0 vs 84.5, 91.6 vs 92.5, and 87.4 vs 90.4 for the three models; AIME 2024 accuracies are 30.0 vs 29.0, 52.8 vs 53.2, and 43.3 vs 43.3. No error bars, standard deviations over seeds, or significance tests are reported anywhere, and the number of evaluation samples used for pass@1 is not stated. The paper's claim of 'competitive accuracy' is therefore unsupported by the reported point estimates, which are equally consistent with ACPO trading accuracy for token compression. Report variance over random seeds or bootstrap confidence intervals, and specify the evaluation sample size and temperature.
  4. [§4.1, Table 1, §C.2] Several baselines are not evaluated in a controlled setting: SFT_Shortest, SimPO_Shortest, and SimPO_DAST are numbers taken from the DAST paper, while THINKPRUNE and L1 are trained from different starting checkpoints and data (L1 variants from DeepScaleR-1.5B-Preview) than the ACPO models (which start from base R1-Distill models with an SFT cold start). These are not apples-to-apples comparisons, so the reported token-reduction advantages over those baselines could reflect the training recipe or base model rather than the ACPO reward design. To substantiate the central efficiency claim, at least one comparison should train ACPO and a baseline from the same starting checkpoint with the same training data, reward computation, and evaluation harness.
minor comments (4)
  1. [Table 1] The ACPO-7B AIME 2024 entry '#Token ACU↑' appears garbled as '45200.17'; it should presumably read 4520 and 0.17, but as printed the table is unreadable in that cell.
  2. [Eq. (8)] ACU is defined as Accuracy/(#Params×#Tokens), but Accuracy is reported as a percentage; the resulting values are not intuitive and no normalization is described. Clarify whether Accuracy is used as a fraction or a percentage and state the units of ACU.
  3. [Figure 4] The caption does not clearly indicate which reasoning path corresponds to ACPO and which to GRPO; the main text says ACPO uses 476 tokens and GRPO uses 1555, so the panels should be labeled explicitly.
  4. [§C.3] The ACPO training prompt instructs the model to emit <think>, <fast_think>, and <slow_think> tags, but the paper does not specify how these special tokens are tokenized or whether they are included in the reported token-length numbers; state this explicitly for reproducibility.

Circularity Check

2 steps flagged · score 4.0 of 10

Efficiency numbers are externally benchmarked, but the difficulty-adaptive fast/slow trend in Figure 3 is the direct optimum of Eq. 6, and the fast/slow labels themselves are defined by a length-difference annotation in §3.1.1, so the cognitive-adaptivity claim is substantially self-referential.

  1. fitted input called prediction [Section 3.2.1, Eq. (6); Section 4.3.2, Figure 3]
    "To further encourage difficulty-aware system switch, we use the sampling success rate p from the TLB estimation as a proxy for task difficulty. For easier queries with higher p, the model is encouraged to allocate a larger proportion of its reasoning path to fast thinking. ... It is observed that, as the problem difficulty increases, the proportion of fast thinking decreases while the slow thinking component increases."

    Eq. (6) defines Rthink,i = rho_fast,i when p > p_thresh and rho_slow,i otherwise, with p being the current policy's group success rate from Eq. (3). The Figure 3 trend is therefore the exact quantity the reward was constructed to maximize; reporting it as evidence of adaptive cognition is reporting the optimization target, not an emergent or independent finding. Because p is computed from the model's own sampled responses, 'task difficulty' is partially the model's current competence, making the adaptivity claim self-referential rather than a test against an external difficulty measure.

  2. self definitional [Section 3.1.1, Response Comparison and Annotation; Appendix B]
    "For those reasoning steps that are present in both short and long responses, we consider them as essential and detailed components that should be marked as slow thinking, as they represent critical, non-omittable reasoning processes. For other steps that appear in the long response but are omitted or summarized in the short response, we view them as trivial steps that should be labeled as fast thinking."

    The annotation defines 'fast thinking' as the steps omitted or summarized in the shorter response and 'slow thinking' as steps shared by short and long responses. SFT then trains the model to emit <fast_think>/<slow_think> tokens according to this rule, and the system-pattern reward reinforces the same mapping. Consequently, the later claim that the tokens make the model's 'cognitive process transparent' and reveal dual-process behavior is a consequence of the labeling definition, not independent evidence; no signal distinguishes a cognitive mode from response-length compression. The token-reduction and accuracy results are unaffected by this definitional step.

full rationale

ACPO's efficiency and accuracy claims in Tables 1-3 are grounded in held-out MATH 500, AIME 2024, and GSM8K evaluations with external answer labels and token counts; those results are not circular and I do not flag them. The circularity is confined to the cognitive-adaptivity interpretation. Equations (3) and (6) use the current policy's own group success rate p as a 'task difficulty' proxy and set a reward that directly pushes rho_fast up when p is high and rho_slow up when p is low. The §4.3.2 observation that fast thinking falls and slow thinking rises with difficulty is thus the optimized objective, not an emergent dual-process phenomenon. In addition, the SFT annotation in §3.1.1 defines fast/slow thinking by which steps survive length compression, so the 'transparent cognitive process' language is a restatement of the annotation rule. The paper's limitation section acknowledges that online difficulty estimation relies on verifiable data and may not generalize, but it does not acknowledge that the adaptivity evidence is partly constructed by the reward. No load-bearing self-citation chain or uniqueness argument appears; citations to the authors' prior dual-process work are background only. Overall score 4: partial circularity in the interpretation/adaptivity claim, with the efficiency half independently supported.

Assumptions & free parameters 3 free parameters · 5 assumptions · 1 invented entities

The framework depends on the annotation proxy, the difficulty proxy, and hand-chosen reward weights. The accuracy reward is externally grounded, but the length and system-pattern rewards are derived from the model's own sampled responses, and the claimed 'transparent cognitive process' has no independent evidence.

free parameters (3)
  • p_thresh = 0.5
    Threshold for easy/hard split in the system pattern reward (Eq. 6). Chosen by hand, no sensitivity analysis reported.
  • reward weights wacc, wlen, wthink = 0.6, 0.3, 0.1
    Weights in Eq. 7 balance accuracy, TLB, and system pattern rewards. No ablation over these values is provided.
  • SFT epoch count = 3
    Cold-start training uses 3 epochs on 745 samples; this is a chosen hyperparameter without reported sensitivity.
assumptions (5)
  • standard math GRPO and its clipped surrogate objective are a valid RL update for this setting.
    Used as the base optimizer in Section 2.1, Eq. 2; this is standard and generally accepted.
  • domain assumption The annotation rule that steps common to both short and long correct responses are slow thinking, and omitted steps are fast thinking.
    Section 3.1.1 defines the dual-process token labels. This is a proxy that length omission corresponds to cognitive mode; no external validation of the labels is given.
  • domain assumption The group success rate p is a reliable online proxy for task difficulty.
    Eqs. 3 and 6 use p to set the length budget and system switch. This conflates the policy's current competence with intrinsic task complexity.
  • domain assumption The MATH 500 five-level difficulty bins reflect true task difficulty.
    Used in Section 4.3.2 to claim difficulty adaptivity; no external difficulty ground truth beyond the dataset's own split is supplied.
  • domain assumption All training queries have verifiable ground-truth answers.
    The accuracy reward (Eq. 4) and the entire TLB mechanism depend on verifiable math answers, limiting the method to tasks with checked labels.
invented entities (1)
  • System-aware reasoning tokens (<fast_think>, </fast_think>, <slow_think>, </slow_think>)
    purpose: Explicitly mark fast and slow thinking segments so the policy can be rewarded for system switch and the reasoning process becomes visible in text.
    These special tokens are introduced by the authors. Their mapping to actual cognitive states is not validated by any external measure; it rests entirely on the GPT-4 annotation proxy described in Section 3.1.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Incentivizing Dual Process Thinking for Efficient Large Language Model Reasoning." pith.science (2026). https://pith.science/paper/EI23DEVY

@misc{pith2026250516315,
  author       = {Pith},
  title        = {Pith review of: Incentivizing Dual Process Thinking for Efficient Large Language Model Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EI23DEVY}},
  note         = {Machine review of arXiv:2505.16315}
}
read the original abstract

Large reasoning models (LRMs) have demonstrated strong performance on complex reasoning tasks, but often suffer from overthinking, generating redundant content regardless of task difficulty. Inspired by the dual process theory in cognitive science, we propose Adaptive Cognition Policy Optimization (ACPO), a reinforcement learning framework that enables LRMs to achieve efficient reasoning through adaptive cognitive allocation and dynamic system switch. ACPO incorporates two key components: (1) introducing system-aware reasoning tokens to explicitly represent the thinking modes thereby making the model's cognitive process transparent, and (2) integrating online difficulty estimation and token length budget to guide adaptive system switch and reasoning during reinforcement learning. To this end, we propose a two-stage training strategy. The first stage begins with supervised fine-tuning to cold start the model, enabling it to generate reasoning paths with explicit thinking modes. In the second stage, we apply ACPO to further enhance adaptive system switch for difficulty-aware reasoning. Experimental results demonstrate that ACPO effectively reduces redundant reasoning while adaptively adjusting cognitive allocation based on task complexity, achieving efficient hybrid reasoning.

Figures

Figures reproduced from arXiv: 2505.16315 by the authors.

Figure 1
Figure 1. The overall framework of ACPO. The upper section illustrates the system explicitization [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Average response length and accuracy across different difficulty levels on MATH 500. 1 2 3 4 5 Difficulty 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 Fast Think Ratio Fast Think Ratio 0.800 0.825 0.850 0.875 0.900 0.925 0.950 0.975 1.000 Slow Think Ratio Slow Think Ratio [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 4
Figure 4. An case study comparing the reasoning process of DeepSeek-R1-Distill-Qwen-1.5B trained [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ConPress: Learning Efficient Reasoning from Multi-Question Contextual Pressure

    cs.CL 2026-02 conditional novelty 6.0 of 10

    Multi-question prompts elicit shorter chain-of-thought traces, and fine-tuning on those traces transfers the compression to single-question reasoning.

  2. Strategic Reflectivism In Intelligent Systems

    cs.AI 2025-05 conditional novelty 4.0 of 10

    Strategic Reflectivism holds that intelligent systems should allocate reflective reasoning tactically, weighing its benefits against its costs.

  3. Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey

    cs.AI 2025-07 conditional novelty 3.0 of 10

    A comprehensive review that categorizes methods for shortening and adaptively triggering chain-of-thought reasoning in large language models.

Reference graph

Works this paper leans on

37 extracted references · 2 canonical work pages · cited by 3 Pith papers

  1. [1]

    A survey of large language models.CoRR, abs/2303.18223, 2023

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. A survey of large language models.CoRR, abs/2303.18223, 2023

    arXiv 2023

  2. [2]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai D...

    arXiv 2025

  3. [3]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, Chuning Tang, Congcong Wang, Dehao Zhang, Enming Yuan, Enzhe Lu, Fengxiang Tang, Flood Sung, Guangda Wei, Guokun Lai, Haiqing Guo, Han Zhu, Hao Ding, Hao Hu, Hao Yang, Hao Zhang, Haotian Yao, Haotian Zhao, Haoyu Lu, Haoze Li, Haoz...

    arXiv 2025

  4. [4]

    Openai o1 system card.arXiv preprint arXiv:2412.16720, 2024

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card.arXiv preprint arXiv:2412.16720, 2024

    arXiv 2024

  5. [5]

    Qwq-32b: Embracing the power of reinforcement learning, March 2025

    Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025

    2025

  6. [6]

    Scaling llm test-time compute opti- mally can be more effective than scaling model parameters.arXiv preprint arXiv:2408.03314, 2024

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute opti- mally can be more effective than scaling model parameters.arXiv preprint arXiv:2408.03314, 2024

    arXiv 2024

  7. [7]

    The impact of reasoning step length on large language models

    Mingyu Jin, Qinkai Yu, Dong Shu, Haiyan Zhao, Wenyue Hua, Yanda Meng, Yongfeng Zhang, and Mengnan Du. The impact of reasoning step length on large language models. InFindings of the Association for Computational Linguistics ACL 2024, pages 1830–1842, 2024. 10

    work page 2024

  8. [8]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022

    2022

Show all 37 references
  1. [9]

    Do NOT think that much for 2+3=? on the overthinking of o1-like llms.CoRR, abs/2412.21187, 2024

    Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. Do NOT think that much for 2+3=? on the overthinking of o1-like llms.CoRR, abs/2412.21187, 2024

    2024 arXiv

  2. [10]

    Scaling of search and learning: A roadmap to reproduce o1 from reinforcement learning perspective.arXiv preprint arXiv:2412.14135, 2024

    Zhiyuan Zeng, Qinyuan Cheng, Zhangyue Yin, Bo Wang, Shimin Li, Yunhua Zhou, Qipeng Guo, Xuanjing Huang, and Xipeng Qiu. Scaling of search and learning: A roadmap to reproduce o1 from reinforcement learning perspective.arXiv preprint arXiv:2412.14135, 2024

    2024 arXiv

  3. [11]

    A survey of efficient reasoning for large reasoning models: Language, multimodality, and beyond.CoRR, abs/2503.21614, 2025

    Xiaoye Qu, Yafu Li, Zhaochen Su, Weigao Sun, Jianhao Yan, Dongrui Liu, Ganqu Cui, Daizong Liu, Shuxian Liang, Junxian He, Peng Li, Wei Wei, Jing Shao, Chaochao Lu, Yue Zhang, Xian- Sheng Hua, Bowen Zhou, and Yu Cheng. A survey of efficient reasoning for large reasoning models:...

    2025

  4. [12]

    Stop overthinking: A survey on efficient reasoning for large language models.CoRR, abs/2503.16419, 2025

    Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Hanjie Chen, and Xia Ben Hu. Stop overthinking: A survey on efficient reasoning for large language models.CoRR, abs/2503.16419, 2025

    2025 arXiv

  5. [13]

    Tokenskip: Controllable chain-of-thought compression in llms.arXiv preprint arXiv:2502.12067, 2025

    Heming Xia, Yongqi Li, Chak Tou Leong, Wenjie Wang, and Wenjie Li. Tokenskip: Controllable chain-of-thought compression in llms.arXiv preprint arXiv:2502.12067, 2025

    2025

  6. [14]

    Self-training elicits concise reasoning in large language models.arXiv preprint arXiv:2502.20122, 2025

    Tergel Munkhbat, Namgyu Ho, Seo Hyun Kim, Yongjin Yang, Yujin Kim, and Se-Young Yun. Self-training elicits concise reasoning in large language models.arXiv preprint arXiv:2502.20122, 2025

    2025 arXiv

  7. [15]

    Demystifying long chain-of-thought reasoning in llms.arXiv preprint arXiv:2502.03373, 2025

    Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying long chain-of-thought reasoning in llms.arXiv preprint arXiv:2502.03373, 2025

    2025 arXiv

  8. [16]

    Thinkprune: Pruning long chain-of-thought of llms via reinforcement learning.arXiv preprint arXiv:2504.01296, 2025

    Bairu Hou, Yang Zhang, Jiabao Ji, Yujian Liu, Kaizhi Qian, Jacob Andreas, and Shiyu Chang. Thinkprune: Pruning long chain-of-thought of llms via reinforcement learning.arXiv preprint arXiv:2504.01296, 2025

    2025 arXiv

  9. [17]

    L1: Controlling how long a reasoning model thinks with reinforcement learning.arXiv preprint arXiv:2503.04697, 2025

    Pranjal Aggarwal and Sean Welleck. L1: Controlling how long a reasoning model thinks with reinforcement learning.arXiv preprint arXiv:2503.04697, 2025

    2025 arXiv

  10. [18]

    Dast: Difficulty-adaptive slow-thinking for large reasoning models

    Yi Shen, Jian Zhang, Jieyun Huang, Shuming Shi, Wenjing Zhang, Jiangze Yan, Ning Wang, Kai Wang, and Shiguo Lian. Dast: Difficulty-adaptive slow-thinking for large reasoning models. arXiv preprint arXiv:2503.04472, 2025

    2025

  11. [19]

    Thinking, fast and slow.Farrar, Straus and Giroux, 2011

    Daniel Kahneman. Thinking, fast and slow.Farrar, Straus and Giroux, 2011

    2011

  12. [20]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.CoRR, abs/2402.03300, 2024

    2024 arXiv

  13. [21]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  14. [22]

    Limo: Less is more for reasoning.arXiv preprint arXiv:2502.03387, 2025

    Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. Limo: Less is more for reasoning.arXiv preprint arXiv:2502.03387, 2025

    2025 arXiv

  15. [23]

    Towards thinking-optimal scaling of test-time compute for llm reasoning.arXiv preprint arXiv:2502.18080, 2025

    Wenkai Yang, Shuming Ma, Yankai Lin, and Furu Wei. Towards thinking-optimal scaling of test-time compute for llm reasoning.arXiv preprint arXiv:2502.18080, 2025

    2025

  16. [24]

    GPT-4 technical report.CoRR, abs/2303.08774, 2023

    OpenAI. GPT-4 technical report.CoRR, abs/2303.08774, 2023

    2023 arXiv

  17. [25]

    Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica

    Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y . Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl. https://pretty-radio-b75.notion.site/ DeepScaleR-Surpas...

  18. [26]

    Omni-math: A universal olympiad level mathematic benchmark for large language models

    Bofei Gao, Feifan Song, Zhe Yang, Zefan Cai, Yibo Miao, Qingxiu Dong, Lei Li, Chenghao Ma, Liang Chen, Runxin Xu, Zhengyang Tang, Benyou Wang, Daoguang Zan, Shanghaoran Quan, Ge Zhang, Lei Sha, Yichang Zhang, Xuancheng Ren, Tianyu Liu, and Baobao Chang. Omni-math: A universal ...

    arXiv

  19. [27]

    Imitate, explore, and self-improve: A reproduction report on slow-thinking reasoning systems.CoRR, abs/2412.09413, 2024

    Yingqian Min, Zhipeng Chen, Jinhao Jiang, Jie Chen, Jia Deng, Yiwen Hu, Yiru Tang, Jiapeng Wang, Xiaoxue Cheng, Huatong Song, Wayne Xin Zhao, Zheng Liu, Zhongyuan Wang, and Ji-Rong Wen. Imitate, explore, and self-improve: A reproduction report on slow-thinking reasoning system...

    2024 arXiv

  20. [28]

    Training verifiers to solve math word problems.CoRR, abs/2110.14168, 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems.CoRR, abs/2110.14168, 2021

    arXiv

  21. [29]

    Measuring mathematical problem solving with the math dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021

    2021 arXiv

  22. [30]

    Cot-valve: Length-compressible chain-of-thought tuning.arXiv preprint arXiv:2502.09601, 2025

    Xinyin Ma, Guangnian Wan, Runpeng Yu, Gongfan Fang, and Xinchao Wang. Cot-valve: Length-compressible chain-of-thought tuning.arXiv preprint arXiv:2502.09601, 2025

    2025 arXiv

  23. [31]

    Simpo: Simple preference optimization with a reference-free reward.Advances in Neural Information Processing Systems, 37:124198–124235, 2024

    Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with a reference-free reward.Advances in Neural Information Processing Systems, 37:124198–124235, 2024

    2024

  24. [32]

    verl: V olcano engine reinforcement learning for llms

    Bytedance Seed MLSys. verl: V olcano engine reinforcement learning for llms. https: //github.com/volcengine/verl, 2025

    2025

  25. [33]

    System-1

    Swarnadeep Saha, Archiki Prasad, Justin Chih-Yao Chen, Peter Hase, Elias Stengel-Eskin, and Mohit Bansal. System-1. x: Learning to balance fast and slow planning with language models. arXiv preprint arXiv:2407.14414, 2024

    2024 arXiv

  26. [34]

    Visual agents as fast and slow thinkers.arXiv preprint arXiv:2408.08862, 2024

    Guangyan Sun, Mingyu Jin, Zhenting Wang, Cheng-Long Wang, Siqi Ma, Qifan Wang, Tong Geng, Ying Nian Wu, Yongfeng Zhang, and Dongfang Liu. Visual agents as fast and slow thinkers.arXiv preprint arXiv:2408.08862, 2024

    2024 arXiv

  27. [35]

    Think more, hallucinate less: Mitigating hallucinations via dual process of fast and slow thinking.arXiv preprint arXiv:2501.01306, 2025

    Xiaoxue Cheng, Junyi Li, Wayne Xin Zhao, and Ji-Rong Wen. Think more, hallucinate less: Mitigating hallucinations via dual process of fast and slow thinking.arXiv preprint arXiv:2501.01306, 2025

    2025 arXiv

  28. [36]

    Dynathink: Fast or slow? a dynamic decision-making framework for large language models.arXiv preprint arXiv:2407.01009, 2024

    Jiabao Pan, Yan Zhang, Chen Zhang, Zuozhu Liu, Hongwei Wang, and Haizhou Li. Dynathink: Fast or slow? a dynamic decision-making framework for large language models.arXiv preprint arXiv:2407.01009, 2024

    2024 arXiv

  29. [37]

    Qwen2.5 technical report.CoRR, abs/2412.15115, 2024

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.