Pith. sign in

REVIEW 4 major objections 5 minor 10 cited by

Think Only When You Need with Large Hybrid-Reasoning Models

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a language model can be trained to choose per query whether to emit a reasoning trace or answer directly, and that this hybrid policy beats both always-thinking and never-thinking baselines.

desk verdict Plausible method, strong benchmark numbers, but the advertised efficiency gain is never measured and its own examples undercut it. read the letter →

arxiv 2505.14631 v2 pith:I3VXI3W2 submitted 2025-05-20 cs.CL

Lingjie Jiang , Xun Wu , Shaohan Huang , Qingxiu Dong , Zewen Chi , Li Dong , Xingxing Zhang , Tengchao Lv
show 2 more authors
Lei Cui Furu Wei
This is my paper
classification cs.CL
keywords LargeHybrid-ReasoningModelsadaptivethinkingoverthinkingreinforcementlearningGRPOgrouppolicyoptimizationhybridaccuracyreasoningefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a language model can be trained to decide, for each user query, whether to engage in explicit reasoning (a trace) or to answer directly (), and that this per-query choice produces a model that reasons like a large reasoning model on hard problems while behaving like a fast instruction-following model on easy ones. The paper introduces Large Hybrid-Reasoning Models (LHRMs), trained in two stages: Hybrid Fine-Tuning (HFT) on a mix of think-style and no-think-style examples as a cold start, followed by online reinforcement learning called Hybrid Group Policy Optimization (HGPO) that implicitly learns the mode-selection policy from reward feedback. It also proposes Hybrid Accuracy (HAcc), a metric that scores whether the model picks the mode a reward model considers better. On 1.5B and 7B models built from the same base model family, evaluated across math, code, and general benchmarks, LHRMs-7B outperforms all compared LLMs, LRMs, and hybrid baselines on average—for example 93.8 on MATH500 and 66.7 on AIME24—while using less thinking on easy questions. If the results hold, the work moves the field from always-thinking or never-thinking systems toward models that budget reasoning effort according to query difficulty.

What carries the argument

The load-bearing object is the hybrid policy $\pi$ that maps each query $q$ to one of two modes, think ($\vdash$) or no-think ($\nvdash$), with a per-mode conditional answer distribution $P(a \mid q, m)$. The mechanism that carries the argument is the two-stage training pipeline: HFT cold-starts both modes to avoid imbalance, and HGPO then optimizes the policy by sampling $N/2$ responses in each mode, scoring them with a reward model, and computing per-token advantages as the sum of an intra-group GRPO term (best response within a mode) and an inter-group term applied only on the mode-selection tokens ($\langle\text{think}\rangle$ / $\langle\text{no\_think}\rangle$), weighted by $\alpha$. The margin $\delta$ in the inter-group reward controls how strongly the model prefers the higher-scoring mode, giving an explicit knob for trading reasoning accuracy against response speed. Hybrid Accuracy closes the loop by measuring how often the model's self-selected mode matches the reward-model-preferred mode across a prompt set.

What would settle it

Take a sample of the evaluation prompts, have human experts label whether a concise direct answer or a thinking trace is genuinely preferable, and compare those labels to the reward model's ground-truth mode in HAcc; if agreement is low, or if a length-debiased scorer flips the preferred mode on easy questions, then the reported adaptive mode selection would not establish a real efficiency-quality gain.

Watch

Extended reading notes

Core claim

The central claim is that the decision of whether to think is itself a learnable policy, not a fixed model property. The paper defines two modes, Thinking and No-Thinking, and trains the model to maximize expected utility by selecting a mode per query. Stage I, Hybrid Fine-Tuning, gives the model both modes on the same query without collapse by training on 1.7M hybrid-formatted examples. Stage II, HGPO, samples responses in both modes, scores them with a reward model, and uses inter-group and intra-group rewards to reinforce both choosing the better mode and producing better answers within that mode. The paper reports that the resulting LHRMs adaptively choose <no_think> for simple queries and <think> for hard ones, improving reasoning and general benchmarks simultaneously, and it claims this is the first model class to do so explicitly.

Load-bearing premise

The load-bearing premise is that the reward model used to score responses and to define Hybrid Accuracy is a reliable judge of which mode—think or no-think—is genuinely better for each query, since the paper asserts HAcc tracks human expert judgment but reports no human correlation study.

Editorial extensions

If this is right

  • At inference, LHRMs can be deployed as a single model that emits <no_think> on easy queries, cutting token consumption and latency without switching to a separate small model.
  • The margin $\delta$ provides a controllable trade-off: higher $\delta$ biases the model toward direct answers (better for real-time responsiveness), lower $\delta$ toward thinking (better for peak accuracy).
  • Hybrid thinking learned through HGPO transfers across domains: RL on math and general data improved code-task thinking ratio and code performance.
  • HAcc gives a quantitative target for "did the model reason when it should have?" that could be used as an evaluation metric or training signal for future adaptive-reasoning systems.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: Because HAcc defines the "correct" mode with the same reward model used in training, the reported adaptive behavior could partly reflect reward-model bias (for example, a preference for verbose traces); a human-labeled version of HAcc would separate genuine efficiency gains from reward overfitting.
  • Editorial inference: The binary think/no-think formulation suggests a natural extension to graded effort control, such as capping thinking length or selecting among multiple reasoning budgets per query, which the current metric does not measure.
  • Editorial inference: If the cross-domain transfer of thinking ratios is robust, HGPO could be run on a small, cheap RL set in one domain and the learned mode policy transferred to another domain, reducing RL cost; this is testable by reversing the domains.
  • Editorial inference: The model's mode decision is implicit and post-hoc measurable only through the emitted tag; adding an explicit difficulty or confidence signal at the start of generation could make the policy inspectable and easier to align with user-specified latency budgets.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Large Hybrid-Reasoning Models (LHRMs), which learn to choose between a <think> reasoning mode and a <no_think> direct-answer mode per query. The authors propose a two-stage pipeline: Hybrid Fine-Tuning (HFT) as a cold start, followed by Hybrid Group Policy Optimization (HGPO), a GRPO-style online RL method with intra-group and inter-group rewards. They also introduce Hybrid Accuracy (HAcc) as a metric for mode-selection quality. Experiments on Qwen2.5-Math 1.5B and 7B report improvements over LLM, LRM, HFT, HFT-DPO, and HFT-RFT baselines on math, code, and general benchmarks, as well as higher HAcc. The abstract, introduction, and conclusion further claim that LHRMs 'significantly improve efficiency' and were validated by 'human studies.'

Significance. If substantiated, adaptive per-query selection between extended reasoning and direct answering is practically important for reducing inference cost and improving user experience, and the proposed two-stage training recipe is a plausible way to obtain such behavior. The paper's strengths are a coherent RL formulation, a reasonable set of ablated baselines (HFT-DPO, HFT-RFT), ablations over advantage estimators and the margin hyperparameter, and strong downstream accuracy results in Table 1. However, the central efficiency claim is never measured, the claimed human studies and the claimed strong correlation of HAcc with human judgment are not present in the manuscript, and HAcc for non-verifiable queries uses the same reward model that provides the training signal, making the evaluation circular. These gaps are load-bearing for the paper's headline contributions.

major comments (4)
  1. [§3.2, §5, Appendix C, Figs. 7 and 9] The central efficiency claim is never quantified: no table or figure reports output token counts, latency, or cost per solved problem for any model. Figures 4-6 give only the fraction of queries routed to <think>, which is a proxy and says nothing about how many tokens the <no_think> responses consume. Appendix C reports that the No-Thinking SFT data have average length 4,897 tokens versus 575 for the Thinking data, roughly eight times longer, and Figs. 7 and 9 show <no_think> responses containing complete step-by-step derivations. These internal observations make the claimed 'significantly improving efficiency' doubtful rather than merely unmeasured. Please add direct output-length and latency comparisons, and report a length-controlled analysis that does not depend on mode tags.
  2. [§1 and §5] The introduction and conclusion claim 'human studies' and state that HAcc 'correlates strongly with human expert judgment,' yet no human evaluation, correlation coefficient, or inter-annotator agreement is reported anywhere in the paper or its appendices. This claim must be either substantiated with an actual human study or removed from the abstract, introduction, and conclusion.
  3. [§2.4, Eq. (13)] For non-verifiable queries, HAcc defines the ground-truth mode mgt by applying the same parametric reward model (Llama-3.1-Tulu-3-8B-RM) used to compute training rewards in Eq. (7). The evaluation metric is therefore circular with the training signal for general-domain tasks, and the claim that HGPO 'enables the model to learn correct hybrid thinking behaviors' is not supported for those tasks. The tie-break rule that favors shorter responses can also label an unsolved hard problem as No-Thinking, rewarding avoidance of thinking. Please validate HAcc against human labels or an independent judge, and report HAcc separately for rule-based and parametric-reward settings.
  4. [§2.2 and Fig. 7] The No-Thinking mode as implemented still emits step-by-step reasoning: Fig. 7 shows a <no_think> response to '5⋆1' that contains a full step-by-step substitution and arithmetic derivation, and Fig. 9 shows a <no_think> response with a full difference-of-squares solution. This is inconsistent with the framing of No-Thinking as a 'direct-answer' mode and raises the concern that the learned behavior is tag-level rather than a genuine shift from system-2 to system-1 computation. Please clarify what No-Thinking is supposed to mean, and measure its actual computational cost.
minor comments (5)
  1. [Abstract] The abstract contains typos: 'Expensive experimental results' should be 'Extensive experimental results,' and 'significantly improvs efficiency' should be 'significantly improves efficiency.'
  2. [§2.1] The problem formulation section has several wording errors: 'Each mode defines a inducing' is missing a word, 'lean' should be 'learn,' 'code start' should be 'cold start,' and 'fundational' should be 'foundational.'
  3. [§3.2] The sentence beginning 'We observe that LHRMs consistently outperforms all comparable baselines across both the 1.5B and 7B model scales' appears twice verbatim; please remove the duplicate.
  4. [§2.4 and throughout] The metric name is written inconsistently as 'Hacc' in some places and 'HAcc' in others (e.g., Table 1 and Eq. 13); please standardize. Also, 'Apendix' is missing a 'p.'
  5. [Eq. (10)] The inter-group advantage is multiplied by an indicator 1{o_t^i ∈ Φ} so that it is applied only to the mode tokens <think> and <no_think>. Please clarify in the text whether this credit assignment is intentional and how the model is expected to propagate the mode-level advantage to the rest of the generated response.

Circularity Check

1 steps flagged · score 3.0 of 10

The general-domain Hybrid Accuracy metric is self-referential: its ground-truth mode is defined by the same parametric reward model that HGPO optimizes against, so it measures policy–reward-model agreement rather than an external hybrid-thinking ability; the reported MATH500 HAcc and benchmark gains remain non-circular.

  1. self definitional [Section 2.4, Eq. (13) and footnote 2; Section 2.3, Eqs. (7)–(10)]
    "A reward model is then employed as a scorer to evaluate and assign scores to both sets of generated responses. The mode with the higher average score is regarded as the ground-truth preferred reasoning mode for each prompt, denoted as mgt. ... For queries with definitive answers, we use rule-based rewards [SWZ+24, GYZ+25] for a better reward estimation; otherwise, a trained parametric reward model is applied."

    For non-definitive queries, Eq. (13)'s ground-truth mgt is the mode with the higher average parametric-reward score. HGPO training assigns inter-group rewards (Eq. 9) using the same parametric reward model Rϕ, so the policy is explicitly optimized to choose Rϕ's preferred mode. HAcc then reports the fraction of autonomous mode choices that match Rϕ's preference; it is a self-consistency statistic, not an independent measure of hybrid thinking. The paper's assertion that HAcc 'correlates strongly with human expert judgment' is unsupported by any human study in the manuscript. This circularity does not affect the reported MATH500 HAcc, which uses rule-based answer checking as an external ground truth, nor the external reasoning-benchmark results.

full rationale

The paper's central reasoning and general-capability results are benchmarked externally (MATH500, AIME24, AMC23, LiveCodeBench, MBPP, AlpacaEval, Arena-Hard), and the reported MATH500 HAcc uses rule-based answer checking, so those numbers are not circular. The mode-selection behavior is at least directly observed via thinking-ratio analyses in Figures 4–6. The main circularity is in the metric definition: for queries without definitive answers, HAcc's 'ground-truth' preferred mode is defined by the same parametric reward model that HGPO optimizes, making the metric a measure of agreement with the training reward rather than an external ground truth. The introduction additionally claims human studies and a strong human correlation for HAcc, but no human study is present; this is an unsupported-evidence problem rather than a circular-derivation problem. The headline efficiency claim ('significantly improving efficiency') is asserted without any token-count or latency measurement, and Appendix C shows No-Thinking SFT data is roughly eight times longer than Thinking data on average; this is a serious validity gap, but it is not a circular derivation. No load-bearing self-citations or imported uniqueness theorems were found.

Assumptions & free parameters 7 free parameters · 5 assumptions · 1 invented entities

The central claim rests on a set of hand-chosen hyperparameters (margin, alpha, sampling count) and on the validity of the reward signal as the arbiter of thinking mode quality. The <no_think> token is an introduced control token with no external evidence beyond this paper. The FastText classifier threshold is an unquantified free parameter in the data construction.

free parameters (7)
  • margin delta (delta in Eq. 9) = 0.2
    Hand-chosen hyperparameter that biases mode selection toward no-think; larger delta favors direct answers. It directly controls the central adaptive behavior.
  • inter-group advantage weight alpha (alpha in Eq. 10) = 1.0
    Hand-chosen weight controlling how strongly the mode token is rewarded for group-level mode quality.
  • KL coefficient beta (beta in Eq. 11) = 0.001
    Standard RL stability parameter from GRPO.
  • clip epsilon (epsilon in Eq. 11) = 0.5
    PPO-style clipping threshold inherited from GRPO.
  • number of rollout samples N per query = 4 (2 per mode)
    Sampling budget for reward estimation; small N makes group average rewards noisy.
  • HAcc tie-break margin = unspecified
    Section 2.4 says a predefined margin decides when to choose the shorter response as ground-truth mode, but the value is not given.
  • FastText classifier decision threshold for no-think data = unspecified
    Used to split WildChat into simple vs. complex queries; no threshold or accuracy is reported, affecting the training distribution.
assumptions (5)
  • domain assumption The reward model (Llama-3.1-Tulu-3-8B-RM) provides a valid scalar utility for general-domain responses, including mode preference.
    HGPO and HAcc both rely on this reward model to decide which thinking mode is better; no human validation is provided.
  • domain assumption Rule-based exact-answer rewards are a sufficient training signal for mode selection on math tasks.
    The RL objective on math uses final-answer correctness as the reward; this assumes the chosen mode is optimal when it leads to a correct answer.
  • domain assumption Emitting the mode token (<think> or <no_think>) at the start of a response is a learnable control decision.
    The method trains the policy to choose the mode token; this assumes the model can generalize mode choice to new queries.
  • standard math Standard PPO/GRPO assumptions: importance sampling, clipping, and KL regularization stabilize the RL update.
    The authors borrow the update rule from GRPO and REINFORCE++, assuming convergence behavior.
  • ad hoc to paper The FastText classifier reliably separates queries that need thinking from those that do not.
    Used to construct the no-think training set; no evaluation of classifier accuracy is provided.
invented entities (1)
  • <no_think> mode token
    purpose: Explicitly marks responses that should not include extended reasoning.
    A new special control token introduced by the paper to signal direct-answer mode. Its effectiveness is only demonstrated through the paper's own experiments, and there is no external falsifiable prediction attached to it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Think Only When You Need with Large Hybrid-Reasoning Models." pith.science (2026). https://pith.science/paper/I3VXI3W2

@misc{pith2026250514631,
  author       = {Pith},
  title        = {Pith review of: Think Only When You Need with Large Hybrid-Reasoning Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I3VXI3W2}},
  note         = {Machine review of arXiv:2505.14631}
}
read the original abstract

Recent Large Reasoning Models (LRMs) have shown substantially improved reasoning capabilities over traditional Large Language Models (LLMs) by incorporating extended thinking processes prior to producing final responses. However, excessively lengthy thinking introduces substantial overhead in terms of token consumption and latency, which is particularly unnecessary for simple queries. In this work, we introduce Large Hybrid-Reasoning Models (LHRMs), the first kind of model capable of adaptively determining whether to perform thinking based on the contextual information of user queries. To achieve this, we propose a two-stage training pipeline comprising Hybrid Fine-Tuning (HFT) as a cold start, followed by online reinforcement learning with the proposed Hybrid Group Policy Optimization (HGPO) to implicitly learn to select the appropriate thinking mode. Furthermore, we introduce a metric called Hybrid Accuracy to quantitatively assess the model's capability for hybrid thinking. Extensive experimental results show that LHRMs can adaptively perform hybrid thinking on queries of varying difficulty and type. It outperforms existing LRMs and LLMs in reasoning and general capabilities while significantly improving efficiency. Together, our work advocates for a reconsideration of the appropriate use of extended thinking processes and provides a solid starting point for building hybrid thinking systems.

Figures

Figures reproduced from arXiv: 2505.14631 by the authors.

Figure 1
Figure 1. Example responses from Qwen2.5-7B-Instruct, DeepSeek-R1-Distill-Qwen-7B, and our [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Demonstration of Hybrid Group Pol￾icy Optimization. HGPO proceeds by: (1) sam￾pling multiple responses for each query q using both reasoning modes; (2) scoring the responses with the reward model and assigning these re￾wards based on Eq. 9; and (3) computing the advantage and policy loss, followed by updat￾ing the policy model. AE denotes advantage estimator and reward assignment denotes Eq. 9. Algorithm 1 Hybrid Gr… view at source ↗
Figure 3
Figure 3. Ablation study on the effects of advantage estimators and margin [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Analysis of thinking ratio of LHRMs within a single domain. Effect of Margin δ in Eq. 9. We investigate how varying the margin δ influences the model’s bias toward the two reasoning modes. As shown in [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Analysis of thinking ratio of LHRMs across different domains. Thinking Ratio Study within Domain [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 7
Figure 7. Figure 7: Example outputs from DeepSeek-R1-Distill-Qwen-7B and our LHRMs-7B model. For [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 6
Figure 6. Figure 6: Ablation study on model size. Case Analysis. We present representative model re￾sponses in [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 8
Figure 8. Figure 8: Token length distributions of Thinking and No-Thinking data in Stage I [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: LHRMs-7B uses the No-Thinking mode to solve easy math problems. [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: LHRMs-7B learned to select the Thinking mode to solve hard code problems. [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ConPress: Learning Efficient Reasoning from Multi-Question Contextual Pressure

    cs.CL 2026-02 conditional novelty 6.0 of 10

    Multi-question prompts elicit shorter chain-of-thought traces, and fine-tuning on those traces transfers the compression to single-question reasoning.

  2. AdaThink-Med: Optimizing Inference-Time Compute for Medical Reasoning via Uncertainty Quantification

    cs.CL 2025-09 conditional novelty 6.0 of 10

    An entropy-guided reward that shortens answers to easy, confident questions and extends reasoning on hard ones reduces medical QA tokens by 4.7x to 6.4x with minimal accuracy loss.

  3. Hierarchical Budget Policy Optimization for Adaptive Reasoning

    cs.AI 2025-07 conditional novelty 6.0 of 10

    Training reasoning models with hierarchical token budgets and budget-aware rewards produces up to 60.6% token reduction with no accuracy loss, and benchmark-level evidence of length adaptation.

  4. How Far Are We from Optimal Reasoning Efficiency?

    cs.CL 2025-06 conditional novelty 6.0 of 10

    The authors define a reasoning efficiency frontier and a gap metric (REG), then train models with REO-RL to shrink the gap by at least 50% with only small accuracy losses.

  5. Efficient Reasoning on the Edge

    cs.LG 2026-03 accept novelty 5.5 of 10

    LoRA adapters, budget-forced GRPO, dynamic switching, parallel verification and FPTQuant enable practical chain-of-thought reasoning on quantized Qwen2.5-7B for edge devices.

  6. KAT-V1: Kwai-AutoThink Technical Report

    cs.CL 2025-07 conditional novelty 5.0 of 10

    KAT-V1-40B is a 40B language model that switches between deep reasoning and direct answering per query, reporting matching-or-better benchmark scores with lower token use.

  7. Matching Game Preferences Through Dialogical Large Language Models: A Perspective

    cs.AI 2025-07 conditional novelty 4.0 of 10

    This perspective paper proposes the D-LLM framework, which couples the authors' GRAPHYP knowledge graphs with LLMs to personalize AI responses and make reasoning traceable, but no empirical validation is presented.

  8. Schema-R1: A reasoning training approach for schema linking in Text-to-SQL Task

    cs.AI 2025-06 conditional novelty 4.0 of 10

    Schema-R1 combines cold-start SFT on 200 CoT samples with GRPO rule-based RL, reporting table and column filter accuracy gains of 10 percentage points or more over a fine-tuned baseline on Spider-dev.

  9. Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle

    cs.CL 2025-09 conditional novelty 3.0 of 10

    A survey that maps reinforcement learning methods, datasets, benchmarks, and open-source tools across the full training lifecycle of large language models, focusing on verifiable-reward reasoning.

  10. Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey

    cs.AI 2025-07 conditional novelty 3.0 of 10

    A comprehensive review that categorizes methods for shortening and adaptively triggering chain-of-thought reasoning in large language models.

Reference graph

Works this paper leans on

56 extracted references · 14 canonical work pages · cited by 10 Pith papers

  1. [1]

    Back to Basics: Revisiting REINFORCE-Style Optimization for Learning from Human Feedback in LLMs

    Arash Ahmadian, Chris Cremer, Matthias Gall \' e , Marzieh Fadaee, Julia Kreutzer, Olivier Pietquin, Ahmet \" U st \" u n, and Sara Hooker. Back to Basics: Revisiting REINFORCE-Style Optimization for Learning from Human Feedback in LLMs . In Lun - Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association...

    work page 2024

  2. [2]

    Aime 2024, 2024

    AI-MO. Aime 2024, 2024

    2024

  3. [3]

    Claude 3.7 sonnet and claude code

    Anthropic. Claude 3.7 sonnet and claude code. https://www.anthropic.com/news/claude-3-7-sonnet, 2025

    work page 2025

  4. [4]

    Program synthesis with large language models

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732 , 2021

    arXiv 2021

  5. [5]

    Le, Christopher Ré, and Azalia Mirhoseini

    Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V. Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling, 2024

    work page 2024

  6. [6]

    Kcts: Knowledge-constrained tree search decoding with token-level hallucination detection, 2023

    Sehyun Choi, Tianqing Fang, Zhaowei Wang, and Yangqiu Song. Kcts: Knowledge-constrained tree search decoding with token-level hallucination detection, 2023

    work page 2023

  7. [7]

    R1-v: Reinforcing super generalization ability in vision-language models with less than \ 3

    Liang Chen, Lei Li, Haozhe Zhao, Yifan Song, and Vinci. R1-v: Reinforcing super generalization ability in vision-language models with less than \ 3. https://github.com/Deep-Agent/R1-V, 2025. Accessed: 2025-02-02

    2025

  8. [8]

    Do not think that much for 2+ 3=? on the overthinking of o1-like llms

    Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, et al. Do not think that much for 2+ 3=? on the overthinking of o1-like llms. arXiv preprint arXiv:2412.21187 , 2024

    arXiv 2024

Show all 56 references
  1. [9]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    2025

  2. [10]

    Length-controlled alpacaeval: A simple way to debias automatic evaluators

    Yann Dubois, Bal \'a zs Galambosi, Percy Liang, and Tatsunori B Hashimoto. Length-controlled alpacaeval: A simple way to debias automatic evaluators. arXiv preprint arXiv:2404.04475 , 2024

    2024 arXiv

  3. [11]

    Open r1: A fully open reproduction of deepseek-r1, January 2025

    Hugging Face. Open r1: A fully open reproduction of deepseek-r1, January 2025

    2025

  4. [12]

    Video-r1: Reinforcing video reasoning in mllms

    Kaituo Feng, Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang, Tianshuo Peng, Benyou Wang, and Xiangyu Yue. Video-r1: Reinforcing video reasoning in mllms. arXiv preprint arXiv:2503.21776 , 2025

    2025 arXiv

  5. [13]

    Gemini 2.5 flash

    Google. Gemini 2.5 flash. https://deepmind.google/technologies/gemini/flash/, 2025

    2025

  6. [14]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948 , 2025

    2025 arXiv

  7. [15]

    Vision-r1: Incentivizing reasoning capability in multimodal large language models

    Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Yao Hu, and Shaohui Lin. Vision-r1: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749 , 2025

    2025 arXiv

  8. [16]

    Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems, 2024

    Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scienti...

    2024

  9. [17]

    Reinforce++: A simple and efficient approach for aligning large language models

    Jian Hu. Reinforce++: A simple and efficient approach for aligning large language models. arXiv preprint arXiv:2501.03262 , 2025

    2025 arXiv

  10. [18]

    Rewarding chatbots for real-world engagement with millions of users, 2023

    Robert Irvine, Douglas Boubert, Vyas Raina, Adian Liusie, Ziyi Zhu, Vineet Mudupalli, Aliaksei Korshuk, Zongyi Liu, Fritz Cremer, Valentin Assassi, Christie-Carol Beauchamp, Xiaoding Lu, Thomas Rialan, and William Beauchamp. Rewarding chatbots for real-world engagement with mi...

    2023

  11. [19]

    Fasttext.zip: Compressing text classification models

    Armand Joulin, Edouard Grave, Piotr Bojanowski, Matthijs Douze, H \'e rve J \'e gou, and Tomas Mikolov. Fasttext.zip: Compressing text classification models. arXiv preprint arXiv:1612.03651 , 2016

    2016 arXiv

  12. [20]

    Livecodebench: Holistic and contamination free evaluation of large language models for code

    Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974 , 2024

    2024 arXiv

  13. [21]

    o pf, Yannic Kilcher, Dimitri Von R \

    Andreas K \"o pf, Yannic Kilcher, Dimitri Von R \"u tte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Duc Nguyen, Oliver Stanley, Rich \'a rd Nagyfi, et al. Openassistant conversations-democratizing large language model alignment. Advances in Neural Info...

    2023

  14. [22]

    From crowdsourced data to high-quality benchmarks: Arena-hard and benchbuilder pipeline

    Tianle Li, Wei-Lin Chiang, Evan Frick, Lisa Dunlap, Tianhao Wu, Banghua Zhu, Joseph E Gonzalez, and Ion Stoica. From crowdsourced data to high-quality benchmarks: Arena-hard and benchbuilder pipeline. arXiv preprint arXiv:2406.11939 , 2024

    2024 arXiv

  15. [23]

    Don't throw away your value model! generating more preferable text with value-guided monte-carlo tree search decoding, 2024

    Jiacheng Liu, Andrew Cohen, Ramakanth Pasunuru, Yejin Choi, Hannaneh Hajishirzi, and Asli Celikyilmaz. Don't throw away your value model! generating more preferable text with value-guided monte-carlo tree search decoding, 2024

    2024

  16. [24]

    A simple model of inference scaling laws, 2024

    Noam Levi. A simple model of inference scaling laws, 2024

    2024

  17. [25]

    Let's verify step by step

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. arXiv preprint arXiv:2305.20050 , 2023

    2023 arXiv

  18. [26]

    Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D

    Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luc...

    2024

  19. [27]

    Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl, 2025

    Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Erran Li, Raluca Ada Popa, and Ion Stoica. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl, 2025. Notion Blog

    2025

  20. [28]

    Video-t1: Test-time scaling for video generation

    Fangfu Liu, Hanyang Wang, Yimo Cai, Kaiyan Zhang, Xiaohang Zhan, and Yueqi Duan. Video-t1: Test-time scaling for video generation. arXiv preprint arXiv:2503.18942 , 2025

    2025 arXiv

  21. [29]

    Evaluating language models for efficient code generation

    Jiawei Liu, Songrun Xie, Junhao Wang, Yuxiang Wei, Yifeng Ding, and Lingming Zhang. Evaluating language models for efficient code generation. arXiv preprint arXiv:2408.06450 , 2024

    2024 arXiv

  22. [30]

    Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation

    Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. Advances in Neural Information Processing Systems , 36:21558--21572, 2023

    2023

  23. [31]

    Aimo-2 winning solution: Building state-of-the-art mathematical reasoning models with openmathreasoning dataset

    Ivan Moshkov, Darragh Hanley, Ivan Sorokin, Shubham Toshniwal, Christof Henkel, Benedikt Schifferer, Wei Du, and Igor Gitman. Aimo-2 winning solution: Building state-of-the-art mathematical reasoning models with openmathreasoning dataset. arXiv preprint arXiv:2504.16891 , 2025

    2025 arXiv

  24. [32]

    Synthetic-1: Two million collaboratively generated reasoning traces from deepseek-r1, 2025

    Justus Mattern, Sami Jaghouar, Manveer Basra, Jannik Straube, Matthew Di Ferrante, Felix Gabriel, Jack Min Ong, Vincent Weisser, and Johannes Hagemann. Synthetic-1: Two million collaboratively generated reasoning traces from deepseek-r1, 2025

    2025

  25. [33]

    Leveraging online olympiad-level math problems for llms training and contamination-resistant evaluation

    Sadegh Mahdavi, Muchen Li, Kaiwen Liu, Christos Thrampoulidis, Leonid Sigal, and Renjie Liao. Leveraging online olympiad-level math problems for llms training and contamination-resistant evaluation. arXiv preprint arXiv:2501.14275 , 2025

    2025 arXiv

  26. [34]

    s1: Simple test-time scaling

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Cand \`e s, and Tatsunori Hashimoto. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393 , 2025

    2025 arXiv

  27. [35]

    Openai gpt-4.5 system card

    OpenAI. Openai gpt-4.5 system card. OpenAI Publication , 2025

    2025

  28. [36]

    Codeforces

    Guilherme Penedo, Anton Lozhkov, Hynek Kydlíček, Loubna Ben Allal, Edward Beeching, Agustín Piqueres Lajarín, Quentin Gallouédec, Nathan Habib, Lewis Tunstall, and Leandro von Werra. Codeforces. https://huggingface.co/datasets/open-r1/codeforces, 2025

    2025

  29. [37]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems , 36:53728--53741, 2023

    2023

  30. [38]

    Stop overthinking: A survey on efficient reasoning for large language models

    Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Hanjie Chen, et al. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503.16419 , 2025

    2025 arXiv

  31. [39]

    Scaling llm test-time compute optimally can be more effective than scaling model parameters

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314 , 2024

    2024 arXiv

  32. [40]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 , 2024

    2024 arXiv

  33. [41]

    Hybridflow: A flexible and efficient rlhf framework

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256 , 2024

    2024 arXiv

  34. [42]

    Qwq: Reflect deeply on the boundaries of the unknown, November 2024

    Qwen Team. Qwq: Reflect deeply on the boundaries of the unknown, November 2024

    2024

  35. [43]

    Open Thoughts

    OpenThoughts Team. Open Thoughts . https://open-thoughts.ai, January 2025

    2025

  36. [44]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems , 35:24824--24837, 2022

    2022

  37. [45]

    Teaching language models to critique via reinforcement learning

    Zhihui Xie, Liyu Chen, Weichao Mao, Jingjing Xu, Lingpeng Kong, et al. Teaching language models to critique via reinforcement learning. arXiv preprint arXiv:2502.03492 , 2025

    2025

  38. [46]

    Towards large reasoning models: A survey of reinforced reasoning with large language models

    F Xu, Q Hao, Z Zong, J Wang, Y Zhang, J Wang, X Lan, J Gong, T Ouyang, F Meng, et al. Towards large reasoning models: A survey of reinforced reasoning with large language models. arxiv. 10.48550. arXiv preprint arXiv.2501.09686 , 2025

    arXiv

  39. [47]

    Kodcode: A diverse, challenging, and verifiable synthetic dataset for coding, 2025

    Zhangchen Xu, Yang Liu, Yueqin Yin, Mingyuan Zhou, and Radha Poovendran. Kodcode: A diverse, challenging, and verifiable synthetic dataset for coding, 2025

    2025

  40. [48]

    Limo: Less is more for reasoning, 2025

    Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. Limo: Less is more for reasoning, 2025

    2025

  41. [49]

    Scaling relationship on learning mathematical reasoning with large language models

    Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Keming Lu, Chuanqi Tan, Chang Zhou, and Jingren Zhou. Scaling relationship on learning mathematical reasoning with large language models. arXiv preprint arXiv:2308.01825 , 2023

    2023 arXiv

  42. [50]

    Qwen2.5 technical report

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 arXiv

  43. [51]

    Qwen2.5-math technical report: Toward mathematical expert model via self-improvement

    An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang. Qwen2.5-math technical report: Toward mathematical expert model via se...

    2024 arXiv

  44. [52]

    Tenenbaum, and Chuang Gan

    Shun Zhang, Zhenfang Chen, Yikang Shen, Mingyu Ding, Joshua B. Tenenbaum, and Chuang Gan. Planning with large language models for code generation, 2023

    2023

  45. [53]

    Wildchat: 1m chat GPT interaction logs in the wild

    Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. Wildchat: 1m chat GPT interaction logs in the wild. In The Twelfth International Conference on Learning Representations , 2024

    2024

  46. [54]

    1.4 million open-source distilled reasoning dataset to empower large language model training, 2025

    Han Zhao, Haotian Wang, Yiping Peng, Sitong Zhao, Xiaoyu Tian, Shuaiting Chen, Yunjie Ji, and Xiangang Li. 1.4 million open-source distilled reasoning dataset to empower large language model training, 2025

    2025

  47. [55]

    Language agent tree search unifies reasoning acting and planning in language models, 2024

    Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman, Haohan Wang, and Yu-Xiong Wang. Language agent tree search unifies reasoning acting and planning in language models, 2024

    2024

  48. [56]

    Llamafactory: Unified efficient fine-tuning of 100+ language models

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System ...

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.