Pith. sign in

REVIEW 4 major objections 5 minor 4 cited by

Parallel-R1: Towards Parallel Thinking via Reinforcement Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Parallel-R1 is the first RL framework to train parallel-thinking behaviors in LLMs on general math, using a progressive curriculum that beats the GRPO baseline by 8.4% on average.

desk verdict Parallel-R1 is a genuinely new RL recipe for instilling parallel thinking, but the headline gains are not yet separable from an uncontrolled inference-compute difference; still worth refereeing. read the letter →

arxiv 2509.07980 v2 pith:VUMJ4IFS submitted 2025-09-09 cs.CL

Tong Zheng , Hongming Zhang , Wenhao Yu , Xiaoyang Wang , Runpeng Dai , Rui Liu , Huiwen Bao , Chengsong Huang
show 2 more authors
Heng Huang Dong Yu
This is my paper
classification cs.CL
keywords parallelthinkingreinforcementlearningGRPOcurriculummathematicalreasoningrewarddesigncold-starttrainingexplorationscaffold
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Parallel-R1 is a training framework that tries to make an LLM genuinely think in parallel—pausing its main reasoning chain, spawning several independent solution attempts, and summarizing them—rather than merely imitating such traces from synthetic data. The core proposal is a progressive curriculum that solves the cold-start problem: teach the model the parallel-thinking format on easy GSM8K problems with a small supervised fine-tuning set, then let reinforcement learning (GRPO on DAPO problems) explore and generalize the behavior on hard competition tasks. The paper reports average accuracy gains of 8.4% over the standard GRPO baseline across AIME25, AIME24, AMC23, and MATH, and a peak AIME25 score of 25.6% obtained by a two-stage RL schedule where parallel thinking is first forced for exploration and then dropped. It also documents a strategic shift during training: the model starts by using parallel paths to search for solutions and later uses them to verify an already-derived answer. If correct, this shows that parallel thinking is a learnable train-time skill, not just an architectural or prompting trick.

What carries the argument

The load-bearing mechanism is the progressive curriculum: an SFT cold-start on Parallel-GSM8K teaches the control tags <Parallel>, <Path>, and <Summary>; a small RL stage on the same GSM8K set stabilizes the format with the reward R_parallel × R_acc; and a final GRPO stage on the DAPO set with accuracy-only reward lets the model learn when and how to branch on hard problems. The paper also tests a structured variant with path-window attention masks and multiverse position encodings to isolate parallel paths, but finds the causal variant works better; the full framework therefore rests on the interaction between the cold-start data and the reward schedule, not on architectural constraints.

What would settle it

Train the same Qwen-3-4B base with GRPO on DAPO using the identical parallel-thinking prompt and the same alternating reward schedule, but without any SFT cold-start; measure the fraction of rollouts that contain a valid <Parallel> block and the final AIME25 accuracy. If valid parallel traces appear and accuracy matches Parallel-R1, cold-start SFT is unnecessary, and the paper's explanation of the cold-start bottleneck is wrong.

Watch

Extended reading notes

Core claim

The central claim is that parallel thinking—generating multiple independent reasoning paths at critical steps and then summarizing them—can be instilled in an LLM by a progressive curriculum that separates format learning from behavioral exploration. The authors show that a powerful model (DeepSeek-R1-0528-Qwen-3-8B) can produce well-formed parallel traces for 83.6% of simple GSM8K problems but for 0.0% of hard DAPO problems, so they use the easy data only to teach the <Parallel>/<Path>/<Summary> format via SFT, then switch to GRPO with an accuracy reward on DAPO so the model can explore and generalize the behavior on hard problems. The resulting Parallel-R1-Seen model averages 48.9 across A

Load-bearing premise

The progressive curriculum assumes that the parallel-thinking format learned on easy GSM8K problems transfers to hard DAPO problems once RL begins; if the format does not generalize beyond easy math, the cold-start stage fails to bootstrap RL and the benchmark gains disappear.

Editorial extensions

If this is right

  • Models trained with this curriculum can be deployed with the parallel-thinking tags at inference, and they will branch and merge on their own, without an external orchestrator or a fixed branching schedule.
  • The same recipe can be applied to any base model: SFT on easy parallel traces, a short RL stage on the same easy set to stabilize the format, then RL on the hard target set.
  • Because the late-stage model uses parallelism for verification, downstream users get an internal consistency check that catches some wrong answers before final output.
  • The two-stage scaffold result implies that even if parallel thinking is too expensive to deploy at scale, spending 200 training steps forcing it can improve the final accuracy of the sequential model, so it should be considered a training-time intervention.
  • Reward design is decisive: purely structural rewards can destroy accuracy, purely outcome rewards fail to elicit parallel structure; the alternating schedule is a practical middle ground for production use.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: the same two-stage curriculum should work for other structured scaffolds (e.g., self-critique, tool-use blocks) if valid traces can be cheaply generated on an easy task; the key is that the format, not the content, is being bootstrapped.
  • The exploration-to-verification shift predicts that a stronger base model will place the <Parallel> block even later in generation and use shorter paths, since it can solve more problems on the first pass and only needs confirmation.
  • A focused study could replace the alternating reward schedule with a decaying parallel-reward coefficient during the RL stage, checking whether the model keeps the verification benefit without the early high-variance exploration being over-reinforced.
  • The near-0% format validity on DAPO suggests the bottleneck is not reward design but the model's inability to even emit a parallel structure on hard problems; any RL algorithm that cannot start from a valid sample will not explore that part of the policy space.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes Parallel-R1, a multi-stage reinforcement learning (RL) framework for teaching a Qwen3-4B base model to produce explicit parallel-thinking traces, using control tags <Parallel>, <Path>, and <Summary>. The training pipeline consists of a cold-start SFT stage on an easy GSM8K parallel-traces dataset, an optional RL stage on the same easy data, and a final RL stage on the harder DAPO dataset. A structured variant (Parallel-R1-Unseen) adds path-window attention masking and multiverse position encodings. The main empirical claim is an 8.4% average accuracy improvement over standard GRPO (48.9 vs 45.1 on AIME25, AIME24, AMC23, and MATH, Table 2). The paper also reports a behavioral shift from early exploration to late verification and proposes parallel thinking as a mid-training scaffold, with a peak AIME25 accuracy of 25.6%.

Significance. If the central claim holds, this is a useful and fairly original empirical contribution: it is one of the first attempts to use RL rather than SFT to instill parallel reasoning in general mathematical tasks, and it provides transparent reward analyses and a behavioral characterization of how the strategy evolves during training. The promised release of model, data, and code would strengthen reproducibility. The paper's key empirical claim is currently undermined, however, by the absence of any inference-compute control and by an undefined baseline in the scaffold experiment. These issues are fixable and do not invalidate the overall approach, but they must be addressed before the reported gains can be attributed to a learned parallel-thinking skill.

major comments (4)
  1. [Section 4.1; Table 2] The headline comparison is not inference-compute matched. Evaluation samples 16 responses per problem, and Parallel-R1's responses can contain multiple <Path> blocks, each effectively an additional candidate reasoning chain. The paper does not report average response length, token count, or FLOPs for any configuration. Thus the 8.4% average gain (48.9 vs 45.1) may reflect a larger effective test-time sample (self-consistency/best-of-k behavior) rather than a learned, transferable parallel-thinking skill. This is load-bearing for the central claim. Please report token-normalized or compute-matched accuracy (e.g., accuracy as a function of total generated tokens, or an equal-token-budget baseline), and include variance across seeds.
  2. [Figure 4; Section 4.5] The '42.9% improvement' claim is not tied to a defined baseline. Figure 4's legend says 'Baseline GRPO', but the only GRPO baseline in Table 2 has AIME25=14.8, which would make 25.6 a 73% improvement (25.6/14.8 − 1), not 42.9%. If the intended baseline is 17.9 (the 'Parallel-R1-Seen w/o RL on GSM8K' row in Table 3), that is an ablation variant, not a standard GRPO baseline, and the figure must say so explicitly. In addition, no error bars or multiple seeds are reported, and the '+200-step shift' used to align the Stage-2 curve makes the comparison difficult to verify.
  3. [Table 3; Section 4.3.2] The 'w/o Parallel Thinking Prompt' ablation does not control for the compute confound. Removing the prompt may reduce the rate of explicit <Parallel> tags, but the model can still generate long, multi-path outputs; without reporting the parallel ratio or token counts for this condition, the observed 1.8% average drop cannot be attributed to the prompt as opposed to changes in response length or other factors. This is directly relevant to the claim that the prompt helps the model 'better understand the reasoning process.'
  4. [Section 4.1; Section 4.2] No statistical reliability information is provided. All reported numbers come from single runs without standard deviations or seeds, despite mean@16 being a stochastic estimate. Given the relatively small benchmark sizes and the absence of any compute control, the reported differences (e.g., 48.9 vs 45.1) may be within run-to-run variance. At minimum, the authors should report multiple seeds or bootstrap confidence intervals.
minor comments (5)
  1. [Throughout] Typographical and naming issues: 'Parallel-SFT-Uneen' (Section 4.1) should be 'Unseen'; 'introduces' (Section 2.2) should be 'introduce'; '¡Path¿' (Section 3.4.1) is an encoding artifact; 'Baseline GRPO' and 'GRPO (DAPO)' are used inconsistently. The terms 'Seen' and 'Unseen' are never defined.
  2. [Figure 3] Please define 'relative position' explicitly in the caption or axis label, and clarify whether the plotted quantity is the mean across all <Parallel> blocks or only the first one. Error bars or a shaded confidence band would help, since the figure is central to the exploration-to-verification claim.
  3. [Section 1; Related Work] The phrase 'first reinforcement learning framework' is too strong given Pan et al. (2025) already applies RL to parallel reasoning on CountDown. The authors acknowledge this in Section 2.1, but the abstract and introduction should qualify the claim as 'first for general mathematical tasks' to avoid overclaiming.
  4. [Appendix A] The Parallel Thinking Prompt instructs the model not to explicitly mention that it is triggering parallel thinking, while Section 3.2 describes the model 'predicting' a <Parallel> token. This is not a contradiction, but the relationship between the prompt instruction and the trained behavior should be clarified.
  5. [Section 3.5.2] The reward notation is introduced separately for the causal and structured variants. Unifying the notation (e.g., defining R_parallel, R_acc, and the alternating schedule once) would improve readability and reduce the chance of misinterpreting the ablation results in Table 4.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claims are empirical benchmark comparisons with transparent reward shaping, not derivations that reduce to their own inputs.

full rationale

The paper's central claims are empirical: Parallel-R1 is trained with SFT cold-start on GSM8K plus GRPO on DAPO, and evaluated on held-out math benchmarks (AIME25, AIME24, AMC23, MATH). The reward functions are training objectives that explicitly incentivize the use of <Parallel>/<Path>/<Summary> structures, and the paper then measures how often those structures appear and how accuracy changes. This is transparent reward shaping plus behavioral tracking, not a fitted parameter renamed as a prediction. Key Finding 1 (83.6% format validity on GSM8K vs 0.0% on DAPO) is an empirical observation about data-generation difficulty, not an input-output tautology. The ablations in Table 3 compare training-stage configurations and are reported as measurements. The 'mid-training exploration scaffold' result in Figure 4 is an intervention study, not a derivation from an assumed uniqueness theorem. Self-citations (e.g., R-Zero, R1-RE, Mixture-of-Thought) appear only in related-work context and are not load-bearing for the main claims. The paper does contain limitations that a reviewer should weigh separately: Figure 4's 'Baseline GRPO' is not precisely defined, and the paper does not report token budgets or compute-matched evaluation, leaving open the possibility that accuracy gains partly reflect extra inference tokens rather than a learned transferable skill. However, that is an external-validity or confound concern, not circularity under any of the enumerated patterns: no equation defines a predicted quantity in terms of the fitted reward values, no central premise depends on a self-citation, and no known result is merely renamed. Therefore the appropriate circularity score is 0.

Assumptions & free parameters 6 free parameters · 5 assumptions · 1 invented entities

The method introduces several hand-chosen hyperparameters (reward schedule, step counts, reward magnitudes) and relies on assumptions about data quality and transfer that are load-bearing for the curriculum. The control tags are novel format entities, but they are internal to the method.

free parameters (6)
  • Alternating reward window W = 10 steps
    Chosen by hand in Section 3.5.2; the schedule alternates ACC/PAR rewards every W steps. Central to the reward design comparison.
  • Alternating reward split (80/20) = 80% ACC, 20% PAR
    Hand-chosen schedule in Section 3.5.2 to balance accuracy and parallel structure during stage-two RL.
  • Tiered reward values = +1.2 / +1.0 / -1.0
    Hand-chosen reward magnitudes in Section 3.5.2 for the alternating scheme; these values calibrate the incentive for parallel usage.
  • Stage 1 RL gradient steps = 35
    GRPO on GSM8K for five epochs, resulting in 35 gradient updates (Section 4.1).
  • Stage 2 RL gradient steps = 300
    GRPO on DAPO for 300 gradient updates (Section 4.1).
  • Number of parallel paths N
    Not specified in the paper. The inference workflow states N independent trajectories are generated, but the value of N is never given, affecting reproducibility.
assumptions (5)
  • standard math Standard RLVR/GRPO framework provides a reliable training signal for reasoning quality
    The paper relies on GRPO (Shao et al., 2024) and verifiable rewards for training; Section 3.4.1.
  • domain assumption DeepSeek-R1-0528-Qwen-3-8B can produce valid parallel-thinking traces on easy GSM8K problems via zero-shot prompting
    Key Finding 1 (Section 3.3, Table 1): 83.7% format validity on GSM8K vs 0.0% on DAPO. The cold-start dataset depends on this.
  • ad hoc to paper Parallel thinking format learned on easy math transfers to hard math via RL
    The progressive curriculum assumes the format instilled on GSM8K generalizes to DAPO and benchmarks through RL (Section 3.4, ablation Table 3).
  • domain assumption The <Parallel> tag can be learned and reliably triggered at inference for parallel decoding
    The inference workflow (Section 3.2) depends on the model emitting the special tag; the token was absent from pretraining (Section 3.4.2).
  • domain assumption Benchmark rewards (AIME, AMC, MATH) are reliable proxies for reasoning ability
    Evaluation uses these benchmarks as the measure of success (Section 4.1).
invented entities (1)
  • <Parallel>, <Path>, <Summary> control tags
    purpose: Structured markers to trigger and delimit parallel thinking blocks during training and inference.
    Introduced by the authors (Section 3.2); no external falsifiable handle exists for these format tokens.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Parallel-R1: Towards Parallel Thinking via Reinforcement Learning." pith.science (2026). https://pith.science/paper/VUMJ4IFS

@misc{pith2026250907980,
  author       = {Pith},
  title        = {Pith review of: Parallel-R1: Towards Parallel Thinking via Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VUMJ4IFS}},
  note         = {Machine review of arXiv:2509.07980}
}
read the original abstract

Parallel thinking has emerged as a novel approach for enhancing the reasoning capabilities of large language models (LLMs) by exploring multiple reasoning paths concurrently. However, activating such capabilities through training remains challenging, as existing methods predominantly rely on supervised fine-tuning (SFT) over synthetic data, which encourages teacher-forced imitation rather than exploration and generalization. Different from them, we propose \textbf{Parallel-R1}, the first reinforcement learning (RL) framework that enables parallel thinking behaviors for complex real-world reasoning tasks. Our framework employs a progressive curriculum that explicitly addresses the cold-start problem in training parallel thinking with RL. We first use SFT on prompt-generated trajectories from easier tasks to instill the parallel thinking ability, then transition to RL to explore and generalize this skill on harder problems. Experiments on various math benchmarks, including MATH, AMC23, and AIME, show that Parallel-R1 successfully instills parallel thinking, leading to 8.4% accuracy improvements over the sequential thinking model trained directly on challenging tasks with RL. Further analysis reveals a clear shift in the model's thinking behavior: at an early stage, it uses parallel thinking as an exploration strategy, while in a later stage, it uses the same capability for multi-perspective verification. Most significantly, we validate parallel thinking as a \textbf{mid-training exploration scaffold}, where this temporary exploratory phase unlocks a higher performance ceiling after RL, yielding a 42.9% improvement over the baseline on AIME25. Our model, data, and code will be open-source at https://github.com/zhengkid/Parallel-R1.

Figures

Figures reproduced from arXiv: 2509.07980 by the authors.

Figure 1
Figure 1. An overview of the proposed framework. (Top) During inference, the model generates in a [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the structured atten￾tion mask and position IDs, where different paths and the summary block have distinct visibility regions. Blank regions indicate to￾kens that cannot attend to each other, while colored regions indicate tokens that can. To explore an alternative solution, we introduce a struc￾tured variant of our framework, Parallel-Unseen. This model incorporates explicit inductive biases into th… view at source ↗
Figure 3
Figure 3. Dynamics of the relative position of the [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Two-stage training with parallel reasoning as a mid-training exploration scaffold. Left axis [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: A qualitative case study showcases that our model uses parallel exploration of two distinct [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]
Figure 6
Figure 6. Figure 6: A qualitative case study demonstrates late-stage verification through parallel thinking. [PITH_FULL_IMAGE:figures/full_fig_p020_6.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning

    cs.CL 2026-07 unverdicted novelty 6.0 of 10

    Scaling zero-RL training to a 1T-parameter model yields competitive math performance and spontaneous cognitive behaviors such as self-verification and structured formatting.

  2. The Scaling Properties of Implicit Deductive Reasoning in Transformers

    cs.AI 2026-05 unverdicted novelty 6.0 of 10

    With shortcut-decorrelated training, bidirectional masking, and a corrective objective, depth scaling makes direct transformer reasoning over Horn clauses match CoT accuracy within the training horizon, but not beyond it.

  3. StatEval: A Comprehensive Benchmark for Large Language Models in Statistics

    cs.CL 2025-10 conditional novelty 6.0 of 10

    StatEval is a new 16,000-question statistics benchmark showing that even strong LLMs score below 60% on research-level statistical proof tasks.

  4. Efficient Reasoning on the Edge

    cs.LG 2026-03 accept novelty 5.5 of 10

    LoRA adapters, budget-forced GRPO, dynamic switching, parallel verification and FPTQuant enable practical chain-of-thought reasoning on quantized Qwen2.5-7B for edge devices.

Reference graph

Works this paper leans on

27 extracted references · 2 canonical work pages · cited by 4 Pith papers

  1. [1]

    Large language monkeys: Scaling inference compute with repeated sampling.arXiv preprint arXiv:2407.21787,

    Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher R´e, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling.arXiv preprint arXiv:2407.21787,

    arXiv

  2. [5]

    Measuring mathematical problem solving with the math dataset.arXiv preprint arXiv:2103.03874,

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset.arXiv preprint arXiv:2103.03874,

    arXiv

  3. [7]

    Divide, Reweight, and Conquer: A Logit Arithmetic Approach for In-Context Learning

    Chengsong Huang, Langlin Huang, and Jiaxin Huang. Divide, reweight, and conquer: A logit arithmetic approach for in-context learning.arXiv preprint arXiv:2410.10074,

    work page Pith review arXiv

  4. [8]

    Efficient test-time scaling via self-calibration.arXiv preprint arXiv:2503.00031, 2025a

    Chengsong Huang, Langlin Huang, Jixuan Leng, Jiacheng Liu, and Jiaxin Huang. Efficient test-time scaling via self-calibration.arXiv preprint arXiv:2503.00031, 2025a. Chengsong Huang, Wenhao Yu, Xiaoyang Wang, Hongming Zhang, Zongxia Li, Ruosen Li, Jiaxin Huang, Haitao Mi, and Dong Yu. R-zero: Self-evolving reasoning llm from zero data.arXiv preprint arXiv...

    arXiv

  5. [10]

    Self-rewarding vision-language model via reasoning decomposition.arXiv preprint arXiv:2508.19652,

    Zongxia Li, Wenhao Yu, Chengsong Huang, Rui Liu, Zhenwen Liang, Fuxiao Liu, Jingxi Che, Dian Yu, Jordan Boyd-Graber, Haitao Mi, et al. Self-rewarding vision-language model via reasoning decomposition.arXiv preprint arXiv:2508.19652,

    arXiv

  6. [11]

    Spiral: Self-play on zero-sum games incentivizes reasoning via multi-agent multi-turn reinforcement learning.arXiv preprint arXiv:2506.24119,

    Bo Liu, Leon Guertler, Simon Yu, Zichen Liu, Penghui Qi, Daniel Balcells, Mickel Liu, Cheston Tan, Weiyan Shi, Min Lin, et al. Spiral: Self-play on zero-sum games incentivizes reasoning via multi-agent multi-turn reinforcement learning.arXiv preprint arXiv:2506.24119,

    arXiv

  7. [12]

    Matthew Macfarlane, Minseon Kim, Nebojsa Jojic, Weijia Xu, Lucas Caccia, Xingdi Yuan, Wanru Zhao, Zhengyan Shi, and Alessandro Sordoni

    Accessed: 2025-07-30. Matthew Macfarlane, Minseon Kim, Nebojsa Jojic, Weijia Xu, Lucas Caccia, Xingdi Yuan, Wanru Zhao, Zhengyan Shi, and Alessandro Sordoni. Instilling parallel reasoning into language models. In2nd AI for Math Workshop @ ICML 2025,

    work page 2025

  8. [13]

    Jiayi Pan, Xiuyu Li, Long Lian, Charlie Snell, Yifei Zhou, Adam Yala, Trevor Darrell, Kurt Keutzer, and Alane Suhr

    URL https://openreview.net/forum?id=a3o4 b3hkwp. Jiayi Pan, Xiuyu Li, Long Lian, Charlie Snell, Yifei Zhou, Adam Yala, Trevor Darrell, Kurt Keutzer, and Alane Suhr. Learning adaptive parallel reasoning with language models.arXiv preprint arXiv:2504.15466,

    arXiv

Show all 27 references
  1. [14]

    Hogwild! inference: Parallel llm generation via concurrent attention.arXiv preprint arXiv:2504.06261,

    Gleb Rodionov, Roman Garipov, Alina Shutova, George Yakushev, Erik Schultheis, Vage Egiazarian, Anton Sinitsin, Denis Kuznedelev, and Dan Alistarh. Hogwild! inference: Parallel llm generation via concurrent attention.arXiv preprint arXiv:2504.06261,

  2. [15]

    Adversarial reasoning at jailbreaking time.arXiv preprint arXiv:2502.01633,

    Mahdi Sabbaghi, Paul Kassianik, George Pappas, Yaron Singer, Amin Karbasi, and Hamed Hassani. Adversarial reasoning at jailbreaking time.arXiv preprint arXiv:2502.01633,

    arXiv

  3. [16]

    Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models.arXiv preprint arXiv:2402.03300,

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models.arXiv preprint arXiv:2402.03300,

    arXiv

  4. [17]

    Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256,

    14 Technical Report Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256,

    arXiv

  5. [18]

    Mobilegui-rl: Advancing mobile gui agent through reinforcement learning in online environment.arXiv preprint arXiv:2507.05720,

    Yucheng Shi, Wenhao Yu, Zaitang Li, Yonglin Wang, Hongming Zhang, Ninghao Liu, Haitao Mi, and Dong Yu. Mobilegui-rl: Advancing mobile gui agent through reinforcement learning in online environment.arXiv preprint arXiv:2507.05720,

    arXiv

  6. [19]

    On the hardness of faithful chain-of-thought reasoning in large language models.arXiv preprint arXiv:2406.10625,

    Sree Harsha Tanneru, Dan Ley, Chirag Agarwal, and Himabindu Lakkaraju. On the hardness of faithful chain-of-thought reasoning in large language models.arXiv preprint arXiv:2406.10625,

    arXiv

  7. [20]

    To code or not to code? adaptive tool integration for math language models via expectation-maximization

    Haozhe Wang, Long Li, Chao Qu, Fengming Zhu, Weidi Xu, Wei Chu, and Fangzhen Lin. To code or not to code? adaptive tool integration for math language models via expectation-maximization. arXiv preprint arXiv:2502.00691, 2025a. Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangz...

    arXiv

  8. [23]

    Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks.arXiv preprint arXiv:2504.05118,

    Yu Yue, Yufeng Yuan, Qiying Yu, Xiaochen Zuo, Ruofei Zhu, Wenyuan Xu, Jiaze Chen, Chengyi Wang, TianTian Fan, Zhengyin Du, et al. Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks.arXiv preprint arXiv:2504.05118,

    arXiv

  9. [24]

    Accessing gpt-4 level mathematical olympiad solutions via monte carlo tree self-refine with llama-3 8b.arXiv preprint arXiv:2406.07394,

    Di Zhang, Xiaoshui Huang, Dongzhan Zhou, Yuqiang Li, and Wanli Ouyang. Accessing gpt-4 level mathematical olympiad solutions via monte carlo tree self-refine with llama-3 8b.arXiv preprint arXiv:2406.07394,

    arXiv

  10. [25]

    Learning to reason via mixture-of-thought for logical reasoning.arXiv preprint arXiv:2505.15817,

    Tong Zheng, Lichang Chen, Simeng Han, R Thomas McCoy, and Heng Huang. Learning to reason via mixture-of-thought for logical reasoning.arXiv preprint arXiv:2505.15817,

    arXiv

  11. [26]

    Dissecting logical reasoning in llms: A fine-grained evaluation and supervision study.arXiv preprint arXiv:2506.04810,

    Yujun Zhou, Jiayi Ye, Zipeng Ling, Yufei Han, Yue Huang, Haomin Zhuang, Zhenwen Liang, Kehan Guo, Taicheng Guo, Xiangqi Wang, et al. Dissecting logical reasoning in llms: A fine-grained evaluation and supervision study.arXiv preprint arXiv:2506.04810,

  12. [27]

    Ttrl: Test-time reinforcement learning.arXiv preprint arXiv:2504.16084,

    15 Technical Report Yuxin Zuo, Kaiyan Zhang, Li Sheng, Shang Qu, Ganqu Cui, Xuekai Zhu, Haozhan Li, Yuchen Zhang, Xinwei Long, Ermo Hua, et al. Ttrl: Test-time reinforcement learning.arXiv preprint arXiv:2504.16084,

    arXiv

  13. [1989]

    Breach in the shield: Unveiling the vulnera- bilities of large language models.arXiv preprint arXiv:2504.03714, 2025a

    Runpeng Dai, Run Yang, Fan Zhou, and Hongtu Zhu. Breach in the shield: Unveiling the vulnera- bilities of large language models.arXiv preprint arXiv:2504.03714, 2025a. Runpeng Dai, Tong Zheng, Run Yang, Kaixian Yu, and Hongtu Zhu. R1-re: Cross-domain relation extraction with r...

    arXiv

  14. [2011]

    Learning to keep a promise: Scal- ing language model decoding parallelism with learned asynchronous decoding.arXiv preprint arXiv:2502.11517,

    Tian Jin, Ellie Y Cheng, Zack Ankner, Nikunj Saunshi, Blake M Elias, Amir Yazdanbakhsh, Jonathan Ragan-Kelley, Suvinay Subramanian, and Michael Carbin. Learning to keep a promise: Scal- ing language model decoding parallelism with learned asynchronous decoding.arXiv preprint a...

    arXiv

  15. [2021]

    Group think: Multiple concurrent reasoning agents collaborating at token level granularity.arXiv preprint arXiv:2505.11107,

    13 Technical Report Chan-Jan Hsu, Davide Buffelli, Jamie McGowan, Feng-Ting Liao, Yi-Chang Chen, Sattar Vakili, and Da-shan Shiu. Group think: Multiple concurrent reasoning agents collaborating at token level granularity.arXiv preprint arXiv:2505.11107,

    arXiv

  16. [2022]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025a

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025a. Xinyu Yang, Yuwei An, Hongyi Liu, Tianqi Chen, and Beidi Chen. Multiverse: Your language...

    arXiv

  17. [2023]

    Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476,

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476,

    arXiv

  18. [2024]

    Aspd: Unlocking adaptive serial-parallel decoding by exploring intrinsic parallelism in llms

    Keyu Chen, Zhifeng Shen, Daohai Yu, Haoqian Wu, Wei Wen, Jianfeng He, Ruizhi Qiao, and Xing Sun. Aspd: Unlocking adaptive serial-parallel decoding by exploring intrinsic parallelism in llms. arXiv preprint arXiv:2508.08895, 2025a. Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jo...

    arXiv

  19. [2025]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948,

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948,

    arXiv

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.