Pith. sign in

REVIEW 4 major objections 5 minor 7 cited by

QwenLong-L1: Towards Long-Context Large Reasoning Models with Reinforcement Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Reinforcement learning with progressive context scaling turns a short-context 32B model into a long-context reasoner that beats several larger proprietary models on document QA.

desk verdict A credible engineering recipe for long-context RL, but the headline comparison is built on a train/eval overlap that no one has ruled out. read the letter →

arxiv 2505.17667 v2 pith:5VBEJZZ4 submitted 2025-05-23 cs.CL

Fanqi Wan , Weizhou Shen , Shengyi Liao , Yingcheng Shi , Chenliang Li , Ziyi Yang , Ji Zhang , Fei Huang
show 2 more authors
Jingren Zhou Ming Yan
This is my paper · ORCID
classification cs.CL
keywords long-contextreasoningreinforcementlearninglargemodelsprogressivecontextscalingdocumentquestionansweringcurriculumRLretrospectivesamplingQwenLong-L1
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Long-context reasoning models are hard to train with reinforcement learning because long inputs shrink output entropy and inflate KL-divergence variance, which makes optimization slow and unstable. This paper argues that both problems are manageable if the context window is scaled gradually rather than opened all at once. The proposed QwenLong-L1 recipe starts from a short-context reasoning model, adds a warm-up supervised fine-tuning stage at 20K tokens, then runs group-relative RL in two length phases, 20K and 60K, while re-sampling the hardest examples from the first phase into the second. On seven long-document question-answering benchmarks, the 32B version reaches an average of 70.7, which the paper reports as above OpenAI-o3-mini and Qwen3-235B-A22B and on par with Claude-3.7-Sonnet-Thinking. If the recipe holds up, it gives open 32B-class models a practical route to deep-research-style tasks that today seem to require much larger or proprietary systems.

What carries the argument

QwenLong-L1 is a progressive context scaling framework for long-context reasoning RL. Its load-bearing mechanism is the curriculum over input length: a warm-up SFT stage at 20K tokens initializes the policy, then group-relative RL runs in two phases with context caps of 20K and 60K, and a difficulty-aware retrospective sampler re-injects the hardest zero-reward examples from the earlier phase into the later one. The reward is a hybrid of rule-based exact-match verification and an LLM-as-a-judge semantic-equivalence check, combined by taking the maximum of the two signals. Because long contexts make a learned value network computationally prohibitive, the framework uses group-relative advantages from GRPO or DAPO instead, with DAPO adding decoupled clip thresholds and dynamic sampling that the paper finds most effective.

What would settle it

Run OpenAI-o3-mini, Qwen3-235B-A22B, and Claude-3.7-Sonnet-Thinking on the same seven benchmarks with the same 120K input allowance QwenLong-L1-32B receives; if o3-mini or Qwen3-235B-A22B then surpasses 70.7, the paper's superiority claim rests on the asymmetric context cap rather than model quality.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that long-context reasoning can be taught to a short-context reasoning model with RL, provided the context length is increased through a curriculum. Starting from R1-Distill-Qwen-14B and R1-Distill-Qwen-32B, QwenLong-L1 gains about 4 and 5 average points over its base checkpoints; the 32B model reaches 70.7, exceeding OpenAI-o3-mini (70.4), Qwen3-235B-A22B (70.6), QwQ-Plus (70.0), and Qwen3-Plus (70.3), and matching Claude-3.7-Sonnet-Thinking (70.7). The paper also reports that warm-up SFT alone yields only small gains, that phased RL keeps entropy higher and KL divergence more stable than single-stage RL, and that RL training progressively increases grounding, subgoal-setting, backtracking, and verification behaviors in the generated reasoning traces.

Load-bearing premise

The load-bearing premise is that capping all proprietary baselines at 50K input tokens while letting QwenLong-L1 read up to 120K is a fair comparison; if that cap hands the model extra evidence the baselines cannot see, the reported wins are not wins.

Editorial extensions

If this is right

  • Open-source 32B-class models can reach the long-document reasoning level of much larger proprietary systems, lowering the compute and API cost barriers for deep-research-style applications.
  • Long-context reasoning RL should be scheduled as a curriculum over input length rather than run in a single stage, because single-stage training shows entropy collapse and KL spikes in the paper's training-dynamics analysis.
  • Warm-up SFT is best treated as an initialization step, not the final training stage, since the ablations show SFT-only models trail RL models by several average points despite generating similar reasoning behaviors.
  • Difficulty-aware retrospective sampling is a cheap mechanism for maintaining exploration, because hard examples from earlier phases keep reward low and policy entropy high during later training.
  • Test-time scaling pays off on this policy: with 16 samples, the 14B model's Pass@2 exceeds both DeepSeek-R1 and OpenAI-o1-preview, indicating the RL-trained policy retains useful exploration diversity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's evaluation gives its own model up to 120K input tokens while capping proprietary baselines at 50K; a natural follow-up is to equalize context length, which would clarify how much of the reported margin is model capability and how much is the asymmetric context cap.
  • The finding that SFT increases reasoning-behavior counts without improving scores suggests those behaviors are correlates of successful training rather than its cause; distilling RL-generated successful traces back into an SFT model would test whether imitation can recover any of the RL gain.
  • The difficulty signal in retrospective sampling is computed once from base-model rewards, so as the policy improves the same examples become easier; re-scoring difficulty periodically during later phases is a direct, testable refinement the paper does not explore.
  • The hybrid reward's max operation is not ablated, so a purely rule-based reward on open-ended DocQA answers is a testable prediction implied by the paper's motivation for combining the two signals.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes QwenLong-L1, a reinforcement learning framework for adapting short-context reasoning models to long-context document question answering. The framework combines warm-up SFT, curriculum-guided phased RL with progressive context scaling, difficulty-aware retrospective sampling, and a hybrid rule/LLM-judge reward. Experiments on seven DocQA benchmarks report that the 32B model achieves an average score of 70.7, outperforming OpenAI-o3-mini and Qwen3-235B-A22B and matching Claude-3.7-Sonnet-Thinking. The paper also presents ablations and analyses of reasoning behavior.

Significance. If the reported results are reliable, the paper would provide a useful recipe for long-context reasoning RL, with a clear formalization of the problem and an open release of code and model weights. The ablations separating the contributions of SFT, phased RL, and retrospective sampling are informative, and the analysis of emergent reasoning behaviors is a nice addition. However, the headline comparative claims rest on an evaluation protocol that is currently not sufficiently rigorous: potential training/evaluation overlap in the Musique benchmark, an asymmetric input-length cap for proprietary baselines, a metric that directly matches the training reward, and no uncertainty estimates. The significance of the contribution hinges on resolving these confounds.

major comments (4)
  1. [Section 3.1 and Section 3.3 (Table 4)] The RL training data includes 200 examples sampled from Musique, while the evaluation set is 200 Musique examples taken from LongBench; the paper does not demonstrate that these two sets are disjoint, or provide any safeguard against overlap. Removing the Musique column from Table 4 changes the average of QwenLong-L1-32B-DAPO from 70.7 to 70.9, while Claude-3.7-Sonnet-Thinking becomes 71.1, OpenAI-o3-mini 71.0, and Qwen3-235B-A22B 71.8. The headline claim that QwenLong-L1 exceeds o3-mini and Qwen3-235B and matches Claude disappears. The authors must either state and empirically verify the disjointness of the 200 RL examples from the 200 LongBench evaluation examples, or re-run the evaluation on a clearly disjoint held-out set.
  2. [Section 3.3 (Configurations)] All proprietary baselines are capped at a maximum input length of 50K tokens, while QwenLong-L1 is evaluated with up to 120K tokens. Since several documents in the benchmarks exceed 50K (DocMath max 176,285; Frames max 117,131; NarrativeQA max 65,357), the baselines are evaluated on truncated inputs, whereas QwenLong-L1 can use the full document. This confounds model quality with input access. Please re-evaluate baselines with their full supported context lengths (or at least the same maximum as QwenLong-L1) and report how the results change.
  3. [Section 3.3 and Section 2.4] The reported metric is max(exact match, LLM-judged accuracy) and is explicitly stated to be 'aligned with the reward function' in Section 2.4. Because the model is trained with the same max(rule, LLM-judge) reward, the evaluation metric is not independent of the training objective. The training judge (Qwen2.5-1.5B) and evaluation judge (DeepSeek-V3) differ, but the max operation counts a correct answer under either criterion, so the model is directly optimized for the reported metric. The paper should also report standard task metrics (e.g., EM and F1 separately) or an evaluation metric not used during training, to permit comparison with published baselines.
  4. [Table 4] All results are point estimates without error bars, confidence intervals, or significance tests. With 200 examples per benchmark, the standard error of a 70% accuracy is about 3.2 percentage points; the differences between QwenLong-L1-32B-DAPO (70.7) and Claude-3.7-Sonnet-Thinking (70.7), OpenAI-o3-mini (70.4), and Qwen3-235B-A22B (70.6) are within one standard error. Please report uncertainty estimates, e.g., via multiple evaluation runs with different sampling temperatures, or bootstrap confidence intervals, and state which differences are statistically meaningful.
minor comments (5)
  1. [Table 2] The prompt template contains the typo 'quesiton' instead of 'question' in the introductory line.
  2. [Section 4.1] The word 'mathcing' should be 'matching'.
  3. [Throughout the manuscript] The PDF appears to contain corrupted character sequences (e.g., '/uni00000027/uni00000052/...') in the author list, after Figure 1, in Section 4.2, and elsewhere; the source file should be recompiled so that these artifacts do not appear in the published version.
  4. [Section 3.1] Please clarify whether the 200 MultiHopRAG and 200 Musique examples are drawn from the training splits or test splits of those datasets, and report the sampling method so readers can assess independence from the evaluation sets.
  5. [Section 3.2 and Table 4] The paper does not report the total number of RL steps or the compute budget; including this information would help readers assess the practical cost of the method.

Circularity Check

1 steps flagged · score 6.0 of 10

The reported benchmark score is explicitly aligned with the RL reward function, so the headline performance claim partly reduces to optimizing the evaluation metric.

  1. fitted input called prediction [Section 3.3 (Evaluation Details), with the reward defined in Section 2.4, Eq. (14)]
    "We report the maximum of exact match and LLM-judged accuracy as the final score, aligned with the reward function in Section 2.4."

    Section 2.4 defines the training reward as r_phi(x,y) = max(r_rule(y), r_LLM(x,y)), and Section 3.3 defines the evaluation score as the maximum of exact match and LLM-judged accuracy, explicitly stating this is 'aligned with' that reward. The exact-match component and the max operation are identical; only the judge model differs (Qwen2.5-1.5B for training, DeepSeek-V3 for evaluation). Therefore the headline 70.7 average, and the claimed margins over o3-mini, Qwen3-235B-A22B, and Claude-3.7-Sonnet-Thinking, are measured on the very function the RL procedure maximizes. The model was fit to maximize this score, so reporting it as an independent capability prediction is partly by construction.

full rationale

The central circularity is the explicit alignment between the evaluation metric and the RL reward. Section 2.4's combined reward is max(exact match, LLM-as-a-judge); Section 3.3's reported score is max(exact match, LLM-judged accuracy), called 'aligned with the reward function.' Since RL directly optimizes this reward, part of the measured improvement on the seven benchmarks is an artifact of optimizing the test metric, not an independent finding. This is partial rather than total circularity because the training and evaluation examples are at least nominally disjoint (DocMath uses a 75/25 split; the other benchmarks are LongBench subsets), and the judge model differs between training and evaluation. Separately, the paper does not demonstrate that the 200 Musique examples in DOCQA-RL-1.6K are disjoint from the 200 LongBench Musique evaluation examples; if overlapping, that would be contamination, but this is an evaluation-hygiene issue rather than a circular derivation and I do not count it as a separate circular step. No load-bearing self-citation chain was found: the framework's ablations are tested against the paper's own baselines, and the method components are not justified by citing the authors' prior work. The asymmetric 50K context cap for proprietary baselines is a fairness concern, not circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new theoretical entities. The framework rests on standard RL objectives and a set of hand-chosen hyperparameters and data-splitting decisions, the most consequential being the 50K context cap for baseline models and the use of an evaluation metric identical in form to the training reward.

free parameters (5)
  • Phase context lengths L1=20K, L2=60K = 20K and 60K tokens
    Curriculum schedule chosen by hand; no sensitivity analysis is provided to show these values are optimal or robust.
  • Proprietary baseline input cap = 50K tokens
    Limits all proprietary models to 50K input while QwenLong-L1 gets 120K; this directly determines whether the reported superiority holds.
  • Number of RL phases K=2 = 2
    Two-phase curriculum; the paper does not test alternative numbers of phases.
  • Retrospective sampling criterion = samples with average accuracy zero
    Difficulty filter chosen by hand; the specific threshold (zero average reward) is arbitrary and not swept.
  • Evaluation metric max(EM, LLM-judged) = maximum of exact match and judge score
    This metric is the same form as the training reward; choosing it for reporting inflates scores relative to either component alone.
assumptions (5)
  • standard math The KL-regularized RL objective (Eq. 1) remains an appropriate objective for long-context reasoning.
    The paper extends the standard objective with a context variable c but does not justify why the same objective transfers to long inputs.
  • domain assumption The reward function max(r_rule, r_LLM) is a faithful proxy for answer correctness.
    Section 2.4 postulates that combining exact matching and a 1.5B-judge covers correct answers; no calibration against human labels is given.
  • domain assumption Progressive context scaling from 20K to 60K transfers to 120K evaluation inputs.
    The paper assumes training at 60K generalizes to 120K at test time, yet the evaluation uses up to 120K inputs (Section 3.3).
  • domain assumption Group-relative advantage (GRPO/DAPO) is stable for long-context outputs.
    The motivation for choosing GRPO over PPO is computational (Section 2.2), not a demonstrated stability guarantee.
  • domain assumption The RL training and evaluation sets are disjoint.
    Section 3.1 samples training examples from DocMath and Musique, which also appear as evaluation benchmarks; the paper does not state that the 25% evaluation subsets are disjoint from the sampled training items.

how reviews work

0 comments
Cite this review

Pith. "Pith review of QwenLong-L1: Towards Long-Context Large Reasoning Models with Reinforcement Learning." pith.science (2026). https://pith.science/paper/5VBEJZZ4

@misc{pith2026250517667,
  author       = {Pith},
  title        = {Pith review of: QwenLong-L1: Towards Long-Context Large Reasoning Models with Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5VBEJZZ4}},
  note         = {Machine review of arXiv:2505.17667}
}
read the original abstract

Recent large reasoning models (LRMs) have demonstrated strong reasoning capabilities through reinforcement learning (RL). These improvements have primarily been observed within the short-context reasoning tasks. In contrast, extending LRMs to effectively process and reason on long-context inputs via RL remains a critical unsolved challenge. To bridge this gap, we first formalize the paradigm of long-context reasoning RL, and identify key challenges in suboptimal training efficiency and unstable optimization process. To address these issues, we propose QwenLong-L1, a framework that adapts short-context LRMs to long-context scenarios via progressive context scaling. Specifically, we utilize a warm-up supervised fine-tuning (SFT) stage to establish a robust initial policy, followed by a curriculum-guided phased RL technique to stabilize the policy evolution, and enhanced with a difficulty-aware retrospective sampling strategy to incentivize the policy exploration. Experiments on seven long-context document question-answering benchmarks demonstrate that QwenLong-L1-32B outperforms flagship LRMs like OpenAI-o3-mini and Qwen3-235B-A22B, achieving performance on par with Claude-3.7-Sonnet-Thinking, demonstrating leading performance among state-of-the-art LRMs. This work advances the development of practical long-context LRMs capable of robust reasoning across information-intensive environments.

Figures

Figures reproduced from arXiv: 2505.17667 by the authors.

Figure 1
Figure 1. Overall results of QWENLONG-L1 across seven long-context reasoning benchmarks. Starting from R1-Distill-Qwen-32B, QWENLONG-L1-32B achieves an average gain of 5.1 points, surpassing OpenAI-o3-mini, Qwen3-235B-A22B, and comparable to Claude-3.7-Sonnet-Thinking. ∗ Corresponding author. Preprint. Work in progress. arXiv:2505.17667v2 [cs.CL] 27 May 2025 [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Comparison of training dynamics between short-context and long-context reasoning RL. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Overview of QWENLONG-L1, which is a novel long-context reasoning RL training frame￾work. The proposed framework integrates group-relative RL algorithms, hybrid reward mechanisms, and progressive context scaling strategies to enable stable adaptation from short-context to long￾context LRMs with robust contextual grounding and multi-step reasoning capabilities. learn effective reasoning patterns essential for long-con… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Pass@K rates of QWENLONG-L1-14B with different sample numbers across all bench￾marks. We show that QWENLONG-L1-14B surpasses DeepSeek-R1 with a small sampling number. 4 Experimental Results 4.1 Main Results [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Ablation studies of progressive context scaling strategy, where “Baseline” refers to the base [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Comparison between differ￾ent models before and after RL, where “Baseline” denotes the base model, “Short SFT” denotes the short-context SFT model, and “Long SFT” denotes the long-context SFT model. In this section, we investigate the questions pertaining to the develo…
Figure 7
Figure 7. Figure 7: The change in reasoning behavior over training steps. We focus on four core reasoning [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Internalized Reasoning for Long-Context Visual Document Understanding

    cs.CV 2026-03 unverdicted novelty 7.0 of 10

    Synthetic page-ranked reasoning traces plus low-strength model merging give a 32B VLM 58.3 on MMLongBenchDoc, beating a 235B teacher while cutting output tokens ~12× versus explicit reasoning.

  2. REFACT: Adaptive Fact Restatement for Compact and Faithful Chain-of-Thought Reasoning

    cs.CL 2026-07 conditional novelty 6.0 of 10

    REFACT teaches LLMs to adaptively cite only the source facts needed during chain-of-thought reasoning, improving faithfulness and shortening reasoning traces.

  3. Copy Less, Ground More: Overcoming Repetitive Copying in Long-Context Reasoning via Evidence-Aware Reinforcement Learning

    cs.CL 2026-07 conditional novelty 6.0 of 10

    GEAR adds a grounding reward for overlap with key evidence and a distractor penalty to the RL accuracy signal, reducing repetitive copying and improving long-context reasoning accuracy by up to 4.6 points over accurac...

  4. ConspirED: A Dataset for Cognitive Traits of Conspiracy Theories and Large Language Model Safety

    cs.CL 2025-08 conditional novelty 6.0 of 10

    A new dataset annotates conspiracy texts with six cognitive traits, and experiments show LLMs reproduce conspiracy reasoning more readily than they deflect it.

  5. MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent

    cs.CL 2025-07 unverdicted novelty 6.0 of 10

    MemAgent uses multi-conversation RL to train a memory agent that reads text in segments and overwrites memory, extrapolating from 8K training to 3.5M token QA with under 5% loss and 95%+ on 512K RULER.

  6. Observation of momentum dependent charge density wave gap in EuTe4

    cond-mat.mes-hall 2025-08 unverdicted novelty 4.0 of 10

    EuTe4 shows a momentum-dependent charge density wave gap at the Fermi level, largest along Gamma-Y and smallest along Gamma-X, plus a low-temperature magnetic phase diagram near TN = 6.9 K.

  7. MA-CBP: A Criminal Behavior Prediction Framework Based on Multi-Agent Asynchronous Collaboration

    cs.CV 2025-08 unverdicted novelty 4.0 of 10

    MA-CBP is a proposed multi-agent AI system that turns live video into text descriptions and summaries and reasons jointly to warn about potential criminal behavior.

Reference graph

Works this paper leans on

59 extracted references · 19 canonical work pages · cited by 7 Pith papers

  1. [1]

    Claude 3.7 sonnet system card, Feburary 2025

    Anthropic. Claude 3.7 sonnet system card, Feburary 2025

    work page 2025

  2. [2]

    LongBench: A bilingual, multitask benchmark for long context understanding

    Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. LongBench: A bilingual, multitask benchmark for long context understanding. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association for C...

    2024

  3. [3]

    Process reinforcement through implicit rewards

    Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, et al. Process reinforcement through implicit rewards. arXiv preprint arXiv:2502.01456, 2025

    arXiv 2025

  4. [4]

    Thinking, fast and slow

    Kahneman Daniel. Thinking, fast and slow. 2017

    2017

  5. [5]

    A dataset of information-seeking questions and answers anchored in research papers

    Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A Smith, and Matt Gardner. A dataset of information-seeking questions and answers anchored in research papers. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4599–4610, 2021

    work page 2021

  6. [6]

    Deepseek-r1-lite-preview is now live: unleashing supercharged reasoning power!, November 2024

    DeepSeek-AI. Deepseek-r1-lite-preview is now live: unleashing supercharged reasoning power!, November 2024

    2024

  7. [7]

    Competitive programming with large reasoning models

    Ahmed El-Kishky, Alexander Wei, Andre Saraiva, Borys Minaev, Daniel Selsam, David Dohan, Francis Song, Hunter Lightman, Ignasi Clavera, Jakub Pachocki, et al. Competitive programming with large reasoning models. arXiv preprint arXiv:2502.06807, 2025

    arXiv 2025

  8. [8]

    Open r1: A fully open reproduction of deepseek-r1, January 2025

    Hugging Face. Open r1: A fully open reproduction of deepseek-r1, January 2025. 20

    work page 2025

Show all 59 references
  1. [9]

    Data engineering for scaling language models to 128k context

    Yao Fu, Rameswar Panda, Xinyao Niu, Xiang Yue, Hannaneh Hajishirzi, Yoon Kim, and Hao Peng. Data engineering for scaling language models to 128k context. In International Conference on Machine Learning, 2024

    2024

  2. [10]

    How to train long-context language models (effectively)

    Tianyu Gao, Alexander Wettig, Howard Yen, and Danqi Chen. How to train long-context language models (effectively). arXiv preprint arXiv:2410.02660, 2024

    2024

  3. [11]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  4. [12]

    Retrieval augmented language model pre-training

    Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. Retrieval augmented language model pre-training. In International conference on machine learning , pages 3929–3938. PMLR, 2020

    2020

  5. [13]

    Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps

    Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6609–6625, 2020

    2020

  6. [14]

    Open-reasoner-zero: An open source approach to scaling reinforcement learning on the base model, 2025

    Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, and Heung-Yeung Shum Xiangyu Zhang. Open-reasoner-zero: An open source approach to scaling reinforcement learning on the base model, 2025

    2025

  7. [15]

    Openai o1 system card

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720, 2024

    2024 arXiv

  8. [16]

    Search-r1: Training llms to reason and leverage search engines with reinforcement learning

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Dong Wang, Hamed Zamani, and Jiawei Han. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516, 2025

    2025 arXiv

  9. [17]

    The narrativeqa reading comprehension challenge

    Tomáš Koˇcisk`y, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette. The narrativeqa reading comprehension challenge. Transac- tions of the Association for Computational Linguistics, 6:317–328, 2018

    2018

  10. [18]

    Fact, fetch, and reason: A unified evaluation of retrieval-augmented generation

    Satyapriya Krishna, Kalpesh Krishna, Anhad Mohananey, Steven Schwarcz, Adam Stambler, Shyam Upadhyay, and Manaal Faruqui. Fact, fetch, and reason: A unified evaluation of retrieval-augmented generation. arXiv preprint arXiv:2409.12941, 2024

    2024 arXiv

  11. [19]

    From quantity to quality: Boosting llm performance with self- guided data selection for instruction tuning

    Ming Li, Yong Zhang, Zhitao Li, Jiuhai Chen, Lichang Chen, Ning Cheng, Jianzong Wang, Tianyi Zhou, and Jing Xiao. From quantity to quality: Boosting llm performance with self- guided data selection for instruction tuning. In Proceedings of the 2024 Conference of the North Amer...

    2024

  12. [20]

    The unlocking spell on base llms: Rethinking alignment via in-context learning

    Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, Nouha Dziri, Melanie Sclar, Khyathi Chandu, Chandra Bhagavatula, and Yejin Choi. The unlocking spell on base llms: Rethinking alignment via in-context learning. In The Twelfth International Conference on Learning Representations, 2023

    2023

  13. [21]

    Deepseek-v3 technical report

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024

    2024 arXiv

  14. [22]

    A comprehensive survey on long context language modeling

    Jiaheng Liu, Dawei Zhu, Zhiqi Bai, Yancheng He, Huanxuan Liao, Haoran Que, Zekun Wang, Chenchen Zhang, Ge Zhang, Jiebin Zhang, et al. A comprehensive survey on long context language modeling. arXiv preprint arXiv:2503.17407, 2025

    2025

  15. [23]

    Understanding r1-zero-like training: A critical perspective

    Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding r1-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783, 2025. 21

    2025 arXiv

  16. [24]

    Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Li Erran Li, Raluca Ada Popa, and Ion Stoica

    Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y . Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl, 2025

    2025

  17. [25]

    s1: Simple test-time scaling

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393, 2025

    2025 arXiv

  18. [26]

    Learning to reason with llms, September 2024

    OpenAI. Learning to reason with llms, September 2024

    2024

  19. [27]

    Introducing deep research, February 2025

    OpenAI. Introducing deep research, February 2025

    2025

  20. [28]

    Openai o3-mini system card, January 2025

    OpenAI. Openai o3-mini system card, January 2025

    2025

  21. [29]

    Tinyzero: Clean, minimal, accessible reproduction of deepseek r1-zero, Janurary 2025

    Jiayi Pan, Junjie Zhang, Xingyao Wang, Lifan Yuan, Hao Peng, and Alane Suhr. Tinyzero: Clean, minimal, accessible reproduction of deepseek r1-zero, Janurary 2025

    2025

  22. [30]

    In-context retrieval-augmented language models

    Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. In-context retrieval-augmented language models. Transactions of the Association for Computational Linguistics, 11:1316–1331, 2023

    2023

  23. [31]

    High- dimensional continuous control using generalized advantage estimation

    John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. High- dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438, 2015

    2015 arXiv

  24. [32]

    Equivalence between policy gradients and soft q-learning

    John Schulman, Xi Chen, and Pieter Abbeel. Equivalence between policy gradients and soft q-learning. arXiv preprint arXiv:1704.06440, 2017

    2017 arXiv

  25. [33]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  26. [34]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  27. [35]

    Defining and characterizing reward gaming

    Joar Skalse, Nikolaus Howe, Dmitrii Krasheninnikov, and David Krueger. Defining and characterizing reward gaming. Advances in Neural Information Processing Systems, 35:9460– 9471, 2022

    2022

  28. [36]

    Multihop-rag: Benchmarking retrieval-augmented generation for multi-hop queries

    Yixuan Tang and Yi Yang. Multihop-rag: Benchmarking retrieval-augmented generation for multi-hop queries. In First Conference on Language Modeling, 2024

    2024

  29. [37]

    Gemini 2.0 flash thinking, December 2024

    Gemini Team. Gemini 2.0 flash thinking, December 2024

    2024

  30. [38]

    Try deep research and our new experimental model in gemini, your ai assistant, December 2024

    Gemini Team. Try deep research and our new experimental model in gemini, your ai assistant, December 2024

    2024

  31. [39]

    Unlocking the potential of reinforcement learning in improving reasoning models, Feburary 2025

    NovaSky Team. Unlocking the potential of reinforcement learning in improving reasoning models, Feburary 2025

    2025

  32. [40]

    Introducing perplexity deep research, February 2025

    Perplexity Team. Introducing perplexity deep research, February 2025

    2025

  33. [41]

    Qwq: Reflect deeply on the boundaries of the unknown, November 2024

    Qwen Team. Qwq: Reflect deeply on the boundaries of the unknown, November 2024

    2024

  34. [42]

    Qwen3: Think deeper, act faster, April 2025

    Qwen Team. Qwen3: Think deeper, act faster, April 2025

    2025

  35. [43]

    Qwq-32b: Embracing the power of reinforcement learning, March 2025

    Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025

    2025

  36. [44]

    Musique: Multihop questions via single-hop question composition

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Musique: Multihop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10:539–554, 2022

    2022

  37. [45]

    Light-r1: Curriculum sft, dpo and rl for long cot from scratch and beyond

    Liang Wen, Yunke Cai, Fenrui Xiao, Xin He, Qi An, Zhenyu Duan, Yimin Du, Junchen Liu, Lifu Tang, Xiaowei Lv, Haosheng Zou, Yongchao Deng, Shousheng Jia, and Xiangzheng Zhang. Light-r1: Curriculum sft, dpo and rl for long cot from scratch and beyond. arXiv preprint arXiv:2503.1...

    2025 arXiv

  38. [46]

    A comparative study on reasoning patterns of openai’s o1 model

    Siwei Wu, Zhongyuan Peng, Xinrun Du, Tuney Zheng, Minghao Liu, Jialong Wu, Jiachen Ma, Yizhi Li, Jian Yang, Wangchunshu Zhou, et al. A comparative study on reasoning patterns of openai’s o1 model. arXiv preprint arXiv:2410.13639, 2024

    arXiv

  39. [47]

    Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning

    Tian Xie, Zitian Gao, Qingnan Ren, Haoming Luo, Yuqian Hong, Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, and Chong Luo. Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning. arXiv preprint arXiv:2502.14768, 2025

    2025 arXiv

  40. [48]

    Effective long-context scaling of foundation models

    Wenhan Xiong, Jingyu Liu, Igor Molybog, Hejia Zhang, Prajjwal Bhargava, Rui Hou, Louis Martin, Rashi Rungta, Karthik Abinav Sankararaman, Barlas Oguz, et al. Effective long-context scaling of foundation models. In Proceedings of the 2024 Conference of the North American Chapte...

    2024

  41. [49]

    Towards large reasoning models: A survey of reinforced reasoning with large language models

    Fengli Xu, Qianyue Hao, Zefang Zong, Jingwei Wang, Yunke Zhang, Jingyi Wang, Xiaochong Lan, Jiahui Gong, Tianjian Ouyang, Fanjin Meng, et al. Towards large reasoning models: A survey of reinforced reasoning with large language models. arXiv preprint arXiv:2501.09686, 2025

    2025 arXiv

  42. [50]

    An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, et al. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.12122, 2024

    2024 arXiv

  43. [51]

    Hotpotqa: A dataset for diverse, explainable multi-hop question answering

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Proces...

    2018

  44. [52]

    React: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations, 2023

    2023

  45. [53]

    Limo: Less is more for reasoning

    Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. Limo: Less is more for reasoning. arXiv preprint arXiv:2502.03387, 2025

    2025 arXiv

  46. [54]

    Dapo: An open-source llm reinforcement learning system at scale

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476, 2025

    2025 arXiv

  47. [55]

    What’s behind ppo’s collapse in long-cot? value optimization holds the secret

    Yufeng Yuan, Yu Yue, Ruofei Zhu, Tiantian Fan, and Lin Yan. What’s behind ppo’s collapse in long-cot? value optimization holds the secret. arXiv preprint arXiv:2503.01491, 2025

    2025 arXiv

  48. [56]

    Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild

    Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild. arXiv preprint arXiv:2503.18892, 2025

    2025 arXiv

  49. [57]

    Docmath-eval: Evaluating math reasoning capabilities of llms in understanding long and specialized documents

    Yilun Zhao, Yitao Long, Hongjun Liu, Ryo Kamoi, Linyong Nan, Lyuhao Chen, Yixin Liu, Xiangru Tang, Rui Zhang, and Arman Cohan. Docmath-eval: Evaluating math reasoning capabilities of llms in understanding long and specialized documents. In Proceedings of the 62nd Annual Meetin...

    2024

  50. [58]

    Judging llm-as-a-judge with mt-bench and chatbot arena

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595–46623, 2023

    2023

  51. [59]

    Lima: Less is more for alignment

    Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 36:55006–55021, 2023. 23

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.