Pith. sign in

REVIEW 3 major objections 5 minor 7 cited by

SFR-DeepResearch: Towards Effective Reinforcement Learning for Autonomously Reasoning Single Agents

T0 review · 3 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read RL training turns a 20B model into a top-scoring deep-research agent

desk verdict A careful RL-for-agents recipe with a genuine length-normalization insight, and a headline HLE number that is plausible but currently unverifiable because training-data overlap with the public eval set is never measured. read the letter →

arxiv 2509.06283 v2 pith:KA476PUX submitted 2025-09-08 cs.AI cs.CL

Xuan-Phi Nguyen , Shrey Pandit , Revanth Gangi Reddy , Austin Xu , Silvio Savarese , Caiming Xiong , Shafiq Joty This is my paper
classification cs.AIcs.CL
keywords reinforcementlearningdeepresearchagentstoolusereasoningmodelslengthnormalizationsynthetictrainingdataHumanity'sLastExamsingle-agentLLM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's central claim is that a single open LLM can be turned into a competitive deep-research agent by running continual reinforcement learning on top of an already reasoning-optimized model, using a minimal tool set and entirely synthetic training data. The authors' best model, SFR-DR-20B, built from gpt-oss-20b, reaches 28.7% on the full text-only Humanity's Last Exam, 82.8 on FRAMES, and 66.0 on GAIA, and on HLE it outperforms the proprietary Deep Research system powered by o3. A second claim is that the RL objective needs a length-normalized advantage: without it, training collapses into repetitive tool calling while accuracy drops. The wider point is that agentic skill can be acquired without multi-agent orchestration or cold-start SFT, while preserving the base model's reasoning ability.

What carries the argument

The load-bearing mechanism is a length-normalized, group-relative advantage in a REINFORCE-style objective: every action step in trajectory i gets advantage (r_i - mean(R)) / (std(R) * T_i), so long trajectories no longer dominate the gradient. It is paired with strategic trajectory filtering that drops invalid rollouts. Around this sits an agentic scaffolding with three minimal tools, a reserved memory buffer with a clean_memory tool that lets the model decide what context to keep, and—for Qwen/QwQ models—a reformulation of the multi-turn tool conversation into one ever-growing contextual user turn. The machinery does two jobs: it keeps long tool-calling trajectories from hijacking training

What would settle it

Search the synthetic training set for near-duplicate or paraphrased HLE, FRAMES, or GAIA questions, then retrain with any leaked items removed and re-run the evaluation; if scores barely move, contamination is not the driver. A complementary check: run SFR-DR-20B on HLE with all web tools disabled—if it still scores near 28.7%, the result likely reflects memorized answers rather than tool-augmented reasoning.

Watch

Extended reading notes

Core claim

The paper claims that an autonomous single-agent deep-research model can be made by continuing RL on a reasoning-optimized LLM instead of cold-starting from a base model. With three minimal tools—search, static page browsing, and stateless Python—and fully synthetic data, SFR-DR-20B scores 28.7% on the full text-only Humanity's Last Exam, 82.8 on FRAMES, and 66.0 on GAIA. The secondary claim is that the RL update must normalize each step's advantage by trajectory length; without it, tool calls explode while HLE accuracy collapses. The paper also finds that recasting multi-turn tool use as a single contextual user turn stabilizes Qwen-family reasoning models.

Load-bearing premise

The load-bearing premise is that the synthetic training questions and the web search environment contain none of the benchmark answers; the paper blocks a few sensitive domains but never measures n-gram or semantic overlap between its LLM-generated training data and the public HLE, FRAMES, and GAIA sets.

Editorial extensions

If this is right

  • Continual RL on reasoning-optimized models can add agentic skills while preserving reasoning; no cold-start SFT from a base model is required.
  • Length-normalizing per-step advantages appears to be a general stabilizer for long-horizon agentic RL, preventing degenerate repetitive tool calls.
  • A single autonomous agent with three primitive tools can match or beat larger multi-agent workflows on deep-research benchmarks.
  • Using a contamination blocklist during evaluation can lower reported scores; numbers from web-agent runs without such precautions are not directly comparable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper never audits its synthetic training data for overlap with HLE, FRAMES, or GAIA; if LLM-generated questions paraphrase public benchmark items, the headline HLE number would be inflated. A near-duplicate audit before and after training is a direct way to test this.
  • The single-turn contextual reformulation works because Qwen-family models are optimized for single-turn reasoning; other reasoning models with similar post-training may benefit from the same trick, which the paper does not generalize.
  • The length-normalized advantage may transfer to other long-horizon agentic tasks such as computer use, where trajectory length also varies wildly; that is untested here.
  • Because the code interpreter is stateless and cannot install packages, success on these benchmarks does not imply success on research tasks that require iterative code state or external libraries; failures there would not refute the core RL claim.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a reinforcement learning recipe for adapting open-weights reasoning LLMs into autonomous single-agent deep research systems. The system uses a minimal tool set (web search, page browsing, stateless Python), a memory-cleaning mechanism, LLM-generated synthetic training data, and a REINFORCE variant with length-normalized advantages and trajectory filtering. The recipe is applied to QwQ-32B, Qwen3-8B, and gpt-oss-20b. The best model, SFR-DR-20B, is reported to achieve 28.7% on the full text-only HLE, 82.8 on FRAMES, and 66.0 on GAIA, surpassing several open single/multi-agent baselines and, on HLE, the proprietary OpenAI Deep Research system. Ablations show that a single-turn reformulated agentic workflow helps Qwen-family models, and that length normalization prevents degenerate tool-calling behavior that otherwise collapses HLE performance.

Significance. If the headline numbers hold under contamination control, this is a significant practical result: a relatively simple, fully synthetic RL pipeline applied to an open reasoning model can reach or exceed proprietary deep-research systems on a demanding benchmark, while preserving the base model's reasoning ability. The paper's strengths include the multi-backbone evaluation, the explicit blocklist during evaluation, and the clear ablation demonstrating that more tool calls do not automatically improve scores. The main weakness is that no code, data, or weights are released, and no training-side contamination analysis is provided, so the central HLE claim cannot currently be verified externally. The result is conditional on that verification.

major comments (3)
  1. [§3.2, §3.3] The contamination blocklist is applied only during rollout/evaluation. The synthetic training data in §3.2 is generated by LLMs and is never tested for overlap with HLE/FRAMES/GAIA. Since HLE is public and gpt-oss-20b has already seen public web data, semantically identical or paraphrased items could appear in the training set; with verifiable ground-truth rewards, this would directly reinforce answer recall. The headline 28.7% HLE (Table 1) could then be inflated by memorization rather than tool-augmented reasoning. Please report n-gram and paraphrase-level overlap between synthetic training questions and the three eval sets, and ideally release data/model so this can be checked independently.
  2. [Table 1, §4.1] All SFR-DR scores are single-run Pass@1 values with no seed variance or confidence intervals, yet the central claim of outperforming OpenAI Deep Research on HLE rests on a 2.1-point gap (28.7 vs 26.6). I ask for multiple evaluation runs (or per-question bootstrap CIs) for at least the headline numbers, and an explicit statement of the exact HLE full-text sample size (the text says 'more than 2100 questions', but HLE subsets are often 500; this should be unambiguous). The mixing of † (reported) and ∗ (re-run with blocklist) baselines also needs a sentence on how contamination precautions affect the comparison direction.
  3. [Eq. (1), §3.3] The normalized advantage A_i,j = (r_i - mean(R))/(std(R)*T_i) divides by the group reward standard deviation. If all rollouts in a group receive identical rewards (e.g., all correct or all incorrect), std(R)=0, making the update undefined. The paper does not state how this case is handled (epsilon, filtering, or modified advantage). Since this is the core training objective, please specify the exact numerical safeguard or exclusion rule used in the reported runs.
minor comments (5)
  1. [§3.1.2] In the template '<user>q; [c1, o1, ..., oo−1]<assistant>', 'oo−1' should be 'o_{i-1}'. Also the abstract renders 'AutonomousSingle-Agent' without a space.
  2. [§2.1] 'in an multi-agent system' should be 'in a multi-agent system'; similar grammatical fixes throughout.
  3. [Figure 2] The caption says 'average training trajectory lengths' but the text discusses tool usage and HLE performance; please clarify what each panel/axis shows and when HLE is measured (during training or after).
  4. [§4.1] The HLE-500 subset is not defined beyond '500 samples'; state which subset or cite the reference that defines it.
  5. [§3.3] The loss is described as clipped-surrogate but the clipping bounds/epsilon are not given; provide the exact PPO/GRPO-style objective for reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the RL objective and benchmark evaluations are externally grounded; self-citations are peripheral.

full rationale

The paper's derivation chain is not circular. The RL objective in Eq. (1) is a REINFORCE-style advantage with length normalization; the reward r_i for short-form QA is defined by a semantic-consistency check against ground-truth answers, and for long-form reports by rubric-based grading. These rewards are not defined in terms of the benchmark scores that the paper claims to predict. The headline results on HLE, FRAMES, and GAIA are measured on public benchmarks after RL training, rather than being fitted or constructed from the training data. The paper's self-citations ([27], [35], [6]) appear only in related-work or context statements, not as load-bearing justification for the training recipe or the reported improvements. The contamination blocklist described in §3.3 and §4 is a disclosed precaution against web-access leakage; while training-side overlap between synthetic data and public benchmarks is a legitimate external-validity concern, it is not a circular derivation—the paper never defines its predictions in terms of the benchmark answers, nor does it rename a fitted parameter as a prediction. No equation in the paper reduces a claimed result to its own input by construction, and no uniqueness claim is imported from the authors' prior work. Therefore no circular steps are identified.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The paper's central result rests mainly on standard RL machinery and on engineering assumptions about reward quality, tool environment, and contamination. The only unverifiable assumptions are the LLM-judge reliability and the absence of benchmark leakage in synthetic data; both are empirical risks rather than mathematical derivation steps.

free parameters (6)
  • memory_buffer_limit Lmem
    Design threshold that triggers clean_memory; not reported; affects trajectory length and final performance.
  • search_top_k = 10
    search_internet returns top-10 results per query; fixed tool behavior selected by authors.
  • rollout_group_size G
    Number of independent rollouts per question used in Eq. 1 normalization; not reported.
  • positive_negative_trajectory_ratio
    Range maintained by dropping over-represented trajectories; values not reported.
  • report_reward_weights
    Weights for factuality, compliance, writing, citation in report reward; values and rubric not given.
  • synthetic_data_mix_and_budget
    Composition and size of synthetic QA/math/code/report datasets; not specified.
assumptions (5)
  • standard math REINFORCE with clipped surrogate and group advantage yields unbiased gradient estimates.
    Invoked in §3.3, Eq. 1; standard RL background.
  • domain assumption LLM-as-judge verifier rewards faithfully measure correctness and report quality.
    Used for all rewards (§3.3); if gameable, RL optimizes a flawed objective.
  • domain assumption The minimal toolset (serper search, static browse, stateless Python) provides enough information to solve tasks.
    Tool outputs are the only external evidence (§3.1); search API quality and page stripping bound performance.
  • ad hoc to paper Omitting previous chain-of-thought tokens for Qwen-family models does not lose information needed later.
    Core design choice in §3.1.2; motivated by observed CoT degradation but not separately ablated.
  • ad hoc to paper Synthetic training data does not meaningfully overlap HLE, FRAMES, or GAIA; domain blocklist suffices.
    Assumed in §3.3 and §4.1; no overlap measurement is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SFR-DeepResearch: Towards Effective Reinforcement Learning for Autonomously Reasoning Single Agents." pith.science (2026). https://pith.science/paper/KA476PUX

@misc{pith2026250906283,
  author       = {Pith},
  title        = {Pith review of: SFR-DeepResearch: Towards Effective Reinforcement Learning for Autonomously Reasoning Single Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KA476PUX}},
  note         = {Machine review of arXiv:2509.06283}
}
read the original abstract

Equipping large language models (LLMs) with complex, interleaved reasoning and tool-use capabilities has become a key focus in agentic AI research, especially with recent advances in reasoning-oriented (``thinking'') models. Such capabilities are key to unlocking a number of important applications. One such application is Deep Research (DR), which requires extensive search and reasoning over many sources. Our work in this paper focuses on the development of native Autonomous Single-Agent models for DR featuring minimal web crawling and Python tool integration. Unlike multi-agent systems, where agents take up pre-defined roles and are told what to do at each step in a static workflow, an autonomous single-agent determines its next action dynamically based on context, without manual directive. While prior work has proposed training recipes for base or instruction-tuned LLMs, we focus on continual reinforcement learning (RL) of reasoning-optimized models to further enhance agentic skills while preserving reasoning ability. Towards this end, we propose a simple RL recipe with entirely synthetic data, which we apply to various open-source LLMs. Our best variant SFR-DR-20B achieves up to 28.7% on Humanity's Last Exam benchmark. In addition, we conduct key analysis experiments to provide more insights into our methodologies.

Figures

Figures reproduced from arXiv: 2509.06283 by the authors.

Figure 1
Figure 1. An example tool calling trajectory by our SFR-DR agentic workflow, catered for QwQ [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Average training trajectory lengths of SFR-DR-8B agents over the course of RL training [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Comparison of (a) average tool usage and (b) average step-level response lengths (tokens) [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Divide and Cooperate: Role-Decomposed Multi-Agent LLM Training with Cross-Agent Learning Signals

    cs.LG 2026-06 unverdicted novelty 6.0 of 10

    DAC decomposes agentic search into cooperative searcher and generator agents with cross-agent signals (abstention reward and hard-positive augmentation), achieving strong QA benchmark performance via LoRA on a shared ...

  2. EvoDS: Self-Evolving Autonomous Data Science Agent with Skill Learning and Context Management

    cs.AI 2026-06 unverdicted novelty 6.0 of 10

    EvoDS adds autonomous skill acquisition via synthesis-validation-reuse and adaptive context compression via learned control within a two-stage multi-agent RL scheme, claiming 28.9% average gains over prior agents on f...

  3. MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling

    cs.CL 2025-11 unverdicted novelty 6.0 of 10

    MiroThinker shows that scaling agent-environment interactions via reinforcement learning lets a 72B open-source model reach up to 81.9% on GAIA and approach commercial performance on research benchmarks.

  4. Lost in the Maze: Overcoming Context Limitations in Long-Horizon Agentic Search

    cs.CL 2025-10 conditional novelty 6.0 of 10

    SLIM separates search and browse tools and summarizes trajectories every 50 turns, beating several open-source agentic search systems on BrowseComp and HLE with fewer tool calls and lower cost.

  5. The Landscape of Agentic Reinforcement Learning for LLMs: A Survey

    cs.AI 2025-09 accept novelty 6.0 of 10

    Survey that defines agentic RL for LLMs via POMDPs, introduces a taxonomy of planning/tool-use/memory/reasoning capabilities and domains, and compiles open environments from over 500 papers.

  6. LiteResearcher: A Scalable Agentic RL Training Framework for Deep Research Agent

    cs.AI 2026-04 unverdicted novelty 5.0 of 10

    LiteResearcher uses a lite virtual world to make agentic RL training scalable and stable, enabling a 4B model to achieve 71.3% on GAIA and 78.0% on Xbench, outperforming larger open-source and commercial systems.

  7. LiteResearcher: A Scalable Agentic RL Training Framework for Deep Research Agent

    cs.AI 2026-04 conditional novelty 5.0 of 10

    Injecting 1% targeted synthetic data into GPT-2's pre-training substantially improves performance on 8 of 9 failing BLiMP grammatical paradigms, indicating data scarcity causes formal linguistic failures.

Reference graph

Works this paper leans on

53 extracted references · 12 canonical work pages · cited by 6 Pith papers

  1. [1]

    gpt-oss-120b & gpt-oss-20b model card.arXiv preprint arXiv:2508.10925, 2025

    Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al. gpt-oss-120b & gpt-oss-20b model card.arXiv preprint arXiv:2508.10925, 2025

    arXiv 2025

  2. [2]

    Back to basics: Revisiting reinforce style optimization for learning from human feedback in llms.arXiv preprint arXiv:2402.14740, 2024

    Arash Ahmadian, Chris Cremer, Matthias Gallé, Marzieh Fadaee, Julia Kreutzer, Olivier Pietquin, Ahmet Üstün, and Sara Hooker. Back to basics: Revisiting reinforce style optimization for learning from human feedback in llms.arXiv preprint arXiv:2402.14740, 2024

    arXiv 2024

  3. [3]

    Open deep search: Democratizing search with open-source reasoning agents.arXiv preprint arXiv:2503.20201, 2025

    Salaheddin Alzubi, Creston Brooks, Purva Chiniya, Edoardo Contente, Chiara von Gerlach, Lucas Irwin, Yihan Jiang, Arda Kaz, Windsor Nguyen, Sewoong Oh, et al. Open deep search: Democratizing search with open-source reasoning agents.arXiv preprint arXiv:2503.20201, 2025

    arXiv 2025

  4. [4]

    A survey on rag with llms.Procedia computer science, 246:3781–3790, 2024

    Muhammad Arslan, Hussam Ghanem, Saba Munawar, and Christophe Cruz. A survey on rag with llms.Procedia computer science, 246:3781–3790, 2024

    2024

  5. [5]

    Mindsearch: Mimicking human minds elicits deep ai searcher.arXiv preprint arXiv:2407.20183, 2024

    Zehui Chen, Kuikun Liu, Qiuchen Wang, Jiangning Liu, Wenwei Zhang, Kai Chen, and Feng Zhao. Mindsearch: Mimicking human minds elicits deep ai searcher.arXiv preprint arXiv:2407.20183, 2024. 10

    arXiv 2024

  6. [6]

    Benchmarking deep search over heterogeneous enterprise data

    Prafulla Kumar Choubey, Xiangyu Peng, Shilpa Bhagavath, Kung-Hsiang Huang, Caiming Xiong, and Chien-Sheng Wu. Benchmarking deep search over heterogeneous enterprise data. arXiv preprint arXiv:2506.23139, 2025

    arXiv 2025

  7. [7]

    Agentic reinforced policy optimization

    Guanting Dong, Hangyu Mao, Kai Ma, Licheng Bao, Yifei Chen, Zhongyuan Wang, Zhongxia Chen, Jiazhen Du, Huiyang Wang, Fuzheng Zhang, et al. Agentic reinforced policy optimization. arXiv preprint arXiv:2507.19849, 2025

    arXiv 2025

  8. [8]

    Retool: Reinforcement learning for strategic tool use in llms.arXiv preprint arXiv:2504.11536, 2025

    Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong. Retool: Reinforcement learning for strategic tool use in llms.arXiv preprint arXiv:2504.11536, 2025

    arXiv 2025

Show all 53 references
  1. [9]

    Try deep research and our new experimental model in gemini, your ai assistant

    Google. Try deep research and our new experimental model in gemini, your ai assistant. 2025. URL https://blog.google/products/gemini/ google-gemini-deep-research/

    2025

  2. [10]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  3. [11]

    Search-time data contamination

    Ziwen Han, Meher Mankikar, Julian Michael, and Zifan Wang. Search-time data contamination. arXiv preprint arXiv:2508.13180, 2025

    2025 arXiv

  4. [12]

    Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps

    Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps. In Donia Scott, Nuria Bel, and Chengqing Zong, editors,Proceedings of the 28th International Conference on Computational Lingui...

    2020 doi

  5. [13]

    Characterizing deep research: A benchmark and formal definition.arXiv preprint arXiv:2508.04183, 2025

    Abhinav Java, Ashmit Khandelwal, Sukruta Midigeshi, Aaron Halfaker, Amit Deshpande, Navin Goyal, Ankur Gupta, Nagarajan Natarajan, and Amit Sharma. Characterizing deep research: A benchmark and formal definition.arXiv preprint arXiv:2508.04183, 2025

    2025 arXiv

  6. [14]

    Search-r1: Training llms to reason and leverage search engines with reinforcement learning.arXiv preprint arXiv:2503.09516, 2025

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-r1: Training llms to reason and leverage search engines with reinforcement learning.arXiv preprint arXiv:2503.09516, 2025

    2025 arXiv

  7. [15]

    Fact, fetch, and reason: A unified evaluation of retrieval-augmented generation.arXiv preprint arXiv:2409.12941, 2024

    Satyapriya Krishna, Kalpesh Krishna, Anhad Mohananey, Steven Schwarcz, Adam Stambler, Shyam Upadhyay, and Manaal Faruqui. Fact, fetch, and reason: A unified evaluation of retrieval-augmented generation.arXiv preprint arXiv:2409.12941, 2024

    2024 arXiv

  8. [16]

    Open deep research github

    LangChain-AI. Open deep research github. 2025. URL https://github.com/ langchain-ai/open_deep_research

    2025

  9. [17]

    Websailor: Navigating super-human reasoning for web agent.arXiv preprint arXiv:2507.02592, 2025

    Kuan Li, Zhongwang Zhang, Huifeng Yin, Liwen Zhang, Litu Ou, Jialong Wu, Wenbiao Yin, Baixuan Li, Zhengwei Tao, Xinyu Wang, et al. Websailor: Navigating super-human reasoning for web agent.arXiv preprint arXiv:2507.02592, 2025

    2025 arXiv

  10. [18]

    Chain-of-agents: End-to-end agent foundation models via multi-agent distillation and agentic rl.arXiv preprint arXiv:2508.13167, 2025

    Weizhen Li, Jianbo Lin, Zhuosong Jiang, Jingyi Cao, Xinpeng Liu, Jiayu Zhang, Zhenqiang Huang, Qianben Chen, Weichen Sun, Qiexiang Wang, et al. Chain-of-agents: End-to-end agent foundation models via multi-agent distillation and agentic rl.arXiv preprint arXiv:2508.13167, 2025

    2025 arXiv

  11. [19]

    Search-o1: Agentic search-enhanced large reasoning models.arXiv preprint arXiv:2501.05366, 2025

    Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. Search-o1: Agentic search-enhanced large reasoning models.arXiv preprint arXiv:2501.05366, 2025

    2025 arXiv

  12. [20]

    Webthinker: Empowering large reasoning models with deep research capability.arXiv preprint arXiv:2504.21776, 2025

    Xiaoxi Li, Jiajie Jin, Guanting Dong, Hongjin Qian, Yutao Zhu, Yongkang Wu, Ji-Rong Wen, and Zhicheng Dou. Webthinker: Empowering large reasoning models with deep research capability.arXiv preprint arXiv:2504.21776, 2025

    2025 arXiv

  13. [21]

    Torl: Scaling tool-integrated rl.arXiv preprint arXiv:2503.23383, 2025

    Xuefeng Li, Haoyang Zou, and Pengfei Liu. Torl: Scaling tool-integrated rl.arXiv preprint arXiv:2503.23383, 2025. 11

    2025 arXiv

  14. [22]

    Openmanus: An open-source framework for building general ai agents, 2025

    Xinbin Liang, Jinyu Xiang, Zhaoyang Yu, Jiayi Zhang, Sirui Hong, Sheng Fan, and Xiao Tang. Openmanus: An open-source framework for building general ai agents, 2025. URL https://doi.org/10.5281/zenodo.15186407

    2025 doi

  15. [23]

    Apigen: Automated pipeline for generating verifiable and diverse function-calling datasets.arXiv preprint arXiv:2406.18518, 2024

    Zuxin Liu, Thai Hoang, Jianguo Zhang, Ming Zhu, Tian Lan, Shirley Kokane, Juntao Tan, Weiran Yao, Zhiwei Liu, Yihao Feng, et al. Apigen: Automated pipeline for generating verifiable and diverse function-calling datasets.arXiv preprint arXiv:2406.18518, 2024

    2024 arXiv

  16. [24]

    Gaia: a benchmark for general ai assistants

    Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. Gaia: a benchmark for general ai assistants. InThe Twelfth International Conference on Learning Representations, 2023

    2023

  17. [25]

    Miromind open deep research

    MiroMind. Miromind open deep research. 2025. URL https://miromind.ai/blog/ miromind-open-deep-research

    2025

  18. [26]

    Kimi-researcher: End-to-end rl training for emerging agentic capabilities

    MoonshotAI. Kimi-researcher: End-to-end rl training for emerging agentic capabilities. 2025. URLhttps://moonshotai.github.io/Kimi-Researcher/

    2025

  19. [27]

    Sfr-rag: Towards contextually faithful llms.arXiv preprint arXiv:2409.09916, 2024

    Xuan-Phi Nguyen, Shrey Pandit, Senthil Purushwalkam, Austin Xu, Hailin Chen, Yifei Ming, Zixuan Ke, Silvio Savarese, Caiming Xong, and Shafiq Joty. Sfr-rag: Towards contextually faithful llms.arXiv preprint arXiv:2409.09916, 2024

    2024 arXiv

  20. [28]

    Gpt-5 system card

    OpenAI. Gpt-5 system card. 2025. URL https://cdn.openai.com/ gpt-5-system-card.pdf

    2025

  21. [29]

    Openai o3 and o4-mini system card

    OpenAI. Openai o3 and o4-mini system card. 2025. URL https:// cdn.openai.com/pdf/2221c875-02dc-4789-800b-e7758f3722c1/ o3-and-o4-mini-system-card.pdf

    2025

  22. [30]

    Deep research system card

    OpenAI. Deep research system card. Technical report, OpenAI, August 2025. URL https: //cdn.openai.com/deep-research-system-card.pdf

    2025

  23. [31]

    Training powerful llm agents with end-to-end reinforcement learning, 2025

    Jie Ouyang, Ruiran Yan, Yucong Luo, Mingyue Cheng, Qi Liu, Zirui Liu, Shuo Yu, and Daoyu Wang. Training powerful llm agents with end-to-end reinforcement learning, 2025. URL https://github.com/0russwest0/Agent-R1

    2025

  24. [32]

    Introducing perplexity deep research

    Perplexity. Introducing perplexity deep research. 2025. URL https://www.perplexity. ai/hub/blog/introducing-perplexitydeep-research

    2025

  25. [33]

    Humanity’s last exam.arXiv preprint arXiv:2501.14249, 2025

    Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, Sean Shi, et al. Humanity’s last exam.arXiv preprint arXiv:2501.14249, 2025

    2025 arXiv

  26. [34]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. arXiv preprint arXiv:2305.18290, 2023

    2023 arXiv

  27. [35]

    Infogent: An agent-based framework for web information aggregation.arXiv preprint arXiv:2410.19054, 2024

    Revanth Gangi Reddy, Sagnik Mukherjee, Jeonghwan Kim, Zhenhailong Wang, Dilek Hakkani- Tur, and Heng Ji. Infogent: An agent-based framework for web information aggregation.arXiv preprint arXiv:2410.19054, 2024

    2024 arXiv

  28. [36]

    Ai agents for computer use: A review of instructionbased computer control, gui automation, and operator assistants.arXiv preprint arXiv:2501.16150, 2025

    Pascal J Sager, Benjamin Meyer, Peng Yan, Rebekka von Wartburg-Kottler, Layan Etaiwi, Aref Enayati, Gabriel Nobel, Ahmed Abdulkadir, Benjamin F Grewe, and Thilo Stadelmann. Ai agents for computer use: A review of instructionbased computer control, gui automation, and operator ...

    2025 arXiv

  29. [37]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  30. [38]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024. 12

    2024 arXiv

  31. [39]

    Webshaper: Agentically data synthesizing via information-seeking formalization.arXiv preprint arXiv:2507.15061, 2025

    Zhengwei Tao, Jialong Wu, Wenbiao Yin, Junkai Zhang, Baixuan Li, Haiyang Shen, Kuan Li, Liwen Zhang, Xinyu Wang, Yong Jiang, et al. Webshaper: Agentically data synthesizing via information-seeking formalization.arXiv preprint arXiv:2507.15061, 2025

    2025 arXiv

  32. [40]

    Kimi k2: Open agentic intelligence.arXiv preprint arXiv:2507.20534, 2025

    Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, et al. Kimi k2: Open agentic intelligence.arXiv preprint arXiv:2507.20534, 2025

    2025 arXiv

  33. [41]

    Qwq-32b: Embracing the power of reinforcement learning, March 2025

    Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025. URL https://qwenlm.github.io/blog/qwq-32b/

    2025

  34. [42]

    Webagent-r1: Training web agents via end-to-end multi-turn reinforcement learning.arXiv preprint arXiv:2505.16421, 2025

    Zhepei Wei, Wenlin Yao, Yao Liu, Weizhi Zhang, Qin Lu, Liang Qiu, Changlong Yu, Puyang Xu, Chao Zhang, Bing Yin, et al. Webagent-r1: Training web agents via end-to-end multi-turn reinforcement learning.arXiv preprint arXiv:2505.16421, 2025

    2025

  35. [43]

    Simple statistical gradient-following algorithms for connectionist reinforcement learning.Machine learning, 8(3):229–256, 1992

    Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning.Machine learning, 8(3):229–256, 1992

    1992

  36. [44]

    A comprehensive survey of deep research: Systems, methodologies, and applications.arXiv preprint arXiv:2506.12594, 2025

    Renjun Xu and Jingwen Peng. A comprehensive survey of deep research: Systems, methodologies, and applications.arXiv preprint arXiv:2506.12594, 2025

    2025 arXiv

  37. [45]

    Simpletir: End-to- end reinforcement learning for multi-turn tool-integrated reasoning

    Zhenghai Xue, Longtao Zheng, Qian Liu, Yingru Li, Zejun Ma, and Bo An. Simpletir: End-to- end reinforcement learning for multi-turn tool-integrated reasoning. https://simpletir. notion.site/report, 2025. Notion Blog

    2025

  38. [46]

    Qwen2.5 technical report

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 arXiv

  39. [47]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    2025 arXiv

  40. [48]

    Cohen, Ruslan Salakhutdinov, and Christopher D

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. HotpotQA: A dataset for diverse, explainable multi-hop question answering. InConference on Empirical Methods in Natural Language Processing (EMNLP), 2018

    2018

  41. [49]

    React: Synergizing reasoning and acting in language models.arXiv preprint arXiv:2210.03629, 2022

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models.arXiv preprint arXiv:2210.03629, 2022

    2022 arXiv

  42. [50]

    Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476, 2025

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476, 2025

    2025 arXiv

  43. [51]

    Agentorchestra: A hierarchical multi-agent framework for general-purpose task solving.arXiv preprint arXiv:2506.12508, 2025

    Wentao Zhang, Ce Cui, Yilei Zhao, Rui Hu, Yang Liu, Yahui Zhou, and Bo An. Agentorchestra: A hierarchical multi-agent framework for general-purpose task solving.arXiv preprint arXiv:2506.12508, 2025

    2025 arXiv

  44. [52]

    Group sequence policy optimization.arXiv preprint arXiv:2507.18071, 2025

    Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, et al. Group sequence policy optimization.arXiv preprint arXiv:2507.18071, 2025

    2025 arXiv

  45. [53]

    Sglang: Efficient execution of structured language model programs.Advances in neural information processing systems, 37:62557–62583, 2024

    Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Livia Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E Gonzalez, et al. Sglang: Efficient execution of structured language model programs.Advances in neural information processing systems, 37:...

    2024

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.