Pith. sign in

REVIEW 5 major objections 4 minor 4 cited by

MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use

T0 review · 5 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 32B model trained with a simulated user beats far larger rivals

desk verdict MUA-RL is a genuinely new RL-for-agentic-tool-use recipe, but its headline gains are measured against GPT-4-family user simulators, so transfer to real users is unproven. read the letter →

arxiv 2508.18669 v1 pith:GEFG2KKU submitted 2025-08-26 cs.AI

Weikang Zhao , Xili Wang , Chengdi Ma , Lingbin Kong , Zhaohua Yang , Mingxiang Tuo , Xiaowei Shi , Yitao Zhai
show 1 more author
Xunliang Cai
This is my paper · ORCID
classification cs.AI
keywords reinforcementlearningLLMagentstoolusemulti-turninteractionsimulateduserGRPOfunctioncallingagenticAI
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MUA-RL claims that putting an LLM-simulated user inside the reinforcement-learning loop, rather than relying on fixed pre-scripted queries, lets a model learn to converse and call tools in the same dynamic way real users demand. The paper reports that a 32B-parameter model trained this way matches or outperforms much larger open models such as DeepSeek-V3-0324 and Qwen3-235B-A22B across four multi-turn tool-use benchmarks. The key move is to reward only final task completion, not intermediate tool-call formats, so the agent freely explores how to clarify, confirm, and correct itself while using real database tools. If the claim holds, it means the missing ingredient in prior agentic RL was not a better reward function or more compute, but a living conversation partner during training.

What carries the argument

The central mechanism is the multi-turn user-interacting rollout: during each RL step, the policy samples a conversation with an LLM-simulated user, interleaving text clarification, tool invocation, and real database responses, while only the final binary task-completion outcome feeds a group-relative advantage signal (GRPO). The simulated user is what makes the optimization genuinely interactive, allowing the agent to learn when to ask for information, when to confirm changes, and when to call tools, rather than following a static script.

What would settle it

Run MUA-RL with a differently constructed user simulator (e.g., one that never asks follow-up questions, or one that randomly alters requests mid-task) and measure whether the benchmark gains disappear or reverse; if they do, the reported improvements are tied to the specific simulator rather than to dynamic user interaction in general. A stronger version would evaluate the trained agents with real human users on the same retail and airline tasks.

Watch

Extended reading notes

Core claim

MUA-RL is a reinforcement-learning framework that, for the first time in agentic tool use, integrates an LLM-simulated user into the RL rollout loop. During training, the policy model converses with a GPT-4o-simulated user, invokes tools against a real operational database, and receives a reward of 1 only if the entire task is ultimately completed, regardless of the exact dialogue path or tool sequence. The paper shows that this dynamic interaction, combined with a lightweight supervised cold-start phase and GRPO, produces models that generalize across benchmarks they were not trained on: MUA-RL-32B achieves 67.3 on TAU2 Retail, 45.4 on TAU2 Airline, 28.3 on TAU2 Telecom, 28.4 on BFCL-V3 Mul

Load-bearing premise

The whole approach depends on GPT-4o, used as the simulated user during training, behaving enough like a real human user—including asking follow-ups, changing requests, and confirming actions—that a policy trained against it will also succeed with the evaluation simulators and with actual users.

Editorial extensions

If this is right

  • A 32B model trained with MUA-RL can match or exceed the non-thinking performance of models several times larger, suggesting that interaction strategy matters more than raw parameter count for agentic tool use.
  • Both the cold-start phase and the RL phase are necessary; ablations removing either one degrade performance, so the method is a pipeline rather than a single trick.
  • Training dynamics show gains come from more structured multi-turn interactions (about 21-23 turns), not from longer responses, so the improvement mechanism differs from test-time scaling in reasoning models.
  • The model learns to reduce reliance on auxiliary tools like Calculate, Think, and Transfer to Human Agent, implying RL can prune unnecessary tool calls and produce more efficient policies.
  • The framework scales across 8B, 14B, and 32B backbones, with larger models showing smoother training and more consistent gains, suggesting the approach is not tied to one model size.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The strongest untested implication is that the simulated user transfers to real humans: if a human study shows similar gains, then simulator fidelity is not the bottleneck; if not, the benchmark results may partly reflect simulator-specific dialogue patterns.
  • A cheap and concrete extension would be to train with a deliberately weaker or narrower user simulator and measure how performance degrades; this would isolate how much of the benefit comes from user diversity versus user quality.
  • The binary-reward design implies the agent learns clarification behavior implicitly, because asking the right questions is the only way to reach the terminal success state; this suggests a general recipe for teaching agents soft skills without hand-crafting rewards for them.
  • The same framework should be testable in dual-control environments beyond telecom, where both the user and the agent can call tools, which is where the paper reports its strongest relative gains.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes MUA-RL, a multi-turn user-interacting agent reinforcement learning framework for tool use. Starting from Qwen3-8B/14B/32B non-thinking models, the authors first perform a cold-start SFT on synthetic tool-use trajectories, then run GRPO with a GPT-4o-simulated user during rollouts and real-time tool execution. Rewards are terminal and binary (task completion only). Evaluations on TAU1-Bench, TAU2-Bench, BFCL-V3 Multi Turn, and ACEBench Agent report improvements over the corresponding base and cold-start models, and the abstract claims the 32B model outperforms or matches much larger open models such as DeepSeek-V3-0324 and Qwen3-235B-A22B. The paper also includes training-dynamics analysis, an ablation study, and plans to release code, data, and model weights.

Significance. If the claims hold, MUA-RL is a practical recipe: injecting an LLM-simulated user into GRPO rollouts with real tool execution improves tool-use agents across three model scales, and the controlled comparison against base/cold-start models is informative. The planned open-source release of code, data, and weights, plus the use of official evaluation harnesses, are clear strengths. However, the headline claim of outperforming larger open models is currently oversold: the abstract's Telecom number is below the stated baseline, the TAU1 evaluation may use training tasks, BFCL gains are not monotone across scales, and all user-interaction evidence comes from the GPT-4 family of simulators. The contribution is therefore plausible but not yet established at the level claimed.

major comments (5)
  1. [§4.1, Table 1] RL training uses "115 retail and 50 airline datasets from TAU1-Bench," and Table 1 reports TAU1-Bench results, but the paper never states that the evaluation tasks are disjoint from the training tasks. If the same tasks are used for GRPO training and evaluation, the TAU1 Retail/Airline comparisons against baselines that never trained on those tasks are not apples-to-apples. Please evaluate on a held-out TAU1 split, or remove/adjust the TAU1 comparisons, and state the split explicitly.
  2. [Abstract, Table 1] The abstract claims MUA-RL-32B achieves 28.3 on TAU2 Telecom, "outperforming or matching" DeepSeek-V3-0324, but Table 1 lists DeepSeek-V3-0324 at 32.9. The 32B model is 4.6 points lower. The claim is only true for MUA-RL-14B (33.4), not for the 32B model highlighted in the abstract. This overstatement must be corrected.
  3. [§4.2.2, Table 2] The text says the MUA-RL series show "steady performance gains across all model scales" on BFCL-V3 Multi Turn, but MUA-RL-8B's Overall Acc is 14.6 versus 15.4 for Qwen3-8B Cold-start. On BFCL Base, MUA-RL-8B (21.0) also drops below cold-start (24.0). The claim of monotone gains is contradicted; the scale-dependence should be qualified.
  4. [§3.3.2, §4.1] The trained policy interacts with GPT-4o-2024-11-20 during RL and with GPT-4.1 during TAU/ACEBench evaluation; no non-GPT-4, out-of-family user simulator or human-user evaluation is provided. The paper also gives no distributional analysis of the simulated user (clarification requests, refusals, mid-task changes of intent). Since the reward is terminal success under this simulator, the benchmark gains could reflect adaptation to GPT-4-family dialogue patterns rather than robust multi-turn user interaction. Please narrow the claims or add evidence of transfer.
  5. [§4.1] Quantitative comparisons lack uncertainty estimates. Section 4.1 says four repeated tests were averaged, but no standard deviations, confidence intervals, or significance tests are reported. Many headline comparisons are small (e.g., TAU2 Retail 67.3 vs 64.7 for DeepSeek-V3-0324; ACEBench 82.5 vs 74.2) and some test sets are small (50 tasks for airline, 50 for ACEBench). Please report per-run variance or otherwise justify that the differences are not noise.
minor comments (4)
  1. [Keywords, Eq. (1)] The keywords contain the typo "Reinforcement Learing." Equation (1) has stray LaTeX "| {z }" markup and is not readable as typeset.
  2. [Table 3] The parenthetical notation in Table 3 should be defined directly in the caption. The row labels "+ MUA-RLw/oRL" and "+ MUA-RLw/ocold-start" are confusing because only the latter is an actual RL variant in the usual sense; the former is the cold-start model.
  3. [§4.1] The text says evaluations are "deterministic" with temperature 0.0 but then averages four repeated tests "to improve confidence." If decoding is deterministic and the user simulator is deterministic, repeated runs add no information; if the user or environment is stochastic, the evaluation is not fully deterministic. Please clarify.
  4. [Conclusion] The conclusion says the framework enables models to handle "dynamic, user-driven interactions," but the paper contains no discussion of the simulator-transfer limitation. A limitations paragraph explicitly noting that all user interactions are LLM-simulated would help readers calibrate the claims.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: reported benchmark scores are external evaluations, not derivations from or refits of the training objective.

full rationale

The paper's central claim is empirical: adding LLM-simulated users to GRPO rollouts improves multi-turn tool-use benchmark scores. The headline numbers (TAU2 Retail/Airline/Telecom, BFCL-V3 Multi Turn, ACEBench Agent) come from external benchmarks evaluated with official code and with user simulators (GPT-4.1) that are not identical to the GPT-4o-2024-11-20 simulator used in RL training. No parameter is fitted to these evaluation benchmarks, and no benchmark score is derived from an equation whose inputs include that same score. The training reward r=1 for ultimate task completion is the standard RL objective, not a hidden refit of the evaluation metric. The only overlap is that TAU1-Bench tasks are used for RL rollouts and also reported in Table 1, but the abstract's headline results are on TAU2 and other benchmarks, and TAU2 modifies tool sets and policies. There are no load-bearing self-citations or imported uniqueness theorems; citations to GRPO, TAU-Bench, BFCL, and ACEBench are external. The legitimate concern about GPT-4o-to-GPT-4.1 simulator similarity and unknown transfer to real users is a validity/generalization risk, not circularity, because the benchmark scores are not definitionally equal to the training reward.

Assumptions & free parameters 10 free parameters · 5 assumptions · 0 invented entities

The framework rests on standard RL machinery plus domain assumptions about user simulation and reward design. No new physical or mathematical entities are postulated. The hand-chosen hyperparameters are not tuned on evaluation benchmarks, but no sensitivity analysis is reported.

free parameters (10)
  • KL coefficient beta = 0.001
    Set by hand for GRPO; no sensitivity analysis is reported.
  • Rollout number G = 8
    Number of responses sampled per query in GRPO; chosen by the authors.
  • RL batch size = 32
    Training batch size for 25 epochs.
  • RL epochs = 25
    Training length; no early stopping or sensitivity analysis.
  • Max interaction turns = 30
    Upper bound on rollout turns to limit compute.
  • Rollout temperature = 1.0
    Sampling temperature for the agent during RL rollouts.
  • Sequence length limit = 32768 tokens
    Context limit during RL training.
  • Cold-start data size = ~2000 trajectories
    Number of synthetic trajectories used in the SFT cold-start stage.
  • Cold-start epochs and batch size = 2 epochs, batch 128
    SFT hyperparameters for the cold-start stage.
  • User simulator model = GPT-4o-2024-11-20
    The LLM that plays the user during RL; defines the training dialogue distribution.
assumptions (5)
  • standard math GRPO advantage estimation is valid for this policy optimization
    Adopted from DeepSeekMath with Eq. 2-3; relies on standard RL assumptions.
  • domain assumption LLM-simulated user responses are a faithful proxy for real user behavior
    Sections 3.3.2 and 4.1; the central premise of the method.
  • domain assumption Binary task-completion reward is sufficient to shape multi-turn tool use
    Section 3.3.3; the reward simplification dropped dialogue content requirements.
  • domain assumption TAU1-Bench databases and checkers provide a realistic training environment
    Section 4.1; RL is trained on TAU1 retail and airline tasks with real databases.
  • domain assumption Evaluation benchmarks measure the capability of interest
    Appendix C; TAU2, BFCL-V3 Multi Turn, and ACEBench Agent are used as proxies for real multi-turn tool use.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use." pith.science (2026). https://pith.science/paper/GEFG2KKU

@misc{pith2026250818669,
  author       = {Pith},
  title        = {Pith review of: MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GEFG2KKU}},
  note         = {Machine review of arXiv:2508.18669}
}
read the original abstract

With the recent rapid advancement of Agentic Intelligence, agentic tool use in LLMs has become increasingly important. During multi-turn interactions between agents and users, the dynamic, uncertain, and stochastic nature of user demands poses significant challenges to the agent's tool invocation capabilities. Agents are no longer expected to simply call tools to deliver a result; rather, they must iteratively refine their understanding of user needs through communication while simultaneously invoking tools to resolve user queries. Existing reinforcement learning (RL) approaches for tool use lack the integration of genuinely dynamic users during the RL training process. To bridge this gap, we introduce MUA-RL (Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use), a novel reinforcement learning framework that, for the first time in the field of agentic tool use, integrates LLM-simulated users into the reinforcement learning loop. MUA-RL aims to enable autonomous learning of models to communicate with users efficiently and use various tools to solve practical problems in dynamic multi-turn interactions. Evaluations are done on several multi-turn tool-using benchmarks (see Figure 1). Specifically, MUA-RL-32B achieves 67.3 on TAU2 Retail, 45.4 on TAU2 Airline, 28.3 on TAU2 Telecom, 28.4 on BFCL-V3 Multi Turn, and 82.5 on ACEBench Agent -- outperforming or matching the performance of larger open-source models such as DeepSeek-V3-0324 and Qwen3-235B-A22B in non-thinking settings.

Figures

Figures reproduced from arXiv: 2508.18669 by the authors.

Figure 1
Figure 1. MUA-RL main results. * Weikang Zhao and Xili Wang contributed equally to this work. † Corresponding authors. arXiv:2508.18669v1 [cs.AI] 26 Aug 2025 [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. A trajectory example in multi-turn tool-using tasks. Here, the agent invokes tools to interact with the database, [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Agentic data synthesis pipeline. (a) Tool execution results simulated by LLMs. (b) Tool execution results [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Three different kinds of the rollout processes. (a) The text-based rollout. (b) The multi-step rollout with tool [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Learning curves of MUA-RL series during RL training. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Trend in the average of general-purpose tools’ invocation times during RL training. [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. CustomerSim: Benchmarking and Aligning Multimodal Language Models as Retail User Simulators

    cs.CL 2026-05 unverdicted novelty 7.0 of 10

    SalesSim benchmarks MLLMs as retail user simulators, finds gaps in persona adherence and over-persuasion, and introduces UserGRPO RL to raise decision alignment by 13.8%.

  2. CurateEvo: Data-Curation Evolving for Agentic Post-Training

    cs.CL 2026-07 conditional novelty 6.0 of 10

    CurateEvo evolves executable data-curation code using failed agent trajectories, improving post-training performance by 3.2 and 2.7 points over baselines on labeled and wild data respectively.

  3. Experience-Evolving Multi-Turn Tool-Use Agent with Hybrid Episodic-Procedural Memory

    cs.LG 2025-12 conditional novelty 6.0 of 10

    An LLM agent that builds a tool-transition graph with state summaries from past experience improves tool selection and RL exploration by large margins on multi-turn benchmarks.

  4. AgentOmnia: Scaling Agentic Models for Full-Scenario Applications

    cs.AI 2026-07 conditional novelty 5.0 of 10

    Post-training an open Qwen3-30B agent on 52,361 verifiable tasks in 5,018 synthesized stateful environments lifts its average across four agent benchmarks from 22.9% to 41.7%.

Reference graph

Works this paper leans on

47 extracted references · 11 canonical work pages · cited by 4 Pith papers

  1. [1]

    Kimi k2: Open agentic intelligence

    Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, et al. Kimi k2: Open agentic intelligence. arXiv preprint arXiv:2507.20534, 2025

    arXiv 2025

  2. [2]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. 11

    arXiv 2024

  3. [3]

    Retool: Reinforcement learning for strategic tool use in llms.arXiv preprint arXiv:2504.11536, 2025

    Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong. Retool: Reinforcement learning for strategic tool use in llms.arXiv preprint arXiv:2504.11536, 2025

    arXiv 2025

  4. [4]

    Gonzalez, and Ion Stoica

    Shiyi Cao, Sumanth Hegde, Dacheng Li, Tyler Griggs, Shu Liu, Eric Tang, Jiayi Pan, Xingyao Wang, Akshay Malik, Graham Neubig, Kourosh Hakhamaneshi, Richard Liaw, Philipp Moritz, Matei Zaharia, Joseph E. Gonzalez, and Ion Stoica. Skyrl-v0: Train real-world long-horizon agents via reinforcement learning, 2025

    2025

  5. [5]

    Ragen: Understanding self-evolution in llm agents via multi-turn reinforcement learning

    Zihan Wang, Kangrui Wang, Qineng Wang, Pingyue Zhang, Linjie Li, Zhengyuan Yang, Xing Jin, Kefan Yu, Minh Nhat Nguyen, Licheng Liu, et al. Ragen: Understanding self-evolution in llm agents via multi-turn reinforcement learning. arXiv preprint arXiv:2504.20073, 2025

    arXiv 2025

  6. [6]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    arXiv 2017

  7. [7]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor

    Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning , pages 1861–1870. Pmlr, 2018

    work page 2018

  8. [8]

    Buy 4 reinforce samples, get a baseline for free! Learn- ing,Learning, Mar 2019

    Wouter Kool, Herkevan Hoof, and Max Welling. Buy 4 reinforce samples, get a baseline for free! Learn- ing,Learning, Mar 2019

    work page 2019

Show all 47 references
  1. [9]

    Dapo: An open-source llm reinforcement learning system at scale

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476, 2025

    2025 arXiv

  2. [10]

    Star: Bootstrapping reasoning with reasoning

    Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. Star: Bootstrapping reasoning with reasoning. Advances in Neural Information Processing Systems , 35:15476–15488, 2022

    2022

  3. [11]

    Reasoning with language model is planning with world model

    Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu. Reasoning with language model is planning with world model. arXiv preprint arXiv:2305.14992, 2023

    2023 arXiv

  4. [12]

    Search-r1: Training llms to reason and leverage search engines with reinforcement learning

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516, 2025

    2025 arXiv

  5. [13]

    Code-r1: Reproducing r1 for code with reliable rewards

    Jiawei Liu and Lingming Zhang. Code-r1: Reproducing r1 for code with reliable rewards. arXiv preprint arXiv:2503.18470, 3, 2025

    2025

  6. [14]

    Vlm-r1: A stable and generalizable r1-style large vision-language model

    Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, et al. Vlm-r1: A stable and generalizable r1-style large vision-language model. arXiv preprint arXiv:2504.07615, 2025

    2025 arXiv

  7. [15]

    Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks

    Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W Cohen. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. arXiv preprint arXiv:2211.12588, 2022

    2022 arXiv

  8. [16]

    Instructerc: Reforming emotion recognition in conversation with a retrieval multi-task llms framework

    Shanglin Lei, Guanting Dong, Xiaoping Wang, Keheng Wang, and Sirui Wang. Instructerc: Reforming emotion recognition in conversation with a retrieval multi-task llms framework. CoRR, 2023

    2023

  9. [17]

    Hammer: Robust function-calling for on-device language models via function masking

    Qiqiang Lin, Muning Wen, Qiuying Peng, Guanyu Nie, Junwei Liao, Jun Wang, Xiaoyun Mo, Jiamu Zhou, Cheng Cheng, Yin Zhao, et al. Hammer: Robust function-calling for on-device language models via function masking. arXiv preprint arXiv:2410.04587, 2024

    2024

  10. [18]

    xlam: A family of large action models to empower ai agent systems

    Jianguo Zhang, Tian Lan, Ming Zhu, Zuxin Liu, Thai Quoc Hoang, Shirley Kokane, Weiran Yao, Juntao Tan, Akshara Prabhakar, Haolin Chen, et al. xlam: A family of large action models to empower ai agent systems. In Proceedings of the 2025 Conference of the Nations of the Americas...

    2025

  11. [19]

    Can a single model master both multi-turn conversations and tool use? coalm: A unified conversational agentic language model

    Emre Can Acikgoz, Jeremiah Greer, Akul Datta, Ze Yang, William Zeng, Oussama Elachqar, Emmanouil Koukoumidis, Dilek Hakkani-Tür, and Gokhan Tur. Can a single model master both multi-turn conversations and tool use? coalm: A unified conversational agentic language model. arXiv ...

    2025 arXiv

  12. [20]

    R1-searcher: Incentivizing the search capability in llms via reinforcement learning

    Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji- Rong Wen. R1-searcher: Incentivizing the search capability in llms via reinforcement learning. arXiv preprint arXiv:2503.05592, 2025

    2025 arXiv

  13. [21]

    Zerosearch: Incentivize the search capability of llms without searching

    Hao Sun, Zile Qiao, Jiayan Guo, Xuanbo Fan, Yingyan Hou, Yong Jiang, Pengjun Xie, Yan Zhang, Fei Huang, and Jingren Zhou. Zerosearch: Incentivize the search capability of llms without searching. arXiv preprint arXiv:2505.04588, 2025

    2025 arXiv

  14. [22]

    Torl: Scaling tool-integrated rl

    Xuefeng Li, Haoyang Zou, and Pengfei Liu. Torl: Scaling tool-integrated rl. arXiv preprint arXiv:2503.23383, 2025. 12

    2025 arXiv

  15. [23]

    Agentic reasoning and tool integration for llms via reinforcement learning

    Joykirat Singh, Raghav Magazine, Yash Pandya, and Akshay Nambi. Agentic reasoning and tool integration for llms via reinforcement learning. arXiv preprint arXiv:2505.01441, 2025

    2025 arXiv

  16. [24]

    Agentinstruct: Toward generative teaching with agentic flows

    Arindam Mitra, Luciano Del Corro, Guoqing Zheng, Shweti Mahajan, Dany Rouhana, Andres Codas, Yadong Lu, Wei-ge Chen, Olga Vrousgos, Corby Rosset, et al. Agentinstruct: Toward generative teaching with agentic flows. arXiv preprint arXiv:2407.03502, 2024

    2024 arXiv

  17. [25]

    Stabletoolbench: Towards stable large-scale benchmarking on tool learning of large language models

    Zhicheng Guo, Sijie Cheng, Hao Wang, Shihao Liang, Yujia Qin, Peng Li, Zhiyuan Liu, Maosong Sun, and Yang Liu. Stabletoolbench: Towards stable large-scale benchmarking on tool learning of large language models. arXiv preprint arXiv:2403.07714, 2024

    2024 arXiv

  18. [26]

    Model context protocol (mcp): Landscape, security threats, and future research directions

    Xinyi Hou, Yanjie Zhao, Shenao Wang, and Haoyu Wang. Model context protocol (mcp): Landscape, security threats, and future research directions. arXiv preprint arXiv:2503.23278, 2025

    2025 arXiv

  19. [27]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  20. [28]

    ByteDance Seed, Jiaze Chen, Tiantian Fan, Xin Liu, Lingjun Liu, Zhiqi Lin, Mingxuan Wang, Chengyi Wang, Xiangpeng Wei, Wenyuan Xu, et al. Seed1. 5-thinking: Advancing superb reasoning models with reinforcement learning. arXiv preprint arXiv:2504.13914, 2025

    2025

  21. [29]

    Learning to reason with search for llms via reinforcement learning

    Mingyang Chen, Tianpeng Li, Haoze Sun, Yijie Zhou, Chenzheng Zhu, Haofen Wang, Jeff Z Pan, Wen Zhang, Huajun Chen, Fan Yang, et al. Learning to reason with search for llms via reinforcement learning. arXiv preprint arXiv:2503.19470, 2025

    2025 arXiv

  22. [30]

    Webthinker: Empowering large reasoning models with deep research capability

    Xiaoxi Li, Jiajie Jin, Guanting Dong, Hongjin Qian, Yutao Zhu, Yongkang Wu, Ji-Rong Wen, and Zhicheng Dou. Webthinker: Empowering large reasoning models with deep research capability. arXiv preprint arXiv:2504.21776, 2025

    2025 arXiv

  23. [31]

    Plm-based world models for text-based games

    Minsoo Kim, Yeonjoon Jung, Dohyeon Lee, and Seung-won Hwang. Plm-based world models for text-based games. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages 1324–1341, 2022

    2022

  24. [32]

    Generative agents: Interactive simulacra of human behavior

    Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology , pages 1–22, 2023

    2023

  25. [33]

    Toolrl: Reward is all tool learning needs

    Cheng Qian, Emre Can Acikgoz, Qi He, Hongru Wang, Xiusi Chen, Dilek Hakkani-Tür, Gokhan Tur, and Heng Ji. Toolrl: Reward is all tool learning needs. arXiv preprint arXiv:2504.13958, 2025

    2025 arXiv

  26. [34]

    Nemotron-research-tool-n1: Tool-using language models with reinforced reasoning

    Shaokun Zhang, Yi Dong, Jieyu Zhang, Jan Kautz, Bryan Catanzaro, Andrew Tao, Qingyun Wu, Zhiding Yu, and Guilin Liu. Nemotron-research-tool-n1: Tool-using language models with reinforced reasoning. arXiv preprint arXiv:2505.00024, 2025

    2025 arXiv

  27. [35]

    Reinforcing multi-turn reasoning in llm agents via turn-level credit assignment

    Siliang Zeng, Quan Wei, William Brown, Oana Frunza, Yuriy Nevmyvaka, Yang Katie Zhao, and Mingyi Hong. Reinforcing multi-turn reasoning in llm agents via turn-level credit assignment. In ICML 2025 Workshop on Computer Use Agents, 2025

    2025

  28. [36]

    Qwen3 technical report

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025

    2025 arXiv

  29. [37]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  30. [38]

    Hybridflow: A flexible and efficient rlhf framework

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256 , 2024

    2024 arXiv

  31. [39]

    tau-bench: A benchmark for tool-agent-user interaction in real-world domains

    Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. tau-bench: A benchmark for tool-agent-user interaction in real-world domains. arXiv preprint arXiv:2406.12045, 2024

    2024 arXiv

  32. [40]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  33. [41]

    tau2-bench: Evaluating conversa- tional agents in a dual-control environment

    Victor Barres, Honghua Dong, Soham Ray, Xujie Si, and Karthik Narasimhan. tau2-bench: Evaluating conversa- tional agents in a dual-control environment. arXiv preprint arXiv:2506.07982, 2025

    2025 arXiv

  34. [42]

    The berkeley function calling leaderboard (bfcl): From tool use to agentic evaluation of large language models

    Shishir G Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Vishnu Suresh, Ion Stoica, and Joseph E Gonzalez. The berkeley function calling leaderboard (bfcl): From tool use to agentic evaluation of large language models. In F orty-second International Conference on Machin...

    2025

  35. [43]

    Acebench: Who wins the match point in tool usage? arXiv preprint arXiv:2501.12851, 2025

    Chen Chen, Xinlong Hao, Weiwen Liu, Xu Huang, Xingshan Zeng, Shuai Yu, Dexun Li, Shuai Wang, Weinan Gan, Yuefeng Huang, et al. Acebench: Who wins the match point in tool usage? arXiv preprint arXiv:2501.12851, 2025

    2025

  36. [44]

    Deepseek-v3 technical report

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024

    2024 arXiv

  37. [45]

    student_id

    Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng, Jiajie Zhang, et al. Glm-4.5: Agentic, reasoning, and coding (arc) foundation models. arXiv preprint arXiv:2508.06471, 2025. 14 Appendix A Cold-start Data Example #1 LLM-sim...

    2025 arXiv

  38. [46]

    term":"Sakura

    **Sakura Matou** (ID: 500) 2. **Mai Sakurajima** (ID: 127222) 3. **Sakura Yamauchi** (ID: 127118) 4. **Sakura Kinomoto** (ID: 2671) 5. **Sakura Haruno** (ID: 145) This shows the first 5 results out of 5,000 total characters found. The search found characters from various popul...

  39. [47]

    02:30:00

    **Sakura Matou** (ID: 500) 2. **Mai Sakurajima** (ID: 127222) 3. **Sakura Yamauchi** (ID: 127118) 4. **Sakura Kinomoto** (ID: 2671) 5. **Sakura Haruno** (ID: 145) 6. **Kyouko Sakura** (ID: 40006) 7. **Chiyo Sakura** (ID: 87271) 8. **Haruka Sakura** (ID: 230204) 9. **Futaba Sak...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.