Pith. sign in

REVIEW 5 major objections 5 minor 3 cited by

Agent Safety Alignment via Reinforcement Learning

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A tri-modal policy trained in a sandbox hardens tool-using agents against user and tool attacks without sacrificing utility.

desk verdict A plausible sandboxed-RL recipe for agent safety whose headline 'safety without utility loss' claim is undermined by metric circularity and a 28-point BFCL drop in the paper's own Table 4. read the letter →

arxiv 2507.08270 v1 pith:Q46NWYNT submitted 2025-07-11 cs.AI cs.CR

Zeyang Sha , Hanling Tian , Zhuoer Xu , Shiwen Cui , Changhua Meng , Weiqiang Wang This is my paper
classification cs.AIcs.CR
keywords LLMagentssafetyalignmentreinforcementlearningtoolusepromptinjectionindirectsandboxtrainingexecute-refuse-verifypolicy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that the safety of tool-using LLM agents can be aligned end-to-end with reinforcement learning, covering both attack channels at once: malicious instructions from users and malicious outputs from compromised tools. It defines a three-way label—benign, malicious, sensitive—for both user prompts and tool responses, and trains a single execute–refuse–verify policy inside a sandbox that simulates tool calls. On public and self-built benchmarks, the paper reports that aligned agents refuse far more malicious and sensitive requests and malicious tool outputs while retaining high utility on normal function-calling tasks. The consequence, if true, is that safety and usefulness can be jointly optimized rather than traded off.

What carries the argument

The load-bearing object is the tri-modal taxonomy of benign, malicious, and sensitive labels applied to both user prompts and tool outputs, bundled into an execute–refuse–verify policy. Training happens in a sandboxed environment that intercepts each `<tool_call>`, executes the requested function, returns the result, and stochastically answers confirmation requests; the reward is a product of structural checks (EOS termination and balanced `<think>` tags, $R_{\mathrm{gen}}(a_t)=\mathrm{EOS}(a_t)\cdot\mathrm{THINK}(a_t)$) and a scenario-specific term $R_{\ell}$, so the final scalar is $R = R_{\mathrm{gen}} \times R_{\ell}$. Sensitive prompts and malicious tool outputs are rewarded only if the agent emits `<tool_check>` and then acts on the simulated `<tool_check_result>`, while malicious prompts are rewarded for containing no tool call plus a rejection-classifier-confirmed refusal. This reward structure is what is supposed to teach the model to distinguish harmless requests from threats without losing everyday tool-using competence.

What would settle it

Present the aligned agent with a held-out set of malicious tool outputs whose wording does not match the training templates, and check whether `<tool_check>` still fires; if refusal drops toward the unaligned baseline while the tag fires mainly on template-like inputs, the model has learned a lexical cue rather than threat recognition. A second decisive check: after the sandbox returns 'yes' to a `<tool_check>` for a genuinely dangerous sensitive call, observe whether the agent proceeds to invoke the tool, which would show the safety guarantee is outsourced to user confirmation.

Watch

Extended reading notes

Core claim

The paper's central claim is that one policy trained in a sandbox can simultaneously execute benign requests immediately, refuse malicious requests outright, and delay sensitive actions behind an explicit confirmation step—and that the same controller applied to tool outputs protects against indirect prompt injection. The evidence is comparative: on Agent SafetyBench, the user-aligned 7B model rises from 15.3 to 69.9 and the internal malicious-prompt test from 0.9 to 99.2; on the tool-side malicious test, the user-tool-aligned 7B model climbs from 0.0 to 92.5; and on BFCL, the user-tool-aligned 7B model still scores 91.3 versus 95.3 for the unaligned model, which the paper interprets as preserving utility. Safety and effectiveness, the paper argues, are jointly optimizable.

Load-bearing premise

The safety scores depend on treating the model's 'double-check' signal as a faithful sign that it truly recognized a threat, when the training reward directly pays the model to emit that signal.

Editorial extensions

If this is right

  • A single execute–refuse–verify policy can cover both the user-input channel and the tool-output channel, so an agent trained this way needs no separate safety wrapper for each threat source.
  • Heuristic input filtering improves some threat scores but costs utility, whereas the aligned models keep high function-calling scores while exceeding the filtered models on most safety tests.
  • Tool-side hardening requires explicit joint training: user-only alignment still leaves malicious tool outputs dangerous on the internal malicious-tool test, while user-tool alignment raises the score substantially.
  • The same verification mechanism that protects sensitive user requests transfers to detecting poisoned tool outputs, suggesting the framework may cover other tool-side attacks such as tool poisoning and memory injection.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If `<tool_check>` is learned as a label-matching shortcut rather than a genuine risk estimate, then on adversarial prompts that avoid the training templates the reported safety margins may shrink; a useful test would be a held-out red-team set with new phrasing.
  • Because the reward allows a sensitive action after a simulated 'yes', the trained policy's real-world safety depends on the human's confirmation being informed; in deployments where users click through prompts, the effective protection may be weaker than the benchmark suggests.
  • The approach could be extended to estimate a calibrated risk probability per tool call instead of a three-way hard label, which would let the agent decide when a double-check is worth the interaction cost.
  • A natural next experiment is to train with a continuous spectrum of sensitivity, or with human preferences over when to verify, to test whether the categorical taxonomy is necessary or merely convenient.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a reinforcement learning framework for aligning tool-using LLM agents against two threat channels: malicious user prompts and malicious tool outputs. The method introduces a tri-modal taxonomy (benign/malicious/sensitive) applied to both user prompts and tool responses, along with a sandboxed training environment and reward shaping that encourages the agent to execute benign requests, refuse malicious ones, and request user confirmation for sensitive ones. The authors train Qwen-2.5 7B and 14B models in three alignment configurations (user-aligned, user-tool aligned, and a prompt-guarded baseline) and evaluate them on Agent SafetyBench, InjecAgent, BFCL, and self-built tests, claiming that safety improves on both threat vectors without sacrificing utility.

Significance. If the central claims were well supported, this work would be a useful step toward training safer tool-using agents, particularly by addressing tool-initiated threats via reinforcement learning and by proposing a unified execute-refuse-verify policy. The use of a sandbox environment for fine-grained reward shaping is a plausible and potentially valuable design choice. However, the significance is currently limited by three factors: the safety metric is circular (it counts the same <tool_check> tag that the reward function directly incentivizes), the utility-preservation claim is contradicted by large drops on public function-calling benchmarks, and the lack of code, data, seeds, and error bars prevents independent verification. The taxonomy and training framework are reasonable, but the experimental evidence does not yet establish that safety and utility can be jointly optimized.

major comments (5)
  1. [Section 4, Tables 1 and 3] The safety metric equates emission of the <tool_check> tag with successful threat recognition, while the reward function in Section 3.3 directly rewards <tool_check> for sensitive and malicious-tool scenarios. This creates circularity: the evaluation measures exactly the behavior that the training reward incentivizes. The paper must provide an independent validation of <tool_check> as a faithful signal of genuine threat recognition, for example by human evaluation of a sample of tag emissions, by measuring actual sensitive tool invocations in a held-out sandbox, or by showing that <tool_check> correlates with refusal of harmful actions in a real environment. Without such validation, the reported safety gains on ASB and InjecAgent may reflect a shallow policy of tag over-emission rather than improved security.
  2. [Section 5.4, Table 4] The claim that utility is preserved is contradicted by the public BFCL results: the 14B user-tool aligned model drops from 95.5 to 67.3 on BFCL and from 74.8 to 42.2 on BFCL-Live, and the 7B user-aligned model drops from 95.3 to 84.3 on BFCL. The text dismisses these as 'do not drop a lot' and instead highlights the self-built utility test, but that test is generated with the same DeepSeek pipeline and taxonomy as the training data, so it may share the trained format biases. The authors need to report the trade-off honestly and demonstrate, with external benchmarks or independent evaluation, that the self-built utility test is not biased toward the trained <tool_check>/<tool_call> behavior.
  3. [Section 5.2, Table 1] User-tool aligned models show substantially lower ASB scores than user-aligned models (7B: 50.1 vs 69.9; 14B: 43.5 vs 88.8), yet the paper does not discuss this degradation. This pattern suggests that adding tool-side alignment can reduce user-side threat resistance, which directly undermines the claim that the framework jointly improves both channels. The authors should analyze this interaction and explain why joint alignment hurts ASB performance, or present evidence that this is not a systematic effect.
  4. [Section 5.3, Table 3] On the public InjecAgent benchmark, the Prompt Guarded baseline achieves far better security scores than the proposed user-tool aligned models (7B: 4.0 vs 28.3; 14B: 0.1 vs 3.0; lower is better). The paper only mentions that Prompt Guarded reduces utility, but it does not acknowledge that a simple input filter outperforms the trained method on this public benchmark. Additionally, the text in Section 5.3 states that unprotected models 'scoring only 15.3 on InjecAgent,' which does not match Table 3 (36.8 for 7B and 12.4 for 14B). This internal inconsistency must be corrected, and the comparison with Prompt Guarded should be discussed fairly.
  5. [Section 3.3 and Section 4] The reward for malicious user prompts depends on the rejection classifier REF, and the safety evaluation for ASB and InjecAgent uses the <tool_check> tag, but no details are given on how REF is trained, what its accuracy is, or how it handles edge cases. REF is a load-bearing component of the reward, and its errors may distort both training and evaluation. The authors should report the classifier's performance, provide sensitivity analysis to its threshold or training set, and ideally make REF public so the results can be reproduced independently.
minor comments (5)
  1. [Section 3.3] The heading 'Rewards for BU' actually describes rewards for malicious user prompts (MU), and 'Rewards for BT' actually describes malicious tool cases (MT); also, the benign tool is referred to as 'GT' in Section 3.1. Please unify the notation and correct these labels.
  2. [Section 3.3, equation for RBU] The reward formulas are incomplete or ambiguous: the product symbol is missing in 'RBU(at) = Rtc(at) REF(at)', and the predicates C(at), A(at), and Rtc are not formally defined. The 'Rewards for BU, BT, ST' formula also omits the required absence-of-double-check condition that the prose states.
  3. [Table 1] The asterisks next to the ASB scores (15.3* and 31.9*) are not explained in the table caption or text; please clarify what they denote or remove them.
  4. [Section 5.2] The text contains typos such as 'malicisous' and 'donot', and the phrase 'we compute the tool call rates' for ASB is confusing because higher scores are presented as better; please specify the exact metric (e.g., refusal rate or safe-action rate) and the direction of improvement.
  5. [Figure 2] The radar plot in Figure 2 lacks numerical labels on the axes, making it impossible to verify the claimed improvements; please add numeric scaling or report the underlying values in a table.

Circularity Check

1 steps flagged · score 6.0 of 10

Safety metric equals the reward token: <tool_check> is both the behavior trained by the reward and the behavior counted as successful threat recognition.

  1. self definitional [Section 3.3 (Rewards for SU and BT) and Section 4 (Datasets and Metrics)]
    "• request explicit user approval via <tool_check> ... The reward function can be formulated similar to the reward for SU: RBT (at, yt) = Rtc(at)[I(C(at) ∧ ((yt=1∧A(at)) ∨ (yt=0∧¬A(at))))] ... Similar to ASB, if the agent invokes <tool_check>, we consider it to have successfully recognized the security threat."

    The reward for sensitive-user (SU) and malicious-tool (BT) scenarios pays credit only when the model emits <tool_check> (the C(at) predicate), and the safety metric in Section 4 counts exactly the same <tool_check> tag as successful threat recognition. The reported ASB and InjecAgent gains are therefore, by construction, a measurement of whether the model emitted the token that the reward reinforced. The evaluation never independently verifies that the tag corresponds to a genuine refusal, verification query, or withheld tool call.

full rationale

The central safety result is partially self-referential. Section 3.3 defines the reward so that sensitive-user and malicious-tool scenarios are scored by the presence of <tool_check> (C(at)), and Section 4 defines the safety metric as counting <tool_check> as successful threat recognition. The reported safety improvements therefore partly state that the model learned the reinforced token, not that an externally validated threat-recognition capability exists; no human or real-environment check ties the tag to genuine refusal or verification. The paper's self-built 'Our Malicious Test', 'Our Sensitive Test', and 'Our Utility Test' are generated with the same DeepSeek pipeline and tri-modal taxonomy used for the training corpus, so they add little independent evidence, and their high scores may reflect in-distribution format matching. External benchmarks (ASB, InjecAgent, BFCL) provide partial independent grounding, and the trained policy of verifying before sensitive actions is a plausible safety behavior, so the circularity is real but not total. The BFCL/BFCL-Live utility drops (e.g., 14B user-tool aligned from 95.5 to 67.3 and 74.8 to 42.2) are dismissed as 'do not drop a lot', which is a correctness and calibration concern rather than a circularity. Overall score 6: one load-bearing safety metric reduces to the reward token by construction.

Assumptions & free parameters 2 free parameters · 6 assumptions · 1 invented entities

The central results depend on the taxonomy labels, tag-based observables, a sandbox that simulates tools, and an auxiliary refusal classifier. None of these is independently validated or released, so the headline numbers rest on several unverified premises.

free parameters (2)
  • stochastic user-confirmation probability = not reported
    The sandbox samples binary yes/no replies to double-check requests; the approval probability is a hand-chosen stochastic parameter that shapes the verification behavior in training.
  • REF rejection classifier parameters = not reported
    The malicious-prompt reward depends on a separately trained refusal classifier, REF, whose parameters are fit to data. No architecture, training set, or accuracy is reported.
assumptions (6)
  • ad hoc to paper The tri-modal benign/malicious/sensitive taxonomy is complete and mutually exclusive for user prompts and tool outputs.
    Section 3.1 introduces BU/MU/SU and BT/MT/ST as the entire label space; no evidence shows that real agent traffic fits these categories cleanly.
  • domain assumption DeepSeek-generated synthetic corpora cover the real distribution of user and tool attacks.
    Section 3.1 generates 20,000 user prompts and 5,000 tool utterances with few-shot DeepSeek plus light manual spot-checking, with no validation against real attack corpora.
  • domain assumption The ReCall sandbox faithfully simulates real-world tool execution and attack behavior.
    Section 3.2 builds the training environment on ReCall; transfer to production MCP or API tools is assumed rather than demonstrated.
  • domain assumption Emission of <tool_call> and <tool_check> tags faithfully reflects actual tool invocation and safety verification.
    Rewards in Section 3.3 and metrics in Section 4 key off these tags, with no validation against real tool calls or human review.
  • ad hoc to paper The rejection classifier REF accurately identifies refusal text.
    Section 3.3 introduces REF as a trained indicator with no architecture, training data, or performance numbers; the malicious-prompt reward depends on it.
  • standard math Clipped policy-gradient RL converges to the intended execute-refuse-verify policy under the composite product reward.
    Section 3.4 applies a standard clipped policy-gradient update with batch-normalized returns, but convergence assumptions are unstated and no seeds or variance are reported.
invented entities (1)
  • Rejection classifier REF
    purpose: Supplies the refusal indicator used in the malicious-user-prompt reward so the model cannot satisfy the reward by avoiding tool calls without also producing a textual refusal.
    No separate validation, architecture, or release is provided; its accuracy is assumed within the training loop.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Agent Safety Alignment via Reinforcement Learning." pith.science (2026). https://pith.science/paper/Q46NWYNT

@misc{pith2026250708270,
  author       = {Pith},
  title        = {Pith review of: Agent Safety Alignment via Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q46NWYNT}},
  note         = {Machine review of arXiv:2507.08270}
}
read the original abstract

The emergence of autonomous Large Language Model (LLM) agents capable of tool usage has introduced new safety risks that go beyond traditional conversational misuse. These agents, empowered to execute external functions, are vulnerable to both user-initiated threats (e.g., adversarial prompts) and tool-initiated threats (e.g., malicious outputs from compromised tools). In this paper, we propose the first unified safety-alignment framework for tool-using agents, enabling models to handle both channels of threat via structured reasoning and sandboxed reinforcement learning. We introduce a tri-modal taxonomy, including benign, malicious, and sensitive for both user prompts and tool responses, and define a policy-driven decision model. Our framework employs a custom-designed sandbox environment that simulates real-world tool execution and allows fine-grained reward shaping. Through extensive evaluations on public and self-built benchmarks, including Agent SafetyBench, InjecAgent, and BFCL, we demonstrate that our safety-aligned agents significantly improve resistance to security threats while preserving strong utility on benign tasks. Our results show that safety and effectiveness can be jointly optimized, laying the groundwork for trustworthy deployment of autonomous LLM agents.

Figures

Figures reproduced from arXiv: 2507.08270 by the authors.

Figure 1
Figure 1. Overview of the proposed frameworks. harm. For instance, in May 2025, researchers disclosed CVE-2025-314912 , a redirect vulnerability in AutoGPT that leaked GitHub OAuth tokens. Attackers could exploit the flaw to obtain write access to private repositories, illustrating how seemingly minor bugs in tool-using agents can escalate into severe real-world breaches. The maintainers released a patched version (v0.6.1) wi… view at source ↗
Figure 2
Figure 2. Radar plot comparing seven evaluation dimensions for aligned vs. unaligned agents. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. $S^3$: Improving Agent Safety through Multi-Stage Defense

    cs.CR 2026-08 conditional novelty 6.0 of 10

    S3 composes stage-specific safety skills through a guard agent, achieving near-zero attack success on six risk types in its own benchmark while preserving benign task completion.

  2. SAFETY SENTRY: Context-Aware Human Intervention via EXECUTE-ASK-REFUSE Routing

    cs.AI 2026-07 conditional novelty 6.0 of 10

    A 4B model fine-tuned on a 9,203-step LLM-annotated corpus with a single tunable threshold yields a three-way EXECUTE/ASK/REFUSE guard that beats zero-shot baselines on in-distribution and held-out agent actions.

  3. S3LoRA: Safe Spectral Sharpness-Guided Pruning in Adaptation of Agent Planner

    cs.AI 2025-08 conditional novelty 5.0 of 10

    S3LoRA prunes LoRA layers with the sharpest spectral update concentration to improve safety in fine-tuned LLM agents without needing base models or extra data.

Reference graph

Works this paper leans on

30 extracted references · 13 canonical work pages · cited by 3 Pith papers

  1. [13]

    Think twice before you act: Enhancing agent behavioral safety with thought correction

    Changyue Jiang, Xudong Pan, and Min Yang. Think twice before you act: Enhancing agent behavioral safety with thought correction. CoRR, abs/2505.11063, 2025

    arXiv 2025

  2. [14]

    Agentalign: Navigating safety alignment in the shift from informative to agentic large language models

    Jinchuan Zhang, Lu Yin, Yan Zhou, and Songlin Hu. Agentalign: Navigating safety alignment in the shift from informative to agentic large language models. CoRR, abs/2505.23020, 2025

    arXiv 2025

  3. [1]

    Narasimhan, and Yuan Cao

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5,

    work page 2023

  4. [2]

    AutoGPT: An open-source autonomous agent framework

    Significant Gravitas. AutoGPT: An open-source autonomous agent framework. https:// github.com/Significant-Gravitas/AutoGPT, 2023. Accessed 2025-07-10

    work page 2023

  5. [3]

    BabyAGI: Experimental self-building autonomous agent

    Yohei Nakajima. BabyAGI: Experimental self-building autonomous agent. https://github. com/yoheinakajima/babyagi, 2023. Accessed 2025-07-10

    work page 2023

  6. [4]

    AgentGPT: Configure and deploy autonomous ai agents

    Reworkd. AgentGPT: Configure and deploy autonomous ai agents. https://github.com/ reworkd/AgentGPT, 2024. Accessed 2025-07-10

    work page 2024

  7. [5]

    AI agents under threat: A survey of key security challenges and future pathways

    Zehang Deng, Yongjian Guo, Changzhou Han, Wanlun Ma, Junwu Xiong, Sheng Wen, and Yang Xiang. AI agents under threat: A survey of key security challenges and future pathways. ACM Comput. Surv., 57(7):182:1–182:36, 2025

    work page 2025

  8. [6]

    Navigating the risks: A survey of security, privacy, and ethics threats in llm-based agents

    Yuyou Gan, Yong Yang, Zhe Ma, Ping He, Rui Zeng, Yiming Wang, Qingming Li, Chunyi Zhou, Songze Li, Ting Wang, Yunjun Gao, Yingcai Wu, and Shouling Ji. Navigating the risks: A survey of security, privacy, and ethics threats in llm-based agents. CoRR, abs/2411.09523, 2024

    arXiv 2024

Show all 30 references
  1. [7]

    Retool: Reinforcement learning for strategic tool use in llms, 2025

    Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong. Retool: Reinforcement learning for strategic tool use in llms, 2025

    2025

  2. [8]

    SEM: reinforcement learning for search-efficient large language models

    Zeyang Sha, Shiwen Cui, and Weiqiang Wang. SEM: reinforcement learning for search-efficient large language models. CoRR, abs/2505.07903, 2025

    2025 arXiv

  3. [9]

    Agent-r: Training language model agents to reflect via iterative self-training

    Siyu Yuan, Zehui Chen, Zhiheng Xi, Junjie Ye, Zhengyin Du, and Jiecao Chen. Agent-r: Training language model agents to reflect via iterative self-training. CoRR, abs/2501.11425, 2025

    2025 arXiv

  4. [10]

    Pan, Wen Zhang, Huajun Chen, Fan Yang, Zenan Zhou, and Weipeng Chen

    Mingyang Chen, Tianpeng Li, Haoze Sun, Yijie Zhou, Chenzheng Zhu, Haofen Wang, Jeff Z. Pan, Wen Zhang, Huajun Chen, Fan Yang, Zenan Zhou, and Weipeng Chen. Research: Learning to reason with search for llms via reinforcement learning, 2025

    2025

  5. [11]

    Agent security bench (ASB): formalizing and benchmarking attacks and defenses in llm-based agents

    Hanrong Zhang, Jingyuan Huang, Kai Mei, Yifei Yao, Zhenting Wang, Chenlu Zhan, Hongwei Wang, and Yongfeng Zhang. Agent security bench (ASB): formalizing and benchmarking attacks and defenses in llm-based agents. In The Thirteenth International Conference on Learning Representa...

    2025

  6. [12]

    Agent-safetybench: Evaluating the safety of LLM agents

    Zhexin Zhang, Shiyao Cui, Yida Lu, Jingzhuo Zhou, Junxiao Yang, Hongning Wang, and Minlie Huang. Agent-safetybench: Evaluating the safety of LLM agents. CoRR, abs/2412.14470, 2024

    2024 arXiv

  7. [15]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    2025

  8. [16]

    Injecagent: Benchmarking indirect prompt injections in tool-integrated large language model agents

    Qiusi Zhan, Zhixiang Liang, Zifan Ying, and Daniel Kang. Injecagent: Benchmarking indirect prompt injections in tool-integrated large language model agents. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Findings of the Association for Computational Linguistics, ACL...

    2024

  9. [17]

    Patil, Huanzhi Mao, Charlie Cheng-Jie Ji, Fanjia Yan, Vishnu Suresh, Ion Stoica, and Joseph E

    Shishir G. Patil, Huanzhi Mao, Charlie Cheng-Jie Ji, Fanjia Yan, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. The berkeley function calling leaderboard (bfcl): From tool use to agentic evaluation of large language models. In F orty-second International Conference on Mach...

    2025

  10. [18]

    Yu, and Ming Zhang

    Junyu Luo, Weizhi Zhang, Ye Yuan, Yusheng Zhao, Junwei Yang, Yiyang Gu, Bohan Wu, Binqi Chen, Ziyue Qiao, Qingqing Long, Rongcheng Tu, Xiao Luo, Wei Ju, Zhiping Xiao, Yifan Wang, Meng Xiao, Chenwu Liu, Jingyang Yuan, Shichang Zhang, Yiqiao Jin, Fan Zhang, Xian Wu, Hanqing Zhao...

    2025 arXiv

  11. [19]

    An in-depth survey of large language model-based artificial intelligence agents

    Pengyu Zhao, Zijian Jin, and Ning Cheng. An in-depth survey of large language model-based artificial intelligence agents. CoRR, abs/2309.14365, 2023

    2023 arXiv

  12. [20]

    Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas L

    Theodore R. Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas L. Griffiths. Cognitive architectures for language agents. Trans. Mach. Learn. Res., 2024, 2024

    2024

  13. [21]

    AutoAgent: A Fully-Automated and Zero-Code Frame- work for LLM Agents, 2025

    Chao Huang Jiabin Tang, Tianyu Fan. AutoAgent: A Fully-Automated and Zero-Code Frame- work for LLM Agents, 2025

    2025

  14. [23]

    Pan, Wen Zhang, Huajun Chen, Fan Yang, Zenan Zhou, and Weipeng Chen

    Mingyang Chen, Tianpeng Li, Haoze Sun, Yijie Zhou, Chenzheng Zhu, Haofen Wang, Jeff Z. Pan, Wen Zhang, Huajun Chen, Fan Yang, Zenan Zhou, and Weipeng Chen. Research: Learning to reason with search for llms via reinforcement learning. CoRR, abs/2503.19470, 2025

    2025 arXiv

  15. [24]

    Deepresearcher: Scaling deep research via reinforcement learning in real-world environments, 2025

    Yuxiang Zheng, Dayuan Fu, Xiangkun Hu, Xiaojie Cai, Lyumanshan Ye, Pengrui Lu, and Pengfei Liu. Deepresearcher: Scaling deep research via reinforcement learning in real-world environments, 2025

    2025

  16. [25]

    Beyond the protocol: Unveiling attack vectors in the model context protocol ecosystem

    Hao Song, Yiming Shen, Wenxuan Luo, Leixin Guo, Ting Chen, Jiashui Wang, Beibei Li, Xiaosong Zhang, and Jiachi Chen. Beyond the protocol: Unveiling attack vectors in the model context protocol ecosystem. CoRR, abs/2506.02040, 2025

    2025 arXiv

  17. [26]

    Progent: Programmable privilege control for LLM agents

    Tianneng Shi, Jingxuan He, Zhun Wang, Linyu Wu, Hongwei Li, Wenbo Guo, and Dawn Song. Progent: Programmable privilege control for LLM agents. CoRR, abs/2504.11703, 2025

    2025 arXiv

  18. [27]

    Cullen, Paul Montague, Sarah M

    Shijie Liu, Andrew C. Cullen, Paul Montague, Sarah M. Erfani, and Benjamin I. P. Rubinstein. Fox in the henhouse: Supply-chain backdoor attacks against reinforcement learning. CoRR, abs/2505.19532, 2025

    2025 arXiv

  19. [28]

    A practical memory injection attack against LLM agents

    Shen Dong, Shaochen Xu, Pengfei He, Yige Li, Jiliang Tang, Tianming Liu, Hui Liu, and Zhen Xiang. A practical memory injection attack against LLM agents. CoRR, abs/2503.03704, 2025

    2025

  20. [29]

    Agentpoison: Red- teaming LLM agents via poisoning memory or knowledge bases

    Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, and Bo Li. Agentpoison: Red- teaming LLM agents via poisoning memory or knowledge bases. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang, editors, Advances in ...

    2024

  21. [30]

    Safeagentbench: A benchmark for safe task planning of embodied LLM agents

    Sheng Yin, Xianghe Pang, Yuanzhuo Ding, Menglan Chen, Yutong Bi, Yichen Xiong, Wenhao Huang, Zhen Xiang, Jing Shao, and Siheng Chen. Safeagentbench: A benchmark for safe task planning of embodied LLM agents. CoRR, abs/2412.13178, 2024. 12

    2024

  22. [2023]

    OpenReview.net, 2023

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.