{"id":"d5e00f13-c5a9-493a-8a90-265c235f2d2f","arxiv_id":"2502.09809","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"AgentGuard repurposes an LLM agent's own orchestrator to find, validate, and constrain unsafe tool-use workflows, demonstrated as a proof-of-concept on Aider.","lead":"This paper describes AgentGuard, a framework that uses an LLM agent's own orchestrator to discover unsafe tool-use workflows, execute them to confirm the risk, and generate security rules to block the behavior. It reports a proof-of-concept on a coding agent, with many failures but a few end-to-end successes.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Feasibility rests entirely on self-reported validation by the same model that proposes the workflows; without an independent oracle on execution logs, a successful end-to-end run is consistent with self-consistent hallucination.","rationale":"The paper's strongest claim is narrow—a proof-of-concept that the orchestrator can evaluate itself well enough to expose unsafe workflows and to generate constraints that block them. The reported failures (SELinux rule generation) are honestly stated, and a single clean end-to-end run would indeed be adequate for a feasibility claim. The soft spot is verification independence, not the general idea: every role in the validation loop is implemented by the same underlying LLM, and the measured 'success' is an unverified self-report. This is a correctness risk, not a novelty or consensus problem. The reader's weakest assumption points at the closely related compliance and hallucination issues as well as at the same-LLM overlap, so the two readings agree on where the risk sits. The conditional verdict is the right one: the missing piece is a concrete external-oracle test that would settle whether the proof-of-concept is real; without that test, accepting the feasibility claim as fully demonstrated would be premature, while rejecting the work entirely would ignore its transparency and the plausibility of the underlying idea.","tokens_in":5882,"tokens_out":4036,"duration_ms":40541,"concrete_test":"Run AgentGuard on the same Aider/ChatGPT-4o configuration, preserving the successful end-to-end runs, and have two independent raters (e.g., a human auditor and a different model not given AgentGuard's prompts) label from raw execution logs whether (i) the reported unsafe outcome occurred, (ii) it was attributable to the generated tool-orchestration sequence, and (iii) after applying the SELinux policy, re-execution was blocked by a denial rather than by the orchestrator silently choosing different actions. Require all test cases that were marked 'validated' in the report, and compute inter-rater agreement. If the independent labels agree with AgentGuard's own reports, the feasibility claim survives this test; if they diverge, the claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"To support the feasibility claim, one needs at least one end-to-end run in which an unsafe workflow is genuinely discovered, genuinely causes harm in execution, and is genuinely blocked by a generated SELinux constraint. In the reported successful runs (Section 4), ChatGPT-4o is simultaneously the workflow generator, the test-case author, the executor, the outcome detector, the SELinux rule writer, and the post-constraint outcome checker. A run that 'passes' is therefore a self-consistency report from the very model whose orchestration is under test. It cannot by itself distinguish a real unsafe outcome from the model's own flawed detection code, nor a genuine SELinux denial from the model's changed choice of tool calls after the constraint is applied. The paper's other deviation is also relevant: the orchestrator refused the safety-evaluation task and had to be jailbroken with Role Augmentation prompting before it would cooperate, so the entity evaluated is not the same as the moderated deployment agent. An independent check on logs is the minimal missing experiment; without it, the proof-of-concept is not yet distinguished from a self-consistent hallucination.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AgentGuard, a pre-deployment safety testing and hardening framework for LLM agents. The key idea is to repurpose the agent's own LLM-based orchestrator as its safety evaluator: the orchestrator identifies unsafe tool-use workflows, generates and executes concrete test cases to validate that they cause unsafe outcomes, a Safety Constraint Expert Agent converts validated workflows into SELinux constraints, and the orchestrator re-executes the test cases to show that the constraints block the unsafe outcomes. The deliverable is an evaluation report containing unsafe workflows, test cases, and validated constraints. The authors evaluate the prototype on the Aider coding agent with ChatGPT-4o on an Ubuntu 22.04 VM with SELinux enabled, reporting that most runs failed at SELinux rule generation or application but that a few successful runs demonstrated proof-of-concept feasibility.","tokens_in":6019,"tokens_out":3533,"duration_ms":35220,"significance":"If the feasibility claim is substantiated, AgentGuard addresses a real need: a standardized pre-deployment process for discovering and mitigating unsafe tool orchestration in LLM agents. The paper's core design rationale is sensible and well articulated: the orchestrator has tool knowledge, has execution privileges, and can generate realistic workflows precisely because it is the same entity that will be exploited at deployment. The four-phase structure and the concrete SELinux embodiment make the proposal actionable, and the authors are transparent about failures, including the orchestrator's initial refusal and the difficulty of generating applicable SELinux rules. The comparison with TooLEmu is useful. However, the empirical evidence is qualitative and self-referential: one target agent, one model, one configuration, no success counts, no example logs, and no independent oracle for whether an unsafe outcome actually occurred and was actually blocked. The paper is best read as a position/exploratory report; its central feasibility claim needs significantly stronger evidence before it can support a general conclusion.","major_comments":[{"comment":"The central feasibility claim, stated in Section 6 as 'we observed successful evaluation results indicating the feasibility of AGENT GUARD', is not supported by the reported evidence. The evaluation section reports qualitative observations (e.g., 'only a very few' rules could be applied) but provides no success counts, no number of identified versus validated workflows, no example test cases, no SELinux rule examples, and no logs of denials or outcome detections. A reader cannot tell how many end-to-end runs succeeded, which workflows were validated, or whether the constraints actually blocked the unsafe effects. At minimum, the paper needs one complete end-to-end trace with independent verification, for example OS-level audit logs showing the unsafe operation occurring before the constraint and being denied after it.","section":"Section 4 and Section 6"},{"comment":"Assumption 3 states that the orchestrator is not compromised and will actively expose unsafe workflows as requested. In practice, the orchestrator initially refused the task and had to be jailbroken with Role Augmentation prompting; it also hallucinated non-existent tool capabilities until Chain-of-Thought prompting was added. This means the entity that was evaluated is not the compliant, unmodified orchestrator assumed by the design, nor is it the same as the deployment-time agent after moderation and hardening. The paper should either evaluate a genuinely compliant orchestrator, or explicitly analyze how the jailbroken state affects the validity and transferability of the generated safety constraints. Without this, the evaluation report may not cover the actual risk surface of the agent as it would behave at deployment.","section":"Section 4, Unsafe Workflow Identification, versus Assumption 3 in Section 2.1"},{"comment":"The validation loop is self-referential. As described in Section 3, the Safety Constraint Expert Agent is 'the orchestrator equipped with knowledge in SELinux from the underlying LLM', so the same model generates the workflows, writes the test cases, executes them, detects unsafe outcomes, writes the SELinux constraints, re-executes the test cases, and checks whether outcomes are blocked. A successful post-constraint run could therefore reflect a genuine SELinux denial, but it could also reflect the model's own changed choice of tool calls or a flawed outcome-detection check. There is no independent oracle on the execution logs. The authors should add at least one run in which an independent mechanism (e.g., a SELinux audit log, a filesystem snapshot, or a human reviewer) confirms both that the unsafe outcome occurred before the constraint and that the constraint, not the model's changed behavior, prevented it afterward.","section":"Section 3 and Section 4, Safety Constraint Validation"}],"minor_comments":[{"comment":"The statement that SELinux rule generation 'appeared to be decent' and 'yielded rules with correct syntax ... for a fair amount of times' is too vague for a scientific report; please provide the number of generated rules, the number syntactically valid, and the number actually applied and validated.","section":"Section 4, Safety Constraint Generation"},{"comment":"Figure 1 and Section 2.2 describe the Prompting Proxy Agent and the Safety Constraint Expert Agent as distinct components, but Section 3 states that the same underlying LLM currently plays both roles. Please clarify which components are actually separate in the implementation and which are the same model under different prompts, as this matters for interpreting the results.","section":"Section 1.2 and Section 3"},{"comment":"The claim that 'the vanilla underlying foundation model (i.e., ChatGPT 4o) does not have content moderation for such a benign task' is asserted without evidence and is difficult to verify from the outside; please either show the relevant experiments or rephrase as a hypothesis about implicit prompt-based moderation.","section":"Section 4, Unsafe Workflow Identification"},{"comment":"Section 2.3 lists 'the difference in the occurrences of unsafe outcomes before and after applying the safety constraints' as a quantitative improvement measure, but the evaluation section does not report this number for any workflow. Either report it for the successful runs or explicitly defer it to future work in Section 6.","section":"Section 2.3 and Section 6"},{"comment":"The manuscript lacks reproducibility details: the exact system prompts, the Role Augmentation prompt template, the Chain-of-Thought prompt, the test-case code, and the SELinux rules are not included. Providing an appendix with these artifacts, or a repository link, would substantially strengthen the paper. There are also several grammatical slips (e.g., 'the orchestrator's should be good at generating tool-use plans' in Section 4) that should be corrected.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"This is clearly an exploratory hackathon report rather than a mature research paper. The core idea is interesting and worth pursuing, but the empirical support for the central feasibility claim is currently a self-consistency report from the very model under test, with no independent oracle and no quantitative reporting. The missing evidence is obtainable within the scope of the manuscript (additional experiments with logs and counts), so I do not recommend rejection; however, the paper in its current form would not meet the evidentiary bar for a strong journal publication. I would also flag that the Role Augmentation jailbreak is a sensitive methodological choice; the authors should make sure they frame it as a tool for evaluation, not as a recommended way to bypass content moderation in deployed agents."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a clearly written hackathon prototype with one genuinely new idea and very little evidential weight. The new idea is to repurpose the target agent's own LLM orchestrator as the safety evaluator: it proposes unsafe tool workflows, executes them in the real environment, and then generates and validates SELinux constraints to block them. That is different from TooLEmu, which emulates tools with an LM. Running against real SELinux on a real VM is also a step up from a simulated sandbox. The paper is honest about the rough edges: it reports that the orchestrator initially refused the task, that hallucination required Chain-of-Thought prompting, and that most generated SELinux rules were unusable. The Role Augmentation observation is a useful incidental finding.\n\nThe soft spots are real and load-bearing. The empirical base is one target agent (Aider), one model (ChatGPT-4o), one configuration, no reported counts, no code, no prompts, no logs. The paper says 'very few' end-to-end runs succeeded. More importantly, in this implementation the same model is the workflow proposer, test-case author, executor, outcome detector, SELinux rule writer, and post-constraint checker. A 'validated' unsafe workflow is therefore self-reported by the very model under test. The stress-test's 'self-consistent hallucination' phrasing is a bit strong when the SELinux denial is real, but the point stands: without an independent oracle reading the execution logs, a successful run cannot distinguish a real unsafe outcome from a bug in the model-written detection code, nor a genuine denial from the model changing its tool calls. The paper also had to jailbreak its own orchestrator before it would cooperate, so the entity evaluated is not identical to the moderated deployment agent. The abstract's 'baseline of safety guarantee' language overstates what a few self-validated workflows support.\n\nThe paper is for people working on LLM agent safety who want an early, honest sketch of a self-evaluating-agent approach. It is not a benchmark or a system paper. I would not cite it as evidence that AgentGuard works, but I might cite it as an early exploration of the idea. It deserves a serious referee if positioned as a workshop or exploratory paper; the core question is worth asking and the limitations are stated clearly. The referee request should insist on reproducible artifacts, quantitative success rates, and an independent log-based oracle. Without those, the central feasibility claim remains plausible but unproven.","headline":"An honest, clearly written hackathon prototype with a genuinely interesting core idea—using the target orchestrator as its own safety evaluator—but the evidence is too thin and self-referential to call the feasibility case closed.","tokens_in":6571,"tokens_out":3307,"would_cite":true,"duration_ms":33168,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AgentGuard claims an LLM agent's own orchestrator can serve as its safety evaluator, discovering unsafe tool workflows, validating them by real execution, and producing tested constraints for deployment.","keywords":["LLM agents","tool orchestration safety","self-evaluation","unsafe workflow discovery","safety constraints","SELinux","agent red-teaming","pre-deployment hardening"],"falsifier":"Take a coding agent with a known unsafe tool sequence that a manual red team can execute to overwrite a protected file; run AgentGuard on that exact configuration many times, and if the pipeline never validates that workflow and never applies a successfully blocking SELinux rule across all runs, the central feasibility claim for the hardening loop is falsified.","tokens_in":5620,"feed_emoji":"🛡️","tokens_out":6408,"duration_ms":58298,"temperature":0.7,"pith_summary":"AgentGuard is a proposed pre-deployment testing and hardening pipeline for LLM agents that use tools. Its central idea is to reuse the agent's own orchestrator as the safety evaluator: the orchestrator already knows the tools, can generate realistic workflows, and holds execution privileges, so the same entity that plans tool calls can be asked to plan unsafe ones, execute them, and confirm the harm. The pipeline runs four phases—identify unsafe workflows, validate them by real execution, generate safety constraints, and validate those constraints—and outputs an evaluation report with confirmed unsafe workflows, test cases, and constraints that are ready to enforce. The paper demonstrates feasibility with a prototype built around a coding agent, reporting that successful end-to-end runs occurred even though many runs failed because the evaluated LLM could not reliably produce applicable SELinux rules.","feed_headline":"Agent's own planner becomes its safety auditor","feed_subtitle":"Four-phase pipeline finds, executes, and blocks unsafe tool workflows before deployment.","key_machinery":"The central object is the LLM-based orchestrator inside the target agent, repurposed as its own safety evaluator. It carries the argument because it already satisfies the three requirements the authors argue any such framework must have: internal knowledge of tool capabilities and usage, the ability to generate workflows that realistically match what the agent would actually do, and the privilege to invoke tools for concrete validation. The pipeline is coordinated by a centralized Prompting Proxy Agent and a Safety Constraint Expert Agent, with feedback loops that request regeneration when generated content is low quality or execution fails. Two prompt-level mechanisms were needed to make the prototype work: Role Augmentation to get past the orchestrator's initial refusal to perform the evaluation task, and Chain-of-Thought prompting, asking the orchestrator to list and reflect on tools before identifying workflows, to suppress hallucinated tool capabilities.","core_discovery":"On its own terms, the paper claims that an LLM-based agentic orchestrator can be repurposed, before deployment, into the safety evaluator of its own tool orchestration. The orchestrator is instructed to reflect on its tools, produce unsafe workflows that violate security principles in given task scenarios, turn those workflows into executable test cases, execute them to check whether the predicted harm actually occurs, and then re-execute them under generated safety constraints to confirm the constraints block the harm. The evaluation report thereby contains validated unsafe workflows, violated security principles, test cases, and validated constraints, which can be enforced as a deployment-time safety baseline. The empirical result is a feasibility proof-of-concept: with a coding agent as the target, the pipeline ran end to end successfully at least a few times, despite many runs failing at the step of applying LLM-generated SELinux rules, a failure the authors attribute to the evaluated LLM rather than to the design.","pith_inferences":["A natural extension is to use the same self-evaluation loop continuously at deployment with a shadow orchestrator, turning each real interaction into a latent test case rather than auditing only once before release.","The initial refusal and tool hallucination observed in the prototype suggest a general precondition for self-auditing: the orchestrator's system prompt must be audited or temporarily stripped of defensive role assignments, and the actual tool registry should be fed back into the prompt before workflow generation.","A more robust design would separate the evaluator model from the target agent's model, or pair the orchestrator with a tool-knowledge retriever, to avoid relying on the same LLM for both planning and rule generation.","An attacker will not ask a compliant orchestrator politely, so the identification phase could be extended with goal-injection prompts that simulate adversarial hijacking, testing workflows the agent would only produce under compromise."],"forward_implications":["A deployable agent can be shipped with a validated set of safety constraints, such as SELinux rules, that confine its behavior even if the agent is later compromised.","Re-running the four phases produces a quantitative measure of safety improvement by comparing unsafe-outcome occurrences before and after constraints are applied.","Collected unsafe workflows and test cases for one agent family can be pooled into a benchmark corpus and shared as threat intelligence for similar agents.","Accumulated evaluation reports can serve as training samples to fine-tune LLMs for safer tool orchestration.","Because the method rests on the general design of LLM agents, it is intended to transfer from software agents to agents that interact with the physical world, such as robotic systems."],"supporting_citations":[{"why":"Supplies the prior LLM-emulated sandbox approach that AgentGuard contrasts with as insufficiently realistic and insufficiently concrete, motivating the self-orchestrator design.","marker":"[5]"},{"why":"Provides the target coding agent used in the feasibility evaluation of the four-phase pipeline.","marker":"[3]"},{"why":"Provides the SELinux embodiment used as the safety constraint mechanism that generated rules must be applicable to and validated against.","marker":"[4]"}],"fun_headline_variants":["Agent's planner audits its own tool workflows","Self-safety evaluation for tool-using agents","LLM orchestrator finds and blocks unsafe tools","Four-phase pipeline validates agent safety constraints","Repurposing the orchestrator as its own auditor"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the target agent's orchestrator is not compromised and will actively comply with requests to expose unsafe workflows, and in the prototype this premise failed initially—the orchestrator refused the task and had to be jailbroken with Role Augmentation—so if a real agent hides or distorts its unsafe workflows, or the same LLM cannot turn constraints into executable rules, the resulting evaluation report does not cover the agent's actual risk surface.","fun_headline_variants_meta":{"raw":{"variants":["Agent's planner audits its own tool workflows","Self-safety evaluation for tool-using agents","LLM orchestrator finds and blocks unsafe tools","Four-phase pipeline validates agent safety constraints","Repurposing the orchestrator as its own auditor"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000267,"raw_usage":{"total_tokens":1600,"prompt_tokens":913,"completion_tokens":687,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":529,"completion_tokens_details":{"reasoning_tokens":618}},"tokens_in":529,"tokens_out":687,"duration_ms":7052,"temperature":1.0,"reasoning_tokens":618,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T20:24:37.184333+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a coding agent with a known unsafe tool sequence that a manual red team can execute to overwrite a protected file; run AgentGuard on that exact configuration many times, and if the pipeline never validates that workflow and never applies a successfully blocking SELinux rule across all runs, the central feasibility claim for the hardening loop is falsified.","supporting_citations":[{"cited_title":"Maddison, and Tatsunori Hashimoto","cited_arxiv_id":null,"evidence_quote":"Supplies the prior LLM-emulated sandbox approach that AgentGuard contrasts with as insufficiently realistic and insufficiently concrete, motivating the self-orchestrator design."},{"cited_title":"Aider: Ai pair programming in your terminal","cited_arxiv_id":null,"evidence_quote":"Provides the target coding agent used in the feasibility evaluation of the four-phase pipeline."},{"cited_title":"Security-enhanced linux (selinux), 2000","cited_arxiv_id":null,"evidence_quote":"Provides the SELinux embodiment used as the safety constraint mechanism that generated rules must be applicable to and validated against."}],"review_version":1}