Pith. sign in

REVIEW 3 major objections 5 minor 66 references

Individually benign CLI commands can compose through shared OS state to compromise LLM coding agents at 96.59% success under ordinary developer tasks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 06:33 UTC pith:2X7STFOY

load-bearing objection CCR is a real, well-separated attack surface; the ~97% ASR is strong evidence that agents will run these compositions, with the main caveat that the 101 paths are curated for agent-executable workflows rather than a random live sample. the 3 major comments →

arxiv 2607.02857 v1 pith:2X7STFOY submitted 2026-07-03 cs.CR cs.SE

MOSAIC: Knowledge-Guided CLI Command Composition Attack in LLM Coding Agents

Jiangrong Wu , Huaijin Wang , Yihao Zhang , Yuhong Nan , Shuai Wang This is my paper
classification cs.CR cs.SE
keywords LLM coding agentsCLI securitycommand compositionCLI command-composition riskagent securityindirect prompt injectionprovenance-aware defense
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

LLM coding agents complete work by issuing ordinary CLI commands that cooperate through shared operating-system state, following classic Unix design. The paper argues this intended composition creates an overlooked attack surface: CLI command-composition risk (CCR), in which each command the agent runs looks legitimate in isolation, yet a producer-consumer handoff across the trace yields remote code execution, credential leakage, or boundary escape. Existing attacks and defenses focus on hostile text or single actions and therefore miss the cross-command relation. MOSAIC turns validated command-state behaviors from CVEs, advisories, and researcher PoCs into reusable summaries, composes them into exploit paths of length two to four, and packages each path as a realistic developer workflow for black-box testing. Across five real CLI coding agents and five backend models in 2,525 trials, the method reaches 96.59% end-to-end success while matched instruction-layer baselines stay near two percent and five representative deployed defenses leave most attacks intact.

Core claim

Real-world LLM coding agents are broadly susceptible to CLI command-composition risk: chains of individually benign CLI commands can, through shared operating-system state, produce security-sensitive capabilities outside the user's task. A knowledge-guided pipeline that distills multi-source CLI security evidence into command-state summaries and composes them into realistic developer workflows achieves near-ceiling black-box attack success under benign prompts, far above instruction-layer baselines and residual risk under current defenses.

What carries the argument

CLI command-composition risk (CCR): the gap that appears only when agent-run commands that are each benign in isolation form a producer-consumer state dependency over shared OS state and yield an out-of-scope capability. MOSAIC carries the argument by distilling CVE, advisory, and PoC evidence into reusable command-state summaries, enumerating feasible producer-consumer paths, and instantiating them as black-box evaluation workflows.

Load-bearing premise

The 101 exploit paths drawn from curated CVEs and PoCs are still live on current CLI tools and representative enough that success on them generalizes to real attacker capability, rather than reflecting a selected set of compositions agents happen to follow when the task is packaged as ordinary developer work.

What would settle it

Re-run the same 101 instances against the five agents and five models on locked modern git, npm, and bash versions; if end-to-end success collapses toward the instruction-layer baseline of a few percent, or if most paths fail because the underlying command-state mechanisms are already patched out, the central claim does not hold.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Instruction-layer filters, single-command scanners, and tool-call information-flow monitors leave residual attack success above 82 percent because they never observe the cross-command state handoff.
  • A defense that works for this class must become provenance-aware at the CLI runtime: tag state written from untrusted resources and block later consumption that creates out-of-scope capability.
  • High success across five agents and five backend models indicates the exposure is structural to CLI-based coding agents, not an artifact of one model family.
  • Structured command-state knowledge substantially raises instance validity and triggerability compared with pure LLM generation or raw retrieval over uncurated records.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Any agent that orchestrates stateful tools—not only shell—may face analogous composition risk when one tool writes session, filesystem, or config state that a later tool trusts.
  • Ephemeral least-privilege environments that reset hooks, environment variables, and package lifecycle scripts between steps could blunt many of the short paths without changing the agent model.
  • The same distilled knowledge base used for attack generation could be inverted into a static or runtime linter that flags known producer-consumer pairs before the agent commits to a workflow.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper identifies CLI command-composition risk (CCR): individually benign CLI commands that, through shared OS state (env vars, hooks, lifecycle scripts, config, etc.), compose into a security-sensitive capability outside the user task. It formalizes producer/consumer state dependencies over a command trace and presents MOSAIC, which crawls multi-source CLI security evidence (NVD, GHSA, Exploit-DB, KEV, blogs), filters to 454 CLI-relevant entries, distills them into command-state summaries clustered into 13 families, enumerates producer-consumer paths of length 2–4, and instantiates them as black-box attack instances (benign prompt + attacker-controlled resource + expected trace + external marker). On five real-world CLI coding agents and five backend LLMs (2,525 trials), MOSAIC reports 96.59% end-to-end ASR under benign developer tasks, far above matched instruction-layer baselines (≤2.18%), with ablations showing the structured KB is essential and five representative defenses leaving residual ASR of 82.57%–96.63%.

Significance. If the result holds, the paper surfaces a structural attack surface orthogonal to instruction-layer prompt injection: the intended Unix composition model that coding agents rely on. Strengths include a large multi-agent/multi-LLM evaluation, dual oracles (semantic trace match + deterministic external-effect markers), matched instruction-layer baselines under the same threat model, structured-KB ablations (Table III), and a defense residual analysis (Table V) that motivates provenance-aware monitoring over state relations. The knowledge-guided methodology is a concrete contribution beyond ad-hoc PoCs and is useful for agent developers self-checking products. The work is timely given the rapid adoption of CLI coding agents and recent real-world compromises cited in the threat model.

major comments (3)
  1. Section V and VIII-A: The central claim that CCR is broadly exploitable rests on 101 evidence-grounded paths distilled from 454 retained entries, with the statement that entries are discarded if a patch removes the underlying state interface. The manuscript does not report an independent liveness check that the retained mechanisms (e.g., core.hooksPath, package lifecycle scripts, env redirection) still fire under the exact CLI versions used by the five agents. Without that verification (or a version matrix), the 96.59% ASR primarily shows that agents execute these particular materializable workflows, not that an attacker planting content can reliably exploit live, unpatched CLI mechanisms. A short appendix confirming mechanism liveness on the evaluation environment would make the generalization claim load-bearing rather than assumed.
  2. Section VIII-B and the failure analysis: Failures are attributed to planning stochasticity (agent takes an alternative valid path), and forcing the intended path still succeeds. Combined with L_max=4 and the length mix (21/35/45), this raises a selection concern: the 101 instances may be those for which a benign developer workflow naturally drives the agent onto the producer-consumer chain. The paper should quantify how instances were filtered for agent-executability versus pure mechanism feasibility, and report ASR when the agent is free to choose any valid workflow without path forcing, so that reported success is not inflated by design for agent-friendly compositions.
  3. Section III-B (L1–L3) and VI-C: Local benignness (L1) and Scope(p) are defined informally. The static validator checks that “the prompt and visible commands remain plausible for the developer task,” but there is no operational criterion or inter-rater check for when a command is within Scope(p) versus when the composed capability is out of scope. Because ASR success depends on this distinction (and on distinguishing CCR from classic supply-chain delivery), a clearer operationalization—or at least explicit examples of borderline cases rejected by the validator—is needed for the formal claim that risk emerges only through composition.
minor comments (5)
  1. Figure 2 and Section II-B: The motivating Git hooksPath example is clear; a short note on whether the same pattern is blocked by modern default Git configs or agent sandboxes would help readers assess residual risk.
  2. Table I: Report absolute counts per length cell (not only percentages) for easier audit of the 105/175/225 trial groups.
  3. Section V-A: The two-dimension LLM rubric and CWE blacklist are described at a high level; listing the whitelist tools and the CWE blacklist (or a pointer to an appendix) would improve reproducibility of the 30,180→454 filter.
  4. Section IX: The provenance-aware defense sketch is useful but brief; even a one-paragraph threat model for the monitor (what it can observe at the agent command layer) would strengthen the discussion.
  5. References and dates: Several “accessed: 2026-…” and model names (gpt-5.1, deepseek-v4-flash, qwen3.7-plus) should be double-checked for consistency with public naming at submission time.

Circularity Check

0 steps flagged

No circularity: high ASR is an external black-box measurement, not forced by the knowledge-base definitions or generation procedure.

full rationale

MOSAIC's central claim is an empirical end-to-end attack success rate (96.59% over 2,525 trials) measured against five real CLI coding agents and five backend LLMs under a black-box threat model. The CLI security knowledge base supplies candidate producer-consumer mechanisms distilled from external CVEs/advisories/PoCs; instance generation and static validation ensure schema/evidence consistency, but success is decided by an independent runtime oracle (TraceOK ∧ EffectOK) that requires the agent to actually invoke the composition and produce a deterministic external marker. Ablations confirm the result is not by construction: LLM-only generation collapses to 0.59% end-to-end ASR. Definitions of CCR (L1–L3) and the L_max=4 bound are methodological choices that do not algebraically force the reported rates. There are no self-definitional equations, fitted parameters re-labeled as predictions, load-bearing self-citations of uniqueness theorems, or renamings of known results. The derivation chain is therefore self-contained against external agent behavior.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 3 invented entities

The central empirical claim rests on a curated, partially manual knowledge pipeline and design bounds rather than free physical constants. Load-bearing choices are the max path length, the CLI-relevance filter rubric, and the assumption that historical CLI mechanisms still present on agent hosts compose under “benign” developer prompts. Invented entities are methodological (CCR, command-state summaries/families, MOSAIC), with independent handle via external agent trials rather than pure postulate.

free parameters (3)
  • L_max (max exploit-path length)
    Hard-coded to 4 commands as a coverage/tractability trade-off (Section VI-A); longer paths are argued to mostly repeat patterns but are not exhaustively searched.
  • CLI relevance filter thresholds / two-dimension LLM rubric
    Author-designed shell-nativeness and substantive-PoC scoring plus whitelist/blacklist reduce 21,149 candidates to 5,583 then 454 confirmed entries (Section V-A); cutoffs are not externally fixed.
  • Exploit-path set size and length mix (101 paths: 21/35/45 at n=2/3/4)
    Instance budget and length distribution are construction choices that define the evaluation surface for the headline ASR.
axioms (5)
  • domain assumption CLI commands form a stateful trace over shared OS state Σ with producer/consumer roles grounded in official CLI documentation (git/npm/bash manuals).
    Section III-A formalizes ci ⇝ cj via Pi ∩ Qj ≠ ∅; this is standard Unix semantics treated as given.
  • domain assumption Attacker can only plant content the agent may reasonably process during benign development (public repo, package metadata, docs, etc.), not control the agent runtime or issue commands directly.
    Threat model Section III-C; required for black-box evaluation design and comparison to IPI baselines.
  • ad hoc to paper A retained CVE/advisory/PoC is usable as mechanism evidence if the underlying state interface still exists after patches that only fix a specific exploit instance.
    Section V-A: entries discarded only if the mechanism/state interface itself is removed; this selection rule is paper-specific.
  • domain assumption Deterministic external-effect markers fire only if the terminal capability truly occurred (code execution, escape, persistence, etc.).
    Oracle design Section VIII-A; standard for capability confirmation but assumes marker placement cannot be spuriously satisfied.
  • ad hoc to paper LLM-assisted extraction/generation plus static validators and manual review yield faithful command-state summaries and feasible instances.
    Sections V–VI; human-in-the-loop fidelity is assumed rather than measured with agreement statistics.
invented entities (3)
  • CLI command-composition risk (CCR) independent evidence
    purpose: Name and formalize the risk that benign agent-run commands compose via shared state into out-of-scope security capabilities (L1–L3).
    Core conceptual contribution; independent handle is empirical ASR on external agents, not a new physical object.
  • Command-state summary / 13 command-state families no independent evidence
    purpose: Reusable abstraction (template, trigger, produced/consumed state, capability, evidence) organizing heterogeneous CVE/PoC knowledge for composition.
    Methodological clustering of 454 entries into 13 families (Section V-B); families are paper constructs over real CLI mechanisms.
  • MOSAIC framework no independent evidence
    purpose: End-to-end pipeline: KB construction, chain generation/validation, black-box runtime oracle for CCR assessment.
    System contribution enabling the reported attack instances; evidence is the evaluation, not external prior existence of MOSAIC.

pith-pipeline@v1.1.0-grok45 · 23425 in / 3673 out tokens · 42301 ms · 2026-07-12T06:33:09.899340+00:00 · methodology

0 comments
read the original abstract

LLM coding agents increasingly complete development tasks by issuing ordinary CLI commands. Following the Unix design, these commands cooperate through shared operating-system state: one command may write state that a later command reads. While this composition is benign and intended, it creates an overlooked exploit surface. Existing attacks and defenses mainly target the instruction layer, where malicious intent appears as hostile text. In contrast, we observe that individually benign commands can form a dangerous producer-consumer state relation across the command trace, exposing what we call CLI command-composition risk (CCR). Given this new attack surface, it is critical to systematically uncover and characterize the impact of CCR in real-world coding agents. However, systematically understanding this risk is quite challenging, because naive command enumeration and end-to-end LLM generation produce mostly invalid workflows. We present MOSAIC, a knowledge-guided framework that distills validated command-state behaviors from CVEs, advisories, and researcher PoCs into reusable summaries, composes them into exploit paths, and instantiates them as realistic developer workflows for black-box agent evaluation. Across five real-world CLI coding agents and five backend LLMs over 2,525 trials, MOSAIC achieves a 96.59% attack success rate under benign developer tasks.

Figures

Figures reproduced from arXiv: 2607.02857 by Huaijin Wang, Jiangrong Wu, Shuai Wang, Yihao Zhang, Yuhong Nan.

Figure 1
Figure 1. Figure 1: Among existing LLM coding agents, user tasks are [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: shows a concrete instance. The user asks the agent to clone and set up an attacker-controlled repository whose README requests a one-line setup step, git config core.hooksPath .githooks, and which ships a .githooks/pre-commit hook contain￾ing malicious_command. The setup command writes the hooks-path directive into the on-disk .git/config; later, when the agent commits during its normal workflow, git commi… view at source ↗
Figure 3
Figure 3. Figure 3: Workflow of MOSAIC: a three-module pipeline that assess the security boundary of LLM coding agent. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

66 extracted references · 16 linked inside Pith

  1. [1]

    Claude Code: Configure permissions,

    Anthropic, “Claude Code: Configure permissions,” https://code.claude. com/docs/en/permissions, 2026, accessed: 2026-06-08

    2026

  2. [2]

    Codex CLI,

    OpenAI, “Codex CLI,” https://developers.openai.com/codex/cli, 2026, accessed: 2026-06-08

    2026

  3. [3]

    Gemini CLI documentation,

    Google, “Gemini CLI documentation,” https://developers.google.com/ gemini-code-assist/docs/gemini-cli, 2026, accessed: 2026-06-08

    2026

  4. [4]

    Getting started with GitHub Copilot CLI,

    GitHub, “Getting started with GitHub Copilot CLI,” https://docs.githu b.com/copilot/how-tos/copilot-cli/cli-getting-started, 2026, accessed: 2026-06-08

    2026

  5. [5]

    Trae Agent: An llm-based agent for software engineering,

    ByteDance, “Trae Agent: An llm-based agent for software engineering,” https://github.com/bytedance/trae-agent, 2026, accessed: 2026-06-08

    2026

  6. [6]

    The UNIX time-sharing system: Foreword,

    M. D. McIlroy, E. N. Pinson, and B. A. Tague, “The UNIX time-sharing system: Foreword,”The Bell System Technical Journal, vol. 57, no. 6, pp. 1899–1904, 1978

    1904

  7. [7]

    E. S. Raymond,The Art of UNIX Programming. Addison-Wesley, 2003

    2003

  8. [8]

    Amazon AI coding agent hacked to inject data- wiping commands,

    BleepingComputer, “Amazon AI coding agent hacked to inject data- wiping commands,” https://www.bleepingcomputer.com/news/security /amazon-ai-coding-agent-hacked-to-inject-data-wiping-commands/, 2025, amazon Q Developer extension v1.84.0; AWS bulletin AWS-2025-

    2025

  9. [9]

    Accessed: 2026-06-26

    2026

  10. [10]

    s1ngularity: Supply chain attack leaks secrets on GitHub,

    Wiz Research, “s1ngularity: Supply chain attack leaks secrets on GitHub,” https://www.wiz.io/blog/s1ngularity-supply-chain-attack, 2025, over 1,000 GitHub tokens leaked; 5,500+ private repositories exposed. Accessed: 2026-06-26

    2025

  11. [11]

    CamoLeak: Critical GitHub Copilot vulnerability leaks private source code,

    Legit Security, “CamoLeak: Critical GitHub Copilot vulnerability leaks private source code,” https://www.legitsecurity.com/blog/camoleak-c ritical-github-copilot-vulnerability-leaks-private-source-code, 2025, cVSS 9.6; reported by Omer Mayraz (Legit Security); GitHub disabled Copilot Chat image rendering on 2025-08-14. Accessed: 2026-06-26

    2025

  12. [12]

    New vulnerability in GitHub Copilot and Cursor: How hackers can weaponize code agents,

    Pillar Security, “New vulnerability in GitHub Copilot and Cursor: How hackers can weaponize code agents,” https://www.pillar.security/blog/n ew-vulnerability-in-github-copilot-and-cursor-how-hackers-can-wea ponize-code-agents, 2025, rules File Backdoor; MITRE ATLAS case AML.CS0041. Accessed: 2026-06-26

    2025

  13. [13]

    Caught in the hook: RCE and API token exfiltration through Claude Code project files,

    Check Point Research, “Caught in the hook: RCE and API token exfiltration through Claude Code project files,” https://research.che ckpoint.com/2026/rce-and-api-token-exfiltration-through-claude-cod e-project-files-cve-2025-59536/, 2025, cVE-2025-59536 (CVSS 8.7), CVE-2026-21852. Accessed: 2026-06-26

    2026

  14. [14]

    CVE-2025-53773: GitHub Copilot and Visual Studio remote code execution via prompt injection,

    NIST National Vulnerability Database, “CVE-2025-53773: GitHub Copilot and Visual Studio remote code execution via prompt injection,” https://nvd.nist.gov/vuln/detail/CVE-2025-53773, 2025, cWE-77; CVSS 7.8; writes chat.tools.autoApprove to enable YOLO mode (per Embrace The Red). Accessed: 2026-06-26

    2025

  15. [16]

    Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for llm agents,

    E. Debenedetti, J. Zhang, M. Balunovi ´c, L. Beurer-Kellner, M. Fis- cher, and F. Tramèr, “Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for llm agents,”arXiv preprint arXiv:2406.13352, 2024

    Pith/arXiv arXiv 2024

  16. [17]

    Demystifying rce vulnerabilities in llm-integrated apps,

    T. Liu, Z. Deng, G. Meng, Y . Li, and K. Chen, “Demystifying rce vulnerabilities in llm-integrated apps,” inProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, 2024, pp. 1716–1730

    2024

  17. [18]

    Wainjectbench: Bench- marking prompt injection detections for web agents,

    Y . Liu, R. Xu, X. Wang, Y . Jia, and N. Z. Gong, “Wainjectbench: Bench- marking prompt injection detections for web agents,”arXiv preprint arXiv:2510.01354, 2025

    arXiv 2025

  18. [19]

    Wasp: Benchmarking web agent security against prompt injection attacks,

    I. Evtimov, A. Zharmagambetov, A. Grattafiori, C. Guo, and K. Chaud- huri, “Wasp: Benchmarking web agent security against prompt injection attacks,”arXiv preprint arXiv:2504.18575, 2025

    Pith/arXiv arXiv 2025

  19. [20]

    "your ai, my shell

    Y . Liu, Y . Zhao, Y . Lyu, T. Zhang, H. Wang, and D. Lo, “"your ai, my shell": Demystifying prompt injection attacks on agentic ai coding editors,” 2025. [Online]. Available: https://arxiv.org/abs/2509.22040

    Pith/arXiv arXiv 2025

  20. [22]

    {StruQ}: Defending against prompt injection with structured queries,

    S. Chen, J. Piet, C. Sitawarin, and D. Wagner, “{StruQ}: Defending against prompt injection with structured queries,” in34th USENIX Security Symposium (USENIX Security 25), 2025, pp. 2383–2400

    2025

  21. [23]

    Secalign: Defending against prompt injection with preference optimization,

    S. Chen, A. Zharmagambetov, S. Mahloujifar, K. Chaudhuri, D. Wag- ner, and C. Guo, “Secalign: Defending against prompt injection with preference optimization,” inProceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security, 2025, pp. 2833–2847

    2025

  22. [24]

    Drift: Dynamic rule-based defense with injection isolation for securing llm agents,

    H. Li, X. Liu, H.-C. Chiu, D. Li, N. Zhang, and C. Xiao, “Drift: Dynamic rule-based defense with injection isolation for securing llm agents,”arXiv preprint arXiv:2506.12104, 2025

    arXiv 2025

  23. [25]

    Prompt flow integrity to prevent privilege escalation in llm agents,

    J. Kim, W. Choi, and B. Lee, “Prompt flow integrity to prevent privilege escalation in llm agents,”arXiv preprint arXiv:2503.15547, 2025

    Pith/arXiv arXiv 2025

  24. [26]

    Progent: Programmable privilege control for llm agents, 2025,

    T. Shi, J. He, Z. Wang, H. Li, L. Wu, W. Guo, and D. Song, “Progent: Programmable privilege control for llm agents, 2025,”URL https://arxiv. org/abs/2504.11703, 2025

    Pith/arXiv arXiv 2025

  25. [27]

    Terminal-Bench: Benchmarking agents on hard, realistic tasks in command line interfaces,

    M. A. Merrillet al., “Terminal-Bench: Benchmarking agents on hard, realistic tasks in command line interfaces,” 2026. [Online]. Available: https://arxiv.org/abs/2601.11868

    Pith/arXiv arXiv 2026

  26. [28]

    Husky: Git hooks made easy,

    typicode, “Husky: Git hooks made easy,” https://github.com/typicode/ husky, 2025, accessed: 2026-06-15

    2025

  27. [29]

    githooks: Hooks used by git,

    Git, “githooks: Hooks used by git,” https://git-scm.com/docs/githooks, 2025, accessed: 2026-06-15

    2025

  28. [30]

    Getting started with git hooks and husky,

    git-tower, “Getting started with git hooks and husky,” https://www.git-t ower.com/blog/git-hooks-husky, 2024, accessed: 2026-06-15

    2024

  29. [31]

    Universal and transferable adversarial attacks on aligned language models,

    A. Zou, Z. Wang, N. Carliniet al., “Universal and transferable adversarial attacks on aligned language models,”arXiv preprint arXiv:2307.15043, 2023

    Pith/arXiv arXiv 2023

  30. [32]

    Ignore previous prompt: Attack techniques for language models,

    F. Perez and I. Ribeiro, “Ignore previous prompt: Attack techniques for language models,” inNeurIPS ML Safety Workshop, 2022

    2022

  31. [33]

    Not what you’ve signed up for: Compromising real-world llm- integrated applications with indirect prompt injection,

    K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz, “Not what you’ve signed up for: Compromising real-world llm- integrated applications with indirect prompt injection,”arXiv preprint arXiv:2302.12173, 2023

    Pith/arXiv arXiv 2023

  32. [34]

    Promptarmor: Simple yet effective prompt injection defenses,

    T. Shi, K. Zhu, Z. Wang, Y . Jia, W. Cai, W. Liang, H. Wang, H. Alzahrani, J. Lu, K. Kawaguchi, B. Alomair, X. Zhao, W. Y . Wang, N. Z. Gong, W. Guo, and D. Song, “Promptarmor: Simple yet effective prompt injection defenses,”arXiv preprint arXiv:2507.15219, 2025

    Pith/arXiv arXiv 2025

  33. [35]

    Llamafirewall: An open source guardrail system for building secure ai agents,

    S. Chennabasappaet al., “Llamafirewall: An open source guardrail system for building secure ai agents,”arXiv preprint arXiv:2505.03574, 2025

    Pith/arXiv arXiv 2025

  34. [36]

    Defending against indirect prompt injection attacks with spotlighting,

    K. Hineset al., “Defending against indirect prompt injection attacks with spotlighting,”arXiv preprint arXiv:2403.14720, 2024

    Pith/arXiv arXiv 2024

  35. [37]

    The instruction hierarchy: Training llms to prioritize privileged instructions,

    E. Wallaceet al., “The instruction hierarchy: Training llms to prioritize privileged instructions,”arXiv preprint arXiv:2404.13208, 2024

    Pith/arXiv arXiv 2024

  36. [38]

    Instructional segment embedding: Improving LLM safety with instruction hierarchy,

    T. Wuet al., “Instructional segment embedding: Improving LLM safety with instruction hierarchy,” inInternational Conference on Learning Representations (ICLR), 2025

    2025

  37. [39]

    Jatmo: Prompt injection defense by task-specific fine- tuning,

    J. Pietet al., “Jatmo: Prompt injection defense by task-specific fine- tuning,” inEuropean Symposium on Research in Computer Security (ESORICS), 2024

    2024

  38. [40]

    The task shield: Enforcing task alignment to defend against indirect prompt injection in llm agents,

    F. Jia, T. Wu, X. Qin, and A. Squicciarini, “The task shield: Enforcing task alignment to defend against indirect prompt injection in llm agents,” arXiv preprint arXiv:2412.16682, 2024

    Pith/arXiv arXiv 2024

  39. [41]

    Contextual agent security: A policy for every purpose,

    L. Tsaiet al., “Contextual agent security: A policy for every purpose,” inWorkshop on Hot Topics in Operating Systems (HotOS), 2025

    2025

  40. [42]

    Shellcheck: A shell script static analysis tool,

    V . Holen and contributors, “Shellcheck: A shell script static analysis tool,” https://github.com/koalaman/shellcheck, 2026, accessed: 2026- 06-11

    2026

  41. [43]

    GTFOBins: Curated unix binaries for bypassing local security restrictions,

    E. Pinna, A. Cardaci, and contributors, “GTFOBins: Curated unix binaries for bypassing local security restrictions,” https://gtfobins.org/, 2026, accessed: 2026-06-11

    2026

  42. [44]

    LOLBAS: Living off the land binaries, scripts and libraries,

    LOLBAS Project, “LOLBAS: Living off the land binaries, scripts and libraries,” https://lolbas-project.github.io/, 2026, accessed: 2026-06-11

    2026

  43. [45]

    Git reference documentation,

    Git, “Git reference documentation,” https://git-scm.com/docs, accessed: 2026-06-12

    2026

  44. [46]

    npm docs: Scripts,

    npm, Inc., “npm docs: Scripts,” https://docs.npmjs.com/cli/v11/using-n pm/scripts, accessed: 2026-06-12

    2026

  45. [47]

    GNU Bash reference manual,

    Free Software Foundation, “GNU Bash reference manual,” https://ww w.gnu.org/software/bash/manual/, accessed: 2026-06-12

    2026

  46. [48]

    SWE-bench: Can language models resolve real-world github issues?

    C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. Narasimhan, “SWE-bench: Can language models resolve real-world github issues?” inInternational Conference on Learning Representations (ICLR), 2024. 11

    2024

  47. [49]

    Claude code: Common workflows,

    Anthropic, “Claude code: Common workflows,” https://code.claude.co m/docs/en/common-workflows, 2025, accessed: 2026-06-12

    2025

  48. [50]

    National Vulnerability Database,

    National Institute of Standards and Technology, “National Vulnerability Database,” https://nvd.nist.gov/, accessed: 2026-06-05

    2026

  49. [51]

    GitHub Advisory Database,

    GitHub, “GitHub Advisory Database,” https://github.com/advisories, accessed: 2026-06-05

    2026

  50. [52]

    Exploit Database,

    OffSec, “Exploit Database,” https://www.exploit-db.com/, accessed: 2026-06-05

    2026

  51. [53]

    Known Exploited Vulnerabilities Catalog,

    Cybersecurity and Infrastructure Security Agency, “Known Exploited Vulnerabilities Catalog,” https://www.cisa.gov/known-exploited-vulnera bilities-catalog, accessed: 2026-06-05

    2026

  52. [54]

    Project Zero Blog,

    Google Project Zero, “Project Zero Blog,” https://projectzero.google/, accessed: 2026-06-05

    2026

  53. [55]

    The Trail of Bits Blog,

    Trail of Bits, “The Trail of Bits Blog,” https://blog.trailofbits.com/, accessed: 2026-06-05

    2026

  54. [56]

    Embrace The Red Blog,

    Embrace The Red, “Embrace The Red Blog,” https://embracethered.co m/blog/, accessed: 2026-06-05

    2026

  55. [57]

    GPT-5.1 model documentation,

    OpenAI, “GPT-5.1 model documentation,” https://developers.openai.co m/api/docs/models/gpt-5.1, 2026, accessed: 2026-06-08

    2026

  56. [58]

    Gemini 2.5 Flash model documentation,

    Google, “Gemini 2.5 Flash model documentation,” https://ai.google.dev/ gemini-api/docs/models/gemini-2.5-flash, 2026, accessed: 2026-06-08

    2026

  57. [59]

    Claude API models overview,

    Anthropic, “Claude API models overview,” https://docs.anthropic.com /en/docs/about-claude/models/overview, 2026, accessed: 2026-06-08

    2026

  58. [60]

    DeepSeek V4 Preview Release,

    DeepSeek, “DeepSeek V4 Preview Release,” https://api-docs.deepseek .com/news/news260424, 2026, accessed: 2026-06-08

    2026

  59. [61]

    Qwen3.7-Plus: Multimodal agent intelligence,

    Qwen Team, “Qwen3.7-Plus: Multimodal agent intelligence,” https://qw en.ai/blog?id=qwen3.7-plus, 2026, accessed: 2026-06-08

    2026

  60. [62]

    Prompt injection attack against llm- integrated applications,

    Y . Liu, G. Deng, Y . Li, K. Wang, Z. Wang, X. Wang, T. Zhang, Y . Liu, H. Wang, Y . Zheng, and Y . Liu, “Prompt injection attack against llm- integrated applications,”arXiv preprint arXiv:2306.05499, 2023

    Pith/arXiv arXiv 2023

  61. [63]

    Formalizing and benchmarking prompt injection attacks and defenses,

    Y . Liu, Y . Jia, R. Geng, J. Jia, and N. Z. Gong, “Formalizing and benchmarking prompt injection attacks and defenses,” in33rd USENIX Security Symposium (USENIX Security 24), 2024, pp. 1831–1847

    2024

  62. [64]

    Injecagent: Benchmark- ing indirect prompt injections in tool-integrated large language model agents,

    Q. Zhan, Z. Liang, Z. Ying, and D. Kang, “Injecagent: Benchmark- ing indirect prompt injections in tool-integrated large language model agents,” inFindings of the Association for Computational Linguistics: ACL 2024, 2024

    2024

  63. [65]

    GPQA: A graduate-level google-proof q&a benchmark,

    D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y . Pang, J. Dirani, J. Michael, and S. R. Bowman, “GPQA: A graduate-level google-proof q&a benchmark,” inConference on Language Modeling (COLM), 2024

    2024

  64. [66]

    Defeating prompt injections by design,

    E. Debenedettiet al., “Defeating prompt injections by design,”arXiv preprint arXiv:2503.18813, 2025

    Pith/arXiv arXiv 2025

  65. [67]

    Securing ai agents with information-flow control,

    M. Costaet al., “Securing ai agents with information-flow control,” arXiv preprint arXiv:2505.23643, 2025

    Pith/arXiv arXiv 2025

  66. [68]

    Semgrep documentation,

    Semgrep, “Semgrep documentation,” https://docs.semgrep.dev/, 2026, accessed: 2026-06-11. 12

    2026