Pith. sign in

REVIEW 4 major objections 6 minor 59 references

FlowGuard: From Signals to Evidence for MCP Security Detection

T0 review · 4 major / 6 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read FlowGuard claims that MCP security detection should verify execution-related risks through runtime evidence and handle semantic risks in metadata and returned content separately, and that its central adjudication rule—a finding is real only

desk verdict A well-built MCP scanner with a thorough evaluation, but the benchmark and the adjudication rule share the same evidence definition, so the absolute F1 numbers should be read with caution. read the letter →

arxiv 2607.14754 v1 pith:LP2LM5FE submitted 2026-07-16 cs.CR

Baichao An , Pei Chen , Geng Hong , Yueyue Chen , Mengying Wu This is my paper · ORCID
classification cs.CR
keywords ModelContextProtocolsecurityevidence-groundeddetectionruntimeprobingcommandinjectionfilesystemaccesstoolpoisoningpromptcredentialleakage
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FlowGuard argues that suspicious signals in MCP interactions are not enough: a credential-like string can be a placeholder, a command-like output can be reflection, and a rejection can be correct defense. Instead, the paper proposes an evidence-grounded scanner that verifies execution-related risks through runtime probing and adjudicates returned content by source attribution and behavioral expectation. On a new 1,880-case executable benchmark, it reports F1 scores of 0.879 for command injection and 0.942 for file system access, and on a real-world corpus it reports 523 findings across 326 servers. If true, this shifts MCP security from flagging suspicious text to confirming actual exploitability.

What carries the argument

The load-bearing mechanism is the adjudication rule R = S ∧ O ∧ ¬E, applied to every candidate response signal: signal identified, system-originated (not echo reflection), and not expected defensive behavior. Around this rule, the pipeline is a closed-loop verification cycle: Triage ranks parameters for risk; Recon sends schema-valid low-impact probes to extract backend fingerprints; Strike generates a single targeted payload per parameter and threat family; Analysis extracts candidate signals; Adjudication applies the evidence rule; Refinement updates a probe history and decides whether to continue, stop, or report. This loop is what converts raw responses into confirmed findings or rejecte

What would settle it

Re-label the 1,880 benchmark cases by inspecting backend source code or having independent human analysts decide vulnerability status, without reference to FlowGuard's adjudication rule, then rerun FlowGuard and measure F1; if the scores drop substantially below 0.879 and 0.942, the performance claim is partly an artifact of circular labeling. A simpler check: any server that returns the probe string verbatim should never be counted as a finding—if a scan reports it as credential leakage, that is a false positive by the paper's own definition.

Watch

Extended reading notes

Core claim

The paper's central claim is that a security finding in an MCP tool should be defined by observable evidence, not by suspicious semantics. FlowGuard operationalizes this with the rule R = S ∧ O ∧ ¬E: a real finding requires a signal that is identified, is system-originated rather than reflected from the probe, and is not expected behavior for the tool. The system combines semantic triage of tool metadata with a two-phase probing strategy—recon to fingerprint the backend using low-impact probes, then strike with targeted payloads—and a history-guided refinement loop that repairs rejected probes. Evaluated on a benchmark that pairs positive and negative cases with explicit evidence, it reaches

Load-bearing premise

The benchmark's ground-truth labels are defined using the same evidence categories that FlowGuard's adjudicator emits (R = S ∧ O ∧ ¬E), so the high F1 scores may partly reflect the scanner being evaluated against its own decision rule rather than an independent ground truth.

Editorial extensions

If this is right

  • Execution-related risks like command injection and file-system access can be confirmed from runtime evidence, closing the gap left by static and metadata-only scanners.
  • Credential leakage detection can stop treating placeholder-like strings as real leaks, reducing false positives while improving recall.
  • Semantic risks such as tool poisoning and prompt injection can be reported separately from execution findings, without asserting that a downstream agent will follow the injected instruction.
  • The recon-and-strike design narrows the probe space, cutting end-to-end latency by up to 2.23× compared with broader dynamic scanning while requiring fewer probes per true positive.
  • Bounded probing with termination rules makes the approach suitable for offline scanning and CI/CD integration.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The adjudication rule is a transferable recipe: any agent-tool protocol that exposes metadata and responses could use 'system-originated and unexpected' as a general finding criterion, not just MCP.
  • The benchmark's positive/negative pairing with explicit evidence is a reusable template for evaluating security scanners independently of the downstream agent model.
  • The reported over-conservative triage for command injection suggests a future variant that retains uncertain parameters in a lightweight probing path to recover missed recall.
  • If the ground-truth labeling concern is resolved, the same benchmark could serve as a community standard for comparing MCP scanners across labs.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents FlowGuard, a black-box MCP security scanner that separates semantic risk detection from execution-evidence verification. It operates through five stages: triage of tool metadata, recon probes to extract backend fingerprints, schema-valid strike probes, response analysis with evidence adjudication using rule R = S ∧ O ∧ ¬E, and history-guided refinement. The authors introduce an executable benchmark of 1,880 MCP cases across five categories (Command Injection, Tool Poisoning, Prompt Injection, Credential Leakage, File System Access) and report F1 scores of 0.879, 0.995, 0.957, 0.864, and 0.942, respectively, outperforming MCPScan, MCP-Scanner, and A.I.G, with up to 2.23× lower latency than A.I.G. On 8,000 real-world MCP servers, FlowGuard reports 523 findings in 326 servers; manual verification of 100 servers confirms concrete evidence in 84, and one case (MarkItDown file:// disclosure) was externally acknowledged. The paper claims that evidence-grounded detection can assess both execution-related and semantic risks in MCP interactions.

Significance. If the results hold, FlowGuard would be a useful contribution: it directly addresses a real limitation of signal-based MCP scanners, and the structured verification loop with explicit adjudication is a principled design. The paper's strengths include extensive ablations (triage/recon/feedback, E1/E2 adjudication), an LLM-backbone sensitivity study, a novel scanner-level benchmark, and one externally confirmed real-world case. The main weakness is that the benchmark's ground truth and FlowGuard's decision rule share the same evidence definition, so the absolute F1 numbers may overstate performance against an independent standard. Reproducibility is also currently limited by the absence of a released artifact. The central idea is defensible; the evaluation needs strengthening before the headline claims can be accepted.

major comments (4)
  1. [§IV-A and §III-D2/Prompt C] The benchmark's ground-truth labels and FlowGuard's adjudication rule are defined by the same operational evidence criterion. Section IV-A labels a case positive when it has 'category-specific evidence, such as command markers, fixture file contents, synthetic canaries, or instruction patterns' and negative when the safe implementation 'should not produce the corresponding evidence.' Prompt C defines a real finding as R = S ∧ O ∧ ¬E (signal identified, system-originated, not expected behavior). These are two formulations of the same test. The F1 values in Table II therefore measure, to a large degree, how faithfully FlowGuard reproduces the authors' evidence taxonomy rather than how well it detects vulnerabilities against an independent ground truth. Hard negatives mitigate this partially but do not remove the overlap. Please (i) release the benchmark and show label generation did not us
  2. [§V-A, Table II] The benchmark is not neutral across detection modalities. Since positive labels are defined by runtime response evidence, static (MCPScan) and metadata-only (MCP-Scanner) baselines are evaluated on a target that their design cannot in principle satisfy; their very low F1 scores in Table II are partly an artifact of benchmark design rather than a measure of detection quality. The comparison against A.I.G, which also probes at runtime, is more meaningful. I recommend restricting the headline comparison to dynamic baselines, or providing modality-appropriate ground truth (e.g., code-level labels for static scanners) so that the reported 'best overall results' reflect detection accuracy rather than modality fit.
  3. [§IV, §V] The paper introduces a new 1,880-case benchmark and a full system but provides no release artifact, code, or benchmark URL. This is a stated contribution, and the central performance claims rest entirely on this self-built benchmark. Without the benchmark generator, label-assignment scripts, and scanner implementation, the claims cannot be independently verified or reused. A reproducibility appendix with these artifacts, or a clear availability statement, is essential before publication.
  4. [Table II, §V-B] Table II's footnote excludes 'Error' runs from denominators. A.I.G has 42 errors on Prompt Injection, 19 on Command Injection, and 2 on File System Access, while FlowGuard has none. If failed runs are non-random (e.g., timeouts on hard cases), this exclusion systematically favors FlowGuard. Please report error-inclusive precision/recall/F1 (treating errors as false negatives) or at least provide per-method error rates. This is needed for fair cross-scanner comparison.
minor comments (6)
  1. [Table III] For Credential Leakage, FlowGuard's Evidence Calls percentage (30.04%) is lower than A.I.G's (40.56%), yet FlowGuard obtains more true positives. Define what counts as a 'security-relevant signal' for each scanner; otherwise the metric is not comparable across systems.
  2. [§III-C2] 'Fluffy's parameter to sink observations' is mentioned without a citation; please add the reference for this knowledge source.
  3. [Figure 5, §V-E4] Report whether probing budget B, timeout, and patience k were held constant across LLM backbones; temperature alone is not sufficient for a controlled comparison.
  4. [Global typo] 'OW ASP' should be 'OWASP' throughout (e.g., §IV-A, §IV-B, references [10], [17]).
  5. [§V-F] Specify the popularity criterion for selecting the top 8,000 MCPZoo servers and report connection/interaction success rates; this affects external validity of the real-world numbers.
  6. [Appendix B, Table VI] The row 'Total 82 24 –' is confusing; clarify the counts and column semantics.

Circularity Check

1 steps flagged · score 4.0 of 10

Benchmark labels and FlowGuard's adjudication rule share the same evidence definition (S∧O∧¬E), so absolute F1 partly measures self-consistency; the one external MarkItDown confirmation provides independent support.

  1. self definitional [Section IV-A (Benchmark Overview) vs. Appendix A, Prompt C (Evidence Adjudication)]
    "We use the following verdict rule: R=S∧O∧¬E. Here,Rdenotesis_real_finding,Sdenotessignal_identified,Odenotesis_system_originated, andEdenotesis_expected_behavior."

    The benchmark's positive labels are assigned when a case has 'category-specific evidence, such as command markers, fixture file contents, synthetic canaries, or instruction patterns' (Section IV-A), and are validated by checking that trigger inputs produce the expected marker while safe variants block the same evidence. This is the same S∧O∧¬E rule: the observed evidence is the signal S; because the evidence is a canary or marker not present in the probe it counts as system-originated O; because it lies outside the tool's declared purpose it is not expected behavior ¬E. Thus the ground-truth oracle and FlowGuard's adjudicator implement the same operational definition of 'real finding.' The reported F1 therefore measures how faithfully FlowGuard reproduces the authors' evidence rule rather

full rationale

This is an empirical systems paper, not a formal derivation, so the circularity analysis focuses on whether the evaluation's ground truth is independent of the system's decision procedure. The main concern is real: the benchmark's execution-evidence labels (command markers, canaries, out-of-bound file reads) are defined by the same S∧O∧¬E criterion encoded in FlowGuard's Prompt C adjudicator. That overlap makes the absolute F1 numbers for Command Injection, Credential Leakage, and File System Access partly a test of self-consistency between two instantiations of the same evidence definition. However, the circularity is only partial: FlowGuard still must discover the triggering probes through blind interaction, and the benchmark construction does not feed FlowGuard's outputs into the labels. Relative comparisons against the baselines remain meaningful because all scanners are measured on the same labels. The real-world evaluation also includes one externally confirmed finding (MarkItDown file:///etc/passwd via the project's issue tracker), which is independent support. I did not find load-bearing self-citation circularity: the MCPZoo reference [19] is an overlapping-author dataset citation, but it supplies the real-world corpus rather than an unverified uniqueness or derivation claim, so it does not raise the score. No fitted-parameter-called-prediction, imported-uniqueness, or ansatz-by-citation pattern is present. Score 4 reflects one definitional overlap in the central evaluation while acknowledging the system has independent content and external anchoring.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

No new physical or formal entities are postulated. The 'risk vectors,' 'fingerprints,' and 'fast-path' rules are internal data structures and heuristics, not independently testable entities. The main burdens are the hand-set probing budget, the uncalibrated LLM subroutines, and the self-authored benchmark and MCPZoo corpus.

free parameters (5)
  • Probing budget B = 5
    Maximum refinement rounds per parameter; set by authors. Figure 4 shows metrics and runtime depend on it.
  • Scan timeout = 900 s
    Per-case timeout; influences evaluation latency and termination.
  • No-new-signal patience k = unspecified
    Algorithm 1 termination parameter; not given numerically in the paper.
  • Fast-path risk scores/thresholds = not reported
    Hand-constructed from CVE parameter names, 'Fluffy' observations, and OWASP Top 25; determines which parameters are probed.
  • LLM sampling temperature = 0.7
    Used for all backbones in Section V-E4; single-run results show no variance across samples.
assumptions (5)
  • domain assumption MCP servers expose observable, stable runtime behavior in response to schema-valid probes.
    The entire evidence loop relies on responses (errors, output, fingerprints) being informative and not adversarially adaptive (Section II-C3).
  • ad hoc to paper The decision rule R = S ∧ O ∧ ¬E is a sufficient and sound definition of a true MCP vulnerability.
    Defined by the authors in Prompt C; it is the scanner's output contract and is also used to define benchmark evidence, creating a circular-validity risk.
  • domain assumption Benchmark labels assigned by design are correct and representative.
    Section IV-A states labels are 'assigned by benchmark design rather than by scanner output'; no independent audit is provided.
  • domain assumption The LLM subroutines give accurate-enough triage, payload generation, and adjudication.
    Ablation across backbones shows F1 ranges 0.327 to 0.898 (Fig. 5), so results depend on LLM quality.
  • domain assumption OWASP/CWE attack-pattern mappings to MCP tool parameters are valid.
    Benchmark categories follow CWE-78, CWE-22, OWASP WSTG; these mappings are assumed to transfer to MCP.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FlowGuard: From Signals to Evidence for MCP Security Detection." pith.science (2026). https://pith.science/paper/LP2LM5FE

@misc{pith2026260714754,
  author       = {Pith},
  title        = {Pith review of: FlowGuard: From Signals to Evidence for MCP Security Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LP2LM5FE}},
  note         = {Machine review of arXiv:2607.14754}
}
read the original abstract

The Model Context Protocol (MCP) enables LLM agents to interact with external tools through metadata exchange, tool invocation, and response consumption. Existing MCP security scanners primarily reason about suspicious semantic signals rather than real execution behaviors, which can lead to unreliable risk assessment. For example, credential-like strings may simply be placeholders rather than actual leakage. This gap requires runtime evidence for execution-related risks and careful semantic analysis for risks carried in metadata or returned content. We present FlowGuard, an evidence-grounded MCP security detection system. FlowGuard combines semantic risk triage, recon-guided payload narrowing, schema-valid probe generation, evidence adjudication, and history-guided refinement. It verifies execution-related risks through runtime evidence and detects semantic risks in tool metadata and returned content. We evaluate FlowGuard on an executable benchmark containing 1,880 MCP cases across five vulnerability categories. FlowGuard achieves F1 scores of 0.879 and 0.942 on the execution-related Command Injection and File System Access categories, respectively. Compared with existing dynamic scanners, FlowGuard reduces end-to-end latency by up to 2.23x. In the real-world evaluation, FlowGuard reports 523 findings across 326 servers. These results show that evidence-grounded detection can assess both execution-related and semantic risks in MCP interactions.

Figures

Figures reproduced from arXiv: 2607.14754 by the authors.

Figure 1
Figure 1. MCP interaction workflow and the attack surfaces analyzed by Flow [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. An overview of FlowGuard. backend calls. This semantic ambiguity makes it difficult to identify which inputs are security-relevant. At the same time, exhaustively probing all parameters is inefficient, increases un￾necessary load on MCP servers, and reduces the likelihood of triggering real vulnerabilities under a limited probing budget. In the triage phase, we combine tool descriptions, parameter names, and JSON sc… view at source ↗
Figure 3
Figure 3. CDF of per-case end-to-end analysis latency on three benchmark categories. The x-axis uses log scale. FlowGuard’s curves are consistently left-shifted, [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Effect of the probing-round budget on detection effectiveness and [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Sensitivity of FlowGuard to different LLM backbones. Bars show [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 7 linked inside Pith

  1. [1]

    MCPScan: Security analysis tool for MCP servers,

    Ant Group, “MCPScan: Security analysis tool for MCP servers,” https: //github.com/antgroup/MCPScan, 2025, GitHub repository. Accessed: 2026-05-01

    2025

  2. [2]

    Snyk Agent Scan,

    Snyk, “Snyk Agent Scan,” https://github.com/snyk/agent-scan, 2025, GitHub repository. Accessed: 2026-05-01

    2025

  3. [3]

    AI-Infra-Guard: A Comprehensive, Intelligent, and Easy-to-Use AI Red Teaming Platform,

    Tencent Zhuque Lab, “AI-Infra-Guard: A Comprehensive, Intelligent, and Easy-to-Use AI Red Teaming Platform,” https://github.com/Tence nt/AI-Infra-Guard, 2025, GitHub repository. Accessed: 2026-05-01

    2025

  4. [4]

    Model Context Protocol,

    “Model Context Protocol,” https://modelcontextprotocol.io/, accessed: 2026-05-01

    2026

  5. [5]

    JSON-RPC 2.0 specification,

    JSON-RPC Working Group, “JSON-RPC 2.0 specification,” https://ww w.jsonrpc.org/specification, 2010, accessed: 2026-05-01

    2010

  6. [6]

    Claude Desktop,

    Anthropic, “Claude Desktop,” https://claude.ai/download, 2025, accessed: 2026-05-01

    2025

  7. [7]

    Cursor: The AI Code Editor,

    Anysphere, “Cursor: The AI Code Editor,” https://www.cursor.com/, 2025, accessed: 2026-05-01

    2025

  8. [8]

    Snipuzz: Black-box fuzzing of iot firmware via message snippet inference,

    X. Feng, R. Sun, X. Zhu, M. Xue, S. Wen, D. Liu, S. Nepal, and Y . Xiang, “Snipuzz: Black-box fuzzing of iot firmware via message snippet inference,” inProceedings of the 2021 ACM SIGSAC conference on computer and communications security, 2021, pp. 337–350

    2021

Show all 59 references
  1. [9]

    Beware of the unexpected: Bimodal taint analysis,

    Y . W. Chow, M. Sch ¨afer, and M. Pradel, “Beware of the unexpected: Bimodal taint analysis,” inProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, 2023, pp. 211–222

    2023

  2. [10]

    OW ASP Benchmark Project,

    OW ASP Foundation, “OW ASP Benchmark Project,” https://owasp.org/ www-project-benchmark/, 2024, accessed: 2026-05-01

    2024

  3. [11]

    Juliet Test Suite v1.3 for C/C++ and Java,

    National Security Agency (NSA) and National Institute of Standards and Technology (NIST), “Juliet Test Suite v1.3 for C/C++ and Java,” https://www.nist.gov/publications/juliet-11-cc-and-java-test-suite, 2017, accessed: 2026-05-01

    2017

  4. [12]

    MCP security bench (MSB): Benchmarking attacks against model context protocol in LLM agents,

    D. Zhang, Z. Li, X. Luo, X. Liu, P. Li, and W. Xu, “MCP security bench (MSB): Benchmarking attacks against model context protocol in LLM agents,” inThe Fourteenth International Conference on Learning Representations, 2026. [Online]. Available: https: //iclr.cc/virtual/2026/pos...

    2026

  5. [13]

    MCPSecBench: A systematic security benchmark and playground for testing model context protocols,

    Y . Yang, C. Gao, D. Wu, Y . Chen, Y . Li, and S. Wang, “MCPSecBench: A systematic security benchmark and playground for testing model context protocols,” 2026. [Online]. Available: https://arxiv.org/abs/2508.13220

    2026

  6. [14]

    MCP- SafetyBench: A benchmark for safety evaluation of large language models with real-world MCP servers,

    X. Zong, Z. Shen, L. Wang, Y . Lan, and C. Yang, “MCP- SafetyBench: A benchmark for safety evaluation of large language models with real-world MCP servers,” 2026. [Online]. Available: https://arxiv.org/abs/2512.15163

    2026

  7. [15]

    CWE-78: Improper Neutralization of Special Elements used in an OS Command (’OS Command Injection’),

    MITRE Corporation, “CWE-78: Improper Neutralization of Special Elements used in an OS Command (’OS Command Injection’),” https: //cwe.mitre.org/data/definitions/78.html, 2024, accessed: 2026-05-01

    2024

  8. [16]

    CWE-22: Improper Limitation of a Pathname to a Restricted Directory (’Path Traversal’),

    ——, “CWE-22: Improper Limitation of a Pathname to a Restricted Directory (’Path Traversal’),” https://cwe.mitre.org/data/definitions/22. html, 2024, accessed: 2026-05-01

    2024

  9. [17]

    OW ASP Web Security Testing Guide: Testing for Path Traversal (WSTG-ATHZ-01),

    OW ASP Foundation, “OW ASP Web Security Testing Guide: Testing for Path Traversal (WSTG-ATHZ-01),” https://owasp.org/www-project-w eb-security-testing-guide/, 2024, accessed: 2026-05-01

    2024

  10. [18]

    MCPTox: A benchmark for tool poisoning attack on real-world MCP servers,

    Z. Wang, Y . Gao, Y . Wang, S. Liu, H. Sun, H. Cheng, G. Shi, H. Du, and X. Li, “MCPTox: A benchmark for tool poisoning attack on real-world MCP servers,” 2025. [Online]. Available: https://arxiv.org/abs/2508.14925

    2025 arXiv

  11. [19]

    MCPZoo: A large-scale dataset of runnable model context protocol servers for ai agent,

    M. Wu, P. Chen, G. Hong, B. An, J. Chen, B. Wan, X. Pan, J. Dai, and M. Yang, “MCPZoo: A large-scale dataset of runnable model context protocol servers for ai agent,” 2025. [Online]. Available: https://arxiv.org/abs/2512.15144

    2025

  12. [20]

    Systematic analysis of MCP security,

    Y . Guo, P. Liu, W. Ma, Z. Deng, X. Zhu, P. Di, X. Xiao, and S. Wen, “Systematic analysis of MCP security,” 2025. [Online]. Available: https://arxiv.org/abs/2508.12538

    2025 arXiv

  13. [21]

    Prompt injection attack to tool selection in LLM agents,

    J. Shi, Z. Yuan, G. Tie, P. Zhou, N. Z. Gong, and L. Sun, “Prompt injection attack to tool selection in LLM agents,” inNetwork and Distributed System Security Symposium, 2026. [Online]. Available: https://www.ndss-symposium.org/ndss-paper/prompt-injection-attack-t o-tool-selec...

    2026

  14. [22]

    MPMA: Preference manipulation attack against model context protocol,

    Z. Wang, R. Zhang, Y . Liu, W. Fan, W. Jiang, Q. Zhao, H. Li, and G. Xu, “MPMA: Preference manipulation attack against model context protocol,” 2025. [Online]. Available: https://arxiv.org/abs/2505.11154

    2025

  15. [23]

    MCP-ITP: An automated framework for implicit tool poisoning in MCP,

    R. Li, Z. Wang, Y . Yao, and X.-Y . Li, “MCP-ITP: An automated framework for implicit tool poisoning in MCP,” 2026. [Online]. Available: https://arxiv.org/abs/2601.07395

    2026

  16. [24]

    Attractive metadata attack: Inducing LLM agents to invoke malicious tools,

    K. Mo, L. Hu, Y . Long, and Z. Li, “Attractive metadata attack: Inducing LLM agents to invoke malicious tools,” inAdvances in Neural Information Processing Systems, 2025. [Online]. Available: https://arxiv.org/abs/2508.02110

    2025

  17. [25]

    Breaking the protocol: Security analysis of the model context protocol specification and prompt injection vulnerabilities in tool-integrated LLM agents,

    N. Maloyan and D. Namiot, “Breaking the protocol: Security analysis of the model context protocol specification and prompt injection vulnerabilities in tool-integrated LLM agents,” 2026. [Online]. Available: https://arxiv.org/abs/2601.17549

    2026

  18. [26]

    Compatibility at a cost: Systematic discovery and exploitation of MCP clause-compliance vulnerabilities,

    N. Yang, W. Bai, and K. Lu, “Compatibility at a cost: Systematic discovery and exploitation of MCP clause-compliance vulnerabilities,”

  19. [27]

    Don’t believe everything you read: Understanding and measuring MCP behavior under misleading tool descriptions,

    Z. Li, B. Ma, X. Dai, M. Xu, Y . Zhang, B. Yan, and K. Li, “Don’t believe everything you read: Understanding and measuring MCP behavior under misleading tool descriptions,” 2026. [Online]. Available: https://arxiv.org/abs/2602.03580

    2026

  20. [28]

    Injecagent: Benchmarking indirect prompt injections in tool-integrated large language model agents,

    Q. Zhan, Z. Liang, Z. Ying, and D. Kang, “Injecagent: Benchmarking indirect prompt injections in tool-integrated large language model agents,” inFindings of the Association for Computational Linguistics: ACL 2024, 2024, pp. 10 471–10 506. [Online]. Available: https: //aclantho...

    2024

  21. [29]

    Identifying the risks of LM agents with an LM-emulated sandbox,

    Y . Ruan, H. Dong, A. Wang, S. Pitis, Y . Zhou, J. Ba, Y . Dubois, C. J. Maddison, and T. Hashimoto, “Identifying the risks of LM agents with an LM-emulated sandbox,” inThe Twelfth International Conference on Learning Representations, 2024. [Online]. Available: https://openrev...

    2024

  22. [30]

    Model context protocol (MCP): Landscape, security threats, and future research directions,

    X. Hou, Y . Zhao, S. Wang, and H. Wang, “Model context protocol (MCP): Landscape, security threats, and future research directions,” ACM Trans. Softw. Eng. Methodol., Feb. 2026, just Accepted. [Online]. Available: https://doi.org/10.1145/3796519

    2026 doi

  23. [31]

    A first look at the security issues in the model context protocol ecosystem,

    X. Li and X. Gao, “A first look at the security issues in the model context protocol ecosystem,” 2026. [Online]. Available: https://arxiv.org/abs/2510.16558

    2026 arXiv

  24. [32]

    Automatic red teaming LLM-based agents with model context protocol tools,

    P. He, C. Li, B. Zhao, T. Du, and S. Ji, “Automatic red teaming LLM-based agents with model context protocol tools,” 2025. [Online]. Available: https://arxiv.org/abs/2509.21011

    2025

  25. [33]

    T-MAP: Red-teaming LLM agents with trajectory-aware evolutionary search,

    H. Lee, S. Park, Y . Choi, S. An, S. Lee, and S. J. Hwang, “T-MAP: Red-teaming LLM agents with trajectory-aware evolutionary search,”

  26. [34]

    MCP Scanner,

    Cisco AI Defense, “MCP Scanner,” https://github.com/cisco-ai-defense /mcp-scanner, 2025, GitHub repository. Accessed: 2026-05-01

    2025

  27. [35]

    Available: https://arxiv.org/abs/2603.22341

    [Online]. Available: https://arxiv.org/abs/2603.22341

  28. [36]

    Mindguard: Intrinsic decision inspection for securing LLM agents against metadata poisoning,

    Z. Wang, H. Du, G. Shi, J. Zhang, H. Cheng, Y . Yao, K. Guo, and X.-Y . Li, “Mindguard: Intrinsic decision inspection for securing LLM agents against metadata poisoning,” 2026. [Online]. Available: https://arxiv.org/abs/2508.20412

    2026

  29. [37]

    mcp-armor: MCP Configuration Scanner with Client- Aware Security Analysis,

    Aira Security, “mcp-armor: MCP Configuration Scanner with Client- Aware Security Analysis,” https://github.com/aira-security/mcp-armor, 2026, GitHub repository. Accessed: 2026-05-01

    2026

  30. [38]

    CASCADE: A cascaded hybrid defense architecture for prompt injection detection in MCP-based systems,

    ˙Ipek Abasıkeles ¸ Turgut and E. G¨um¨us ¸, “CASCADE: A cascaded hybrid defense architecture for prompt injection detection in MCP-based systems,” 2026. [Online]. Available: https://arxiv.org/abs/2604.17125

    2026 arXiv

  31. [39]

    MCP-Guard: A multi-stage defense-in-depth framework for securing model context protocol in agentic AI,

    W. Xing, Z. Qi, Y . Qin, Y . Li, C. Chang, J. Yu, C. Lin, Z. Xie, and M. Han, “MCP-Guard: A multi-stage defense-in-depth framework for securing model context protocol in agentic AI,” 2026. [Online]. Available: https://arxiv.org/abs/2508.10991

    2026

  32. [40]

    Auditing MCP servers for over-privileged tool capabilities,

    C. Huang, X. Huang, and A. M. Fard, “Auditing MCP servers for over-privileged tool capabilities,” 2026. [Online]. Available: https://arxiv.org/abs/2603.21641

    2026

  33. [41]

    MCP Safety Audit: LLMs with the Model Context Protocol Allow Major Security Exploits,

    B. Radosevich and J. Halloran, “MCP Safety Audit: LLMs with the Model Context Protocol Allow Major Security Exploits,” https://www. arxiv.org/abs/2504.03767, 2025, MCPSafetyScanner

    2025 arXiv

  34. [42]

    From component manipulation to system compromise: Understanding and detecting malicious MCP servers,

    Y . Huang, Z. Zhao, B. Chen, S. Wu, Z. Zhou, Y . Cao, X. Hu, and X. Peng, “From component manipulation to system compromise: Understanding and detecting malicious MCP servers,” 2026. [Online]. Available: https://arxiv.org/abs/2604.01905

    2026 arXiv

  35. [43]

    MCP- SandboxScan: W ASM-based secure execution and runtime analysis for MCP tools,

    Z. Tan, R. Hao, J. Singer, Y . Tang, and C. Anagnostopoulos, “MCP- SandboxScan: W ASM-based secure execution and runtime analysis for MCP tools,” 2026. [Online]. Available: https://arxiv.org/abs/2601.01241

    2026 arXiv

  36. [44]

    Large language model guided protocol fuzzing

    R. Meng, M. Mirchev, M. B ¨ohme, and A. Roychoudhury, “Large language model guided protocol fuzzing.” inNDSS, 2024

    2024

  37. [45]

    MCPShield: A security cognition layer for adaptive trust calibration in model context protocol agents,

    Z. Zhou, Y . Zhang, H. Cai, M. Aloqaily, O. Bouachir, L. Pang, P. Mehrotra, K. Wang, and Q. Wen, “MCPShield: A security cognition layer for adaptive trust calibration in model context protocol agents,”

  38. [46]

    Available: https://arxiv.org/abs/2602.14281

    [Online]. Available: https://arxiv.org/abs/2602.14281

  39. [47]

    YuraScanner: Leveraging LLMs for task-driven web app scanning,

    A. Stafeev, T. Recktenwald, G. De Stefano, S. Khodayari, and G. Pellegrino, “YuraScanner: Leveraging LLMs for task-driven web app scanning,” inNetwork and Distributed System Security Symposium (NDSS), 2025. [Online]. Available: https://www.ndss-symposium.org/n dss-paper/yurasc...

    2025

  40. [48]

    Make agent defeat agent: Automatic detection of {Taint-Style}vulnerabilities in{LLM-based}agents,

    F. Liu, Y . Zhang, J. Luo, J. Dai, T. Chen, L. Yuan, Z. Yu, Y . Shi, K. Li, C. Zhouet al., “Make agent defeat agent: Automatic detection of {Taint-Style}vulnerabilities in{LLM-based}agents,” in34th USENIX Security Symposium (USENIX Security 25), 2025, pp. 3767–3786

    2025

  41. [49]

    Black widow: Blackbox data-driven web scanning,

    B. Eriksson, G. Pellegrino, and A. Sabelfeld, “Black widow: Blackbox data-driven web scanning,” in2021 IEEE Symposium on Security and Privacy (SP). IEEE, 2021, pp. 1125–1142

    2021

  42. [51]

    MarkItDown,

    Microsoft, “MarkItDown,” https://github.com/microsoft/markitdown, 2025, accessed: 2026-05-01. APPENDIXA PROMPTTEMPLATES This appendix lists the LLM prompt locations used by FlowGuard. Each prompt is used as a bounded structured subroutine in the pipeline described in Section I...

    2025

  43. [52]

    Respect explicit schema constraints, including type, enum, pattern, minimum and maximum length, numeric range, and format

  44. [53]

    Prefer compact, high-signal probes over generic enumeration

    Specialize the payload to the observed operating system, backend language, database, and threat family. Prefer compact, high-signal probes over generic enumeration

  45. [54]

    For FS and CRED risks, prefer benchmark-defined canary files, synthetic secrets, environment markers, application configuration markers, or other non-destructive runtime indicators when compatible with the schema

  46. [55]

    For DB and PI risks, choose a compact, constraint-aware payload that is likely to produce an observable signal in the response

  47. [56]

    is_system_originated

    Avoid destructive or intrusive behavior. Do not generate payloads that delete files, modify persistent state, ex- filtrate real credentials, establish persistence, or contact external hosts. Output Rules:Output only the raw payload value. Do not output JSON, markdown, explanat...

  48. [57]

    Infer how the target is currently interpreting the parameter at runtime

  49. [58]

    Extract actionable clues from the history, such as accepted structure, revealed paths, file names, table names, column names, environment-variable names, protocol hints, coer- cion hints, and failure modes

  50. [59]

    decision

    Decide the single most informative next probe for the current hypothesis while minimizing side effects. Planning rules. •If the history indicates schema, type, format, or protocol validation failure, repair the payload so it preserves the test intent while better satisfying th...

  51. [2026]

    Available: https://arxiv.org/abs/2603.10163

    [Online]. Available: https://arxiv.org/abs/2603.10163

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.