REVIEW 3 major objections 3 minor 23 references
ChainWatch catches multi-step MCP attacks by tracking tool-call sequences through a six-stage kill chain, catching chains that pass per-call checks.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 15:03 UTC pith:4NULVX7T
load-bearing objection Honest design spec for session-level MCP attack detection; the framework is plausible but the evaluation is authored illustration, not evidence, and the abstract overstates what is demonstrated. the 3 major comments →
ChainWatch: A Kill Chain-Aligned Sequential Detection Framework for Multi-Step Attacks in MCP-Based AI Agent Systems
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
ChainWatch's central claim is that attack intent invisible at the level of a single tool call becomes visible as a sequence of kill-chain stage labels. Each call is mapped to a 20-dimensional feature vector (tool category, parameter sensitivity, data flow, temporal rhythm, output characteristics); a Hidden Markov Model treats the six kill-chain stages as hidden states and assigns a stage to every call; and five rules scan a window of ten calls for signatures such as reconnaissance followed by sensitive access, cross-server data flow, read-then-transmit exfiltration, rapid stage jumps, and late-stage configuration writes. In the paper's five scenario traces, every attack chain triggers at lea
What carries the argument
The six-stage MCP kill chain (Reconnaissance, Trust Building, Injection, Escalation, Lateral Movement, Exfiltration) plus a Hidden Markov Model over those stages forms the core mechanism. The HMM converts the problem of 'is this session an attack?' into 'what hidden stage is each call in?' by emitting continuous 20-dimensional feature vectors. The transition matrix encodes the prior that attacks move mostly forward, rarely jump more than two stages, and occasionally repeat earlier behaviour. Five pattern rules then turn the stage sequence into alerts, with R3 (read-then-transmit) and R5 (late-stage configuration write) set to block pending calls.
Load-bearing premise
The load-bearing premise is that the illustrative feature values assigned to calls in the five scenarios faithfully represent what real MCP tool calls produce, so the HMM would actually assign those kill-chain stages to real sessions; if raw calls do not map that way, the rules fire on the wrong calls and the detection claim collapses.
What would settle it
Collect labelled session traces from a multi-turn MCP safety benchmark plus benign enterprise workflows, run the 20-dimensional feature extraction on the raw calls, fit the HMM with Baum-Welch, and check whether inferred stages match the assumed kill-chain labels and whether benign multi-server sessions trigger R2. If real feature vectors do not reproduce the stage sequences in the five scenarios, or benign workflows trigger R2 at an unacceptable rate, the paper's detection claim fails.
If this is right
- If ChainWatch works as designed, MCP deployments gain a detection layer that per-call filters cannot provide, because it uses session context rather than individual call content.
- The three attack classes—direct sequential, indirect injection, hybrid—all leave kill-chain signatures, so one framework covers credential harvesting, financial fraud, prompt-injection chains, and cross-agent escalation.
- A sliding window of ten calls appears sufficient, since documented attack chains span four to seven calls, allowing detection before a chain completes.
- ChainWatch is positioned as complementary to per-call defenses rather than a replacement, so existing MCP security tooling remains useful.
- CRITICAL rules (R3, R5) are designed to block the pending call, meaning successful detection could interrupt exfiltration and configuration-write steps in real time.
Where Pith is reading between the lines
- If the feature-to-stage mapping can be learned from real traces, the same HMM-plus-rules architecture could transfer to other agent protocols where tool calls are logged, not just MCP.
- The paper's own acknowledged risk—R2 false positives on legitimate multi-server workflows—suggests the immediate testable extension is a benign-workflow false-positive benchmark, which would be a stronger evaluation than the scenario traces.
- Because the HMM is first-order Markov, attacks that deliberately cycle stages or use very long trust-building phases could evade the fixed transition assumptions; a natural extension is comparing this HMM against recurrent or graph-based sequence models on the same traces.
- The framework's value may ultimately be as a telemetry layer: even before rules are tuned, stage labels give security analysts a compact description of what a session is doing, which could power alert triage.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. ChainWatch is presented as a sequential detection framework for multi-step attacks in MCP-based AI agent systems. The paper proposes a six-stage MCP kill chain, a 20-dimensional feature extraction schema, an HMM-based stage classifier, and five session-level detection rules. The evaluation traces five documented attack scenarios through the framework, claiming that ChainWatch 'would detect' attacks that evade per-call defenses. The paper is explicitly a design specification (Sections IV.A, V.A, VI), with HMM parameters pending estimation and feature values assigned for illustration.
Significance. If the detection pipeline could be realized, ChainWatch would address a real and currently under-served threat class: multi-step attacks composed of individually benign tool calls. The paper's contributions include a structured taxonomy (DSA, IIC, HMSA), a kill chain adapted to MCP, and a concrete feature schema. The authors are transparent that this is not a deployed system and that empirical validation is required. However, as presented, the central detection claim is not demonstrated: the evaluation is by construction, because the stage labels are assigned by the authors rather than inferred by the HMM. The architecture and taxonomy are plausible and worth publishing as a design/position paper, but the manuscript currently overstates what is shown.
major comments (3)
- [Section V.B (with Abstract and Section VI)] The evaluation outcomes are authored by the authors' choices. In each scenario, stage labels are 'illustrative assignments' (V.B) chosen to match the attack narrative; no feature vectors are supplied and no Viterbi/forward inference is run. For example, in S1 the authors assign get_balance to Stage 1, list_payees to Stage 2, add_payee to Stage 4, and transfer_funds to Stage 6, so R4 and R3 fire by construction. The Abstract's claim that the framework 'detects' attacks and Section VI's claim that 'the five scenarios demonstrate' detection are therefore unsupported. This is load-bearing: the central contribution is the detection claim. The manuscript should either present a real or simulated instantiation of the pipeline with actual HMM inference, or reframe all claims as 'traceability of the designed rules under assumed stage labels'.
- [Section IV.C] The HMM is defined as λ=(S,Σ,A,B,π), but A, B, and π are not specified; the paper states that transition values are 'design choices pending Baum-Welch estimation from labelled trace data.' Without a concrete emission model (e.g., Gaussian mixture over the 20-dimensional vectors) and transition matrix, the stage classifier does not exist as a mechanism, and the detection rules R1–R5 cannot be evaluated on raw tool calls. This is not a minor parameter-setting issue: the entire pipeline hinges on the HMM mapping feature vectors to the six stages. The manuscript must either specify the model family and initial parameterization, or explicitly state that the HMM component is a placeholder and restrict the paper's claims accordingly.
- [Section VI] The false-positive risk of R2 for legitimate multi-server workflows is acknowledged, but no analysis is given for this or for the sensitivity of the thresholds k=10, m=5, or the WARNING/CRITICAL design. Since the paper aims to show that the rules detect attacks while not raising unacceptable alert rates, the absence of any baseline or failure-mode analysis leaves the practical detection claim unquantified. At minimum, the paper should provide a speculative analysis of which benign patterns would cross each rule and clearly state that no claim of operational viability is made.
minor comments (3)
- [Section V.A] The language is inconsistent: Section V.A says the evaluation is 'scenario analyses' with 'illustrative assignments,' but Sections V.B and VI repeatedly use 'would detect' and 'demonstrate.' The authors should align the wording with the design-specification caveat, for example by saying 'the rules are designed to fire on these assumed stage labels' rather than 'ChainWatch would detect.'
- [Figure 4 caption] The caption states that ChainWatch raises WARNING (R4) and CRITICAL (R3), but the accompanying text does not spell out which calls in the figure trigger these rules. A brief walkthrough (e.g., which stage assignments cause the jump and the read-to-network pattern) would make the figure self-contained.
- [Section II.D] The related work on multi-step detection in traditional security is relevant and useful, but the connection could be sharpened: the paper should clarify which specific techniques from Holgado et al. and Navarro et al. are transferred to MCP and which are newly adapted. This would strengthen the positioning of the HMM-based approach.
Circularity Check
ChainWatch's detection demonstrations in §V.B reduce to the authors' own illustrative stage assignments; with HMM parameters unfitted and no trace data, the rule firings are authored, not computed.
specific steps
-
self definitional
[§V.B (S1–S5), with §IV.C and §V.A as context]
"Feature values are illustrative assignments based on the parameter content described in each source. ... ChainWatch would see: get_balance at Stage 1, list_payees at Stage 2, add_payee as a high-sensitivity WRITE jumping to Stage 4, then transfer_funds as a chained NETWORK call at Stage 6. R4 would fire at call 3 — the stage jumped two positions. R3 would fire at call 4."
The HMM that is supposed to produce stage labels has no fitted parameters: 'Specific transition values are design choices pending Baum-Welch estimation from labelled trace data' (§IV.C), and §V.A admits no labelled trace data exists. The §V.B stage labels are illustrative assignments chosen by the authors to match each attack narrative. Since feature vectors are not given and no Viterbi/forward inference is run, the resulting R1/R3/R4/R5 firings are predetermined by those hand-assigned labels—the 'detection' is equivalent to the assumption that the calls occupy those stages. The scenario outcome is therefore constructed, not computed.
full rationale
The paper is transparent that this is a design specification, not a deployed system: §IV.A states 'this section does not document a system that is currently in operation,' and §VI says the distance to a deployed system is 'primarily empirical.' The taxonomy, feature schema, and rule set are coherent and not circular in themselves; no self-citation chain or imported-uniqueness argument is used. However, the paper's central demonstration—that ChainWatch 'would detect' S1–S5—is not an inference from raw tool calls. The HMM transition values are 'design choices pending Baum-Welch estimation from labelled trace data' (§IV.C), no such trace data exists (§V.A), and the §V.B feature/stage values are 'illustrative assignments based on the parameter content described in each source.' Under those conditions, the stage labels assigned to each call are authored to match the attack narrative, so the rule firings (R1/R3/R4/R5) follow by construction rather than from any computed classification. The evaluation therefore reduces to a consistency check of the rules against hand-labeled scenarios, which is partial circularity in the detection claim—though the authors do flag the missing empirical step. If the paper were read purely as an architectural proposal with no detection demonstration, the circularity would be near zero; as written, the detection claim is partially forced by the authors' own illustrative assignments.
Axiom & Free-Parameter Ledger
free parameters (8)
- HMM transition matrix A =
unspecified (design choice)
- HMM emission distributions B =
unspecified
- Initial stage distribution π =
unspecified
- Sliding window size k =
10
- Step threshold m =
5
- Parameter Sensitivity (PS) weights =
unspecified
- Output Characteristics (OC) thresholds =
unspecified
- R2 multi-server threshold =
2
axioms (5)
- domain assumption Per-call defenses are active and functioning; the attacker cannot observe ChainWatch's thresholds or window parameters.
- domain assumption MCP servers are potentially adversarial; MCP client and host are trusted.
- ad hoc to paper Documented multi-step attacks are faithfully captured by the six-stage kill chain and the 20-dimensional feature schema.
- domain assumption HMM-based kill-chain stage inference transfers from IDS alert sequences to MCP tool-call sequences.
- domain assumption Benign MCP workflows do not frequently exhibit the transition patterns targeted by R1–R5.
read the original abstract
The Model Context Protocol (MCP) is an open-source standard that allows AI agents to connect to external tools, databases, and services. While this connectivity enables powerful agent capabilities, it also introduces multi-step attacks that existing per-call defenses cannot reliably detect. Attackers can compose individually benign tool invocations into malicious sequences that evade isolated inspection. This paper presents ChainWatch, a sequential detection framework for identifying multi-step attacks in MCP-based AI agent systems. ChainWatch models attack progression using a six-stage kill chain and applies a Hidden Markov Model (HMM) to classify tool-call sequences. Detection rules are triggered when a session exhibits suspicious progression across multiple stages. The framework is supported by a structured threat model covering direct sequential attacks, indirect prompt injection chains, and hybrid multi-stage attacks. A 20-dimensional feature extraction schema captures behavioral signals from tool interactions. We demonstrate the approach using five representative attack scenarios from the security literature, showing how ChainWatch detects attack chains that evade traditional per-call security mechanisms.
Figures
Reference graph
Works this paper leans on
-
[1]
Introducing the Model Context Protocol,
Anthropic, “Introducing the Model Context Protocol,” Nov
-
[2]
Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directions,
X. Hou, Y . Zhao, S. Wang, and H. Wang, “Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directions,” arXiv preprint arXiv:2503.23278, Mar. 2025
Pith/arXiv arXiv 2025
-
[3]
MCP Security Notification: Tool Poison- ing Attacks and GitHub MCP Prompt Injection,
Invariant Labs, “MCP Security Notification: Tool Poison- ing Attacks and GitHub MCP Prompt Injection,” Apr.– May 2025. [Online]. Available: https://invariantlabs.ai/blog/ mcp-security-notification-tool-poisoning-attacks
2025
-
[4]
WhatsApp MCP Exploited via Rug-Pull At- tack,
Invariant Labs, “WhatsApp MCP Exploited via Rug-Pull At- tack,” Apr. 2025. [Online]. Available: https://invariantlabs.ai/blog/ whatsapp-mcp-exploited
2025
-
[5]
STAC: When Innocent Tools Form Dangerous Chains to Jailbreak LLM Agents,
Z. Li, J. He, C. Shang, D. Kulshreshtha, X. Xian, Y . Zhang, H. Su, S. Swamy, and Y . Qi, “STAC: When Innocent Tools Form Dangerous Chains to Jailbreak LLM Agents,” arXiv preprint arXiv:2509.25624, Sep. 2025
Pith/arXiv arXiv 2025
-
[6]
R. Brodt, N. Feldman, and B. Nassi, “The Promptware Kill Chain,” arXiv preprint arXiv:2601.09625, Jan. 2026
arXiv 2026
-
[7]
Z. Zhou, Y . Zhang, H. Cai, M. Aloqaily, O. Bouachir, L. Pang, P. Mehrotra, K. Wang, and Q. Wen, “MCPShield: A Security Cognition Layer for Adaptive Trust Calibration in Model Context Protocol Agents,” arXiv preprint arXiv:2602.14281, Feb. 2026
arXiv 2026
-
[8]
X. Wenpeng, Q. Zhonghao, Q. Yupeng, L. Yilin, C. Caini, Y . Jiahui, L. Changting, X. Zhenzhen, and H. Meng, “MCP-Guard: A Multi-Stage Defense-in-Depth Framework for Securing Model Context Protocol in Agentic AI,” arXiv preprint arXiv:2508.10991, Aug. 2025
arXiv 2025
-
[9]
Y . Wang, “MindGuard: Tracking, Detecting, and Attributing MCP Tool Poisoning Attack via Decision Dependence Graph,” arXiv preprint arXiv:2508.20412, Aug. 2025
arXiv 2025
-
[10]
MCPTox: A Benchmark for Tool Poisoning on Real-World MCP Servers,
Z. Wang, Y . Gao, Y . Wang, S. Liu, H. Sun, H. Cheng, G. Shi, H. Du, and X. Li, “MCPTox: A Benchmark for Tool Poisoning on Real-World MCP Servers,” inProc. AAAI Conf. Artif. Intell., vol. 40, no. 42, pp. 35811–35819, 2026
2026
-
[11]
Beyond the Protocol: Empirical Analysis of MCP Attack Scenarios,
S. Song et al., “Beyond the Protocol: Empirical Analysis of MCP Attack Scenarios,” arXiv preprint arXiv:2506.02040, Jun. 2025
Pith/arXiv arXiv 2025
-
[12]
A systematic survey on multi- step attack detection,
J. Navarro, A. Deruyver, and P. Parrend, “A systematic survey on multi- step attack detection,”Computers & Security, vol. 76, pp. 214–249, 2018
2018
-
[13]
Real-time multistep attack prediction based on hidden Markov models,
P. Holgado, V . A. Villagr´a, and L. V ´azquez, “Real-time multistep attack prediction based on hidden Markov models,”IEEE Trans. Dependable Secure Comput., vol. 17, no. 1, pp. 134–147, Jan. 2020
2020
-
[14]
KAIROS: Practical Intrusion Detection and Investigation using Whole- system Provenance,
Z. Cheng, Q. Lv, J. Liang, Y . Wang, D. Sun, T. Pasquier, and X. Han, “KAIROS: Practical Intrusion Detection and Investigation using Whole- system Provenance,” inProc. IEEE Symp. Security Privacy (S&P), 2024, p. 3533
2024
-
[15]
MAAC: Multi-step Attack Alert Correlation using Semantic Vectorization,
J. Wang, “MAAC: Multi-step Attack Alert Correlation using Semantic Vectorization,” inProc. IEEE INFOCOM, 2021
2021
-
[16]
MCP Safety Audit: LLMs with the Model Context Protocol Allow Major Security Vulnerabilities,
B. Radosevich and J. Halloran, “MCP Safety Audit: LLMs with the Model Context Protocol Allow Major Security Vulnerabilities,” arXiv preprint arXiv:2504.03767, Apr. 2025
Pith/arXiv arXiv 2025
-
[17]
X. Zong, Z. Sun, L. Wang, Y . Lan, and C. Yang, “MCP-SafetyBench: A Benchmark for Safety Evaluation of Large Language Models with Real- World MCP Servers,” arXiv preprint arXiv:2512.15163, Dec. 2025
arXiv 2025
-
[18]
V . S. Narajala and I. Habler, “Enterprise-Grade Security for the Model Context Protocol (MCP): Frameworks and Mitigation Strategies,” arXiv preprint arXiv:2504.08623, Apr. 2025
Pith/arXiv arXiv 2025
-
[19]
R. F. D. Rosario, “Temporal Attack Pattern Detection in Multi-Agent AI Workflows: An Open Framework for Training Trace-Based Security Models,” arXiv preprint arXiv:2601.00848, Jan. 2026
arXiv 2026
-
[20]
From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflows,
M. A. Ferrag, N. Tihanyi, D. Hamouda, L. Maglaras, A. Lakas, and M. Debbah, “From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflows,”ICT Express, vol. 12, no. 2, p. 353, 2025
2025
-
[21]
Cross-Agent Privilege Escalation: When Agents Free Each Other,
J. Rehberger, “Cross-Agent Privilege Escalation: When Agents Free Each Other,” Embrace The Red, Sep. 2025. [Online]. Available: https://embracethered.com/blog/posts/2025/ cross-agent-privilege-escalation-agents-that-free-each-other/
2025
-
[22]
Poison Everywhere: No Output from Your MCP Server is Safe,
S. Kosman, “Poison Everywhere: No Output from Your MCP Server is Safe,” CyberArk Labs, Dec. 2025. [Online]. Available: https://www.cyberark.com/resources/threat-research-blog/ poison-everywhere-no-output-from-your-mcp-server-is-safe
2025
-
[2024]
Available: https://www.anthropic.com/news/ model-context-protocol
[Online]. Available: https://www.anthropic.com/news/ model-context-protocol
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.