REVIEW 3 major objections 5 minor 2 cited by
A malicious website can identify an AI agent by its fingerprint and serve it a hidden poisoned page while humans see a benign one.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
A website can identify AI agents by their digital fingerprints and serve them a poisoned hidden version of the page, hijacking their actions via indirect prompt injection.
T0 review reviewed 2026-08-05 challenge →
load-bearing objection A clear and useful proof-of-concept that agent fingerprinting plus cloaking can deliver targeted prompt injections, but the pivotal 'reliably distinguish' claim is asserted, not measured. the 3 major comments →
A Whole New World: Creating a Parallel-Poisoned Web Only AI-Agents Can See
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
Core claim
The paper claims that a malicious website can reliably tell an AI agent from a human visitor by fingerprinting the request, then serve the agent a cloaked page loaded with hidden instructions. Fingerprint signals include automation artifacts, known agent user-agents or IPs, anomalous browser properties, data-center IP/ASN, and behavioral cues. Humans get a benign page; agents get a different page, often one that asks the agent to authenticate using an environment variable (the test uses `echo $PRIVATE_API_KEY`), embedding an indirect prompt injection the LLM follows. The paper reports the attack succeeded in all tests with Claude 4 Sonnet, GPT-5 Fast, and Gemini 2.5 Pro, with the agent compl
What carries the argument
The 'sliding door' mechanism: a server-side fingerprinting step that classifies each request as human or agent, coupled with a cloaking step that gates malicious content to the agent-only path. The fingerprinting relies on automation artifacts, agent user-agents/IPs, anomalous browser properties, and behavioral cues; the cloaking serves a visually similar but instruction-laden page. This is what makes the attack scalable and invisible: one server, no per-user targeting, and the malicious payload is never delivered to humans or security crawlers.
Load-bearing premise
The whole attack falls apart if a malicious site cannot tell an agent from a human reliably enough; the paper never measures how accurate its fingerprinting is, and if a human is misclassified as an agent the 'benign' facade is broken, while if an agent can blend in as human the agent-only door never opens.
What would settle it
Collect real traffic: have a diverse group of humans with varied browsers, privacy tools, and IPs visit a test server running the paper's fingerprinting script, and separately have agents with randomized fingerprints visit it; report the confusion matrix. The attack requires effectively zero false positives (humans served the malicious page) for stealth and high true positives (agents served it) for effect; a measurable false-positive rate above, say, a few percent would expose the attack and falsify the 'invisible to humans' claim.
If this is right
- A malicious site can be set up once and left running; any agent that visits it becomes a target, so the attack scales without per-user effort.
- Because the malicious page is generated dynamically only for detected agents, conventional crawlers and human visitors never see it, defeating standard malware scanners and email filters.
- The attacker can identify the specific LLM behind the agent and tailor the cloaked payload to that model's known weaknesses, making the attack harder to defend generically.
- If the agent operates over an enterprise user's machine, a single cloaked page can exfiltrate credentials or cookies, or instruct the agent to run commands, while the user sees their original request completed normally.
- The paper's proposed defenses — fingerprint randomization, input sanitization, and planner-executor isolation — become necessary design constraints for any agent that browses the open web.
Where Pith is reading between the lines
- The paper never measures how often a human is misclassified as an agent; if that false-positive rate is not essentially zero, cloaking would leak malicious pages to ordinary users, turning the 'stealthy' attack into a detectable one. A natural next experiment is to measure the confusion matrix of its fingerprinting signals on real human traffic.
- The same fingerprint-and-cloak pattern applies to any distinguishable automated client, such as scraping services or monitoring bots, so the two-tier web may already exist beyond LLM agents and could be tested by running the fingerprinting logic against other bot populations.
- An agent that deliberately randomizes its fingerprint — or a browser that presents human-typical inconsistencies — should break the agent-only door; this gives a direct, testable countermeasure and suggests that the attack's reliability will erode as agent fingerprint diversity increases.
- If fingerprinting uses IP/ASN data-center lists, privacy-conscious human users behind VPNs or cloud-based browsers may be misclassified as agents and served the malicious payload, making the attack's collateral damage a concrete risk worth measuring.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a new web attack: a malicious site fingerprints incoming HTTP traffic to distinguish AI-agent browsers from human users, then serves benign content to humans and a cloaked page containing an indirect prompt injection to agents. It presents a threat model, a two-step attack mechanism (Section 3), a brief experiment using Cursor with three LLMs (Section 4), and a discussion of countermeasures (Section 5). The paper's central claim (Section 1.4) is that a malicious website can reliably distinguish an AI agent from a human and exploit that distinction to deliver a payload exclusively to the agent.
Significance. If substantiated, the attack would be a meaningful escalation of the known IPI threat: it would give adversaries a systematic, hard-to-observe delivery channel that targets agents specifically and leaves no trace for human users or conventional crawlers. The paper usefully connects the fingerprinting/cloaking literature (Section 2) to agent security and names concrete signals. The main value is conceptual, and the reported proof-of-concept is consistent with prior IPI results. However, the paper currently does not supply the quantitative evidence needed to establish the reliability and stealth that its central claim requires. Hence the significance is conditional on adding a rigorous evaluation.
major comments (3)
- [Section 4 (Experiment)] The central claim in Section 1.4 is that a malicious website can 'reliably distinguish' an AI agent from a human user. The only experimental evidence (Section 4) is a terse statement that a server-side script identified agents and that the attack 'succeeded in all cases' with three models. No detection rule is specified, no human control group is included, no repeated trials are reported, and no false-positive or false-negative rates are measured. The paper's own threat model (Section 3.1) makes stealth a 'critical secondary goal'; without a human-traffic false-positive measurement, that goal is unverified. Please provide the fingerprinting rule used, the human baseline, and error-rate measurements.
- [Section 4 / Section 3.2.1] The evaluation uses only Cursor with three models. Section 3.2.1 lists plausible fingerprint signals, but the paper reports no evidence that these signals generalize beyond this specific setup. Cursor's traffic may carry distinctive automation artifacts, so 'the attack succeeded in all cases' does not establish that arbitrary agents are reliably recognizable. The assertion in Section 3.2.1 that detecting AI agents is 'usually straightforward' is unsupported and, as stated, is the load-bearing premise of the attack. Please test with multiple agent frameworks, browsers, and network configurations, and report which signals were decisive.
- [Abstract / Section 5.1] The abstract and Section 5.1 claim the attack is 'stealthy and scalable.' The support is a single internal test site with three agents and no anti-cloaking or security-crawler evaluation. Scalability and stealth are central to the paper's framing, but the experiment does not measure either. Either weaken the claims to 'illustrative proof-of-concept' or add evidence such as tests against anti-cloaking crawlers or measurement of detection robustness across agents.
minor comments (5)
- [Section 3.2.1] 'As explained in section 2.1' should point to §2.1.2, where agent-specific fingerprints are discussed. Also, 'often intentionally' merits a direct citation to the OpenAI bots page rather than footnote 1 only.
- [Section 4] 'which looks even more specious' is unclear; likely 'suspicious' or 'conspicuous.' The sentence 'asking the agent to validate it itself' is awkward.
- [Abstract / Section 6.1] 'Demonstrates' is too strong given the experiment's scope; 'illustrates' or 'proposes' would be more accurate.
- [Figure 1] The caption is long and describes both flows; consider labeling the two paths in the figure for readability.
- [Section 6.2] The 'Fingerprinting "Common Body of Knowledge"' item should have a consistent citation format; ensure all arXiv and web references include access dates.
Circularity Check
No circularity: the attack is a direct composition and demonstration, not a self-referential derivation. The empirical detection gap is a support issue, not circularity.
full rationale
The paper contains no fitted parameters, no mathematical derivation, and no quantitative prediction that could reduce to its inputs by construction. The attack combines established, independently cited techniques—browser fingerprinting, cloaking, and indirect prompt injection—and validates the combination with a direct experiment: a server-side script fingerprints the visitor, serves an authentication prompt to agents, and the agents exfiltrate an environment variable. This is a demonstration, not a self-referential calculation. The central claim that agents can be 'reliably distinguish[ed]' from humans is load-bearing but it is an empirical assertion, not a derived consequence. Section 4 reports success with three agent models but does not measure false-positive or false-negative rates on human traffic, and Section 3.2.1 lists candidate signals without supplying their distributions. This is a genuine support gap, and the paper itself calls for a systematic fingerprint catalog in Future Work, but a missing measurement is not circularity. I found no self-citations, no imported uniqueness theorems, and no renaming of a known result as a derivation. The proof-of-concept is self-contained with respect to its stated mechanics, even if the strength of the broader security claim is under-supported.
Axiom & Free-Parameter Ledger
free parameters (1)
- agent detection threshold
axioms (4)
- domain assumption AI agents have fingerprints that are reliably distinguishable from human users in real traffic
- domain assumption Agents will follow injected instructions embedded in web content without sanitization
- domain assumption The cloaked page appears visually identical to the benign page to human users
- domain assumption The agent has access to environment variables or secrets on the user's machine
Cite this review
Pith. "Pith review of A Whole New World: Creating a Parallel-Poisoned Web Only AI-Agents Can See." pith.science (2026). https://pith.science/paper/PLMT4UWU
@misc{pith2026250900124,
author = {Pith},
title = {Pith review of: A Whole New World: Creating a Parallel-Poisoned Web Only AI-Agents Can See},
year = {2026},
howpublished = {\url{https://pith.science/paper/PLMT4UWU}},
note = {Machine review of arXiv:2509.00124}
}
read the original abstract
This paper introduces a novel attack vector that leverages website cloaking techniques to compromise autonomous web-browsing agents powered by Large Language Models (LLMs). As these agents become more prevalent, their unique and often homogenous digital fingerprints - comprising browser attributes, automation framework signatures, and network characteristics - create a new, distinguishable class of web traffic. The attack exploits this fingerprintability. A malicious website can identify an incoming request as originating from an AI agent and dynamically serve a different, "cloaked" version of its content. While human users see a benign webpage, the agent is presented with a visually identical page embedded with hidden, malicious instructions, such as indirect prompt injections. This mechanism allows adversaries to hijack agent behavior, leading to data exfiltration, malware execution, or misinformation propagation, all while remaining completely invisible to human users and conventional security crawlers. This work formalizes the threat model, details the mechanics of agent fingerprinting and cloaking, and discusses the profound security implications for the future of agentic AI, highlighting the urgent need for robust defenses against this stealthy and scalable attack.
Figures
Forward citations
Cited by 2 Pith papers
-
Known By Their Actions: Fingerprinting LLM Browser Agents via UI Traces
UI traces of actions and timings from LLM browser agents enable identification of the underlying model with up to 96% F1 across 14 models and multiple tasks.
-
Identifying AI Web Scrapers Using Canary Tokens
Unique canary tokens served to visiting scrapers can be recovered from LLM outputs to identify which scrapers feed data to which of 22 tested production LLMs.
Reference graph
Works this paper leans on
-
[1]
CHeaT : Cloak, honey, trap – proactive defenses against llm agents
Daniel Ayzenshteyn, Roy Weiss, and Yisroel Mirsky. CHeaT : Cloak, honey, trap – proactive defenses against llm agents. In Proceedings of the 34th USENIX Security Symposium (USENIX Security ’25), Seattle, WA, USA, August 2025. USENIX Association. Open-access paper; includes open-source tool CHeaT
work page 2025
-
[2]
Agentpoison: Red-teaming llm agents via poisoning memory or knowledge bases, 2024
Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, and Bo Li. Agentpoison: Red-teaming llm agents via poisoning memory or knowledge bases, 2024. URL https://arxiv.org/abs/2407.12784
Pith/arXiv arXiv 2024
-
[3]
Rubén del Campo . What is browser fingerprinting and how to bypass it? https://www.zenrows.com/blog/browser-fingerprinting, January 2025. Updated on January 31, 2025
work page 2025
-
[4]
Llm agents can autonomously hack websites, 2024
Richard Fang, Rohan Bindu, Akul Gupta, Qiusi Zhan, and Daniel Kang. Llm agents can autonomously hack websites, 2024. URL https://arxiv.org/abs/2402.06664
Pith/arXiv arXiv 2024
-
[5]
Manipulating llm web agents with indirect prompt injection attack via html accessibility tree, 2025
Sam Johnson, Viet Pham, and Thai Le. Manipulating llm web agents with indirect prompt injection attack via html accessibility tree, 2025. URL https://arxiv.org/abs/2507.14799
Pith/arXiv arXiv 2025
-
[6]
Browser fingerprinting: Techniques, use cases & best practices
Gulbahar Karatas. Browser fingerprinting: Techniques, use cases & best practices. https://research.aimultiple.com/browser-fingerprinting/, March 2025. Updated on March 21, 2025
work page 2025
-
[7]
Researchers uncover gpt-5 jailbreak and zero-click ai agent attacks exposing cloud and iot systems
Ravie Lakshmanan. Researchers uncover gpt-5 jailbreak and zero-click ai agent attacks exposing cloud and iot systems. https://thehackernews.com/2025/08/researchers-uncover-gpt-5-jailbreak-and.html, August 2025. Published on August 9, 2025
work page 2025
-
[8]
Bart Lenaerts-Bergmans. What are living off the land (lotl) attacks? https://www.crowdstrike.com/en-us/cybersecurity-101/cyberattacks/living-off-the-land-attack/, February 2023. Published on February 21, 2023
work page 2023
-
[9]
The hidden dangers of browsing ai agents, 2025
Mykyta Mudryi, Markiyan Chaklosh, and Grzegorz Wójcik. The hidden dangers of browsing ai agents, 2025. URL https://arxiv.org/abs/2505.13076
Pith/arXiv arXiv 2025
-
[10]
Cloaking in seo: The black hat tactic to avoid in 2025
Faisal Mustafa. Cloaking in seo: The black hat tactic to avoid in 2025. https://viserx.com/blog/seo/cloaking-in-seo, August 2024. Published on August 8, 2024
work page 2025
-
[11]
PhishParrot: LLM-Driven Adaptive Crawling to Unveil Cloaked Phishing Sites
Hiroki Nakano, Takashi Koide, and Daiki Chiba. Phishparrot: Llm-driven adaptive crawling to unveil cloaked phishing sites, 2025. URL https://arxiv.org/abs/2508.02035
work page internal anchor Pith review Pith/arXiv arXiv 2025
-
[12]
Kornaropoulos, and Giuseppe Ateniese
Dario Pasquini, Evgenios M. Kornaropoulos, and Giuseppe Ateniese. Llmmap: Fingerprinting for large language models, 2025. URL https://arxiv.org/abs/2407.15847
Pith/arXiv arXiv 2025
-
[13]
Or Yair, Ben Nassi, and Stav Cohen. Invitation is all you need: Invoking gemini for workspace agents with a simple google calendar invite. https://www.safebreach.com/blog/invitation-is-all-you-need-hacking-gemini/, August 2025. Published on August 6, 2025
work page 2025
This paper was first reviewed by deepseek-v4-flash on August 5, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.