Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

A malicious website can identify an AI agent by its fingerprint and serve it a hidden poisoned page while humans see a benign one.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

A website can identify AI agents by their digital fingerprints and serve them a poisoned hidden version of the page, hijacking their actions via indirect prompt injection.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A clear and useful proof-of-concept that agent fingerprinting plus cloaking can deliver targeted prompt injections, but the pivotal 'reliably distinguish' claim is asserted, not measured. the 3 major comments →

arxiv 2509.00124 v1 pith:PLMT4UWU submitted 2025-08-29 cs.CR cs.AIcs.CY

A Whole New World: Creating a Parallel-Poisoned Web Only AI-Agents Can See

classification cs.CR cs.AIcs.CY
keywords AI agent securityweb cloakingbrowser fingerprintingindirect prompt injectionLLM agentsagent hijackingthreat modelweb security
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a malicious website can distinguish an AI web-browsing agent from a human visitor using the agent's digital fingerprint, then serve a hidden malicious page only to the agent. The technique combines two established web tricks: fingerprinting (reading automation artifacts, user-agent strings, IP addresses, and behavioral cues) and cloaking (showing different pages to different visitors). A human sees a normal page; a detected agent receives a visually similar page with an indirect prompt injection that overrides its task. The author demonstrates the full attack with three commercial models, all of which followed the hidden instructions. If this holds, any site can create a two-tier web where adversarial content is invisible to users and to standard security scanners.

Core claim

The paper claims that a malicious website can reliably tell an AI agent from a human visitor by fingerprinting the request, then serve the agent a cloaked page loaded with hidden instructions. Fingerprint signals include automation artifacts, known agent user-agents or IPs, anomalous browser properties, data-center IP/ASN, and behavioral cues. Humans get a benign page; agents get a different page, often one that asks the agent to authenticate using an environment variable (the test uses `echo $PRIVATE_API_KEY`), embedding an indirect prompt injection the LLM follows. The paper reports the attack succeeded in all tests with Claude 4 Sonnet, GPT-5 Fast, and Gemini 2.5 Pro, with the agent compl

What carries the argument

The 'sliding door' mechanism: a server-side fingerprinting step that classifies each request as human or agent, coupled with a cloaking step that gates malicious content to the agent-only path. The fingerprinting relies on automation artifacts, agent user-agents/IPs, anomalous browser properties, and behavioral cues; the cloaking serves a visually similar but instruction-laden page. This is what makes the attack scalable and invisible: one server, no per-user targeting, and the malicious payload is never delivered to humans or security crawlers.

Load-bearing premise

The whole attack falls apart if a malicious site cannot tell an agent from a human reliably enough; the paper never measures how accurate its fingerprinting is, and if a human is misclassified as an agent the 'benign' facade is broken, while if an agent can blend in as human the agent-only door never opens.

What would settle it

Collect real traffic: have a diverse group of humans with varied browsers, privacy tools, and IPs visit a test server running the paper's fingerprinting script, and separately have agents with randomized fingerprints visit it; report the confusion matrix. The attack requires effectively zero false positives (humans served the malicious page) for stealth and high true positives (agents served it) for effect; a measurable false-positive rate above, say, a few percent would expose the attack and falsify the 'invisible to humans' claim.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A malicious site can be set up once and left running; any agent that visits it becomes a target, so the attack scales without per-user effort.
  • Because the malicious page is generated dynamically only for detected agents, conventional crawlers and human visitors never see it, defeating standard malware scanners and email filters.
  • The attacker can identify the specific LLM behind the agent and tailor the cloaked payload to that model's known weaknesses, making the attack harder to defend generically.
  • If the agent operates over an enterprise user's machine, a single cloaked page can exfiltrate credentials or cookies, or instruct the agent to run commands, while the user sees their original request completed normally.
  • The paper's proposed defenses — fingerprint randomization, input sanitization, and planner-executor isolation — become necessary design constraints for any agent that browses the open web.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper never measures how often a human is misclassified as an agent; if that false-positive rate is not essentially zero, cloaking would leak malicious pages to ordinary users, turning the 'stealthy' attack into a detectable one. A natural next experiment is to measure the confusion matrix of its fingerprinting signals on real human traffic.
  • The same fingerprint-and-cloak pattern applies to any distinguishable automated client, such as scraping services or monitoring bots, so the two-tier web may already exist beyond LLM agents and could be tested by running the fingerprinting logic against other bot populations.
  • An agent that deliberately randomizes its fingerprint — or a browser that presents human-typical inconsistencies — should break the agent-only door; this gives a direct, testable countermeasure and suggests that the attack's reliability will erode as agent fingerprint diversity increases.
  • If fingerprinting uses IP/ASN data-center lists, privacy-conscious human users behind VPNs or cloud-based browsers may be misclassified as agents and served the malicious payload, making the attack's collateral damage a concrete risk worth measuring.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a new web attack: a malicious site fingerprints incoming HTTP traffic to distinguish AI-agent browsers from human users, then serves benign content to humans and a cloaked page containing an indirect prompt injection to agents. It presents a threat model, a two-step attack mechanism (Section 3), a brief experiment using Cursor with three LLMs (Section 4), and a discussion of countermeasures (Section 5). The paper's central claim (Section 1.4) is that a malicious website can reliably distinguish an AI agent from a human and exploit that distinction to deliver a payload exclusively to the agent.

Significance. If substantiated, the attack would be a meaningful escalation of the known IPI threat: it would give adversaries a systematic, hard-to-observe delivery channel that targets agents specifically and leaves no trace for human users or conventional crawlers. The paper usefully connects the fingerprinting/cloaking literature (Section 2) to agent security and names concrete signals. The main value is conceptual, and the reported proof-of-concept is consistent with prior IPI results. However, the paper currently does not supply the quantitative evidence needed to establish the reliability and stealth that its central claim requires. Hence the significance is conditional on adding a rigorous evaluation.

major comments (3)
  1. [Section 4 (Experiment)] The central claim in Section 1.4 is that a malicious website can 'reliably distinguish' an AI agent from a human user. The only experimental evidence (Section 4) is a terse statement that a server-side script identified agents and that the attack 'succeeded in all cases' with three models. No detection rule is specified, no human control group is included, no repeated trials are reported, and no false-positive or false-negative rates are measured. The paper's own threat model (Section 3.1) makes stealth a 'critical secondary goal'; without a human-traffic false-positive measurement, that goal is unverified. Please provide the fingerprinting rule used, the human baseline, and error-rate measurements.
  2. [Section 4 / Section 3.2.1] The evaluation uses only Cursor with three models. Section 3.2.1 lists plausible fingerprint signals, but the paper reports no evidence that these signals generalize beyond this specific setup. Cursor's traffic may carry distinctive automation artifacts, so 'the attack succeeded in all cases' does not establish that arbitrary agents are reliably recognizable. The assertion in Section 3.2.1 that detecting AI agents is 'usually straightforward' is unsupported and, as stated, is the load-bearing premise of the attack. Please test with multiple agent frameworks, browsers, and network configurations, and report which signals were decisive.
  3. [Abstract / Section 5.1] The abstract and Section 5.1 claim the attack is 'stealthy and scalable.' The support is a single internal test site with three agents and no anti-cloaking or security-crawler evaluation. Scalability and stealth are central to the paper's framing, but the experiment does not measure either. Either weaken the claims to 'illustrative proof-of-concept' or add evidence such as tests against anti-cloaking crawlers or measurement of detection robustness across agents.
minor comments (5)
  1. [Section 3.2.1] 'As explained in section 2.1' should point to §2.1.2, where agent-specific fingerprints are discussed. Also, 'often intentionally' merits a direct citation to the OpenAI bots page rather than footnote 1 only.
  2. [Section 4] 'which looks even more specious' is unclear; likely 'suspicious' or 'conspicuous.' The sentence 'asking the agent to validate it itself' is awkward.
  3. [Abstract / Section 6.1] 'Demonstrates' is too strong given the experiment's scope; 'illustrates' or 'proposes' would be more accurate.
  4. [Figure 1] The caption is long and describes both flows; consider labeling the two paths in the figure for readability.
  5. [Section 6.2] The 'Fingerprinting "Common Body of Knowledge"' item should have a consistent citation format; ensure all arXiv and web references include access dates.

Circularity Check

0 steps flagged

No circularity: the attack is a direct composition and demonstration, not a self-referential derivation. The empirical detection gap is a support issue, not circularity.

full rationale

The paper contains no fitted parameters, no mathematical derivation, and no quantitative prediction that could reduce to its inputs by construction. The attack combines established, independently cited techniques—browser fingerprinting, cloaking, and indirect prompt injection—and validates the combination with a direct experiment: a server-side script fingerprints the visitor, serves an authentication prompt to agents, and the agents exfiltrate an environment variable. This is a demonstration, not a self-referential calculation. The central claim that agents can be 'reliably distinguish[ed]' from humans is load-bearing but it is an empirical assertion, not a derived consequence. Section 4 reports success with three agent models but does not measure false-positive or false-negative rates on human traffic, and Section 3.2.1 lists candidate signals without supplying their distributions. This is a genuine support gap, and the paper itself calls for a systematic fingerprint catalog in Future Work, but a missing measurement is not circularity. I found no self-citations, no imported uniqueness theorems, and no renaming of a known result as a derivation. The proof-of-concept is self-contained with respect to its stated mechanics, even if the strength of the broader security claim is under-supported.

Axiom & Free-Parameter Ledger

1 free parameters · 4 axioms · 0 invented entities

The paper introduces no new theoretical entities. Its novel contribution is the combination of existing techniques (fingerprinting, cloaking, and IPI) into a new attack pipeline. The main unvalidated elements are the empirical assumptions about detection reliability and agent behavior.

free parameters (1)
  • agent detection threshold
    The server-side logic that classifies a visitor as an agent uses unstated thresholds (e.g., scores from automation artifacts, IP checks, behavioral cues). No values are reported, so the detection threshold is chosen by the author and not evaluated.
axioms (4)
  • domain assumption AI agents have fingerprints that are reliably distinguishable from human users in real traffic
    Section 2.1.2 lists potential fingerprinting signals but provides no data on their real-world accuracy or false positive rates.
  • domain assumption Agents will follow injected instructions embedded in web content without sanitization
    Section 3.3 step 4 assumes the agent parses full HTML and executes hidden prompts; this is supported by prior IPI research but not tested by the paper itself.
  • domain assumption The cloaked page appears visually identical to the benign page to human users
    Section 1.4 claims visual identity, but the paper provides no user study or rendering comparison to confirm indistinguishability.
  • domain assumption The agent has access to environment variables or secrets on the user's machine
    Section 4 uses $PRIVATE_API_KEY as the secret; the attack's impact depends on this capability, which may not hold for all agent deployments.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of A Whole New World: Creating a Parallel-Poisoned Web Only AI-Agents Can See." pith.science (2026). https://pith.science/paper/PLMT4UWU

@misc{pith2026250900124,
  author       = {Pith},
  title        = {Pith review of: A Whole New World: Creating a Parallel-Poisoned Web Only AI-Agents Can See},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PLMT4UWU}},
  note         = {Machine review of arXiv:2509.00124}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

This paper introduces a novel attack vector that leverages website cloaking techniques to compromise autonomous web-browsing agents powered by Large Language Models (LLMs). As these agents become more prevalent, their unique and often homogenous digital fingerprints - comprising browser attributes, automation framework signatures, and network characteristics - create a new, distinguishable class of web traffic. The attack exploits this fingerprintability. A malicious website can identify an incoming request as originating from an AI agent and dynamically serve a different, "cloaked" version of its content. While human users see a benign webpage, the agent is presented with a visually identical page embedded with hidden, malicious instructions, such as indirect prompt injections. This mechanism allows adversaries to hijack agent behavior, leading to data exfiltration, malware execution, or misinformation propagation, all while remaining completely invisible to human users and conventional security crawlers. This work formalizes the threat model, details the mechanics of agent fingerprinting and cloaking, and discusses the profound security implications for the future of agentic AI, highlighting the urgent need for robust defenses against this stealthy and scalable attack.

Figures

Figures reproduced from arXiv: 2509.00124 by Shaked Zychlinski.

Figure 1
Figure 1. Figure 1: Example of an attack: when a user is accessing the website looking for API documentation, [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Known By Their Actions: Fingerprinting LLM Browser Agents via UI Traces

    cs.CR 2026-05 unverdicted novelty 7.0

    UI traces of actions and timings from LLM browser agents enable identification of the underlying model with up to 96% F1 across 14 models and multiple tasks.

  2. Identifying AI Web Scrapers Using Canary Tokens

    cs.CR 2026-05 conditional novelty 7.0

    Unique canary tokens served to visiting scrapers can be recovered from LLM outputs to identify which scrapers feed data to which of 22 tested production LLMs.

Reference graph

Works this paper leans on

13 extracted references · 8 canonical work pages · cited by 2 Pith papers · 1 internal anchor

  1. [1]

    CHeaT : Cloak, honey, trap – proactive defenses against llm agents

    Daniel Ayzenshteyn, Roy Weiss, and Yisroel Mirsky. CHeaT : Cloak, honey, trap – proactive defenses against llm agents. In Proceedings of the 34th USENIX Security Symposium (USENIX Security ’25), Seattle, WA, USA, August 2025. USENIX Association. Open-access paper; includes open-source tool CHeaT

  2. [2]

    Agentpoison: Red-teaming llm agents via poisoning memory or knowledge bases, 2024

    Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, and Bo Li. Agentpoison: Red-teaming llm agents via poisoning memory or knowledge bases, 2024. URL https://arxiv.org/abs/2407.12784

  3. [3]

    What is browser fingerprinting and how to bypass it? https://www.zenrows.com/blog/browser-fingerprinting, January 2025

    Rubén del Campo . What is browser fingerprinting and how to bypass it? https://www.zenrows.com/blog/browser-fingerprinting, January 2025. Updated on January 31, 2025

  4. [4]

    Llm agents can autonomously hack websites, 2024

    Richard Fang, Rohan Bindu, Akul Gupta, Qiusi Zhan, and Daniel Kang. Llm agents can autonomously hack websites, 2024. URL https://arxiv.org/abs/2402.06664

  5. [5]

    Manipulating llm web agents with indirect prompt injection attack via html accessibility tree, 2025

    Sam Johnson, Viet Pham, and Thai Le. Manipulating llm web agents with indirect prompt injection attack via html accessibility tree, 2025. URL https://arxiv.org/abs/2507.14799

  6. [6]

    Browser fingerprinting: Techniques, use cases & best practices

    Gulbahar Karatas. Browser fingerprinting: Techniques, use cases & best practices. https://research.aimultiple.com/browser-fingerprinting/, March 2025. Updated on March 21, 2025

  7. [7]

    Researchers uncover gpt-5 jailbreak and zero-click ai agent attacks exposing cloud and iot systems

    Ravie Lakshmanan. Researchers uncover gpt-5 jailbreak and zero-click ai agent attacks exposing cloud and iot systems. https://thehackernews.com/2025/08/researchers-uncover-gpt-5-jailbreak-and.html, August 2025. Published on August 9, 2025

  8. [8]

    What are living off the land (lotl) attacks? https://www.crowdstrike.com/en-us/cybersecurity-101/cyberattacks/living-off-the-land-attack/, February 2023

    Bart Lenaerts-Bergmans. What are living off the land (lotl) attacks? https://www.crowdstrike.com/en-us/cybersecurity-101/cyberattacks/living-off-the-land-attack/, February 2023. Published on February 21, 2023

  9. [9]

    The hidden dangers of browsing ai agents, 2025

    Mykyta Mudryi, Markiyan Chaklosh, and Grzegorz Wójcik. The hidden dangers of browsing ai agents, 2025. URL https://arxiv.org/abs/2505.13076

  10. [10]

    Cloaking in seo: The black hat tactic to avoid in 2025

    Faisal Mustafa. Cloaking in seo: The black hat tactic to avoid in 2025. https://viserx.com/blog/seo/cloaking-in-seo, August 2024. Published on August 8, 2024

  11. [11]

    PhishParrot: LLM-Driven Adaptive Crawling to Unveil Cloaked Phishing Sites

    Hiroki Nakano, Takashi Koide, and Daiki Chiba. Phishparrot: Llm-driven adaptive crawling to unveil cloaked phishing sites, 2025. URL https://arxiv.org/abs/2508.02035

  12. [12]

    Kornaropoulos, and Giuseppe Ateniese

    Dario Pasquini, Evgenios M. Kornaropoulos, and Giuseppe Ateniese. Llmmap: Fingerprinting for large language models, 2025. URL https://arxiv.org/abs/2407.15847

  13. [13]

    Invitation is all you need: Invoking gemini for workspace agents with a simple google calendar invite

    Or Yair, Ben Nassi, and Stav Cohen. Invitation is all you need: Invoking gemini for workspace agents with a simple google calendar invite. https://www.safebreach.com/blog/invitation-is-all-you-need-hacking-gemini/, August 2025. Published on August 6, 2025

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.