Prismata cuts web-agent prompt-injection attack success from 85.5% to 0.7% via Biba-inspired DOM trust labeling and mechanical least-privilege confinement without site annotations.
Ad- vweb: Controllable black-box attacks on vlm-powered web agents
11 Pith papers cite this work. Polarity classification is still indexing.
11
Pith papers citing it
abstract
Foundation model-based agents are increasingly used to automate complex tasks, enhancing efficiency and productivity. However, their access to sensitive resources and autonomous decision-making also introduce significant security risks, where successful attacks could lead to severe consequences. To systematically uncover these vulnerabilities, we propose AdvAgent, a black-box red-teaming framework for attacking web agents. Unlike existing approaches, AdvAgent employs a reinforcement learning-based pipeline to train an adversarial prompter model that optimizes adversarial prompts using feedback from the black-box agent. With careful attack design, these prompts effectively exploit agent weaknesses while maintaining stealthiness and controllability. Extensive evaluations demonstrate that AdvAgent achieves high success rates against state-of-the-art GPT-4-based web agents across diverse web tasks. Furthermore, we find that existing prompt-based defenses provide only limited protection, leaving agents vulnerable to our framework. These findings highlight critical vulnerabilities in current web agents and emphasize the urgent need for stronger defense mechanisms. We release code at https://ai-secure.github.io/AdvAgent/.
citation-role summary
background 1
citation-polarity summary
roles
background 1polarities
background 1representative citing papers
HLL is a new benchmark that evaluates eight frontier multimodal agents on closed-loop interactive CAPTCHA solving, showing sharp performance drops under realism stressors and trace validation.
ShieldNet detects supply-chain poisoned tools in LLM agents by monitoring network interactions with a MITM proxy and lightweight classifier, reaching 0.995 F1 and 0.8% false positives on a new benchmark of 25+ attack types.
SkillTrojan backdoors skill-based agents by partitioning an encrypted payload across benign-looking skills that reassemble and execute only under a predefined trigger.
Introduces an app-content instrumentation framework and benchmark showing that examined GUI agents suffer 42.0% and 36.1% average misleading rates from third-party content in dynamic and static tests respectively.
Progent introduces a privilege-control framework for AI agents that uses LLM-generated symbolic rules over tools, SMT-solver-enforced monotonic updates, and deterministic checks to reduce attack success rates on AgentDojo and ASB benchmarks.
MIRAGE creates perceptually benign adversarial images using diffusion and curvature-aware optimization to enable targeted prompt injection attacks on web agents like SeeAct and OpenClaw within attacker-controlled boundaries.
WARD is a guard model trained on 177K web samples and adversarially hardened via attacker-guard co-evolution to achieve high recall on prompt injections with low false positives and no added latency.
A survey that defines Computer-Using Agents for safety analysis, categorizes their threats, proposes a taxonomy of defensive strategies, and summarizes benchmarks and datasets for evaluating CUA safety and performance.
A survey that taxonomizes threats to agentic AI, reviews benchmarks and evaluation methods, discusses technical and governance defenses, and identifies open challenges.
LLM agents enable universal interoperability by serving as automatic translators and adapters between proprietary digital services.
citing papers explorer
-
Prismata: Confining Cross-Site Prompt Injection in Web Agents
Prismata cuts web-agent prompt-injection attack success from 85.5% to 0.7% via Biba-inspired DOM trust labeling and mechanical least-privilege confinement without site annotations.
-
HLL: Can Agents Cross Humanity's Last Line of Verification?
HLL is a new benchmark that evaluates eight frontier multimodal agents on closed-loop interactive CAPTCHA solving, showing sharp performance drops under realism stressors and trace validation.
-
ShieldNet: Network-Level Guardrails against Emerging Supply-Chain Injections in Agentic Systems
ShieldNet detects supply-chain poisoned tools in LLM agents by monitoring network interactions with a MITM proxy and lightweight classifier, reaching 0.995 F1 and 0.8% false positives on a new benchmark of 25+ attack types.
-
SkillTrojan: Backdoor Attacks on Skill-Based Agent Systems
SkillTrojan backdoors skill-based agents by partitioning an encrypted payload across benign-looking skills that reassemble and execute only under a predefined trigger.
-
Mobile GUI Agents under Real-world Threats: Are We There Yet?
Introduces an app-content instrumentation framework and benchmark showing that examined GUI agents suffer 42.0% and 36.1% average misleading rates from third-party content in dynamic and static tests respectively.
-
Progent: Securing AI Agents with Privilege Control
Progent introduces a privilege-control framework for AI agents that uses LLM-generated symbolic rules over tools, SMT-solver-enforced monotonic updates, and deterministic checks to reduce attack success rates on AgentDojo and ASB benchmarks.
-
MIRAGE: Stealthy Visual Prompt Injection for Vulnerability Detection in Web Agents
MIRAGE creates perceptually benign adversarial images using diffusion and curvature-aware optimization to enable targeted prompt injection attacks on web agents like SeeAct and OpenClaw within attacker-controlled boundaries.
-
WARD: Adversarially Robust Defense of Web Agents Against Prompt Injections
WARD is a guard model trained on 177K web samples and adversarially hardened via attacker-guard co-evolution to achieve high recall on prompt injections with low false positives and no added latency.
-
A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron?
A survey that defines Computer-Using Agents for safety analysis, categorizes their threats, proposes a taxonomy of defensive strategies, and summarizes benchmarks and datasets for evaluating CUA safety and performance.
-
Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges
A survey that taxonomizes threats to agentic AI, reviews benchmarks and evaluation methods, discusses technical and governance defenses, and identifies open challenges.
-
LLM Agents Are the Antidote to Walled Gardens
LLM agents enable universal interoperability by serving as automatic translators and adapters between proprietary digital services.