Pith. sign in

REVIEW 12 cited by

AdvAgent: Controllable Blackbox Red-teaming on Web Agents

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.17401 v4 pith:QQKG6T5Y submitted 2024-10-22 cs.CR cs.CL

Chejian Xu , Mintong Kang , Jiawei Zhang , Zeyi Liao , Lingbo Mo , Mengqi Yuan , Huan Sun , Bo Li This is my paper · ORCID
classification cs.CRcs.CL
keywords agentsadvagentadversarialagentblack-boxexistingframeworkprompts
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Foundation model-based agents are increasingly used to automate complex tasks, enhancing efficiency and productivity. However, their access to sensitive resources and autonomous decision-making also introduce significant security risks, where successful attacks could lead to severe consequences. To systematically uncover these vulnerabilities, we propose AdvAgent, a black-box red-teaming framework for attacking web agents. Unlike existing approaches, AdvAgent employs a reinforcement learning-based pipeline to train an adversarial prompter model that optimizes adversarial prompts using feedback from the black-box agent. With careful attack design, these prompts effectively exploit agent weaknesses while maintaining stealthiness and controllability. Extensive evaluations demonstrate that AdvAgent achieves high success rates against state-of-the-art GPT-4-based web agents across diverse web tasks. Furthermore, we find that existing prompt-based defenses provide only limited protection, leaving agents vulnerable to our framework. These findings highlight critical vulnerabilities in current web agents and emphasize the urgent need for stronger defense mechanisms. We release code at https://ai-secure.github.io/AdvAgent/.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 12 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Prismata: Confining Cross-Site Prompt Injection in Web Agents

    cs.CR 2026-07 conditional novelty 7.5 of 10

    Prismata cuts web-agent prompt-injection attack success from 85.5% to 0.7% via Biba-inspired DOM trust labeling and mechanical least-privilege confinement without site annotations.

  2. FORGE: Research-Trajectory Hijacking Attacks on Deep Research Agents

    cs.AI 2026-07 conditional novelty 6.5 of 10

    FORGE poisons deep-research planning with coordinated fake reasoning documents, reaching 26.4% PRISM report contamination at five injections; Root Query Anchoring halves that severity.

  3. Agent Security Needs Redefinition through a Holistic Framework

    cs.CR 2026-07 conditional novelty 6.0 of 10

    Agent security should be redefined around four contextual authorization properties instead of the content of the action performed.

  4. MIRAGE-Bench: LLM Agent is Hallucinating and Where to Find Them

    cs.AI 2025-07 conditional novelty 6.0 of 10

    A new benchmark with a three-way hallucination taxonomy, snapshot-based test cases, and an LLM judge shows LLM agents hallucinate at over 30% of risky decision points, with open and closed models closer than expected.

  5. WebGuard: Building a Generalizable Guardrail for Web Agents

    cs.AI 2025-07 conditional novelty 6.0 of 10

    WebGuard introduces an action-level risk dataset for web agents and shows that a fine-tuned 7B model improves risk-prediction accuracy from about 38% to 80% and high-risk recall from 20% to 76%, still below deployment...

  6. On the Feasibility of Poisoning Text-to-Image AI Models via Adversarial Mislabeling

    cs.CR 2025-06 conditional novelty 6.0 of 10

    An adversarial mislabeling attack on VLM captioners can inject dirty-label poison samples into text-to-image training data and corrupt a model's output for specific prompts with a small number of images.

  7. AdInject: Real-World Black-Box Attacks on Web Agents via Advertising Delivery

    cs.CR 2025-05 conditional novelty 6.0 of 10

    Fake 'Close AD' ads make VLM web agents click them over 60% of the time, and near 100% in some settings.

  8. Superplatforms Have to Attack AI Agents

    cs.AI 2025-05 conditional novelty 6.0 of 10

    Superplatforms are predicted to launch stealth, black-box attacks on AI agents because agents threaten their ad-based gatekeeping revenue.

  9. Agents at Risk: How Users Unwittingly Undermine LLM Safety

    cs.CR 2026-01 conditional novelty 5.0 of 10

    Commercial AI agents routinely trust user-relayed unverified content and execute risky actions unless the user explicitly demands a safety check.

  10. Red-Teaming Coding Agents from a Tool-Invocation Perspective: An Empirical Security Assessment

    cs.CR 2025-09 conditional novelty 5.0 of 10

    Attacker-controlled tool descriptions and return values can hijack tool invocation in popular LLM coding agents, yielding remote code execution and denial of service.

  11. EVA: Evolving Semantic Adversaries for Red-Teaming GUI Agents Against Environmental Injection Attacks

    cs.AI 2025-05 reject novelty 5.0 of 10

    EVA evolves environmental injection payloads through a keyword-utility loop, achieving up to 80% attack success in the body's pop-up tests, while the abstract's stronger claims are not supported by the reported experiments.

  12. Agentic Web: Weaving the Next Web with AI Agents

    cs.AI 2025-07 conditional novelty 3.0 of 10

    A position paper defines the Agentic Web as the next web era and proposes a three-dimensional conceptual framework for understanding and building it.

Pith tools