GUI agents can transform live web interfaces in real-time via DOM manipulations to deliver contextual assistance directly within the application.
Canonical reference
Decepticon: How dark patterns manipulate web agents
Canonical reference. 80% of citing Pith papers cite this work as background.
5
Pith papers citing it
Background
80% of classified citations
citation-role summary
background 5
citation-polarity summary
years
2026 5roles
background 5representative citing papers
SnapGuard detects prompt injection attacks on screenshot-based web agents via visual stability indicators and contrast-polarity textual signals, reaching F1 0.75 while running 8x faster than GPT-4o with no added memory cost.
PageGuide grounds LLM answers in webpage DOM elements using visual overlays for find, guide, and hide modes, yielding measurable gains in a 94-user study.
Oversight strategy in computer-use agents shapes exposure to problematic actions more reliably than correction success, with plan-based approaches reducing occurrences but not uniformly improving interventions.
The paper develops a unified framework that organizes computer-use agent reliability around perception-decision-execution layers and creation-deployment-operation-maintenance stages to map security and alignment interventions.
citing papers explorer
-
Beyond Chat and Clicks: GUI Agents for In-Situ Assistance via Live Interface Transformation
GUI agents can transform live web interfaces in real-time via DOM manipulations to deliver contextual assistance directly within the application.
-
SnapGuard: Lightweight Prompt Injection Detection for Screenshot-Based Web Agents
SnapGuard detects prompt injection attacks on screenshot-based web agents via visual stability indicators and contrast-polarity textual signals, reaching F1 0.75 while running 8x faster than GPT-4o with no added memory cost.
-
PageGuide: Browser extension to assist users in navigating a webpage and locating information
PageGuide grounds LLM answers in webpage DOM elements using visual overlays for find, guide, and hide modes, yielding measurable gains in a 94-user study.
-
Comparing Human Oversight Strategies for Computer-Use Agents
Oversight strategy in computer-use agents shapes exposure to problematic actions more reliably than correction success, with plan-based approaches reducing occurrences but not uniformly improving interventions.
-
Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliability
The paper develops a unified framework that organizes computer-use agent reliability around perception-decision-execution layers and creation-deployment-operation-maintenance stages to map security and alignment interventions.