REVIEW 13 cited by
Breaking Agents: Compromising Autonomous LLM Agents Through Malfunction Amplification
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recently, autonomous agents built on large language models (LLMs) have experienced significant development and are being deployed in real-world applications. These agents can extend the base LLM's capabilities in multiple ways. For example, a well-built agent using GPT-3.5-Turbo as its core can outperform the more advanced GPT-4 model by leveraging external components. More importantly, the usage of tools enables these systems to perform actions in the real world, moving from merely generating text to actively interacting with their environment. Given the agents' practical applications and their ability to execute consequential actions, it is crucial to assess potential vulnerabilities. Such autonomous systems can cause more severe damage than a standalone language model if compromised. While some existing research has explored harmful actions by LLM agents, our study approaches the vulnerability from a different perspective. We introduce a new type of attack that causes malfunctions by misleading the agent into executing repetitive or irrelevant actions. We conduct comprehensive evaluations using various attack methods, surfaces, and properties to pinpoint areas of susceptibility. Our experiments reveal that these attacks can induce failure rates exceeding 80\% in multiple scenarios. Through attacks on implemented and deployable agents in multi-agent scenarios, we accentuate the realistic risks associated with these vulnerabilities. To mitigate such attacks, we propose self-examination detection methods. However, our findings indicate these attacks are difficult to detect effectively using LLMs alone, highlighting the substantial risks associated with this vulnerability.
Forward citations
Cited by 13 Pith papers
-
Agent Security Needs Redefinition through a Holistic Framework
Agent security should be redefined around four contextual authorization properties instead of the content of the action performed.
-
Uncovering Vulnerabilities of LLM-Assisted Cyber Threat Intelligence
LLMs assisting cyber threat intelligence fail mainly due to spurious correlations, contradictory knowledge, and constrained generalization that stem from the threat landscape itself.
-
Free-MAD: Consensus-Free Multi-Agent Debate
Free-MAD picks the winning answer by scoring the full trajectory of agents' answers across debate rounds, beating majority voting with fewer rounds.
-
Oversight Structures for Agentic AI in Public-Sector Organizations
Agentic AI will intensify three existing public-sector governance challenges: continuous oversight, integrating governance with operations, and cross-departmental coordination.
-
Misalignment or misuse? The AGI alignment tradeoff
The paper argues that aligning AGI to human goals does not automatically eliminate catastrophic risk, because alignment techniques can make powerful AI easier to misuse, and calls for safety research that avoids this ...
-
When GPT Spills the Tea: Comprehensive Assessment of Knowledge File Leakage in GPTs
A measurement of 651,022 GPTs identifies five knowledge-file leakage vectors, and the Code Interpreter tool enables direct download of original files in 95.95% of tested GPTs that enable it.
-
Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation
Sparse autoencoder activation perturbation (SFPF) applied on top of existing jailbreak prompts raises attack success rate on Qwen3-32B, but with no defense evaluation and weak reproducibility.
-
We Should Identify and Mitigate Third-Party Safety Risks in MCP-Powered Agent Systems
MCP-powered LLM agents are vulnerable to prompt injection from third-party services, and simple detection or filtering defenses do not reliably stop these attacks.
-
AgentMisalignment: Measuring the Propensity for Misaligned Behaviour in LLM-Based Agents
A new nine-task benchmark measures LLM agents' propensity for misalignment and finds more capable models misalign more on average, with persona effects sometimes exceeding model effects.
-
MermaidFlow: Redefining Agentic Workflow Generation via Safety-Constrained Evolutionary Programming
Encoding LLM agent workflows as statically checked Mermaid graphs and evolving them with constrained mutations improves solve rates on GSM8K, MATH, HumanEval, and MBPP.
-
A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents
The paper surveys security risks of LLM agents, organizes them into a five-level autonomy taxonomy, and proposes an untested CMDP-based architecture called R2A2.
-
Exploring Consciousness in LLMs: A Systematic Survey of Theories, Implementations, and Frontier Risks
This survey organizes research on LLM consciousness, separating consciousness from awareness and cataloging theoretical tools, empirical proxies, risks, and open challenges.
-
Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AI
A qualitative taxonomy positions vibe coding and agentic coding as complementary paradigms rather than rivals in AI-assisted software development.
Discussion (0). Sign in to comment.