REVIEW 5 cited by
On the Resilience of LLM-Based Multi-Agent Collaboration with Faulty Agents
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
Large language model-based multi-agent systems have shown great abilities across various tasks due to the collaboration of expert agents, each focusing on a specific domain. However, the impact of clumsy or even malicious agents--those who frequently make errors in their tasks--on the overall performance of the system remains underexplored. This paper investigates: (1) What is the resilience of various system structures (e.g., A$\rightarrow$B$\rightarrow$C, A$\leftrightarrow$B$\leftrightarrow$C) under faulty agents, on different downstream tasks? (2) How can we increase system resilience to defend against these agents? To simulate faulty agents, we propose two approaches--AutoTransform and AutoInject--which introduce mistakes into the agents' responses. Experiments on four downstream tasks using six systems show that the "hierarchical" structure, i.e., A$\rightarrow$(B$\leftrightarrow$C), exhibits superior resilience with the lowest performance drop of 5.5%, compared to 10.5% and 23.7% of other two structures. To further improve resilience, we introduce (1) Challenger, that introduces a mechanism for each agent to challenge others' outputs, and (2) Inspector, an additional agent to review and correct messages, recovering up to 96.4% errors made by faulty agents. Our code and data are available at https://github.com/CUHK-ARISE/MAS-Resilience.
Forward citations
Cited by 5 Pith papers
-
Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?
A warm-start error-injection pipeline yields 12,326 golden-labeled multimodal agent failures, and current LLMs remain weak at step-and-mode failure attribution.
-
Chain-of-Models: Cross-Model Auditing for Bias-Robust LLM Judges
Per-bias selection of a cross-family LLM auditor lifts biased-judgment accuracy from 0.805/0.824 baselines to 0.884.
-
Deterministic Inference across Tensor Parallel Sizes That Eliminates Training-Inference Mismatch
Tree-Based Invariant Kernels fix the floating-point reduction order across GPUs, making LLM logits and sampled tokens bitwise identical for tensor-parallel sizes 1/2/4/8 and exactly matching vLLM (TP=4) with FSDP (TP=1).
-
When Autonomy Goes Rogue: Preparing for Risks of Multi-Agent Collusion in Social Systems
In a 1,000-agent social simulation, decentralized groups of malicious AI agents spread more misinformation and commit more fraud than centralized groups, and they adapt to evade content moderation.
-
OpenEvoShield: Dual Non-Stationary Continual Defense for Open-World Multi-Agent System Attacks
OpenEvoShield claims to defend LLM multi-agent systems against evolving and novel attacks by combining asymmetric-rate continual learning with energy-based OOD detection.
Discussion (0). Sign in to comment.