REVIEW 5 cited by
LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaboration
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
abstract
Large language models (LLMs) have demonstrated impressive performance in understanding language and executing complex reasoning tasks. However, LLMs with long context windows have been notorious for their expensive training costs and high inference latency. Even the most advanced models such as GPT-4 and Claude2 often make mistakes when processing inputs of over $100k$ tokens, a phenomenon also known as \textit{lost in the middle}. In this paper, we propose \textsc{LongAgent}, a method based on multi-agent collaboration, which scales LLMs (e.g., LLaMA) to a context of 128K and demonstrates potential superiority in long-text processing compared to GPT-4. In \textsc{LongAgent}, a leader is responsible for understanding user intent and directing team members to acquire information from documents. Due to members' hallucinations, it is non-trivial for a leader to obtain accurate information from the responses of dozens to hundreds of members. To address this, we develop an \textit{inter-member communication} mechanism to resolve response conflicts caused by hallucinations through information sharing. Our experimental results indicate that \textsc{LongAgent} offers a promising alternative for long-text processing. The agent team instantiated with LLaMA-7B achieves significant improvements in tasks such as 128k-long text retrieval, multi-hop question answering, compared to GPT-4.
Forward citations
Cited by 5 Pith papers
-
VCA: Video Curious Agent for Long Video Understanding
A training-free video agent that combines segment-level tree search, self-generated intrinsic rewards, and a fixed memory buffer achieves higher long-video QA accuracy with fewer observed frames than prior agent baselines.
-
HERA: Improving Long Document Summarization using Large Language Models with Context Packaging and Reordering
HERA packages event-related paragraphs and reorders them so LLMs produce more faithful and fluent summaries of long documents without fine-tuning.
-
Eliciting In-context Retrieval and Reasoning for Long-context Large Language Models
A new benchmark with hard confounders shows long-context LMs struggle at in-context retrieval, and a retrieve-then-generate fine-tune plus attention-probing decoding substantially improves them.
-
SWIRL: A Staged Workflow for Interleaved Reinforcement Learning in Mobile GUI Control
A multi-agent RL workflow that interleaves single-agent updates, applied to mobile GUI control, achieves SOTA zero-shot performance and a +14.8 MATH500 gain.
-
A Survey on LLM-based Multi-Agent System: Recent Advances and New Frontiers in Application
This survey organizes recent LLM-based multi-agent research into task-solving, simulation, and agent-evaluation applications, and identifies efficiency and evaluation gaps as key open problems.
Discussion (0). Continue with ORCID to comment.