REVIEW 6 cited by
In Defense of RAG in the Era of Long-Context Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Overcoming the limited context limitations in early-generation LLMs, retrieval-augmented generation (RAG) has been a reliable solution for context-based answer generation in the past. Recently, the emergence of long-context LLMs allows the models to incorporate much longer text sequences, making RAG less attractive. Recent studies show that long-context LLMs significantly outperform RAG in long-context applications. Unlike the existing works favoring the long-context LLM over RAG, we argue that the extremely long context in LLMs suffers from a diminished focus on relevant information and leads to potential degradation in answer quality. This paper revisits the RAG in long-context answer generation. We propose an order-preserve retrieval-augmented generation (OP-RAG) mechanism, which significantly improves the performance of RAG for long-context question-answer applications. With OP-RAG, as the number of retrieved chunks increases, the answer quality initially rises, and then declines, forming an inverted U-shaped curve. There exist sweet points where OP-RAG could achieve higher answer quality with much less tokens than long-context LLM taking the whole context as input. Extensive experiments on public benchmark demonstrate the superiority of our OP-RAG.
Forward citations
Cited by 6 Pith papers
-
Jet-Long: Efficient Long-Context Extension with Dynamic Bifocal RoPE
Jet-Long is a tuning-free bifocal RoPE method that dynamically sets remote group size from sequence length, recovering the base model within the pretrained window and beating prior zero-shot extenders on RULER, HELMET...
-
JointRank: Rank Large Set with Single Pass
JointRank partitions candidates into overlapping blocks, ranks each block in parallel with an LLM, and reconstructs a global ranking by aggregating the resulting pairwise comparisons.
-
LaRA: Benchmarking Retrieval-Augmented Generation and Long-Context LLMs -- No Silver Bullet for LC or RAG Routing
LaRA shows that RAG versus long-context performance depends on model size, context length, task, and chunk settings, with no universal winner.
-
Scaling Multi-Document Event Summarization: Evaluating Compression vs. Full-Text Approaches
Full-context and retrieval-based methods outperform hierarchical and incremental compression for large-scale multi-document summarization, though compression methods show strong intermediate information retention.
-
GSM-Infinite: How Do Your LLMs Behave over Infinitely Increasing Context Length and Reasoning Complexity?
A new synthetic benchmark reveals that LLM reasoning accuracy decays sigmoidally with problem complexity and that repeated sampling has poor scaling efficiency.
-
Position: Episodic Memory is the Missing Piece for Long-Term LLM Agents
The authors propose episodic memory, with five defining properties, as the unifying framework needed for LLM agents to learn and remember over long time horizons.
Discussion (0). Continue with ORCID to comment.