Pith. sign in

REVIEW 10 cited by

Retrieval meets Long Context Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.03025 v2 pith:2HW5FCXT submitted 2023-10-04 cs.CL cs.AIcs.IRcs.LG

Peng Xu , Wei Ping , Xianchao Wu , Lawrence McAfee , Chen Zhu , Zihan Liu , Sandeep Subramanian , Evelina Bakhturina
show 2 more authors
Mohammad Shoeybi Bryan Catanzaro
This is my paper
classification cs.CLcs.AIcs.IRcs.LG
keywords contextwindowlongllmstasksretrievalretrieval-augmentationwhile
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Extending the context window of large language models (LLMs) is getting popular recently, while the solution of augmenting LLMs with retrieval has existed for years. The natural questions are: i) Retrieval-augmentation versus long context window, which one is better for downstream tasks? ii) Can both methods be combined to get the best of both worlds? In this work, we answer these questions by studying both solutions using two state-of-the-art pretrained LLMs, i.e., a proprietary 43B GPT and Llama2-70B. Perhaps surprisingly, we find that LLM with 4K context window using simple retrieval-augmentation at generation can achieve comparable performance to finetuned LLM with 16K context window via positional interpolation on long context tasks, while taking much less computation. More importantly, we demonstrate that retrieval can significantly improve the performance of LLMs regardless of their extended context window sizes. Our best model, retrieval-augmented Llama2-70B with 32K context window, outperforms GPT-3.5-turbo-16k and Davinci003 in terms of average score on nine long context tasks including question answering, query-based summarization, and in-context few-shot learning tasks. It also outperforms its non-retrieval Llama2-70B-32k baseline by a margin, while being much faster at generation. Our study provides general insights on the choice of retrieval-augmentation versus long context extension of LLM for practitioners.

Discussion (0). Sign in to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. When Knowledge Changes: Metamorphic Testing of RAG Systems with Mutations

    cs.SE 2026-07 conditional novelty 6.5 of 10

    Eleven corpus mutations expose 4.9–10.2% metamorphic violations in RAG pipelines, with an oracle F1 of 0.927–1.000 versus at most 0.570 for RAGAS.

  2. Token Reduction Is Not Cost Reduction

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Token reduction of tool output does not reliably lower billed coding-agent cost; prompt-cache traffic dominates, and aggressive compression can also destroy task success.

  3. Exploratory Retrieval-Augmented Planning For Continual Embodied Instruction Following

    cs.AI 2025-09 conditional novelty 6.0 of 10

    ExRAP couples LLM planning with a temporal knowledge-graph memory and information-based exploration, improving success and efficiency for continual embodied instruction following.

  4. Positional Biases Shift as Inputs Approach Context Window Limits

    cs.CL 2025-08 conditional novelty 6.0 of 10

    The lost-in-the-middle effect in LLMs appears mainly when inputs fill up to half the model's context window; beyond that, accuracy favors information closest to the end.

  5. Context is Gold to find the Gold Passage: Evaluating and Training Contextual Document Embeddings

    cs.IR 2025-05 conditional novelty 6.0 of 10

    A new benchmark (ConTEB) and training method (InSeNT) show that context-aware chunk embeddings greatly improve retrieval on context-dependent queries, with minimal computational overhead.

  6. NovelHopQA: Diagnosing Multi-Hop Reasoning Failures in Long Narrative Contexts

    cs.CL 2025-05 conditional novelty 6.0 of 10

    NovelHopQA is a new benchmark that pairs long novel excerpts with 1-4 hop questions and shows LLM accuracy drops consistently with both context length and reasoning depth.

  7. 100-LongBench: Are de facto Long-Context Benchmarks Literally Evaluating Long-Context Ability?

    cs.CL 2025-05 conditional novelty 5.0 of 10

    100-LongBench and LongScore evaluate LLMs at controlled context lengths and rank models by relative performance drop from a short-context baseline, not by raw accuracy.

  8. SELF: Self-Extend the Context Length With Logistic Growth Function

    cs.CL 2025-05 conditional novelty 5.0 of 10

    SELF replaces Self-Extend's fixed token grouping with a logistic-growth grouping schedule, giving mixed but sometimes large gains on long-context benchmarks.

  9. Interfaze: The Future of AI is built on Task-Specific Small Models

    cs.AI 2026-02 reject novelty 4.0 of 10

    Interfaze-Beta uses small specialist models and tools to build a compact context that a general-purpose LLM answers from, reporting competitive benchmark scores without reproducible evidence.

  10. DRAG: Distilling RAG for SLMs from LLMs to Transfer Knowledge and Mitigate Hallucination via Evidence and Graph-based Distillation

    cs.CL 2025-06 reject novelty 4.0 of 10

    A small model prompted with evidence and knowledge graphs generated by GPT-4o scores much higher on QA benchmarks, but the result is not true distillation and may be contaminated by teacher answer leakage.

Pith tools