Pith. sign in

REVIEW 9 cited by

MemoRAG: Boosting Long Context Processing with Global Memory-Enhanced Retrieval Augmentation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2409.05591 v3 pith:6ZGDMFRN submitted 2024-09-09 cs.CL cs.AI

Hongjin Qian , Zheng Liu , Peitian Zhang , Kelong Mao , Defu Lian , Zhicheng Dou , Tiejun Huang This is my paper
classification cs.CLcs.AI
keywords longmemoragcontextglobalprocessingretrievalcontextsexpensive
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Processing long contexts presents a significant challenge for large language models (LLMs). While recent advancements allow LLMs to handle much longer contexts than before (e.g., 32K or 128K tokens), it is computationally expensive and can still be insufficient for many applications. Retrieval-Augmented Generation (RAG) is considered a promising strategy to address this problem. However, conventional RAG methods face inherent limitations because of two underlying requirements: 1) explicitly stated queries, and 2) well-structured knowledge. These conditions, however, do not hold in general long-context processing tasks. In this work, we propose MemoRAG, a novel RAG framework empowered by global memory-augmented retrieval. MemoRAG features a dual-system architecture. First, it employs a light but long-range system to create a global memory of the long context. Once a task is presented, it generates draft answers, providing useful clues for the retrieval tools to locate relevant information within the long context. Second, it leverages an expensive but expressive system, which generates the final answer based on the retrieved information. Building upon this fundamental framework, we realize the memory module in the form of KV compression, and reinforce its memorization and cluing capacity from the Generation quality's Feedback (a.k.a. RLGF). In our experiments, MemoRAG achieves superior performances across a variety of long-context evaluation tasks, not only complex scenarios where traditional RAG methods struggle, but also simpler ones where RAG is typically applied.

Discussion (0). Sign in to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MemSIF: From Structured Interactions to Dual-Track Fact Memory for LLM Agents

    cs.AI 2026-08 conditional novelty 6.0 of 10

    MemSIF improves long-term LLM agent memory by combining structured interaction organization with dual-track fact memory, reporting top Total ACC on LoCoMo and LongMemEval-S across five backbones.

  2. Agents-K1: Towards Agent-native Knowledge Orchestration

    cs.AI 2026-06 unverdicted novelty 6.0 of 10

    Agents-K1 is an end-to-end pipeline with a multimodal parser, 4B GRPO-trained extractor, and agent CLI that builds scientific knowledge graphs from full papers and was run on 2.46 million documents to produce Scholar-KG.

  3. VulRTex: A Reasoning-Guided Approach to Identify Vulnerabilities from Rich-Text Issue Report

    cs.SE 2025-09 conditional novelty 6.0 of 10

    A retrieval-augmented LLM approach that identifies vulnerability-related issue reports and CWE types from screenshots and code snippets, improving F1 by 11 points and AUPRC by 20 points over baselines.

  4. EraRAG: Efficient and Incremental Retrieval Augmented Generation for Growing Corpora

    cs.IR 2025-06 conditional novelty 6.0 of 10

    EraRAG uses hyperplane-based locality-sensitive hashing to build a hierarchical retrieval graph whose affected regions only are re-summarized when new documents arrive, cutting update cost by up to an order of magnitude.

  5. ConMem: Contribution-Aware Memory for Long-Horizon Manufacturing Inspection Logs

    cs.AI 2026-07 conditional novelty 5.5 of 10

    Role-segmented inspection memories ranked by approximated Shapley contribution yield 76% early-risk QA accuracy with ~88% fewer tokens than naive 8K-context LLMs.

  6. GOSU: Retrieval-Augmented Generation with Global-Level Optimized Semantic Unit-Centric Framework

    cs.CL 2025-08 reject novelty 5.0 of 10

    GOSU globally merges semantic units from text chunks into a unit-centric knowledge graph and uses three-tier keyword retrieval to improve RAG generation quality, according to LLM-judge win rates.

  7. SafeDriveRAG: Towards Safe Autonomous Driving with Knowledge Graph-based Retrieval-Augmented Generation

    cs.AI 2025-07 conditional novelty 5.0 of 10

    SafeDrive228K is a 228K-example multimodal QA benchmark for traffic safety, and a graph-based RAG method improves VLM accuracy on it by 4.7 to 14.6 points across five models.

  8. Enterprise Large Language Model Evaluation Benchmark

    cs.AI 2025-06 reject novelty 5.0 of 10

    A 14-task enterprise LLM benchmark built mostly from GPT-4o-generated labels and scored by GPT-4o-as-judge shows open-source models closing the reasoning gap, but the dataset is not public and the evaluation is partly...

  9. Toward Efficient Agents: Memory, Tool learning, and Planning

    cs.AI 2026-01 conditional novelty 3.0 of 10

    A survey that organizes efficiency techniques for LLM agents into memory, tool learning, and planning, and consolidates benchmarks and metrics for measuring cost-performance trade-offs.

Pith tools