REVIEW 9 cited by
Walking Down the Memory Maze: Beyond Context Limit through Interactive Reading
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) have advanced in large strides due to the effectiveness of the self-attention mechanism that processes and compares all tokens at once. However, this mechanism comes with a fundamental issue -- the predetermined context window is bound to be limited. Despite attempts to extend the context window through methods like extrapolating the positional embedding, using recurrence, or selectively retrieving essential parts of the long sequence, long-text understanding continues to be a challenge. We propose an alternative approach which instead treats the LLM as an interactive agent, allowing it to decide how to read the text via iterative prompting. We introduce MemWalker, a method that first processes the long context into a tree of summary nodes. Upon receiving a query, the model navigates this tree in search of relevant information, and responds once it gathers sufficient information. On long-text question answering tasks our method outperforms baseline approaches that use long context windows, recurrence, and retrieval. We show that, beyond effective reading, MemWalker enhances explainability by highlighting the reasoning steps as it interactively reads the text; pinpointing the relevant text segments related to the query.
Forward citations
Cited by 9 Pith papers
-
Asynchronous Collective Tree Exploration: a Distributed Algorithm, and a new Lower Bound
Distributed asynchronous robot teams can explore any tree in 2n + O(k^2 2^k D) moves, and no asynchronous algorithm can beat competitive ratio Ω(log^2 k).
-
Is Progressive Disclosure All You Need for Long-Context Agents?
Progressive disclosure is redundant for a strong agent reading one book, but decisive when an agent must navigate a 20-book library; one flat routing level beats deeper hierarchies.
-
Gavel: Agent Meets Checklist for Evaluating LLMs on Long-Context Legal Summarization
Gavel uses a 26-item checklist, residual-fact, and style scoring to show that top LLMs cover only about half the key content in long legal case summaries and omit information more often than they invent it.
-
HGMEM: Hypergraph-based Working Memory to Improve Multi-step RAG for Long-Context Complex Relational Modeling
A working memory represented as a hypergraph, whose hyperedges are updated, inserted, and progressively merged by the LLM, improves multi-step RAG on long-context sense-making benchmarks.
-
Select, Read, and Write: A Multi-Agent Framework of Full-Text-based Related Work Generation
A multi-agent reader-selector-writer framework improves generated related-work sections by reading full texts in a graph-guided order and compressing key information into shared memory.
-
QwenLong-CPRS: Towards $\infty$-LLMs with Dynamic Context Optimization
QwenLong-CPRS is a 7B instruction-guided compressor that shrinks long contexts to query-relevant spans, boosting downstream LLM accuracy and cutting prefill cost.
-
VecTree-RAG: An Agentic Retrieval-Augmented Generation Framework Combining Vector and Tree Retrieval for Efficiency and Accuracy
An agentic RAG framework that splits corpus-level discovery (vector retrieval) from within-paper evidence localization (tree navigation) reports top scores on QASPER, LitQA2, and a new MOSAIC benchmark.
-
MemOS: An Operating System for Memory-Augmented Generation (MAG) in Large Language Models
A unified memory-operating-system design for LLMs, built around a MemCube abstraction, is presented without any experimental validation.
-
Memory-Augmented Transformers: A Systematic Review from Neuroscience Principles to Enhanced Model Architectures
Memory-augmented Transformer research is organized into a three-axis taxonomy bridging neuroscience memory concepts to network designs, but no new result is produced.
Discussion (0). Continue with ORCID to comment.