Pith. sign in

REVIEW 3 major objections 5 minor 44 references

PI-Mem: Pushing Long-Context Reasoning to 3.6M Tokens with Parallel-Iterative Memory

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read PI-Mem claims that reading all chunks in parallel and iteratively refining a shared memory makes long-context reasoning both more accurate and faster than recurrent-memory methods, with results on HotpotQA up to 3.6M tokens.

desk verdict The workflow idea is real and worth building on, but the headline 3.6M accuracy margin is within sampling noise and the paper needs multi-seed numbers before the claim stands. read the letter →

arxiv 2608.03048 v1 pith:YGUI4C2P submitted 2026-08-04 cs.CL cs.AI

Dawei Liu , Haixu Song , Shuang Cheng , Shijie Wang , Haozheng Hou , Kaifeng Liu , Ermo Hua , Zhonghang Yuan
show 4 more authors
Zhijie Zhong Yuchen Fan Biqing Qi Bowen Zhou
This is my paper · ORCID
classification cs.CLcs.AI
keywords long-contextreasoningparallelmemoryiterativerefinementreinforcementlearningmulti-hopquestionansweringRULERbenchmarkoverwriteturnefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper takes aim at the standard way of giving a language model a very long context: compress the document into a running memory that is rewritten chunk by chunk. That recurrent scheme, the authors argue, has two avoidable defects — later chunks can overwrite early evidence, and the chunk-to-chunk dependency forces latency to grow with document length. PI-Mem replaces it with a bounded number of parallel read–select–merge turns: each turn reads every chunk against the same shared memory, keeps only new evidence, and merges it into a compact memory for the next turn. On RULER HotpotQA at up to 3.6 million tokens, PI-Mem reports gains of +6.25 and +7.81 absolute points over MemAgent on two backbones while running 6.1x and 2.1x faster. The paper's claim is that memory-based long-context reasoning does not have to sacrifice accuracy for speed.

What carries the argument

The load-bearing object is the read–select–merge turn. In turn $k$, each chunk $c_i$ is read in parallel conditioned on the shared memory $m^{(k-1)}$ and must emit an explicit <check>yes</check>/<check>no</check> signal; observations with a yes signal form $O^{(k)}$, and a separate merge call compresses $O^{(k)}$ together with $m^{(k-1)}$ into a compact memory $m^{(k)}$. The loop exits when $O^{(k)}=\emptyset$ or $k=K$, and the final answer is generated from the question and the final memory only. The workflow is trained end-to-end with group-relative policy optimization, broadcasting a trajectory-level accuracy-plus-turn-efficiency reward $R_i=r_{\text{acc}}(a_i)+\lambda_{\text{turn}}\frac{K-k_i}{K-1}$ to every model call in the trajectory. The asymptotic cost of chunk reading is $O(KN(C+M)^2)$ under quadratic attention, but the serial depth is $O(K)$ rather than $O(N)$, which is where the latency gain comes from.

What would settle it

Take a multi-hop QA set with longer, real documents and compare PI-Mem against MemAgent after shuffling chunk order across repeated runs. If the parallel-iterative mechanism works by preventing overwrite, PI-Mem's accuracy should be nearly invariant to chunk order while MemAgent's should swing; if PI-Mem also degrades sharply under shuffling, the reported advantage is not coming from parallel reads. Alternatively, train PI-Mem without the turn-efficiency reward on the same data and check whether the turn count and latency gap to MemAgent disappears at 3.6M tokens.

Watch

Extended reading notes

Core claim

The central claim is that the accuracy loss and the latency of memory-based long-context QA both come from ordering, not from memory itself. PI-Mem conditions every chunk read on the current global memory $m^{(k-1)}$, filters observations to those the model tags with <check>yes</check>, and merges the survivors into $m^{(k)}$; the cycle repeats until no chunk yields new evidence or a maximum of $K$ turns is reached. Because all chunk reads within a turn are independent, the serial critical path shortens from $O(N)$ chunks to $O(K)$ turns, while the compact final memory protects earlier evidence from being overwritten. Training with a trajectory-level reinforcement-learning objective that adds a turn-efficiency reward makes the model exit early once enough evidence is gathered. In the reported experiments, this design outperforms the recurrent-memory baseline on RULER HotpotQA across lengths up to 3.6M tokens, with larger gains at longer contexts, and on the MV-NIAH task it retains nearly all ground-truth values in the final memory where the baseline loses a substantial share.

Load-bearing premise

The whole result depends on the RL-trained policy generalizing from synthetic HotpotQA training samples at 140K tokens (1000 documents) to contexts 25x longer and to RULER tasks that are not part of training; if that transfer fails, the ultra-long-context gains would be an artifact of the benchmark construction.

Editorial extensions

If this is right

  • Answer generation becomes context-free: the final call sees only the question and the consolidated memory, so inference cost for the answer step does not grow with document length.
  • Latency is bounded by the number of turns rather than the number of chunks; even at 3.6M tokens the reported end-to-end speedup over vanilla inference reaches 16.9x on Qwen3.5 and 14.4x on Qwen2.5.
  • Evidence retention is measurably better: on MV-NIAH, the final memory keeps essentially all target values, whereas the recurrent baseline loses a substantial fraction, consistent with the overwrite mechanism the paper claims to fix.
  • The accuracy gains appear to grow with context length and extend to RULER tasks outside the HQA training distribution, suggesting the parallel-iterative structure rather than benchmark memorization drives the result.
  • On LongBench v2, the workflow improves over the recurrent baseline overall, with the largest gain on hard and long questions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The read–select–merge loop is independent of how each chunk is read, so it could be stacked on sparse-attention or retrieval-based chunk readers to cut the per-chunk cost further; the paper does not test this combination.
  • The turn-efficiency reward is a step-level training signal that rewards any trajectory for finishing with fewer turns; the same principle could apply to other multi-call agentic loops (tool use, iterative retrieval, multi-agent debate), not just long-context QA.
  • The evidence-coverage metric on MV-NIAH is a direct, cheap probe of the overwrite failure; future work could use coverage at intermediate turns as a diagnostic to decide when K should be increased.
  • A stress test the authors do not run is real 3.6M-token documents with naturally noisy, repetitive, or adversarial structure; the synthetic HotpotQA-by-construction inputs are cleanly generated, so the select step may be less challenged than it would be in the wild.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes PI-Mem, a parallel-iterative memory mechanism for long-context reasoning. Instead of recurrently updating a memory chunk-by-chunk, PI-Mem reads all chunks in parallel in each of at most K read-select-merge turns, selects chunks that yield new evidence, merges them into a shared memory, and then produces the final answer from the consolidated memory. The workflow is trained end-to-end with GRPO using an accuracy reward plus a turn-efficiency bonus. Experiments on RULER HotpotQA and OOD tasks with Qwen3.5-35B-A3B and Qwen2.5-7B report accuracy gains over MemAgent (e.g., +6.25 and +7.81 absolute points at 3.6M tokens) and wall-clock speedups (6.1x and 2.1x vs MemAgent; up to 16.9x vs Vanilla), along with ablations, evidence-coverage diagnostics, and case studies.

Significance. If the reported gains are reproducible, PI-Mem is a valuable addition to long-context reasoning: it addresses a real failure mode of recurrent memory (overwriting early evidence) with a simple, parallelizable architecture, and the RL turn-efficiency reward is a sensible mechanism for controlling inference cost. The paper is unusually complete in its evaluation: two backbones, context lengths to 3.6M tokens, ablations of every component (select, merge, turn reward, K, chunk/memory size), a direct evidence-retention measurement, wall-clock latency and FLOP tables, and detailed case studies, with code released. The main weaknesses are statistical: the headline 3.6M HQA margins are within sampling error for n=64 single-run evaluations, and the Qwen2.5 baseline comparison is against an official checkpoint whose training budget may differ from the freshly trained PI-Mem. These are fixable with additional runs or more cautious claims, and they do not invalidate the overall pattern, especially the stronger OOD results.

major comments (3)
  1. [Technical Supplement §2.1; Tables 1 and 3] The manuscript states in the Technical Supplement that 'Each model reported in this work is trained in a single run, and each evaluation sample is evaluated once per method.' Under this protocol, the abstract's headline differences at 3.6M tokens — 76.56 vs 70.31 for Qwen3.5 and 81.25 vs 73.44 for Qwen2.5 — correspond to 4 and 5 correct answers out of 64 RULER HQA samples. The standard error of each difference is roughly 7–8 percentage points, so a 95% confidence interval for either margin includes zero. The across-length averages (81.25 vs 76.56 and 84.06 vs 77.50) are more stable, but the specific 3.6M advantage claimed in the abstract is not statistically distinguishable from chance as reported. Please provide multiple seeds or bootstrap confidence intervals across the 64 samples, or reframe the headline claim around the aggregate results.
  2. [Abstract and Conclusion vs Technical Supplement §2.3 (Tables 6–7)] The statement that PI-Mem 'breaks the accuracy–efficiency trade-off' is stronger than the evidence supports. As the Technical Supplement's own asymptotic analysis shows, PI-Mem performs O(KN(C+M)^2) chunk-reading work versus O(N(C+M)^2) for MemAgent, and the measured aggregate FLOPs at 3.6M tokens are 3,365 vs 2,055 PFLOPs (Qwen3.5) and 7,634 vs 4,463 PFLOPs (Qwen2.5). The speedup is in wall-clock latency, achieved through parallelism, not in total computation. The claim should be qualified as a wall-clock latency-accuracy trade-off (or a latency/parallelism benefit), with the FLOP overhead acknowledged in the main text; otherwise the current wording overstates the result.
  3. [Training Setup (main text) and Technical Supplement §1] For Qwen2.5-7B, PI-Mem is trained with 240 rollout steps and 16 rollouts per prompt on the released MemAgent training data, while the comparison baseline is the officially released MemAgent checkpoint, whose training schedule and data splits are not specified in this manuscript. Because the Qwen2.5 MemAgent baseline achieves an average HQA score (77.50) close to that of the much larger Qwen3.5 MemAgent baseline (76.56) trained for only 80 steps, the reader cannot exclude the possibility that part of the +7.81 advantage at 3.6M reflects a difference in training budget rather than the PI-Mem mechanism. Please report the training configuration of the official checkpoint, or retrain MemAgent under the same 240-step protocol as PI-Mem, and release the resulting baseline.
minor comments (5)
  1. [Figure 3] The figure reports speedups relative to Vanilla for all three methods, but the text highlights PI-Mem's speedup relative to MemAgent (6.1x/2.1x); adding the MemAgent-relative ratio to the figure or caption would make the main comparison easier to read.
  2. [Evaluation Setup, RAG baseline] The RAG baseline uses top-6 BM25 chunks with no ablation on k; since RAG is a lower-bound-style baseline, a sentence acknowledging the sensitivity of RAG to k would be useful.
  3. [Reward Design, Eq. (4)] The turn-efficiency coefficient λ_turn is fixed at 0.2 with no sensitivity analysis; given that the turn reward is one of the paper's contributions, reporting at least one other value (e.g., 0.1 or 0.4) would strengthen the claim that the choice is robust.
  4. [Technical Supplement §2.1] The statement that each model is trained in a single run and evaluated once should appear in the main text limitations as well, so that readers who do not consult the supplement are not misled about the confidence of the headline numbers.
  5. [Abstract] The abstract names the baseline only as 'the recurrent-memory baseline'; naming MemAgent would be more precise.

Circularity Check

1 steps flagged · score 2.0 of 10

Mostly self-contained; one minor by-construction element in the turn-efficiency reward ablation.

  1. self definitional [Method, Eq. (4); Ablation Study, Figure 6]
    "We augment answer accuracy with a turn-efficiency bonus: rturn(τi) = K−ki / K−1 ; Ri = racc(ai) + λturn rturn(τi). ... adding the turn reward produces a substantially larger reduction in the number of read-select-merge turns."

    Eq. (4) defines the turn bonus to increase monotonically as the turn count ki decreases, so the reduced mean turn count reported in Figure 6 (left) is an objective that was optimized, not an independently predicted consequence. The non-tautological part of the ablation is the right panel, which shows that the accuracy reward is maintained, and that result is empirical. The paper's headline accuracy and latency comparisons against MemAgent do not rest on this ablation, so the circularity is minor and auxiliary.

full rationale

PI-Mem's central claims are empirical and externally grounded: the parallel read-select-merge workflow (Algorithm 1, Eqs. 1-2) is a new mechanism, and its reported accuracy and speedup numbers are compared against external baselines (MemAgent, GRU-Mem, ReMemR1) and direct-inference methods on RULER and LongBench v2. HQA is explicitly labeled in-distribution, while the RULER OOD tasks provide independent grounding; the 3.6M-token HQA results are a length extrapolation from at most 140K-token training contexts, not a fitted value. RL uses an accuracy-plus-turn-efficiency objective (Eq. 4), and the only by-construction element is that the turn-efficiency reward directly rewards fewer turns; the informative accuracy-preservation result is empirical. Self-citations (e.g., QwenLong-L1.5 in related work) are contextual, not load-bearing, and no uniqueness theorem or ansatz is imported from the authors' prior work. The Technical Supplement's single-run/single-evaluation disclosure is a statistical-evidence limitation (no error bars), not a circularity.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The method introduces no new physical or mathematical entities. The free parameters are workflow hyperparameters (lambda_turn, K, chunk size, memory size) and baseline settings (YaRN scale, RAG top-k). The key assumptions are about LLM format compliance, memory sufficiency, training-to-evaluation transfer, and the attention-cost model used for asymptotic comparisons.

free parameters (6)
  • lambda_turn = 0.2
    Weights the turn-efficiency reward against the accuracy reward in Eq. 4. Chosen by hand (only one value reported, no sweep).
  • Maximum turns K = 3 (main), 5 (ablation)
    Bounds the number of read-select-merge turns. K=3 is used for both models; an ablation varies K from 1 to 5 to justify the choice.
  • Chunk size = 15,000 (Qwen3.5), 5,000 (Qwen2.5)
    Affects the parallelism and per-call context length. The Qwen3.5 value is set by the longer training context; the Qwen2.5 value follows MemAgent. An ablation shows robustness from 5K to 25K for Qwen3.5.
  • Memory size (max output length of merge call) = 4,096 (Qwen3.5), 1,024 (Qwen2.5)
    Bounds the merged memory length. The paper ablates 2K/4K/8K for Qwen3.5 and finds no consistent degradation.
  • YaRN scaling factor = 4.0
    Positional-encoding extrapolation factor for the YaRN baseline, set per common practice.
  • RAG top-k = 6
    Number of BM25-retrieved chunks for the RAG baseline.
assumptions (6)
  • domain assumption LLM outputs obey the <check>yes/no</check> format after RL training.
    The select step filters chunk observations by the check tag; if the model often emits malformed tags, the workflow degrades. Prompt templates and RL are used to enforce this.
  • domain assumption The merged memory is a sufficient statistic for answering the question.
    Final answer is generated only from q and m* (Algorithm 1, line 13). Any evidence lost in merging cannot be recovered.
  • domain assumption HotpotQA-derived synthetic training data transfers to longer contexts and OOD RULER tasks.
    Training uses 140K or 28K token samples, but evaluation goes to 3.6M tokens. The paper's OOD results are meant to validate this transfer.
  • domain assumption Self-attention cost model O((NC)^2) vs O(N(C+M)^2) and O(KN(C+M)^2) is the right basis for comparing these workflows.
    Technical supplement Table 6 compares algorithmic FLOPs under quadratic attention, an approximation for hybrid-attention models like Qwen3.5-35B-A3B.
  • standard math GRPO with trajectory-level advantage and DAPO-style aggregation optimizes the workflow as intended.
    The RL update in Eq. 6-8 is a standard clipped surrogate objective with token-level KL. No new math is claimed.
  • domain assumption The HotpotQA evaluation metric and RULER task construction are accepted benchmarks.
    The paper relies on established benchmarks (HotpotQA, RULER, LongBench v2) as ground truth for accuracy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PI-Mem: Pushing Long-Context Reasoning to 3.6M Tokens with Parallel-Iterative Memory." pith.science (2026). https://pith.science/paper/YGUI4C2P

@misc{pith2026260803048,
  author       = {Pith},
  title        = {Pith review of: PI-Mem: Pushing Long-Context Reasoning to 3.6M Tokens with Parallel-Iterative Memory},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YGUI4C2P}},
  note         = {Machine review of arXiv:2608.03048}
}
abstract

Long-context reasoning remains a critical bottleneck for large language models, as recent recurrent-memory approaches face two inherent challenges: sequential chunk-wise updates can overwrite early critical evidence with later irrelevant content, and serial inter-chunk dependencies limit parallelism and cause latency to increase with context length. To address these issues, we propose PI-Mem (Parallel-Iterative Memory), a mechanism that processes all chunks in parallel and iteratively refines a shared memory over a bounded number of turns. In each turn, PI-Mem reads all chunks in parallel conditioned on the current memory, selects new or complementary evidence from each chunk, and merges the selected evidence into a compact shared memory for the next turn. To discourage redundant turns, we optimize the workflow through reinforcement learning with an auxiliary turn-efficiency reward, enabling the model to adaptively exit once sufficient evidence has been accumulated. We evaluate PI-Mem with Qwen3.5-35B-A3B and Qwen2.5-7B on the HotpotQA benchmark across context lengths up to 3.6 million tokens and find that it outperforms the recurrent-memory baseline by +6.25 and +7.81 absolute points while achieving 6.1$\times$ and 2.1$\times$ inference speedups, respectively. These results demonstrate that PI-Mem breaks the accuracy--efficiency trade-off in long-context reasoning and provides a scalable approach to complex multi-hop question answering over extremely long documents.

Figures

Figures reproduced from arXiv: 2608.03048 by the authors.

Figure 1
Figure 1. Top: Recurrent-memory workflows process chunks [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the PI-Mem workflow. PI-Mem reads all chunks in parallel conditioned on a shared global memory and [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 6
Figure 6. Effect of the turn-efficiency reward on Qwen2.5- [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 20 canonical work pages

  1. [1]

    arXiv preprint arXiv:2507.02259 , year =

    MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent , author =. arXiv preprint arXiv:2507.02259 , year =

    arXiv

  2. [2]

    2026 , eprint =

    When to Memorize and When to Stop: Gated Recurrent Memory for Long-Context Reasoning , author =. 2026 , eprint =

    work page 2026

  3. [3]

    2026 , eprint =

    IterResearch: Rethinking Long-Horizon Agents with Interaction Scaling , author =. 2026 , eprint =

    work page 2026

  4. [4]

    2026 , eprint =

    Kimi K2.5: Visual Agentic Intelligence , author =. 2026 , eprint =

    work page 2026

  5. [5]

    2025 , eprint =

    Qwen2.5 Technical Report , author =. 2025 , eprint =

    2025

  6. [6]

    2025 , eprint =

    DAPO: An Open-Source LLM Reinforcement Learning System at Scale , author =. 2025 , eprint =

    work page 2025

  7. [7]

    arXiv preprint arXiv:2503.20783 , year =

    Understanding R1-Zero-Like Training: A Critical Perspective , author =. arXiv preprint arXiv:2503.20783 , year =

    arXiv

  8. [8]

    2024 , eprint =

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models , author =. 2024 , eprint =

    2024

Show all 44 references
  1. [9]

    and Stoica, Ion and Gonzalez, Joseph E

    Packer, Charles and Wooders, Sarah and Lin, Kevin and Fang, Vivian and Patil, Shishir G. and Stoica, Ion and Gonzalez, Joseph E. , journal =

  2. [10]

    Chhikara, Prateek and Khant, Dev and Aryan, Saket and Singh, Taranjeet and Yadav, Deshraj , journal =

  3. [11]

    arXiv preprint arXiv:2506.15841 , year =

    MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents , author =. arXiv preprint arXiv:2506.15841 , year =

    arXiv

  4. [12]

    Wang, Yu and Takanobu, Ryuichi and Liang, Zhiqi and Mao, Yuzhen and Hu, Yuanzhe and McAuley, Julian and Wu, Xiaojian , journal =

  5. [13]

    arXiv preprint arXiv:2510.11967 , year =

    Scaling Long-Horizon LLM Agent via Context-Folding , author =. arXiv preprint arXiv:2510.11967 , year =

  6. [14]

    2025 , eprint =

    AgentFold: Long-Horizon Web Agents with Proactive Context Management , author =. 2025 , eprint =

    2025

  7. [15]

    2026 , eprint =

    AgentSwing: Adaptive Parallel Context Management Routing for Long-Horizon Web Agents , author =. 2026 , eprint =

    2026

  8. [16]

    2026 , eprint =

    Memory as Action: Autonomous Context Curation for Long-Horizon Agentic Tasks , author =. 2026 , eprint =

    2026

  9. [17]

    2025 , eprint =

    QwenLong-L1: Towards Long-Context Large Reasoning Models with Reinforcement Learning , author =. 2025 , eprint =

    2025

  10. [18]

    2026 , eprint =

    SPELL: Self-Play Reinforcement Learning for Evolving Long-Context Language Models , author =. 2026 , eprint =

    2026

  11. [19]

    arXiv preprint arXiv:2510.19363 , year =

    LoongRL: Reinforcement Learning for Advanced Reasoning over Long Contexts , author =. arXiv preprint arXiv:2510.19363 , year =

  12. [20]

    arXiv preprint arXiv:2603.02146 , year =

    LongRLVR: Long-Context Reinforcement Learning Requires Verifiable Context Rewards , author =. arXiv preprint arXiv:2603.02146 , year =

  13. [21]

    arXiv preprint arXiv:2512.12967 , year =

    QwenLong-L1.5: Post-Training Recipe for Long-Context Reasoning and Memory Management , author =. arXiv preprint arXiv:2512.12967 , year =

  14. [22]

    arXiv preprint arXiv:2509.23040 , year =

    Look Back to Reason Forward: Revisitable Memory for Long-Context LLM Agents , author =. arXiv preprint arXiv:2509.23040 , year =

  15. [23]

    2026 , eprint =

    Dynamic Long Context Reasoning over Compressed Memory via End-to-End Reinforcement Learning , author =. 2026 , eprint =

    2026

  16. [24]

    arXiv preprint arXiv:2602.02704 , year =

    InfMem: Learning System-2 Memory Control for Long-Context Agent , author =. arXiv preprint arXiv:2602.02704 , year =

  17. [25]

    Bowen Peng and Jeffrey Quesnelle and Honglu Fan and Enrico Shippole , booktitle =. Ya. 2024 , url =

    2024

  18. [26]

    arXiv preprint arXiv:2306.15595 , year =

    Extending Context Window of Large Language Models via Positional Interpolation , author =. arXiv preprint arXiv:2306.15595 , year =

    arXiv

  19. [27]

    arXiv preprint arXiv:2502.11089 , year =

    Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention , author =. arXiv preprint arXiv:2502.11089 , year =

    arXiv

  20. [28]

    arXiv preprint arXiv:2502.13189 , year =

    MoBA: Mixture of Block Attention for Long-Context LLMs , author =. arXiv preprint arXiv:2502.13189 , year =

    arXiv

  21. [29]

    International Conference on Learning Representations , year =

    Gated Delta Networks: Improving Mamba2 with Delta Rule , author =. International Conference on Learning Representations , year =. 2412.06464 , archiveprefix =

    arXiv

  22. [30]

    arXiv preprint arXiv:2507.06261 , year =

    Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities , author =. arXiv preprint arXiv:2507.06261 , year =

    arXiv

  23. [31]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics , year =

    Evaluating Very Long-Term Conversational Memory of LLM Agents , author =. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics , year =

  24. [32]

    arXiv preprint arXiv:2406.05925 , year =

    Hello Again! LLM-Powered Personalized Agent for Long-Term Dialogue , author =. arXiv preprint arXiv:2406.05925 , year =

    arXiv

  25. [33]

    Advances in Neural Information Processing Systems , year =

    SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering , author =. Advances in Neural Information Processing Systems , year =

  26. [34]

    Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks , url =

    Lewis, Patrick and Perez, Ethan and Piktus, Aleksandra and Petroni, Fabio and Karpukhin, Vladimir and Goyal, Naman and K\". Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks , url =. Advances in Neural Information Processing Systems , editor =

  27. [35]

    arXiv preprint arXiv:2410.08164 , year =

    Agent S: An Open Agentic Framework that Uses Computers Like a Human , author =. arXiv preprint arXiv:2410.08164 , year =

    arXiv

  28. [36]

    2025 , eprint =

    Qwen2.5-1M Technical Report , author =. 2025 , eprint =

    2025

  29. [37]

    2025 , eprint =

    LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory , author =. 2025 , eprint =

    2025

  30. [38]

    2024 , eprint =

    Qwen2.5-Coder Technical Report , author =. 2024 , eprint =

    2024

  31. [39]

    International Conference on Learning Representations , year =

    ReAct: Synergizing Reasoning and Acting in Language Models , author =. International Conference on Learning Representations , year =

  32. [40]

    2025 , eprint =

    A Comprehensive Survey on Long Context Language Modeling , author =. 2025 , eprint =

    2025

  33. [41]

    2025 , eprint =

    Context Length Alone Hurts LLM Performance Despite Perfect Retrieval , author =. 2025 , eprint =

    2025

  34. [42]

    2024 , eprint =

    RULER: What's the Real Context Size of Your Long-Context Language Models? , author =. 2024 , eprint =

    2024

  35. [43]

    arXiv preprint arXiv:2412.15204 , year =

    LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks , author =. arXiv preprint arXiv:2412.15204 , year =

    arXiv

  36. [44]

    2018 , eprint =

    HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering , author =. 2018 , eprint =

    2018

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.