Pith. sign in

REVIEW 3 major objections 6 minor 26 references

Long-context memory can be organized along transformer depth, not only along tokens, by caching mid-layer residual states and recomputing only the upper layers on a fixed retrieved pack.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 12:53 UTC pith:XEJTL7PS

load-bearing objection Solid systems paper: constant model-side read via mid-depth residual cache is real; the LoCoMo win over full context is mostly retrieval, not depth magic. the 3 major comments →

arxiv 2607.28263 v1 pith:XEJTL7PS submitted 2026-07-30 cs.CL

Understanding Is Done Early: A Depth Division of Labor in Large Language Models and Its Use for Unbounded-Context Memory

Hanzuo Liu , Xuan Qi , Chunyu Liu , Haotian Zhong , Yulong Wang , Rayying , Key , Alex Lamb
show 1 more author
Mingyu Gao
This is my paper
classification cs.CL
keywords long-context memorytransformer depthresidual streamKV cachebounded retrievalself-distillationCoMemlayer-axis reuse
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Transformers do not use depth uniformly: lower and middle layers make semantic content accessible, while upper layers increasingly specialize representations for the current query and prediction. This paper turns that division of labor into CoMem, which writes each context chunk only through an intermediate layer, caches the residual tensor, retrieves a fixed number of relevant tensors, and recomputes only the upper layers with the query present. For a fixed retrieval budget, model-side read compute and memory no longer grow with how much context has been stored. On a frozen backbone with a light self-distillation adapter trained on plain text, the method stays competitive on long-context and dialogue-memory benchmarks while cutting peak memory and prefill cost sharply at long lengths. A sympathetic reader cares because unbounded context becomes a depth-and-retrieval design problem rather than an ever-growing full-depth cache.

Core claim

Semantic information becomes usable near mid-depth while upper layers are more query-conditioned, so a model can cache one intermediate residual state per token, retrieve a bounded relevant pack, and resume only the upper transformer. For fixed retrieval breadth, online read compute and memory are then independent of stored-context length, and long-context memory can be organized along the layer axis rather than only the token axis.

What carries the argument

CoMem (Comprehension Memory): write each chunk through layers [0:j] and cache the residual hj; retrieve a fixed top-k pack (flagship: iterative BM25); recompute layers [j:L] with full cross-chunk attention and the query present. Split depth j is the quality–cost knob; a rank-32 self-distillation LoRA repairs readout fidelity at deeper j without updating the backbone.

Load-bearing premise

Lexical retrieval over chunk text is good enough to surface the evidence the upper layers need, so selection—not the fidelity of the cached mid-depth state—is the main long-range bottleneck.

What would settle it

On tasks where the needed evidence shares little surface form with the query, or exceeds the fixed retrieved pack, measure whether CoMem falls far behind full context while an oracle or strong dense selector does not; if so, the constant-read property can hold while the accuracy claims fail.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Split depth becomes a deployable knob: deeper writes prepay work and cut per-query recompute, at the cost of frozen readout fidelity that light adaptation can repair.
  • One mid-depth residual per token stores far less than full-depth KV (about 1/18 the bytes on the evaluated 8B model).
  • Bounded relevance plus full upper-layer cross-chunk attention can match or beat full-context on some dialogue-memory settings by excluding distractors.
  • The same depth partition keeps a roughly constant model-side read as stored context grows, including on sparse MoE backbones.
  • In-window tasks can still pay a compression tax: bounded read is not uniform accuracy dominance whenever full context fits.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If content depth stays near mid-network across scale while zero-shot readable depth deepens, very large models may need little or no adapter for deep caching.
  • Serving systems could treat mid-layer residual stores as first-class reusable objects, amortizing write cost across many queries on the same document.
  • Pretraining that deliberately shapes compressible mid-depth interfaces could reduce reliance on post-hoc self-distillation.
  • Replacing lexical chunk selection with dense or residual-key retrieval is the natural control for paraphrased and multi-hop evidence.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper argues that transformer depth is used unevenly—semantic content becomes accessible by mid-depth while upper layers specialize for the query—and turns that observation into CoMem: write each context chunk only through split depth j, cache the residual tensor hj, retrieve a fixed top-k pack (flagship iterative BM25, k=12, c=512), and recompute only layers [j:L] with full cross-chunk attention. For fixed retrieval budget, model-side read compute and memory are independent of stored length. On a frozen continued-trained Qwen3-8B under a unified chat-template-free protocol, a rank-32 PG19 self-distillation LoRA at j=12 yields RULER 97.05 and LoCoMo Judge 38.27 (vs KV-Direct 34.59), with adapter-free H20 controls reporting 18.26 GB vs 89.36 GB and 7.83× full-write-inclusive prefill at 128k. Controlled same-pack depth sweeps, selector ablations, block-diagonal vs full attention, and an adapter-free arm separate retrieval, split depth, and distillation.

Significance. If the efficiency measurements and depth-partition design hold, the work supplies a clean complementary axis for long-context memory: reuse along layers rather than only along tokens. Strengths that raise the contribution above a pure systems note include (i) direct hardware measurements of bounded read working set and prefill (Table 3), (ii) same-pack depth and j=0 retrieval controls that make the quality–cost trade-off falsifiable (Tables 8–9), (iii) task-free PG19 bidirectional-KL LoRA that repairs readout without benchmark labels, (iv) conversation-cluster bootstrap and independent-judge checks on LoCoMo, and (v) ports to sparse MoE backbones. The result is practically relevant for multi-query long stores and conceptually useful as evidence that mid-depth residuals are a compressible, resumable memory unit.

major comments (3)
  1. [Abstract; Table 1; §5.5; Table 8] Abstract and Table 1 pair CoMem’s LoCoMo 38.27 and RULER 97.05 against KV-Direct 34.59 / 78.80 in a way that invites attributing the accuracy edge to depth partitioning. Table 8 holds the retrieved pack fixed: j=0 full recompute already scores LoCoMo 41.59—above distilled j=12 (38.27) and KV-Direct (34.59)—while frozen j=6/9/12 fall to 32.78/29.15/24.52. Depth is therefore a real query-cost knob (Table 9: ~29% lower Read at j=12) that incurs readout loss partially repaired by distillation, not the source of the dialogue gain over full context. The abstract, contributions, and Table 1 should lead with j=0-on-pack as a primary baseline and state explicitly that bounded selection drives the LoCoMo edge while depth supplies the efficiency–fidelity trade-off.
  2. [Abstract; Table 1; Table 15; §5 Baselines and position range] The RULER headline contrast (97.05 vs KV-Direct 78.80) mixes in-window cells with unextended native-RoPE stress cells where KV-Direct collapses to 0 at 128k (Table 15). The paper labels those cells stress references and disclaims YaRN comparisons, but the five-benchmark Avg and the abstract still absorb that collapse. Report an in-window (e.g., ≤40,960) RULER macro beside the full 8k–128k macro, or exclude post-native full-context zeros from any head-to-head average used in the abstract.
  3. [§3 (I3); Figure 3; Table 5; Table 17; §7] Motivation I3 and Figure 3 conclude that selection—not cached-state fidelity—is the long-range bottleneck because BM25≈oracle on RULER needles. That diagnostic does not automatically extend to multi-fact or low-lexical-overlap dialogue: Table 5 shows wider retrieval can hurt variable tracking via distractors, BABILong qa1/qa2 expose a clear in-window compression tax versus full context (Table 17), and Limitations already flag surface-form retrieval. Either add a dense/learned retriever control on LoCoMo/BABILong or narrow I3 to “on the reported needle sweep, under lexical overlap.”
minor comments (6)
  1. [Figure 1] Figure 1’s accuracy curve uses the distilled flagship while the 7.83×/18.26 GB inset is adapter-free; the caption states this, but a single panel legend marking “distilled quality / adapter-free efficiency” would reduce misreading.
  2. [§4 Notation and storage; Eq. (1)] Eq. (1) writes |hj|/|KV| = d/(2L dkv) = nq/(2L nkv); a one-line derivation that full KV stores 2L nkv-dimensional vectors per token would help readers verify the 1/18 Qwen3-8B figure.
  3. [Table 1] Table 1 Avg is an unweighted mean of heterogeneous metrics (RULER 15-cell macro, LoCoMo judge, LongBench F1, etc.). Consider dropping Avg or marking it as descriptive only.
  4. [Table 1; Table 2; §5] MemoryLLM is repeatedly caveated as out-of-backbone and out-of-template; consider moving it to an appendix diagnostic row so main tables stay within-backbone.
  5. [Title page; passim] Minor prose inconsistencies: title uses “Understanding Is Done Early” while the running header compresses spacing (“UNDERSTANDINGISDONEEARLY”); “residual tensor” / “residual states” alternate; arXiv date line says July 31, 2026.
  6. [Appendix B.2] Appendix B judge endpoint lacks a dated snapshot (acknowledged). Pinning model version or releasing raw judge judgments would improve auditability.

Circularity Check

0 steps flagged

No significant circularity: length-independent read is an explicit design property, distillation is unsupervised transfer, and quality/efficiency claims are external measurements.

full rationale

CoMem’s central engineering claim—that for fixed retrieval breadth k and chunk size c the model-side Read pack (and thus online FLOPs/memory) does not grow with stored context—is stated as a direct consequence of the Write/Select/Read construction (Eq. 2, Algorithm 1), not as an empirical “prediction” fitted from the same quantity. Self-distillation uses only plain PG-19 bidirectional KL to a frozen j=0 teacher, with no retrieval, task labels, or benchmark supervision, so RULER/LoCoMo gains are transfer evaluations against external scorers and baselines (KV-Direct, StreamingLLM, InfLLM, etc.), not refits of the test metrics. Depth-division motivation rests on in-paper probes, causal truncation, and split sweeps plus an external citation (Lad et al.), not a self-authored uniqueness theorem that forces the architecture. Efficiency numbers are hardware measurements (H20 prefill/decode/memory), not normalized identities. Related-work systems (HCache, KV-Direct) are baselines or partial precursors, not load-bearing self-citations that define the result. Ordinary method self-consistency (same backbone, fixed pack ablations) is not circularity under the stated criteria.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 1 invented entities

The central claim rests on standard transformer mechanics, an empirical depth-division premise, and several engineering knobs (split depth, chunk size, retrieval breadth, distillation objective) chosen by the authors. No new physical entities are postulated; CoMem is an architectural construction. Free parameters are deployment/training choices that affect operating point but are ablated rather than hidden.

free parameters (5)
  • split depth j (flagship j=12/36) = 12 of 36 layers
    Hand-chosen operating point trading write precompute vs read fidelity; ablated at 0/6/9/12 but flagship accuracy depends on this choice plus adapter.
  • retrieval budget k and iteration (top-12, hop 4, 3 rounds) = k=12, h=4, auto 3 rounds
    Fixes the constant read length (~6.5k); broader ~17k expansion hurts variable tracking from 16k onward—budget is a tuned knob, not derived.
  • chunk size c = 512 tokens
    512-token chunks set storage granularity and pack length; Table 14 shows accuracy/cost sensitivity across 128–1024.
  • LoRA rank, α, bidirectional KL weights, LR schedule = r=32, α=64, L=0.6 KL(p||q)+0.4 KL(q||p), LR 8e-5, 4000 steps
    Adapter hyperparameters and 0.6/0.4 KL mix are training choices that substantially move readable depth and benchmark scores at fixed j.
  • BM25 k1, b and tokenization choices = k1=1.5, b=0.75
    Lexical selector hyperparameters (k1=1.5, b=0.75, no stemming/stopwords) determine which residuals enter the pack.
axioms (5)
  • domain assumption Decoder-only transformer residual stream and causal self-attention with RoPE behave as in the Qwen3-8B (and cited Hy3 MoE) implementation; recomputing layers [j:L] from hj resumes the forward pass.
    Foundation of Write/Read correctness; j=0 equivalence checked to max|Δlogit|<1e-4; MoE routing self-test max|Δlogit|=0.
  • domain assumption Semantic content becomes accessible near mid-depth while upper layers are more query-conditioned, so a query-blind cached hj is a valid reusable memory unit if upper layers are recomputed with the query.
    Motivation §3 and Lad et al. 2024; supported by probes/truncation/depth sweeps but still an empirical regularity, not a theorem.
  • domain assumption For the evaluated tasks, bounded relevance selection over chunk text can substitute for full-context attention without destroying answerability when the gold chunk is retrieved.
    Oracle selector hits 100 on reported needle cells; flagship depends on BM25 approximating oracle often enough.
  • ad hoc to paper Comparing against unextended native-RoPE full context (no YaRN) is a valid stress reference for claiming continued usability beyond the native window, without claiming superiority to length-extended full context.
    Explicitly stated in §5 and Limitations; load-bearing for long-length narrative versus KV-Direct zeros at 128k.
  • standard math Standard optimization and information-retrieval mathematics (AdamW, BM25/IDF, KL distillation, LoRA reparameterization) apply as used.
    Training and retrieval sections invoke these without modification to their definitions.
invented entities (1)
  • CoMem (Comprehension Memory) depth-partitioned residual store independent evidence
    purpose: Name the write-shallow / retrieve / recompute-upper memory architecture and its external hj cache.
    Engineering system built from existing transformer states, not a new latent physical or mathematical object; independent_evidence is the shipped method and benchmarks.

pith-pipeline@v1.2.0-daily-grok45 · 27921 in / 4046 out tokens · 78693 ms · 2026-07-31T12:53:25.905096+00:00 · methodology

0 comments
read the original abstract

Transformer depth is not used uniformly: lower and middle layers build semantic representations, while upper layers increasingly specialize them for prediction. We turn this division of labor into CoMem (Comprehension Memory), which writes each context chunk only through an intermediate layer, retrieves a fixed number of cached residual states, and recomputes the query-conditioned upper layers over the resulting pack. For a fixed retrieval budget, model-side read compute and memory are independent of stored-context length. We evaluate a continued-trained Qwen3-8B base LM under a unified chat-template-free protocol. The backbone is frozen; the flagship trains only a rank-32 self-distillation LoRA on plain PG19, and we report an adapter-free arm separately. CoMem reaches 97.05 on RULER and 38.27 on LoCoMo versus 34.59 for full-context KV-Direct; the dialogue-memory advantage survives conversation-cluster resampling and an independent judge. Results on additional long-context and long-document tasks expose both the benefits of bounded retrieval and its in-window compression tax. Controlled depth sweeps show that deeper caching lowers per-query recomputation but incurs a fidelity loss that self-distillation substantially repairs. In a separate adapter-free efficiency control on an NVIDIA H20 at 128k, CoMem uses 18.26 GB rather than 89.36 GB and achieves a 7.83x prefill speedup. These results show that long-context memory can be organized along the layer axis, not only the token axis.

Figures

Figures reproduced from arXiv: 2607.28263 by Alex Lamb, Chunyu Liu, Hanzuo Liu, Haotian Zhong, Key, Mingyu Gao, Rayying, Xuan Qi, Yulong Wang.

Figure 1
Figure 1. Figure 1: CoMem in one picture. Semantic information becomes usable by mid-depth, so CoMem caches the per-token residual tensor at depth j and recomputes the upper layers on a fixed retrieved pack. For fixed k, chunk size, and query length, the model-side online read working set is independent of stored-context length. The accuracy curve uses the distilled flagship, whereas the 7.83×/18.26 GB inset is a separate ada… view at source ↗
Figure 2
Figure 2. Figure 2: The depth division of labor sets a bounded, tunable cache depth. (a) On Qwen3-8B, zero-shot needle recall stays perfect while the split j is shallow and then collapses once the cache enters the query￾conditioned band; a light self-distilled adapter pushes the readable depth deeper. Panel (a) is an independent trend sweep; [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Selection, not fidelity, is the observed long￾range bottleneck. On the chat-template-free RULER single-needle sweep, Oracle and BM25 remain at 100, while recency and reader attention fall at 32k. Full selector results are in Appendix [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: CoMem pipeline. WRITE encodes each 512-token chunk through [0:j] and caches hj . SELECT retrieves top-k relevant chunks. READ packs their hiddens with a sink and the query, then recomputes [j:L] with full cross-chunk attention. The flagship reads approximately 6.5k tokens regardless of stored-context length. match a frozen j=0 teacher on the teacher’s top-64 logit support. Let p and q be their renormalized… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

26 extracted references · 23 linked inside Pith

  1. [4]

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean

    The Llama 3 herd of models.arXiv preprint arXiv:2407.21783. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean

    Pith/arXiv arXiv

  2. [6]

    Edward J

    RULER: What’s the real context size of your long-context language models? arXiv preprint arXiv:2404.06654. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Pith/arXiv arXiv

  3. [7]

    Yuri Kuratov et al

    RAGCache: Efficient knowledge caching for retrieval-augmented generation.arXiv preprint arXiv:2404.12457. Yuri Kuratov et al

    Pith/arXiv arXiv

  4. [8]

    The remarkable robustness of LLMs: Stages of inference? arXiv preprint arXiv:2406.19384. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Hein- rich Kuettler, Mike Lewis, Wen-tau Yih, Tim Rock- taschel, Sebastian Riedel, and Douwe Kiela

    Pith/arXiv arXiv

  5. [9]

    Xiaolin Lin, Jingcun Wang, Olga Kondrateva, Yiyu Shi, Bing Li, and Grace Li Zhang

    SnapKV: LLM knows what you are looking for before generation.arXiv preprint arXiv:2404.14469. Xiaolin Lin, Jingcun Wang, Olga Kondrateva, Yiyu Shi, Bing Li, and Grace Li Zhang

    Pith/arXiv arXiv

  6. [10]

    Akide Liu, Jing Liu, Zizheng Pan, Yefei He, Gholam- reza Haffari, and Bohan Zhuang

    Com- pressKV: Semantic-retrieval-guided KV-cache com- pression for resource-efficient long-context LLM in- ference.arXiv preprint arXiv:2606.24467. Akide Liu, Jing Liu, Zizheng Pan, Yefei He, Gholam- reza Haffari, and Bohan Zhuang. 2024a. MiniCache: KV cache compression in depth dimension for large language models.arXiv preprint arXiv:2405.14366. Di Liu, ...

    Pith/arXiv arXiv

  7. [11]

    Amirkeivan Mohtashami and Martin Jaggi

    Evaluating very long- term conversational memory of LLM agents.arXiv preprint arXiv:2402.17753. Amirkeivan Mohtashami and Martin Jaggi

    Pith/arXiv arXiv

  8. [12]

    Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole

    Landmark attention: Random-access infinite con- text length for transformers.arXiv preprint arXiv:2305.16300. Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole

    Pith/arXiv arXiv

  9. [13]

    Kaleem Ullah Qasim, Jiashu Zhang, Muhammad Kafeel Shaheen, Razan Alharith, and Heying Zhang

    YaRN: Efficient context window extension of large language models.arXiv preprint arXiv:2309.00071. Kaleem Ullah Qasim, Jiashu Zhang, Muhammad Kafeel Shaheen, Razan Alharith, and Heying Zhang

    Pith/arXiv arXiv

  10. [14]

    arXiv preprint arXiv:2603.19664

    The residual stream is all you need: On the redun- dancy of the KV cache in transformer inference. arXiv preprint arXiv:2603.19664. Jiahao Qiu, Xuan Qi, Hongru Wang, Xinzhe Juan, Yimin Wang, Zelin Zhao, Jiayi Geng, Jiacheng Guo, Peihang Li, Jingzhe Shi, et al. 2025a. Alita-g: Self- evolving generative agent for agent generation.arXiv preprint arXiv:2510.2...

    arXiv

  11. [17]

    Jiaming Tang, Yilong Zhao, Kan Zhu, Guangxuan Xiao, Baris Kasikci, and Song Han

    You only cache once: Decoder- decoder architectures for language models.arXiv preprint arXiv:2405.05254. Jiaming Tang, Yilong Zhao, Kan Zhu, Guangxuan Xiao, Baris Kasikci, and Song Han

    Pith/arXiv arXiv

  12. [18]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N

    Focused transformer: Contrastive training for context scaling.arXiv preprint arXiv:2307.03170. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin

    Pith/arXiv arXiv

  13. [20]

    Haoyi Wu and Kewei Tu

    MemoryLLM: Towards self- updatable large language models.arXiv preprint arXiv:2402.04624. Haoyi Wu and Kewei Tu

    Pith/arXiv arXiv

  14. [21]

    arXiv preprint arXiv:2405.10637

    Layer-condensed KV cache for efficient inference of large language models. arXiv preprint arXiv:2405.10637. Chaojun Xiao et al. 2024a. InfLLM: Training-free long- context extrapolation with an efficient context mem- ory.arXiv preprint arXiv:2402.04617. Guangxuan Xiao et al. 2024b. Efficient streaming lan- guage models with attention sinks. InICLR. An Yang et al

    Pith/arXiv arXiv

  15. [23]

    Howard Yen, Tianyu Gao, and Danqi Chen

    CacheBlend: Fast large language model serving for RAG with cached knowledge fu- sion.arXiv preprint arXiv:2405.16444. Howard Yen, Tianyu Gao, and Danqi Chen

    Pith/arXiv arXiv

  16. [24]

    Zhenyu Zhang et al

    Long context com- pression with activation beacon.arXiv preprint arXiv:2401.03462. Zhenyu Zhang et al

    Pith/arXiv arXiv

  17. [25]

    InNeurIPS

    H2O: Heavy-hitter oracle for efficient generative inference of large language models. InNeurIPS. ModelLcontent-jreadout-crash gap (/L) (/L) (∆/L) 0.6B 28 13 (0.46)∼2.6 (0.09)0.37 1.7B 28 13 (0.46)∼6 (0.21) 0.25 4B 36 16 (0.44)∼11 (0.31) 0.14 8B 36 16 (0.44)∼11 (0.31) 0.14 14B 40 18 (0.45)∼15 (0.38) 0.08 32B 64 27 (0.42)>27 (>0.42)≈0 Table 7:Depth partitio...

    arXiv

  18. [92]

    †MemoryLLM uses a different chat-tuned backbone

    The frozen CoMem control used 48 generation tokens rather than 16; both budgets exceed the single-number answer length. †MemoryLLM uses a different chat-tuned backbone. Method Task 0k 1k 2k 4k 8k 16k 32k Mean CoMem + LoRA qa19880 68 68 46 17 12 55.6 qa2 26 44 43 44 23 8 1 27.0 qa5 6876 76 756860 58 68.7 CoMem frozen (j=9) qa19859 71 57 7 1 4 42.4 qa2 53 1...

    2019

  19. [2015]

    Cheng-Ping Hsieh et al

    Distilling the knowledge in a neural network.arXiv preprint arXiv:1503.02531. Cheng-Ping Hsieh et al

    Pith/arXiv arXiv

  20. [2017]

    InAdvances in Neural Information Pro- cessing Systems, volume 30, pages 5998–6008

    Attention is all you need. InAdvances in Neural Information Pro- cessing Systems, volume 30, pages 5998–6008. Qingyue Wang et al. 2023a. Recursively summariz- ing enables long-term dialogue memory in large lan- guage models.arXiv preprint arXiv:2308.15022. Weizhi Wang, Li Dong, Hao Cheng, Xiaodong Liu, Xifeng Yan, Jianfeng Gao, and Furu Wei. 2023b. Augmen...

    Pith/arXiv arXiv

  21. [2019]

    Stephen Robertson and Hugo Zaragoza

    Compressive transformers for long-range sequence modelling.arXiv preprint arXiv:1911.05507. Stephen Robertson and Hugo Zaragoza

    Pith/arXiv arXiv 1911

  22. [2021]

    arXiv preprint arXiv:2104.09864

    RoFormer: En- hanced transformer with rotary position embedding. arXiv preprint arXiv:2104.09864. Yutao Sun et al

    Pith/arXiv arXiv

  23. [2023]

    arXiv preprint arXiv:2308.14508

    LongBench: A bilingual, mul- titask benchmark for long context understanding. arXiv preprint arXiv:2308.14508. Zefan Cai et al

    Pith/arXiv arXiv

  24. [2024]

    Shiwei Gao, Youmin Chen, and Jiwu Shu

    PyramidKV: Dynamic KV cache compression based on pyramidal information funnel- ing.arXiv preprint arXiv:2406.02069. Shiwei Gao, Youmin Chen, and Jiwu Shu

    Pith/arXiv arXiv

  25. [2025]

    Jiayi Yao et al

    Qwen3 technical report.arXiv preprint arXiv:2505.09388. Jiayi Yao et al

    Pith/arXiv arXiv

  26. [2026]

    Aaron Grattafiori et al

    Training transform- ers for KV cache compressibility.arXiv preprint arXiv:2605.05971. Aaron Grattafiori et al

    Pith/arXiv arXiv