Pith. sign in

REVIEW 4 major objections 5 minor 16 references

SeDeM claims that long-context question answering can be made both cheaper and more accurate by decoding from selected, decompressed hidden-state memories instead of from the full context or from compressed memory tokens.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

SeDeM stores long contexts as compact query-independent memory blocks, selects only query-relevant blocks, and decompresses them into intermediate decoder hidden states, beating compression baselines on four long-context QA benchmarks.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection A real architectural idea with coherent system results, but the mechanism claim is confounded: selection and decompression are ablated together. the 4 major comments →

arxiv 2608.00311 v1 pith:BFUBODJ6 submitted 2026-07-31 cs.CL

SeDeM: Selective Decompression of Hidden-State Memories for Long-Context Question Answering

Maryam Haghifam , Jason Cong , Yizhou Sun This is my paper
classification cs.CL
keywords long-context QAcontext compressionhidden-state memoryselective decompressiontop-k selectionmemory blocksintermediate-layer injectionefficient inference
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SeDeM argues that the failure point in soft context compression is not storage but conditioning: forcing a decoder to generate from highly compressed memory tokens. The method stores a long context as compact per-segment blocks of hidden states, selects the blocks most relevant to the query, and expands only those blocks into hidden states injected at a middle decoder layer. On four long-context question-answering benchmarks, the paper reports that this beats all evaluated compression baselines at both 1B and 3B model scale, and with the 3B backbone it exceeds full-context fine-tuning on three of the four datasets. It also reports lower time-to-first-token and higher decoding throughput than direct memory-token conditioning. The central claim is that selective decompression recovers most of the benefit of full-context reasoning while spending compute proportional to the selection budget rather than the context length.

Core claim

The central claim is that decoupling compact memory storage from decoder conditioning works better than making compressed memories double as decoder input. The paper constructs a query-independent memory bank by running a frozen encoder over context segments, mean-pooling local chunks of intermediate-layer hidden states, and projecting them into compact memory blocks. A late-interaction selector scores blocks against the query and keeps the top K; an MLP decompressor expands each selected memory vector back into several decoder-dimension hidden states and injects them at an intermediate decoder layer, after which a frozen decoder with lightweight adapters generates the answer. The reported r

What carries the argument

The central object is a memory bank of segment-level hidden-state blocks with a separate expansion pathway. Each context segment is encoded once to an intermediate layer, collapsed by non-overlapping mean pooling plus a learned projection into a small number of memory vectors per segment. A ColBERT-style late-interaction scorer (max over slots, sum over query tokens, averaged over heads) ranks blocks, and a shared two-layer MLP reshapes each selected memory vector into multiple decoder-dimension states for injection at an intermediate decoder layer. Because compression is done by pooling rather than by learned attention tokens, memory construction is query-independent and parallelizable; bec

Load-bearing premise

The load-bearing premise is that the decompressor's outputs land close enough to the decoder's native hidden-state distribution at the injection layer that the upper decoder layers behave as if they had processed the selected context directly; the paper's own Appendix A.13 explicitly calls the local Lipschitz condition supporting this a strong assumption for Transformers, so if injected states are out-of-distribution, accuracy degrades no matter how good the selector is.

What would settle it

Measure the actual per-state reconstruction gap between decompressed hidden states and the decoder's own hidden states for the same selected segments, or run an oracle experiment that replaces the decompressor with the true full-context hidden states for the selected segments. If QA accuracy does not improve substantially under oracle injection, selection is the real bottleneck; if accuracy still falls short of full-context quality, reconstruction error at the injection layer is the bottleneck.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Decoder compute during generation scales with the number of selected blocks K rather than with the full context length N.
  • The same query-independent memory bank can answer multiple queries over the same context without re-running context compression.
  • The selection budget K acts as a tunable quality-efficiency trade-off: small budgets help single-hop or distractor-heavy tasks, while larger budgets help multi-hop tasks before saturating.
  • Removing the decompression step and feeding memory tokens directly to the decoder is substantially weaker, implying the expansion step carries most of the benefit.
  • A smaller encoder can build memories that a larger decoder consumes, separating context-processing cost from generation capacity.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural untested extension the paper leaves implicit is adaptive per-query K: letting selector confidence or answer likelihood decide how many blocks to expand instead of using a fixed budget.
  • Because the memory bank is query-independent and the selector is a light scoring head, the architecture resembles latent retrieval; precomputing banks once for static corpora and running only selection and decompression at query time would make the reported efficiency gains larger.
  • The stability argument in the appendix suggests a diagnostic before adopting the method on a new backbone: measure the distributional gap between decompressed and genuinely processed hidden states at the injection layer, and if it is large, tune the injection depth or increase reconstruction weight first.
  • The paper's own limitation that the 3B model stays below full-context fine-tuning on one multi-hop dataset suggests the method currently favors settings where selected evidence is sufficient; a hybrid that falls back to full context when selector confidence is low could broaden the method's reach.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes SeDeM, a long-context QA framework that decouples compact memory storage from decoder conditioning. A frozen LLM encoder extracts hidden states at an intermediate layer; each context segment is compressed by local mean pooling and a shared projection into memory blocks; a query-conditioned late-interaction selector chooses top-K blocks; and a learned decompressor expands only the selected blocks into hidden states injected at an intermediate decoder layer, after which the decoder continues to generate. Training is two-stage: Stage 1 reconstructs compressed/decompressed states against projected encoder states with next-token and distillation losses, and Stage 2 adds evidence-supervised selection, QA loss, and LoRA adaptation. Experiments on 2WikiMultiHopQA, MuSiQue, QASPER, and HotpotQA-Distractor with Llama-3.2-1B/3B report higher F1/ROUGE-L than the reimplemented compression baselines, and with the 3B backbone, SeDeM exceeds full-context fine-tuning on three datasets. The paper also reports lower time-to-first-token and improved throughput relative to ICAE, plus ablations on decompression, selection budget, layer depth, granularity, and transfer.

Significance. If the central result holds, SeDeM makes a useful architectural contribution: query-independent compact memories can be reused across queries, storage is decoupled from decoder conditioning, and a smaller encoder can serve a larger decoder. The paper is transparent about its limitations, provides extensive baseline reproduction details in Appendix A.8, includes parameter/FLOP accounting, and openly frames Appendix A.13's Lipschitz argument as motivational rather than as proof. The efficiency numbers and cross-model transfer are interesting. However, the headline attribution of the QA gains to selective decompression is not yet established by the current experiments, and the comparison to baselines is confounded by unequal decoder-side budgets and by the selector's access to gold evidence labels during training.

major comments (4)
  1. [§6.1, Tables 2 and 5; §7.1, Table 4] The main comparison does not isolate selective decompression from evidence selection. Table 4 ablates decompression only in a selector-disabled, all-blocks setting and uses a different ('internal non-final extraction-layer') configuration than the main model in Table 2; Table 5 shows that changing the selection budget K alone moves HotpotQA-Distractor F1 by about 5 points. Since the Table 2 SEDEM rows use top-K selection supervised by gold evidence labels and the compression baselines do not select evidence, the reported margin over ICAE/500x could come largely from the selector and from the larger effective decoder budget (Table 8) rather than from hidden-state decompression. A matched control is needed: decode from the raw tokens (or raw extracted hidden states) of the same top-K selected segments chosen by the SeDeM selector, with the same decoder and LoRA configuration, and compare a
  2. [§5 vs Appendix A.8.5 and Table 8] The compression budgets are not matched across methods. 500xCompressor is globally truncated at 500 tokens (A.8.5), ICAE conditions on 128 memory tokens, and SeDeM decodes from 128K positions, with K=8 giving about 1104 decoder-side tokens on QASPER (Table 8). The claim of 'higher QA scores than the evaluated compression baselines' is therefore a comparison across different effective context budgets as well as different mechanisms. The authors should either equalize the decoder-side budget/number of conditioning positions or present a budget-matched variant (e.g., ICAE and 500x with the same selected K segments) so that the marginal benefit of decompression is separated from the benefit of more conditioning tokens.
  3. [Table 2 and §6.1] All headline numbers are single-run point estimates. The claims of surpassing full-context fine-tuning on three datasets and of ranking above all compression baselines rest on F1 differences that in some cases are only a few points (e.g., QASPER 26.74 vs 23.44; HotpotQA 58.30 vs 45.09). Without multiple seeds or error bars, run-to-run variance could change the ordering. Report means and standard deviations over at least three seeds for SeDeM and for the closest baselines on the main comparisons, or provide a sensitivity analysis for the top three rows of Table 2.
  4. [Table 4 caption] The decompression ablation's reference row is not the main SeDeM configuration: the caption states that 'the SEDEM reference in this controlled ablation uses the internal non-final extraction-layer configuration; Table 2 reports the designated main configuration.' This makes the ablation internally controlled, but it does not directly support the main-table claim that selective decompression is the source of the gains in Table 2. Either re-run the Table 4 comparison with the exact main configuration, or explicitly state in §7.1 that the ablation is a configuration-level diagnostic and not a component analysis of the reported main results.
minor comments (5)
  1. [Table 2 footnote] The '—' entries for Activation Beacon ROUGE-L are explained as 'a corrected evaluation for which the paired corrected ROUGE-L value was not available.' Please clarify what 'corrected evaluation' refers to and why only F1 is re-verified; as written, the footnote is hard to interpret.
  2. [Figure 1] Figure 1 contains rendering artifacts in the submitted text (placeholder strings such as '/gid00001/gid00015'); the figure needs to be regenerated with the proper labels and arrowheads.
  3. [Abstract and body] The name is spelled 'SeDeM' in the abstract and running title but 'SEDEM' throughout the body and tables/appendices. Standardize the capitalization.
  4. [§7.2] The paragraph beginning 'Block-structured memories support selection' is a one-sentence standalone lead-in before the learnable-query compressor discussion; it reads as a heading rather than a result. Either move this sentence into the following paragraph or add a concrete comparison there.
  5. [Appendix A.13] The stability bound is clearly labeled as motivational, which is appropriate. However, the text would benefit from one explicit sentence stating that the bound does not constrain the trained decompressor in this paper and that the empirical losses in Eqs. (9)–(11) are the actual mechanism for state matching.

Circularity Check

0 steps flagged

No significant circularity: the central QA claims are empirical benchmark results; the sole theoretical appendix is explicitly non-load-bearing motivation.

full rationale

The paper's strongest claim (higher QA scores than compression baselines, and exceeding full-context fine-tuning on three datasets with the 3B backbone) is an empirical benchmark comparison reported in Table 2, not a derivation from fitted constants. The compressor, selector, and decompressor are trained with the stated objectives in Section 4 (Eqs. 9-14), and the QA metrics are measured on held-out validation/test splits; no fitted parameter is renamed as a prediction. The only analytical argument, Appendix A.13, is explicitly framed as 'architectural motivation' and says the Lipschitz assumption 'is a strong assumption for Transformer layers in general' and that the bound 'should be interpreted only in the local regime.' Thus it is not load-bearing for the empirical claims. The only same-author citation in the text is HMT (He et al., 2025), which appears as related work and is reimplemented as a baseline (Section 5, Appendix A.8.3); it is not used to justify the architecture, to import a uniqueness theorem, or to forbid alternatives. The Limitations section acknowledges dependence on gold evidence labels, empirically chosen extraction/injection layers, and below-reference MuSiQue performance; these are validity/risk limitations, not circularity. A separate experimental-design concern is noted in Section 7.1: the no-decompression ablation removes selection together with decompression and lacks a matched raw-token selected-context control, so attributing gains specifically to decompression is not fully isolated. That is a confounding-control issue, not an equation-level circularity under the stated rules. Overall, the central claims are self-contained empirical evaluations with external baselines, so no circular step is exhibited.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

The central result is empirical; it rests on pretrained-backbone transfer, on evidence-label supervision for the selector, and on the assumption that injected decompressed states are close enough to native decoder states. Hyperparameters K, T, C, ℓ_inject, and ℓ_extract are hand-chosen, and the paper's own ablations show they materially affect quality.

free parameters (6)
  • Selection budget K = K=2 for HotpotQA-Dist and 2WikiMHQA, K=4 for MuSiQue, K=8 for QASPER
    Chosen per dataset; Table 5 shows HotpotQA-Dist degrades for K>2 and 2WikiMHQA saturates at K=8, so this choice directly controls the main results.
  • Compression factor C and segment length T = C=4, T=128
    Fixed storage ratio 4×; Figure 2 shows monotonic quality decay with C and T=512 outperforming T=128 in the QASPER selector-disabled sweep, so the main operating point is not the best in its own ablation.
  • Injection layer ℓ_inject = 10
    Empirically chosen; A.12 shows F1 varies from 49.04 at ℓ_inject=16 to 61.91 at ℓ_inject=10 on a 3k/1k subset, and no principled selection procedure is given.
  • Extraction layer mixture = {10,12,14,16}, normalized weights 0.214/0.144/0.373/0.269
    Chosen over {4,10,16,22}; A.12 shows the gain comes mainly from using multiple intermediate layers, not from the learned weighting.
  • Selection loss weights and hyperparameters = λ_ret=1.0, λ_rec=0.1, τ=0.07, γ=2.0, λ_margin=0.5
    Set by hand; no sensitivity analysis is reported, and these directly shape the selector that drives top-K retrieval.
  • LoRA rank and alpha = r=64, α=128
    Fixed; Table 6 shows LoRA adds a small benefit, but no sweep is reported.
axioms (5)
  • domain assumption Pretrained Llama-3.2-Base frozen encoder hidden states at ℓ_extract contain enough evidence to answer QA after mean-pooling.
    §5: same-backbone settings use frozen Llama-3.2-Base; the compressor only sees layer ℓ_extract states, so evidence absent there cannot be recovered.
  • domain assumption The decompressor can learn hidden states close enough to native decoder layer-ℓ_inject states that the upper decoder stack remains well-behaved.
    §3.3 and A.13: A.13 explicitly calls the local Lipschitz/manifold-closeness assumption 'a strong assumption for Transformer layers in general'; if it fails, quality gains from injection may not transfer.
  • domain assumption Gold evidence annotations correctly identify the segments needed for answers, and the evidence-to-segment mapping is reliable.
    A.6.1: each dataset's annotations are mapped to positive segments; QASPER marks a segment positive if it overlaps at least one evidence span, which can be noisy.
  • domain assumption The top-K selected blocks contain the evidence required for the answer.
    Potential Risks: 'if selection fails or relevant evidence is omitted, the decoder may produce incomplete or unsupported answers'; HotpotQA-Dist's decline at K>2 shows selection quality is load-bearing.
  • domain assumption The reimplemented baselines are faithful enough for comparison (e.g., ICAE two-stage, AB continual pretraining, 500x truncation).
    A.8 describes substitutions (SlimPajama for RedPajama, arxiv-abstracts for Kaggle arXiv) and compatibility patches; these can advantage or disadvantage baselines relative to SEDEM.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of SeDeM: Selective Decompression of Hidden-State Memories for Long-Context Question Answering." pith.science (2026). https://pith.science/paper/BFUBODJ6

@misc{pith2026260800311,
  author       = {Pith},
  title        = {Pith review of: SeDeM: Selective Decompression of Hidden-State Memories for Long-Context Question Answering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BFUBODJ6}},
  note         = {Machine review of arXiv:2608.00311}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Long-context inference with large language models (LLMs) is costly: self-attention during prefill scales quadratically with sequence length, and the key-value (KV) cache grows with the number of processed tokens. Larger context windows also do not ensure reliable evidence use. Context compression reduces this cost, but many soft-compression methods use LLMs as compressors and rely on compact memory tokens both to preserve information and to condition the decoder. We propose SeDeM, a selective decompression framework that decouples compact memory storage from decoder conditioning. An LLM extracts hidden states from a chosen intermediate Transformer layer, a lightweight compressor stores them as memory blocks, a query-conditioned selector selects relevant blocks, and a decompressor expands only the selected blocks into hidden states compatible with an intermediate decoder layer. Thus, the decoder avoids both full-context processing and direct generation from highly compressed memory slots. On four long-context QA benchmarks, SeDeM achieves higher QA scores than the evaluated compression baselines in both 1B and 3B same-backbone settings, and with the 3B backbone exceeds full-context fine-tuning on three datasets. The learned selector uses block-level evidence supervision during training. SeDeM also reduces online time-to-first-token and improves autoregressive decoding throughput relative to ICAE.

Figures

Figures reproduced from arXiv: 2608.00311 by Jason Cong, Maryam Haghifam, Yizhou Sun.

Figure 1
Figure 1. Figure 1: Overview of SEDEM. The model stores long-context information in a compact memory bank, selects query-relevant blocks, selectively decompresses them into decoder-compatible hidden states, and injects the re￾constructed states into the decoder for answer generation. compatible with an intermediate decoder layer. A frozen LLM maps each context segment into hid￾den states, and a lightweight compressor stores e… view at source ↗
Figure 2
Figure 2. Figure 2: SEDEM granularity ablation on QASPER under selector-disabled evaluation. Left: varying seg￾ment length T at fixed compression factor C = 4, which keeps the total memory budget fixed. Right: varying compression factor C at fixed segment length T = 128, where larger C applies stronger compression and uses fewer memory slots. The results show that moderate segment lengths improve SEDEM quality, while overly a… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

16 extracted references · 5 linked inside Pith

  1. [4]

    Zifan He, Yingqi Cao, Zongyue Qin, Neha Prakriya, Yizhou Sun, and Jason Cong

    The llama 3 herd of models.Preprint, arXiv:2407.21783. Zifan He, Yingqi Cao, Zongyue Qin, Neha Prakriya, Yizhou Sun, and Jason Cong

    Pith/arXiv arXiv

  2. [5]

    HMT: Hierar- chical memory transformer for efficient long context language processing. InProceedings of the 2025 Conference of the Nations of the Americas Chap- ter of the Association for Computational Linguistics: Human Language Technologies (V olume 1: Long Papers), pages 8068–8089, Albuquerque, New Mex- ico. Association for Computational Linguistics. C...

    2025

  3. [6]

    InProceedings of the 2023 Con- ference on Empirical Methods in Natural Language Processing, pages 13358–13376, Singapore

    LLMLingua: Compress- ing prompts for accelerated inference of large lan- guage models. InProceedings of the 2023 Con- ference on Empirical Methods in Natural Language Processing, pages 13358–13376, Singapore. Asso- ciation for Computational Linguistics. Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dong- sheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu

    2023

  4. [7]

    In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6342–6353, Singapore

    Compressing context to enhance in- ference efficiency of large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6342–6353, Singapore. Association for Computa- tional Linguistics. Yuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, a...

    2023

  5. [8]

    InProceedings of the 63rd Annual Meeting of the Association for Com- putational Linguistics (V olume 1: Long Papers), pages 25081–25091, Vienna, Austria

    500xCompressor: Generalized prompt compression for large language models. InProceedings of the 63rd Annual Meeting of the Association for Com- putational Linguistics (V olume 1: Long Papers), pages 25081–25091, Vienna, Austria. Association for Computational Linguistics. Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paran- jape, Michele Bevilacqua, Fabio P...

    2024

  6. [9]

    InFindings of the As- sociation for Computational Linguistics: ACL 2024, pages 963–981, Bangkok, Thailand

    LLMLingua- 2: Data distillation for efficient and faithful task- agnostic prompt compression. InFindings of the As- sociation for Computational Linguistics: ACL 2024, pages 963–981, Bangkok, Thailand. Association for Computational Linguistics. Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis

    2024

  7. [10]

    The{10,12,14,16} mixture improves over the best single-layer set- ting, while learned and uniform mixtures per- form almost identically

    Mixing extraction layers.Table 15 evaluates whether combining extraction layers improves the compressed representation. The{10,12,14,16} mixture improves over the best single-layer set- ting, while learned and uniform mixtures per- form almost identically. This suggests that the gain comes mainly from using multiple interme- diate layers, not from learnin...

    2052

  8. [11]

    Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, Zhangyang Wang, and Beidi Chen

    Long context compression with activation beacon.arXiv preprint arXiv:2401.03462. Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, Zhangyang Wang, and Beidi Chen. 2023.H 2O: Heavy-hitter oracle for efficient generative inference of large language models. InAdvances in N...

    Pith/arXiv arXiv 2023

  9. [13]

    Deviations from the original AB setup.Our AB reproduction differs from Zhang et al. (2024) in four respects: (i) SlimPajama-6B replaces RedPajama-1T-Sample under the same 200M- token budget; (ii) AdamW replaces DeepSpeed FusedAdam for hardware compatibility; (iii) Hot- potQA is evaluated in the distractor setting rather than a single-context formulation; ...

    2024

  10. [14]

    Global context truncation caveat.A key im- plementation constraint of 500x is that the 500- token context limit is applied globally during both SFT and inference

    The question-answer budget is set tomax_qa_len=64; 15 when necessary, the question is truncated first to ensure that the answer retains at least 8 tokens. Global context truncation caveat.A key im- plementation constraint of 500x is that the 500- token context limit is applied globally during both SFT and inference. This is consequential for in- terpretin...

    arXiv 2025

  11. [16]

    The 500xCompres- sor estimate is not directly comparable, since its recipe processes only a single 500-token chunk

    SEDEM reduces to- tal FLOPs because encoding stops atℓ extract and the decoder processes only theKreconstructed blocks aboveℓ inject; ICAE and Activation Beacon require full-depth processing, and HMT pays for segmented backbone passes. The 500xCompres- sor estimate is not directly comparable, since its recipe processes only a single 500-token chunk. A.10 ...

    2022

  12. [128]

    A.7 Implementation Details Our method is implemented in PyTorch with Hug- gingFace Transformers

    Adapters are inserted into all four at- tention projections{W Q, WK, WV , WO}in every decoder layer. A.7 Implementation Details Our method is implemented in PyTorch with Hug- gingFace Transformers. All runs use bfloat16 mixed precision and FlashAttention-2 or SDPA at- tention where available. Generation is greedy with 12 Dataset Train / Val Source Evidenc...

    2024

  13. [2023]

    InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3829–3846, Singapore

    Adapting language mod- els to compress contexts. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3829–3846, Singapore. Association for Computational Linguistics. Tao Ge, Jing Hu, Lei Wang, Xun Wang, Si-Qing Chen, and Furu Wei

    2023

  14. [2024]

    Alexis Chevalier, Alexander Wettig, Anirudh Ajith, and Danqi Chen

    Pyra- midKV: Dynamic KV cache compression based on pyramidal information funneling.arXiv preprint arXiv:2406.02069. Alexis Chevalier, Alexander Wettig, Anirudh Ajith, and Danqi Chen

    Pith/arXiv arXiv

  15. [2025]

    Zefan Cai, Yichi Zhang, Bofei Gao, Yuliang Liu, Yucheng Li, Tianyu Liu, Keming Lu, Wayne Xiong, Yue Dong, Junjie Hu, and Wen Xiao

    CompLLM: Com- pression for long context Q&A.arXiv preprint arXiv:2509.19228. Zefan Cai, Yichi Zhang, Bofei Gao, Yuliang Liu, Yucheng Li, Tianyu Liu, Keming Lu, Wayne Xiong, Yue Dong, Junjie Hu, and Wen Xiao

    Pith/arXiv arXiv

  16. [2026]

    Peitian Zhang, Zheng Liu, Shitao Xiao, Ninglu Shao, Qiwei Ye, and Zhicheng Dou

    Fix the struc- tural bottleneck: Context compression via ex- plicit information transmission.arXiv preprint arXiv:2602.03784. Peitian Zhang, Zheng Liu, Shitao Xiao, Ninglu Shao, Qiwei Ye, and Zhicheng Dou

    Pith/arXiv arXiv

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.