Pith. sign in

REVIEW 4 major objections 6 minor 25 cited by

StreamMem is a training-free, query-agnostic KV cache compression method that holds streaming video memory to a fixed 6K tokens per layer while matching or beating larger-memory and query-aware baselines on long-video benchmarks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A training-free, query-agnostic KV cache compression method for streaming video MLLMs, using chat-template attention as a saliency proxy, matches or beats prior streaming methods at a fixed 6K memory budget.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Solid incremental contribution to streaming video KV compression; the query-agnostic proxy is a reasonable heuristic, but the evaluation would benefit from more rigor before calling it SOTA. the 4 major comments →

arxiv 2508.15717 v1 pith:WNY3IKNR submitted 2025-08-21 cs.CV cs.AI

StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understanding

Yanlai Yang , Zhuokai Zhao , Satya Narayan Shukla , Aashu Singh , Shlok Kumar Mishra , Lizhu Zhang , Mengye Ren This is my paper
classification cs.CV cs.AI
keywords streaming video understandingKV cache compressionquery-agnosticmultimodal large language modelsmemory-constrained inferencevisual token pruningframe merginglong video QA
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

StreamMem tries to solve a practical blocker for long-video AI assistants: when a video streams in, the model's key-value cache grows without bound until it exceeds memory. The paper argues that you do not need to keep every visual token or know the question in advance; a fixed set of chat-template tokens can act as a generic query, and their attention scores reveal which visual tokens matter. By pruning to those salient tokens, merging each frame into a compact prototype, and filtering redundant frames, StreamMem holds the cache at a constant size. On MLVU, EgoSchema, VideoMME, and two streaming QA benchmarks, this fixed-memory method outperforms other query-agnostic streaming compressors and stays competitive with methods that use the actual question or far more memory. A sympathetic reader would take away that a bounded, query-agnostic memory is enough for strong long-video understanding, at least for the tasks tested.

Core claim

The central claim is that in streaming video understanding, a fixed-size KV cache can be maintained without knowing the downstream question, and without fine-tuning, by using the cross-attention scores between visual tokens and the model's own chat-template tokens ("<|im_end|><|im_start|>assistant\n") as a query-agnostic saliency signal. The paper shows that these generic-attention scores select visual content that overlaps heavily with what specific questions attend to, and that combining top-k pruning with a weighted frame-level KV prototype preserves both fine-grained details and global temporal context. With a memory budget of 6K tokens per layer, StreamMem outperforms LiveVLM on LLaVA-O

What carries the argument

The load-bearing mechanism is the saliency metric: cross-attention scores between chat-template tokens and visual tokens, computed per layer via Equation 1, serve as a proxy for query relevance in the absence of the real question. This signal drives two complementary compression operations: top-k pruning of individual visual tokens, and frame-wise weighted merging (Equation 2) that condenses each frame into a single prototype key-value pair inserted at the frame's middle position. A lightweight input-frame filtering step based on cosine similarity of vision embeddings removes redundant frames before encoding, and YaRN position scaling preserves spatial-temporal position consistency across st

Load-bearing premise

The attention scores of a fixed set of chat-template tokens reliably indicate which visual tokens are informative for any downstream question, even though the real question is unknown during encoding.

What would settle it

Construct a video benchmark where the eventual question targets content that a generic video description would not mention, such as 'how many times does the red car appear?' or 'what color was the third background object?', and measure whether the tokens dropped by the chat-template proxy are exactly the ones needed; a sharp accuracy drop compared to using the true query would show the proxy fails on such content.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Memory use stays constant no matter how long the video is, so the method can run on edge devices with a hard memory ceiling and avoid offloading or out-of-memory failures.
  • Because no fine-tuning is required, StreamMem can be dropped onto any new open-source MLLM, as demonstrated with LLaVA-OneVision, Qwen2-VL, and Qwen2.5-VL.
  • A 24K-token budget beats a full 50K KV cache on MLVU, suggesting that much of a long video's KV cache is redundant and that streaming encoding with compression is competitive with, or better than, uniform frame sampling.
  • The query-agnostic proxy largely closes the gap to query-aware compression on holistic and single-detail tasks, though a gap remains on multi-detail tasks that require exact counts or ordering.
  • The method is compatible with existing streaming pipelines and baselines such as ReKV and LiveVLM, offering a direct replacement for their memory management while retaining higher or equal accuracy.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the proxy query is derived from the model's own chat template, the same trick may transfer to any chat-formatted MLLM, but its reliability likely depends on how much video-captioning data the model saw during pretraining; models trained mostly on instruction-following data may give weaker saliency signals.
  • The success of a fixed 6K-token memory suggests that long-form video understanding may not need content-addressed retrieval at all; if a compact summary of salient events is enough, then simpler, cheaper memory designs than ReKV-style offloading are viable.
  • A testable extension would be to combine StreamMem's query-agnostic memory with a late, query-aware refinement pass: keep the compressed stream as a base, then use the actual question to re-attend to any residual tokens, which could close the multi-detail gap without giving up streaming benefits.
  • The paper's proxy-validation (Table 4) is limited to MLVU; a broader stress test on videos where the question targets content a generic caption ignores, such as exact object counts or rarely mentioned background actions, would reveal where the proxy's assumptions break.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. StreamMem proposes a training-free, query-agnostic KV cache compression method for streaming video understanding. At each stream step it filters redundant frames, encodes the remaining frames with the MLLM, and compresses the combined memory and new keys/values by (i) pruning visual tokens according to cross-attention scores between fixed chat-template tokens and visual tokens and (ii) inserting frame-level prototype KV entries formed by attention-weighted averaging. It maintains a nominally fixed per-layer KV budget and uses YaRN positional scaling. The paper reports experiments on MLVU, EgoSchema, VideoMME, and RVS-Ego/RVS-Movie using LLaVA-OneVision-7B, Qwen2-VL-7B, and Qwen2.5-VL-3B, claiming state-of-the-art performance among query-agnostic streaming methods and competitiveness with query-aware methods.

Significance. The paper targets an important practical problem: bounded-memory streaming video QA without knowing the query in advance. The proposed components are simple and training-free, and the empirical study is broad: three model families, three offline benchmarks, two streaming benchmarks, and several ablations. Credit is due for explicitly ablating the proxy-query choice (Table 4) and for disclosing the MLVU metric-protocol inconsistency (Appendix A), which are often hidden. If the proxy-query assumption holds outside MLVU and the memory bound is actually enforced, StreamMem would be a useful plug-and-play method. However, the current evidence is not sufficient to establish the headline 'state-of-the-art' claim because of the issues listed below.

major comments (4)
  1. [Section 4.2 / Algorithm 1] Algorithm 1 does not, as written, enforce the fixed memory budget that is the paper's central contribution. The pruning step (`if |K| > M`) occurs before `Append Ki, Vi, si` and `Insert Merge(Ki), Merge(Vi)`; if the previous memory already has size M, the new KVs and prototypes are added without subsequent pruning, so the cache grows beyond M. This is inconsistent with the description in Section 4.2 ('merged ... and passed through a compression module') and makes the method's memory guarantees unclear. Please correct the pseudocode or specify the precise order and budget accounting, including per-layer budgets.
  2. [Section 4.2 / Table 4] The central query-agnostic assumption is that attention scores from fixed chat-template tokens to visual tokens are a good proxy for arbitrary downstream questions. Table 4 shows that on MLVU the chat-template proxy is worse than the true query by 2.0 points on holistic tasks (78.8 vs 80.8) and by 3.4 points on multi-detail tasks (43.0 vs 46.4). Multi-detail tasks (action order, action count) are exactly those requiring information that a generic description may not emphasize. The margins in Table 1 are small (e.g., 66.9 vs 66.3; EgoSchema tie at 63.0), so a similar proxy gap on VideoMME or EgoSchema could reverse the reported improvements. The proxy is validated on only one benchmark; please test it on the other benchmarks or provide other evidence that the pruned tokens are not needed for diverse questions.
  3. [Section 5.1 / Tables 6, 7 / Appendix A] The hyperparameters δ = 0.95 and the YaRN scaling factors λ are chosen on the MLVU dev set (Tables 6-7) and then reported on the same dev set in Table 1; EgoSchema is also reported on its dev set. No multiple seeds, error bars, or significance tests are provided, and several headline comparisons are within 0.3-1.0 points. Under these conditions, 'consistently retains high utility' and 'significantly outperforms' are not supported. Please report results on held-out/test portions (or at least a clear train/dev/test separation) and include variance/error bars over several runs.
  4. [Appendix A / Table 1] Appendix A states that MLVU 'All' is computed with overall accuracy for LLaVA-OneVision experiments but with task-averaged accuracy for Qwen2-VL and Qwen2.5-VL experiments. Because Table 1 mixes both protocols in the same column, the reader cannot compare rows across backbones, and the sentence in Section 5.2 that StreamMem 'outperforms the baselines on all benchmarks' is only true under the per-baseline protocol. Please report all MLVU numbers under both protocols, or at least make the protocol explicit in the table and restrict the claim to within-protocol comparisons.
minor comments (6)
  1. [Section 6 / Appendix A] Typographical errors: 'visusal' in the conclusion should be 'visual'; 'undestanding' in Appendix A should be 'understanding'.
  2. [Section 5.2] The phrase 'significantly outperforms' is used without statistical tests. Consider 'outperforms in these runs' or add significance testing.
  3. [Section 5.2 / Table 1] ReKV is described as an 'upper bound' but it uses 0.5 fps sampling and does not see the full video; it is a strong baseline rather than a true oracle.
  4. [Section 1 / Table 2] The introduction claims StreamMem 'surpasses state-of-the-art streaming video models', but in Table 2 StreamMem's RVS-Ego accuracy (57.6) is slightly below InfiniPot-V (57.9) and well below ReKV (63.7), and its RVS-Movie accuracy (52.7) is below ReKV (54.4). The abstract's 'competitive' wording is more appropriate.
  5. [Section 5.2] The paper claims memory efficiency but reports only KV token counts; actual GPU memory usage and wall-clock time would strengthen the practical story.
  6. [Algorithm 1] The score matrix s has one row per transformer layer, and Topk(s, k=M) is ambiguous. Specify how per-layer budgets are enforced and how scores from old and new KVs are aligned.

Circularity Check

0 steps flagged

No significant circularity: StreamMem's core proxy-query heuristic is validated empirically against a true-query oracle, and no derivation reduces to its inputs.

full rationale

The paper is an empirical compression system with no claimed formal derivation from assumptions to conclusions. The chat-template attention proxy in Eq. (1) is defined explicitly as a generic-query surrogate, and Table 4 directly compares it against the true-query oracle, showing a measurable gap and acknowledging the limitation. This is a validation of an external heuristic, not a reduction of the target result to its inputs. The only self-citations (e.g., Zhang et al. 2024d, 2024e, 2025c) are used for inspiration or as general prior work on token merging and vision backbones; they are not load-bearing premises, and no uniqueness theorem or forced-choice argument is imported from them. Hyperparameter choices and ablation results are standard empirical tuning on the MLVU benchmark, and the main SOTA claims rest on comparisons against external baselines (LiveVLM, InfiniPot-V) on held-out benchmarks, not on a derivation from the proxy. No equation reduces to another by construction, and no fitted parameter is renamed as a prediction. Therefore the paper is self-contained with respect to circularity.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 0 invented entities

No new entities are introduced. The method relies on the proxy-query assumption and on hand-tuned hyperparameters (δ, λ, KV budget) whose values were partly selected on the same dev sets used for evaluation.

free parameters (3)
  • frame filtering threshold δ = 0.95
    Chosen as the 'sweet spot' on MLVU dev in Table 6; controls how aggressively near-duplicate frames are averaged.
  • YaRN scaling factor λ = 8 for LLaVA-OneVision, 2 for Qwen2-VL, 1 for Qwen2.5-VL
    Selected per model based on default context length and validated on MLVU (Table 7); affects positional generalization for streaming segments.
  • KV memory budget M = 6K tokens/layer (main); 12K/24K in scaling experiments
    Fixed by design to 6K in main experiments; not fitted, but the method's accuracy depends on this manually chosen budget.
axioms (3)
  • domain assumption Chat template tokens act as a generic query proxy because MLLM pretraining makes the model attend to informative content when prompted with the chat template.
    Invoked in Section 4.2; the paper states the template implicitly prompts a generic video description. Table 4 provides empirical support on MLVU but it is not guaranteed for arbitrary questions.
  • domain assumption Cross-attention scores from the chat template tokens to visual tokens are a valid saliency measure for pruning.
    Equation 1 defines importance scores from these attention weights; the whole method rests on this correlation.
  • domain assumption Cosine similarity between vision-encoder frame embeddings reliably measures temporal redundancy.
    Section 4.1 uses a cosine similarity threshold to merge frames; validity depends on the embedding space, not proven.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understanding." pith.science (2026). https://pith.science/paper/WNY3IKNR

@misc{pith2026250815717,
  author       = {Pith},
  title        = {Pith review of: StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WNY3IKNR}},
  note         = {Machine review of arXiv:2508.15717}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Multimodal large language models (MLLMs) have made significant progress in visual-language reasoning, but their ability to efficiently handle long videos remains limited. Despite recent advances in long-context MLLMs, storing and attending to the key-value (KV) cache for long visual contexts incurs substantial memory and computational overhead. Existing visual compression methods require either encoding the entire visual context before compression or having access to the questions in advance, which is impractical for long video understanding and multi-turn conversational settings. In this work, we propose StreamMem, a query-agnostic KV cache memory mechanism for streaming video understanding. Specifically, StreamMem encodes new video frames in a streaming manner, compressing the KV cache using attention scores between visual tokens and generic query tokens, while maintaining a fixed-size KV memory to enable efficient question answering (QA) in memory-constrained, long-video scenarios. Evaluation on three long video understanding and two streaming video question answering benchmarks shows that StreamMem achieves state-of-the-art performance in query-agnostic KV cache compression and is competitive with query-aware compression approaches.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 25 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. An Efficient Streaming Video Understanding Framework with Agentic Control

    cs.CV 2026-05 unverdicted novelty 7.0

    R3-Streaming uses cascaded control with age-aware memory forgetting and TB-GRPO reinforcement learning to reach SOTA scores of 57.92 on OVO-Bench and 76.36 on StreamingBench with 95-96% fewer visual tokens.

  2. CoRDS: Coreset-based Representative and Diverse Selection for Streaming Video Understanding

    cs.CV 2026-05 unverdicted novelty 7.0

    CoRDS selects a compact KV-cache subset via joint-space coreset coverage and log-det diversity to outperform token-wise heuristics on long-video VLM benchmarks.

  3. Semantic-Aware Adaptive Visual Memory for Streaming Video Understanding

    cs.CV 2026-05 unverdicted novelty 7.0

    SAVEMem improves streaming video understanding scores by adding semantic awareness to memory compression and query-adaptive retrieval without any model training.

  4. Don't Pause! Every prediction matters in a streaming video

    cs.CV 2026-04 unverdicted novelty 7.0

    SPOT-Bench tests real-time streaming video perception with timeliness metrics, exposing limitations in current models and introducing AsynKV as an improved baseline.

  5. Mosaic: Cross-Modal Clustering for Efficient Video Understanding

    cs.PF 2026-04 unverdicted novelty 7.0

    Mosaic uses cross-modal clusters as the unit for KVCache organization in VLMs to achieve up to 1.38x speedup in streaming long-video understanding.

  6. STAC: Plug-and-Play Spatio-Temporal Aware Cache Compression for Streaming 3D Reconstruction

    cs.CV 2026-03 unverdicted novelty 7.0

    STAC compresses KV caches in streaming 3D reconstruction transformers via temporal token preservation with decayed attention, spatial voxel compression, and chunked multi-frame optimization, delivering 10x memory redu...

  7. ChronoStitch: Training-Free Composition of Visual KV Memories for Long-Horizon Temporal Reasoning

    cs.CV 2026-07 conditional novelty 6.0

    ChronoStitch re-bases stored video-chunk KV keys into a global multimodal RoPE frame and selectively recomputes a small slice of high-deviation tokens, recovering most of the joint-prefill temporal-reasoning gap at 3....

  8. FOLIO: Focused Semantic Memory for Streaming Video Understanding

    cs.CV 2026-07 conditional novelty 6.0

    Entity-centered focus-guided streaming memory lifts Qwen3-VL-8B to 82.0/69.1 Perception/Backward on OVO-Bench and 74.5 on StreamingBench while cutting writer tokens by ~32%.

  9. Kamera: Unified Position-Invariant Multimodal KV Cache for Training-Free Reuse

    cs.DC 2026-06 unverdicted novelty 6.0

    Kamera stores a low-rank patch with each position-free KV chunk to restore cross-chunk conditioning lost in naive reuse, enabling cheap reordering, sliding windows, and recall across attention mechanisms.

  10. How Well Can Your Video Model Remember? Measuring Memory-Budget Trade-offs in Long Video Understanding

    cs.CV 2026-06 unverdicted novelty 6.0

    Fits a model where logit-accuracy scales linearly in log frame budget B with distance-dependent exponent α(D) that decays log-linearly with temporal distance D, based on 155k binary predictions across ten models.

  11. FlowNar: Scalable Streaming Narration for Long-Form Videos

    cs.CV 2026-05 unverdicted novelty 6.0

    FlowNar achieves bounded memory and 3x higher throughput for streaming narration on Ego4D, EgoExo4D, and EpicKitchens100 by combining dynamic historical context removal with a Cross Linear Attentive Memory module.

  12. An Efficient Streaming Video Understanding Framework with Agentic Control

    cs.CV 2026-05 unverdicted novelty 6.0

    R3-Streaming uses cascaded control, age-aware memory forgetting, and TB-GRPO reinforcement learning to reach SOTA scores on streaming video benchmarks while cutting visual token usage by 95-96%.

  13. CodecSight: Leveraging Video Codec Signals for Efficient Streaming VLM Inference

    cs.DC 2026-04 unverdicted novelty 6.0

    CodecSight reuses video codec signals for online patch pruning before the vision transformer and selective KV-cache refresh in the LLM, delivering up to 3x higher throughput and 87% lower GPU compute than prior baseli...

  14. Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously

    cs.CV 2026-03 conditional novelty 6.0

    A 7B video model that generates intermediate text thoughts during playback, before the query arrives, improves streaming-video QA accuracy while keeping query-time latency near real-time.

  15. HERMES: KV Cache as Hierarchical Memory for Efficient Streaming Video Understanding

    cs.CV 2026-01 unverdicted novelty 6.0

    HERMES organizes the KV cache into a hierarchical memory to enable real-time streaming video understanding in MLLMs, achieving 10x faster TTFT and up to 11.4% accuracy gains on streaming benchmarks with 68% fewer tokens.

  16. LiveVLM: Efficient Online Video Understanding via Streaming-Oriented KV Cache and Retrieval

    cs.CV 2025-05 unverdicted novelty 6.0

    LiveVLM introduces VSB and PaR to compress and retrieve KV cache in streaming video LLMs, enabling LLaVA-OneVision to reach SOTA accuracy among training-free query-agnostic and training-based online models.

  17. ViCoStream: Streaming VideoLLMs Can Run Beyond 100 FPS with Stage-Wise Coordinated Inference

    cs.CV 2026-06 unverdicted novelty 5.0

    ViCoStream is a new coordinated pipeline framework for streaming VideoLLMs that achieves 134 FPS video throughput and less than 50 ms TTFT on A100 while keeping accuracy near full-history baselines.

  18. Linear Scaling Video VLMs for Long Video Understanding

    cs.CV 2026-05 unverdicted novelty 5.0

    StateKV is an inference-time technique that replaces quadratic self-attention prefill in video VLMs with a fixed-capacity importance-based recurrent state, keeping accuracy near full attention on long-video benchmarks...

  19. OmniMem: Perturbation-aware Memory Compression for Streaming Audio-Visual LLMs

    cs.AI 2026-05 unverdicted novelty 5.0

    OmniMem achieves 2-4% higher accuracy than training-free baselines on long video benchmarks for audio-visual LLMs by using modality-aware KV cache allocation and perturbation-aware state selection, with further gains ...

  20. MuKV: Multi-Grained KV Cache Compression for Long Streaming Video Question-Answering

    cs.CV 2026-05 conditional novelty 5.0

    MuKV adds multi-grained KV cache compression at patch-frame-segment levels plus semi-hierarchical retrieval to raise accuracy and cut memory in long video question-answering.

  21. Decouple and Cache: KV Cache Construction for Streaming Video Understanding

    cs.CV 2026-05 unverdicted novelty 5.0

    DSCache decouples cumulative past and instant KV caches with position-agnostic encoding to adapt offline VideoVLLMs to streaming video, delivering 2.5% average accuracy gains on QA benchmarks.

  22. cuRAMSES: Scalable AMR Optimizations for Large-Scale Cosmological Simulations

    astro-ph.GA 2026-04 conditional novelty 5.0

    cuRAMSES replaces Hilbert-curve domain decomposition with recursive k-section partitioning and adds Morton-key hashing plus spatial binning to cut communication volume and accelerate feedback routines by up to 260x wh...

  23. cuRAMSES: Scalable AMR Optimizations for Large-Scale Cosmological Simulations

    astro-ph.GA 2026-04 conditional novelty 5.0

    Recursive k-section domain decomposition, Morton-key hashing, and GPU dispatch cut communication and memory bottlenecks in RAMSES while preserving conservation to ~0.5%.

  24. Towards a Dynamic and Fixed-budget Memory Bank for Efficient Streaming Video Understanding

    cs.CV 2026-06 unverdicted novelty 4.0

    CausalMem constructs a dynamic fixed-budget memory bank for streaming video in MLLMs via online semantic basis updates, achieving 20x token compression and accuracy gains on benchmarks when applied to LLaVA-OneVision ...

  25. Watch, Remember, Reason: Human-View Video Understanding with MLLMs

    cs.CV 2026-06 unverdicted novelty 4.0

    This is a survey that frames video MLLM research via a human-view formulation of perceptual representations, memory states, reasoning traces, and predictions, then reviews methods, datasets, benchmarks, and open problems.

Reference graph

Works this paper leans on

25 extracted references · 3 canonical work pages · cited by 23 Pith papers · 1 internal anchor

  1. [2]

    Videollm-online: Online video large language model for streaming video

    Joya Chen, Zhaoyang Lv, Shiwei Wu, Kevin Qinghong Lin, Chenan Song, Difei Gao, Jia-Wei Liu, Ziteng Gao, Dongxing Mao, and Mike Zheng Shou. Videollm-online: Online video large language model for streaming video. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18407–18418, 2024a. Liang Chen, Haozhe Zhao, Tianyu Liu...

    Pith/arXiv arXiv

  2. [3]

    Beyond RAG: Task-Aware KV Cache Compression for Comprehensive Knowledge Reasoning

    Giulio Corallo, Orion Weller, Fabio Petroni, and Paolo Papotti. Beyond rag: Task-aware kv cache compression for comprehensive knowledge reasoning.arXiv preprint arXiv:2503.04973,

    work page internal anchor Pith review Pith/arXiv arXiv

  3. [4]

    A simple and effectivel_2 norm-based strategy for kv cache compression.arXiv preprint arXiv:2406.11430,

    Alessio Devoto, Yu Zhao, Simone Scardapane, and Pasquale Minervini. A simple and effectivel_2 norm-based strategy for kv cache compression.arXiv preprint arXiv:2406.11430,

    Pith/arXiv arXiv

  4. [6]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In9th International Conference on Learning Representations, ICLR 2021, V...

    work page 2021

  5. [7]

    Not all heads matter: A head-level kv cache compression method with integrated retrieval and reasoning.arXiv preprint arXiv:2410.19258,

    Yu Fu, Zefan Cai, Abedelkadir Asi, Wayne Xiong, Yue Dong, and Wen Xiao. Not all heads matter: A head-level kv cache compression method with integrated retrieval and reasoning.arXiv preprint arXiv:2410.19258,

    arXiv

  6. [8]

    Model tells you what to discard: Adaptive kv cache compression for llms.arXiv preprint arXiv:2310.01801,

    Suyu Ge, Yunan Zhang, Liyuan Liu, Minjia Zhang, Jiawei Han, and Jianfeng Gao. Model tells you what to discard: Adaptive kv cache compression for llms.arXiv preprint arXiv:2310.01801,

    Pith/arXiv arXiv

  7. [9]

    Squeezed attention: Accelerating long context length llm inference.arXiv preprint arXiv:2411.09688,

    Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh, Monishwaran Maheswaran, June Paik, Michael W Mahoney, Kurt Keutzer, and Amir Gholami. Squeezed attention: Accelerating long context length llm inference.arXiv preprint arXiv:2411.09688,

    arXiv

  8. [10]

    Gpt-4o system card.arXiv preprint arXiv:2410.21276,

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276,

    Pith/arXiv arXiv

  9. [11]

    Kvzip: Query-agnostic kv cache compression with context reconstruction.arXiv preprint arXiv:2505.23416, 2025a

    11 Jang-Hyun Kim, Jinuk Kim, Sangwoo Kwon, Jae W Lee, Sangdoo Yun, and Hyun Oh Song. Kvzip: Query-agnostic kv cache compression with context reconstruction.arXiv preprint arXiv:2505.23416, 2025a. Minsoo Kim, Kyuhong Shim, Jungwook Choi, and Simyung Chang. Infinipot: Infinite context processing on memory- constrained llms. arXiv preprint arXiv:2410.01518,

    arXiv

  10. [12]

    Infinipot-v: Memory-constrained kv cache compression for streaming video understanding.arXiv preprint arXiv:2506.15745, 2025b

    Minsoo Kim, Kyuhong Shim, Jungwook Choi, and Simyung Chang. Infinipot-v: Memory-constrained kv cache compression for streaming video understanding.arXiv preprint arXiv:2506.15745, 2025b. Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer.ar...

    arXiv

  11. [13]

    Video-xl-pro: Reconstructive token compression for extremely long video understanding.arXiv preprint arXiv:2503.18478, 2025a

    Xiangrui Liu, Yan Shu, Zheng Liu, Ao Li, Yang Tian, and Bo Zhao. Video-xl-pro: Reconstructive token compression for extremely long video understanding.arXiv preprint arXiv:2503.18478, 2025a. Xuyang Liu, Yiyu Wang, Junpeng Ma, and Linfeng Zhang. Video compression commander: Plug-and-play inference acceleration for video large language models.arXiv preprint...

    Pith/arXiv arXiv

  12. [15]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193,

    Pith/arXiv arXiv

  13. [16]

    Keydiff: Key similarity- based kv cache eviction for long-context llm inference in resource-constrained environments

    Junyoung Park, Dalton Jones, Matthew J Morse, Raghavv Goel, Mingu Lee, and Chris Lott. Keydiff: Key similarity- based kv cache eviction for long-context llm inference in resource-constrained environments. arXiv preprint arXiv:2504.15364,

    arXiv

  14. [17]

    Yarn: Efficient context window extension of large language models

    Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole. Yarn: Efficient context window extension of large language models. arXiv preprint arXiv:2309.00071,

    Pith/arXiv arXiv

  15. [18]

    Longvu: Spatiotemporal adaptive compression for long video-language understanding

    Xiaoqian Shen, Yunyang Xiong, Changsheng Zhao, Lemeng Wu, Jun Chen, Chenchen Zhu, Zechun Liu, Fanyi Xiao, Balakrishnan Varadarajan, Florian Bordes, et al. Longvu: Spatiotemporal adaptive compression for long video-language understanding. arXiv preprint arXiv:2410.17434,

    Pith/arXiv arXiv

  16. [19]

    Moviechat+: Question-aware sparse memory for long video question answering.arXiv preprint arXiv:2404.17176,

    Enxin Song, Wenhao Chai, Tian Ye, Jenq-Neng Hwang, Xi Li, and Gaoang Wang. Moviechat+: Question-aware sparse memory for long video question answering.arXiv preprint arXiv:2404.17176,

    Pith/arXiv arXiv

  17. [20]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024a

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024a. Xiao Wang, Qingyi Si, Jianlong Wu, Shiyu Zhu, Li Cao, and Liqiang Nie. Retake: Reducing temporal and knowl...

    Pith/arXiv arXiv

  18. [21]

    Visual context window extension: A new perspective for long video understanding

    Hongchen Wei and Zhenzhong Chen. Visual context window extension: A new perspective for long video understanding. arXiv preprint arXiv:2409.20018,

    Pith/arXiv arXiv

  19. [22]

    Effective long-context scaling of foundation models.arXiv preprint arXiv:2309.16039,

    Wenhan Xiong, Jingyu Liu, Igor Molybog, Hejia Zhang, Prajjwal Bhargava, Rui Hou, Louis Martin, Rashi Rungta, Karthik Abinav Sankararaman, Barlas Oguz, et al. Effective long-context scaling of foundation models.arXiv preprint arXiv:2309.16039,

    Pith/arXiv arXiv

  20. [23]

    Think: Thinner key cache by query-driven pruning.arXiv preprint arXiv:2407.21018,

    Yuhui Xu, Zhanming Jie, Hanze Dong, Lei Wang, Xudong Lu, Aojun Zhou, Amrita Saha, Caiming Xiong, and Doyen Sahoo. Think: Thinner key cache by query-driven pruning.arXiv preprint arXiv:2407.21018,

    Pith/arXiv arXiv

  21. [24]

    Videollama 3: Frontier multimodal foundation models for image and video understanding

    Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, et al. Videollama 3: Frontier multimodal foundation models for image and video understanding. arXiv preprint arXiv:2501.13106, 2025a. Haoji Zhang, Yiqin Wang, Yansong Tang, Yong Liu, Jiashi Feng, Jifeng Dai, and Xiaojie Jin. Flash...

    Pith/arXiv arXiv

  22. [28]

    overall accuracy

    The vision processor resizes each image such that the width and height are both divisible by 28; each frame is encoded into up to 130 tokens for Qwen2-VL and Qwen2.5-VL, and 196 tokens for LLaVA-OneVision. MLVU evaluation details. We would like to note that there are two different ways prior papers report results on the MLVU benchmark (Zhou et al., 2025):...

    work page 2025

  23. [2023]

    Livevlm: Efficient online video understanding via streaming-oriented kv cache and retrieval.arXiv preprint arXiv:2505.15269,

    Zhenyu Ning, Guangda Liu, Qihao Jin, Wenchao Ding, Minyi Guo, and Jieru Zhao. Livevlm: Efficient online video understanding via streaming-oriented kv cache and retrieval.arXiv preprint arXiv:2505.15269,

    Pith/arXiv arXiv

  24. [2024]

    Streaming video question-answering with in-context video kv-cache retrieval.arXiv preprint arXiv:2503.00540,

    Shangzhe Di, Zhelun Yu, Guanghao Zhang, Haoyuan Li, Tao Zhong, Hao Cheng, Bolin Li, Wanggui He, Fangxun Shu, and Hao Jiang. Streaming video question-answering with in-context video kv-cache retrieval.arXiv preprint arXiv:2503.00540,

    Pith/arXiv arXiv

  25. [2025]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923,

    Pith/arXiv arXiv

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.