Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

MadaKV: Adaptive Modality-Perception KV Cache Eviction for Efficient Multimodal Long-Context Inference

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MadaKV claims that a modality-aware, head-specific KV cache eviction rule keeps multimodal long-context accuracy within half a point of the full cache while using only 20% of the cache memory.

desk verdict A genuinely per-head, per-modality eviction scheme undermined by a budget update that doesn't actually conserve the cache budget, putting the headline memory/accuracy comparisons on shaky ground. read the letter →

arxiv 2506.15724 v1 pith:N6TPQPY2 submitted 2025-06-06 cs.LG cs.AIcs.CL

Kunxi Li , Zhonghua Jiang , Zhouzhou Shen , Zhaode Wang , Chengfei Lv , Shengyu Zhang , Fan Wu , Fei Wu This is my paper
classification cs.LGcs.AIcs.CL
keywords KVcacheevictionmultimodallargelanguagemodelslong-contextinferencemodalitypreferenceattentionsparsitymemorycompressionacceleration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes MadaKV, a scheme for shrinking the key-value cache of multimodal language models by deciding which tokens to evict based on each attention head's preference for text versus visual tokens. The central claim is that this modality-aware, head-specific eviction keeps accuracy nearly intact on long-context multimodal tasks while cutting cache memory by 80% to 95% and speeding up decoding by a factor of 1.3 to 1.5. The method estimates modality preference from attention scores that a few proxy tokens at the end of the prompt pay to each cached token, then splits each head's cache budget between modalities in proportion to that preference. A second component adjusts the budget from layer to layer, compensating when a layer overspends or underspends its cache. Experiments on two multimodal models and a long-context benchmark report that MadaKV stays within about half a point of full-cache accuracy at a 20% budget, and outperforms baselines that evict tokens uniformly or always prioritize text.

What carries the argument

The central mechanism is a ratio-symmetric budget split: for each attention head, the cache budget is divided between visual and text tokens in proportion to a preference score, so a head that mostly attends to text keeps mostly text tokens. The preference score is computed from a small set of proxy tokens (the last few tokens of the prompt, typically the question), summing their attention to every cached token. The second mechanism is a layer-level compensation loop: after computing how many tokens each head would need to cover a threshold fraction $\theta$ of the preference mass, the difference between that need and the current budget is accumulated, and the budget for the next layer is adjusted by $K_l / (L-l)$. Together these define the eviction order: tokens with low proxy-attention are evicted first, but only after per-modality and per-layer budgets are set.

What would settle it

Measure, on a set of long-context tasks, the overlap between the tokens MadaKV evicts and the tokens that actually receive high attention from the model's generated decoding steps; if many evicted tokens are strongly attended to later, the proxy-token preference metric is unreliable. A direct counterexperiment is a needle-in-a-haystack task where the answer is carried by a visual token that the question's last few tokens ignore during prefill: MadaKV would allocate no budget to that modality and would fail to answer, contradicting the paper's claim that budget follows true usefulness.

Watch

Extended reading notes

Core claim

The paper's central discovery is that an eviction rule which is both modality-aware and head-aware preserves the information needed for long-context multimodal generation at cache budgets as low as 5-20%. The evidence comes from two observations: attention to text tokens is sharply concentrated while attention to visual tokens is diffuse, and different heads allocate very different proportions of their attention to text versus visuals. MadaKV encodes these observations as a preference metric $w_v = \sum_{i \in X_v} \psi(i)$ and $w_t = \sum_{i \in X_t} \psi(i)$, where $\psi(i)$ is the attention an averaged proxy token at the end of the prompt pays to token $i$, and each head's budget is $\phi^{l,h}_v = (w_v/(w_v+w_t))\phi^l$. Working alongside this, a hierarchical compensation term $K_l = \sum_h (k^{l,h}_v + k^{l,h}_t - \phi^l)$ measures whether a layer needs more or fewer tokens than its budget and shifts the next layer's budget by $K_l / (L-l)$. The paper argues that this coupling is what lets the method avoid the cascading loss caused by naive early-layer eviction, and that the combined rule is why accuracy stays close to the full cache while the cache shrinks by 80% to 95%.

Load-bearing premise

The eviction order is driven entirely by attention scores from a few proxy tokens at the end of the prompt, and the paper assumes these scores reveal which tokens every later decoding step will need; if the question's attention is not representative of what the model actually queries, the evicted tokens are the wrong ones.

Editorial extensions

If this is right

  • If correct, multimodal long-context serving can run with 5-20% of the KV cache memory, allowing longer contexts or larger batches on the same hardware.
  • Decoding latency improvements of 1.3-1.5x follow directly from the smaller cache, because fewer cache entries are read at each step.
  • The method requires no fine-tuning, so it can be applied at inference time to existing vision-language models as a plug-in.
  • On a text-needle task, a 20% cache budget with MadaKV matches the accuracy of a 60% budget under fixed text-prioritization eviction, showing that recovered tokens carry task-critical information.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to re-estimate modality preferences periodically during decoding, since multi-turn or reasoning-heavy tasks may reformulate the query after the prefill proxies are fixed.
  • The same preference-proportional budget rule could be applied to finer granularities than two modalities, e.g., spatial image regions or frame groups in video, which the paper lists as untested.
  • If attention sparsity profiles are stable across similar inputs, the layer compensation term $K_l$ could be precomputed from a small calibration set, removing the prefill-time cost of estimating it per instance.
  • On tasks with many images (the benchmark averages 15.2 per sample), the diffuse visual attention may require a larger visual share than the proxy metric yields; the paper's 20% budget may not transfer to video-length contexts without rebalancing.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes MadaKV, a training-free KV cache eviction method for multimodal LLMs. It combines Modality Preference Adaptation (MPA), which uses per-head attention scores from proxy tokens to split a per-layer budget between visual and textual tokens, with Hierarchical Compression Compensation (HCC), an inter-layer recurrence that adjusts the per-layer budget based on the previous layer's overspend or underspend. The method is evaluated on nine MileBench tasks using LLaVA-v1.5-7B/13B and Qwen2.5-VL-7B, with reported accuracy close to full cache at 20% budget, an ablation study, and speed/memory measurements. The headline claims are a 1.3 to 1.5 times decoding-latency improvement and an 80% to 95% reduction in KV-cache memory footprint while maintaining performance.

Significance. The core idea—making cache eviction modality-aware at the attention-head level—is timely and plausible, and the paper is clearly structured. The LLaVA rows in Table 1 are internally consistent, the ablation in Table 3 supports the contribution of both MPA and HCC, and the method has no learned parameters fitted to the benchmark outcomes. However, two load-bearing issues prevent the results from being accepted as stated: the Qwen2.5-VL rows in Table 1 report averages incompatible with the listed task scores, and the HCC budget update in Eq. (11) does not conserve the global cache budget, making the same-budget accuracy comparisons and the memory-reduction claims uncontrolled. These are correctness concerns rather than presentation issues.

major comments (3)
  1. [3.3, Eq. (11); Table 2] The claim that HCC 'ensures that the overall cache budget is maintained' is not supported by the recurrence. With K_l = sum_h (k^{l,h}_v + k^{l,h}_t - phi^l), Eq. (11) gives phi^{l+1} = phi^l - K_l/(L-l), and summing over layers yields a realized total cache of H L phi^1 - (H-1) * sum_{l=1}^{L-1} K_l + K_L, which is not equal to the intended H L phi^1 unless the K_l vanish. The per-layer budget phi^l can also exceed 1 or become negative unless clipping is specified, and no clipping is described. Table 2 is consistent with this issue: the nominal 20% budget consumes 0.41 GiB versus 1.63 GiB full cache (25.2%, not 20%), and the nominal 5% budget consumes 0.16 GiB (9.8%, not 5%). Consequently, comparisons at equal nominal budgets in Table 1 and Figure 4 are not controlled, and the 80-95% memory-reduction claim should be expressed in terms of measured cache size. Please either modify HCC to conserve the global budget exactly or report and compare the actual cache footprints for all configurations.
  2. [Table 1, Qwen2.5-VL rows] The Qwen2.5-VL rows contain averages that do not match the nine listed task scores. For Full Cache, the nine values sum to 380.05, giving a mean of 42.23, yet the table reports an Average of 63.34. The same inconsistency appears in every Qwen2.5-VL row: for StreamingLLM the listed scores sum to 335.87 (mean 37.32) against a reported 55.98, and the other rows show similar gaps. The LLaVA rows are consistent, so this is not a generic formatting problem in the Average column. Please correct the task scores or the averages, and verify the Qwen2.5-VL experiments against the original inference logs; as printed, the Qwen results cannot be interpreted.
  3. [3.3, Eq. (7)] The eviction order depends entirely on the proxy-token importance scores psi(i) computed from Eq. (7). The paper selects 'a few tokens from the end of the prompt' as proxy tokens, with the rationale that they typically represent task-specific questions, but it provides no evidence that these tokens cover the queries the decoder will actually attend to during generation. If the proxy-token assumption fails for a task or instance, the eviction order is wrong regardless of how well the budget formulas are implemented. Please add a sensitivity analysis of proxy-token count and position, or an evaluation of proxy-set coverage (for instance, comparing eviction orders induced by different proxy choices), to demonstrate that the method is robust to this choice.
minor comments (5)
  1. [Table 1] Several cells in Table 1 have missing spaces between numbers, e.g., '45.5064.0045.50', '53.5074.5051.00', '62.5061.5061.00', '42.0028.9736.66', and '62.0061.5061.50'. These should be separated for readability.
  2. [Figure 4] The y-axis label in the left panel reads 'Sppt-the-Diff', which appears to be a typo for 'Spot-the-Diff'.
  3. [4.5] The text states that HCC leads to a 3.07% improvement in the TN task, but the differences in Table 3 are 2.80 points (MPA+HCC vs. MPA only) or 3.04 points (HCC only vs. neither). Please clarify how the 3.07% figure is computed.
  4. [4.2] The sentence 'MadaKV achieves a 6.11% improvement in performance' in the TN task is not directly traceable to Table 1; the absolute differences from the listed baselines are approximately 6.0-6.3 points. Please specify the baseline used for the percentage improvement.
  5. [4.4] The latency and memory measurements in Table 2 are reported as means over 20 randomly selected data entries without standard deviations or error bars. Given the small sample, reporting variance would strengthen the efficiency claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the eviction rules are empirically tested heuristics, not quantities derived from the target accuracy metrics.

full rationale

MadaKV's derivation chain is not circular under the definitions in the review protocol. The modality budget split (Eq. 8) and the per-layer compensation (Eq. 11) are computed from attention scores of proxy tokens (Eq. 7), not from the accuracy numbers in Tables 1 and 3; the reported accuracies are empirical outcomes that could have gone the other way, so no fitted-input-called-prediction or self-definitional step is present. The HCC recurrence is claimed to maintain the overall cache budget; under the paper's own accounting the total planned cache is conserved (summing the recurrence gives total usage across layers equal to L times the initial budget), so the memory claims are not tautological. The related-work citations that include overlapping authors (e.g., Zhou et al. 2025a-d) are background references and do not carry the load of the central claim. The one legitimate concern, that Section 3.2 uses MileBench to motivate the design and Section 4 evaluates on MileBench, is a test-distribution overlap issue, not circularity, because no parameter is fitted to the benchmark outcomes and the eviction rule could succeed or fail on new inputs. I therefore find no specific reduction of a prediction to its inputs and assign score 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method uses no new physical or architectural entities. Its contributions are two algorithmic constructs, the modality preference weight and the layer compensation term, both defined in the paper and measurable from attention weights. The ledger therefore records only the underspecified hyperparameters and the borrowed attention-as-importance assumptions that the central claim depends on.

free parameters (3)
  • attention retention threshold theta
    Threshold in Eq. (9) controlling how much per-modality attention mass must be retained before eviction; the value is never reported, so the algorithm is underspecified and performance depends on it.
  • proxy token count
    Section 3.3 says only 'a few tokens from the end of the prompt are selected as proxy tokens'; the exact number is not given, and the quality of the importance estimate depends on it.
  • initial layer cache budget phi_l = 20% in the main table; 5-60% in the budget sweep
    User-controlled compression ratio that sets the operating point for all accuracy, latency, and memory results; it is chosen by hand rather than learned.
assumptions (4)
  • domain assumption Attention scores from proxy tokens measure the future usefulness of cached tokens.
    Equation (7) defines token importance as the attention received from P, a few end-of-prompt tokens; this heuristic is inherited from text-only eviction and is not validated for multimodal long context.
  • domain assumption End-of-prompt proxy tokens represent task-specific queries.
    Section 3.3 states 'We select a few tokens from the end of the prompt as proxy tokens, as they typically represent task-specific questions'; no sensitivity analysis is provided.
  • domain assumption Each attention head has a modality preference that can be summarized by the ratio of summed attention to visual and text tokens.
    Equations (6) through (8) allocate the cache budget between modalities using w_v divided by (w_v plus w_t) computed once during prefill; attention preferences may shift during decoding.
  • ad hoc to paper The layer-budget update phi^{l+1} = phi^l minus K_l divided by (L minus l) is a sound way to distribute overspend or savings across remaining layers.
    Equation (11) is introduced without derivation or an optimality argument; the 1 over (L minus l) weighting is a design choice rather than a proven compensation rule.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MadaKV: Adaptive Modality-Perception KV Cache Eviction for Efficient Multimodal Long-Context Inference." pith.science (2026). https://pith.science/paper/N6TPQPY2

@misc{pith2026250615724,
  author       = {Pith},
  title        = {Pith review of: MadaKV: Adaptive Modality-Perception KV Cache Eviction for Efficient Multimodal Long-Context Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N6TPQPY2}},
  note         = {Machine review of arXiv:2506.15724}
}
read the original abstract

This paper introduces MadaKV, a modality-adaptive key-value (KV) cache eviction strategy designed to enhance the efficiency of multimodal large language models (MLLMs) in long-context inference. In multimodal scenarios, attention heads exhibit varying preferences for different modalities, resulting in significant disparities in modality importance across attention heads. Traditional KV cache eviction methods, which are tailored for unimodal settings, fail to capture modality-specific information, thereby yielding suboptimal performance. MadaKV addresses these challenges through two key components: modality preference adaptation and hierarchical compression compensation. By dynamically sensing modality information within attention heads and adaptively retaining critical tokens, MadaKV achieves substantial reductions in KV cache memory footprint and model inference decoding latency (1.3 to 1.5 times improvement) while maintaining high accuracy across various multimodal long-context tasks. Extensive experiments on representative MLLMs and the MileBench benchmark demonstrate the effectiveness of MadaKV compared to existing KV cache eviction methods.

Figures

Figures reproduced from arXiv: 2506.15724 by the authors.

Figure 1
Figure 1. Comparison between Modal-Agnostic Spar￾sity (a) and MadaKV’s Modal Perception Sparsity (b). tensors from past tokens, reducing the decoding process to linear time complexity. However, as the context length increases, the KV cache’s memory footprint increases significantly (Shi et al., 2024). Building on prior research that highlights the spar￾sity within attention mechanisms, where only a subset of tokens significan… view at source ↗
Figure 2
Figure 2. (a) The sparsity differences among tokens of different modalities. TN: Text Needle In A Haystack; ED: [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Overview of MadaKV. Modality Preference Adaptation identifies the modality preferences of attention [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Comparison results for various cache budgets. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Examples of modal significance in different [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Attention patterns of LLava-v1.5-7B on ALFRED. [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PhyCheck: Fine-Grained Evidence-Grounded Dataset for Physical Law Understanding in Video-LLMs

    cs.CV 2026-08 conditional novelty 7.0 of 10

    PhyCheck is a 69,825-pair video QA benchmark that tests and improves Video-LLMs' ability to judge whether events obey physical laws, with fine-grained evidence questions and a context-sensitivity pilot.

Reference graph

Works this paper leans on

57 extracted references · 17 canonical work pages · cited by 1 Pith paper

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  2. [2]

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. 2022. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716--23736

    2022

  3. [3]

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966

    arXiv 2023

  4. [4]

    Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150

    arXiv 2020

  5. [5]

    Zefan Cai, Yichi Zhang, Bofei Gao, Yuliang Liu, Tianyu Liu, Keming Lu, Wayne Xiong, Yue Dong, Baobao Chang, Junjie Hu, et al. 2024. Pyramidkv: Dynamic kv cache compression based on pyramidal information funneling. arXiv preprint arXiv:2406.02069

    arXiv 2024

  6. [6]

    Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. 2016. Training deep nets with sublinear memory cost. arXiv preprint arXiv:1604.06174

    arXiv 2016

  7. [7]

    Yilong Chen, Guoxia Wang, Junyuan Shang, Shiyao Cui, Zhenyu Zhang, Tingwen Liu, Shuohuan Wang, Yu Sun, Dianhai Yu, and Hua Wu. 2024. Nacl: A general and effective kv cache eviction framework for llms at inference time. arXiv preprint arXiv:2408.03675

    arXiv 2024

  8. [8]

    Rewon Child, Scott Gray, Alec Radford, and Ilya Sutskever. 2019. Generating long sequences with sparse transformers. arXiv preprint arXiv:1904.10509

    arXiv 2019

Show all 57 references
  1. [9]

    Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Davis, Afroz Mohiuddin, Lukasz Kaiser, et al. 2020. Rethinking attention with performers. arXiv preprint arXiv:2009.14794

    2020 arXiv

  2. [10]

    Luciano Floridi and Massimo Chiriatti. 2020. Gpt-3: Its nature, scope, limits, and consequences. Minds and Machines, 30:681--694

    2020

  3. [11]

    Suyu Ge, Yunan Zhang, Liyuan Liu, Minjia Zhang, Jiawei Han, and Jianfeng Gao. 2023. Model tells you what to discard: Adaptive kv cache compression for llms. arXiv preprint arXiv:2310.01801

    2023 arXiv

  4. [12]

    Chi Han, Qifan Wang, Hao Peng, Wenhan Xiong, Yu Chen, Heng Ji, and Sinong Wang. 2024. Lm-infinite: Zero-shot extreme length generalization for large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Lingui...

    2024

  5. [13]

    Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh, Michael W Mahoney, Yakun Sophia Shao, Kurt Keutzer, and Amir Gholami. 2024. Kvquant: Towards 10 million context length llm inference with kv cache quantization. arXiv preprint arXiv:2401.18079

    2024 arXiv

  6. [14]

    Mehrdad Hosseinzadeh and Yang Wang. 2021. https://doi.org/10.1109/CVPR46437.2021.00275 Image change captioning by learning from an auxiliary task . In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021 , pages 2725--2734. Computer ...

    2021

  7. [15]

    Qingqiu Huang, Yu Xiong, Anyi Rao, Jiaze Wang, and Dahua Lin. 2020. https://doi.org/10.1007/978-3-030-58548-8\_41 Movienet: A holistic dataset for movie understanding . In Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Par...

    2020 doi

  8. [16]

    Harsh Jhamtani and Taylor Berg - Kirkpatrick. 2018. https://doi.org/10.18653/V1/D18-1436 Learning to describe differences between pairs of similar images . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 ...

    2018 doi

  9. [17]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825

    2023 arXiv

  10. [18]

    Zixiao Kong, Xianquan Wang, Shuanghong Shen, Keyu Zhu, Huibo Xu, and Yu Su. 2025. https://doi.org/10.1609/aaai.v39i23.34611 Scholargec: Enhancing controllability of large language model for chinese academic grammatical error correction . Proceedings of the AAAI Conference on A...

    2025 doi

  11. [19]

    Dongxu Li, Junnan Li, and Steven Hoi. 2024. Blip-diffusion: Pre-trained subject representation for controllable text-to-image generation and editing. Advances in Neural Information Processing Systems, 36

    2024

  12. [20]

    Yongqi Li, Wenjie Li, and Liqiang Nie. 2022. https://doi.org/10.18653/V1/2022.ACL-LONG.290 Mmcoqa: Conversational question answering over text, tables, and images . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Paper...

    2022 doi

  13. [21]

    Hao Liu, Wilson Yan, Matei Zaharia, and Pieter Abbeel. 2024 a . World model on million-length video and language with blockwise ringattention. CoRR

    2024

  14. [22]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. Advances in neural information processing systems, 36:34892--34916

    2023

  15. [23]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2024 b . Visual instruction tuning. Advances in neural information processing systems, 36

    2024

  16. [24]

    Zichang Liu, Aditya Desai, Fangshuo Liao, Weitao Wang, Victor Xie, Zhaozhuo Xu, Anastasios Kyrillidis, and Anshumali Shrivastava. 2024 c . Scissorhands: Exploiting the persistence of importance hypothesis for llm kv cache compression at test time. Advances in Neural Informatio...

    2024

  17. [25]

    Minesh Mathew, Dimosthenis Karatzas, and C. V. Jawahar. 2021. https://doi.org/10.1109/WACV48630.2021.00225 Docvqa: A dataset for VQA on document images . In IEEE Winter Conference on Applications of Computer Vision, WACV 2021, Waikoloa, HI, USA, January 3-8, 2021 , pages 2199-...

    2021

  18. [26]

    AI Meta. 2024. Introducing meta llama 3: The most capable openly available llm to date. Meta AI

    2024

  19. [27]

    Jie Ren, Samyam Rajbhandari, Reza Yazdani Aminabadi, Olatunji Ruwase, Shuangyan Yang, Minjia Zhang, Dong Li, and Yuxiong He. 2021. \ Zero-offload \ : Democratizing \ billion-scale \ model training. In 2021 USENIX Annual Technical Conference (USENIX ATC 21), pages 551--564

    2021

  20. [28]

    Minsoo Rhu, Natalia Gimelshein, Jason Clemons, Arslan Zulfiqar, and Stephen W Keckler. 2016. vdnn: Virtualized deep neural networks for scalable, memory-efficient neural network design. In 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), pages 1-...

    2016

  21. [29]

    Ying Sheng, Lianmin Zheng, Binhang Yuan, Zhuohan Li, Max Ryabinin, Beidi Chen, Percy Liang, Christopher R \'e , Ion Stoica, and Ce Zhang. 2023. Flexgen: High-throughput generative inference of large language models with a single gpu. In International Conference on Machine Lear...

    2023

  22. [30]

    Luohe Shi, Hongyi Zhang, Yao Yao, Zuchao Li, and Hai Zhao. 2024. Keep the cost down: A review on methods to optimize llm's kv-cache consumption. arXiv preprint arXiv:2407.18003

    2024 arXiv

  23. [31]

    Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. 2020. https://doi.org/10.1109/CVPR42600.2020.01075 ALFRED: A benchmark for interpreting grounded instructions for everyday tasks . In 2020 IEEE/CVF Conf...

    2020

  24. [32]

    Dingjie Song, Shunian Chen, Guiming Hardy Chen, Fei Yu, Xiang Wan, and Benyou Wang. 2024 a . Milebench: Benchmarking mllms in long context. arXiv preprint arXiv:2404.18532

    2024 arXiv

  25. [33]

    Lin Song, Yukang Chen, Shuai Yang, Xiaohan Ding, Yixiao Ge, Ying-Cong Chen, and Ying Shan. 2024 b . Low-rank approximation for sparse attention in multi-modal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13763--13773

    2024

  26. [34]

    Hao Tan, Franck Dernoncourt, Zhe Lin, Trung Bui, and Mohit Bansal. 2019. https://doi.org/10.18653/V1/P19-1182 Expressing visual relationships via language . In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July ...

    2019 doi

  27. [35]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  28. [36]

    Szymon Tworkowski, Konrad Staniszewski, Miko aj Pacek, Yuhuai Wu, Henryk Michalewski, and Piotr Mi o \'s . 2024. Focused transformer: Contrastive training for context scaling. Advances in Neural Information Processing Systems, 36

    2024

  29. [37]

    Zhongwei Wan, Ziang Wu, Che Liu, Jinfa Huang, Zhihong Zhu, Peng Jin, Longyue Wang, and Li Yuan. 2024. Look-m: Look-once optimization in kv cache for efficient multimodal long-context inference. arXiv preprint arXiv:2406.18139

    2024 arXiv

  30. [38]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. 2024. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191

    2024 arXiv

  31. [39]

    Sinong Wang, Belinda Z Li, Madian Khabsa, Han Fang, and Hao Ma. 2020. Linformer: Self-attention with linear complexity. arXiv preprint arXiv:2006.04768

    2020 arXiv

  32. [40]

    Xianquan Wang, Likang Wu, Zhi Li, Haitao Yuan, Shuanghong Shen, Huibo Xu, Yu Su, and Chenyi Lei. 2025. https://doi.org/10.1145/3690624.3709275 Mitigating redundancy in deep recommender systems: A field importance distribution perspective . In Proceedings of the 31st ACM SIGKDD...

    2025

  33. [41]

    A Waswani, N Shazeer, N Parmar, J Uszkoreit, L Jones, A Gomez, L Kaiser, and I Polosukhin. 2017. Attention is all you need. In NIPS

    2017

  34. [42]

    Bo Wu, Shoubin Yu, Zhenfang Chen, Josh Tenenbaum, and Chuang Gan. 2021. https://datasets-benchmarks-proceedings.neurips.cc/paper/2021/hash/5ef059938ba799aaa845e1c2e8a762bd-Abstract-round2.html STAR: A benchmark for situated reasoning in real-world videos . In Proceedings of th...

    2021

  35. [44]

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. 2023 b . https://api.semanticscholar.org/CorpusID:263310483 Efficient streaming language models with attention sinks . ArXiv, abs/2309.17453

    2023 arXiv

  36. [45]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024 a . Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115

    2024 arXiv

  37. [46]

    Dongjie Yang, XiaoDong Han, Yan Gao, Yao Hu, Shilin Zhang, and Hai Zhao. 2024 b . Pyramidinfer: Pyramid kv cache compression for high-throughput llm inference. arXiv preprint arXiv:2405.12532

    2024 arXiv

  38. [47]

    June Yong Yang, Byeongwook Kim, Jeongin Bae, Beomseok Kwon, Gunho Park, Eunho Yang, Se Jung Kwon, and Dongsoo Lee. 2024 c . No token left behind: Reliable kv cache compression via importance-aware mixed precision quantization. arXiv preprint arXiv:2402.18096

    2024 arXiv

  39. [48]

    Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. 2023. A survey on multimodal large language models. arXiv preprint arXiv:2306.13549

    2023 arXiv

  40. [49]

    Rongzhi Zhang, Kuang Wang, Liyuan Liu, Shuohang Wang, Hao Cheng, Chao Zhang, and Yelong Shen. 2024 a . Lorc: Low-rank compression for llms kv cache with a progressive compression strategy. arXiv preprint arXiv:2410.03111

    2024 arXiv

  41. [50]

    Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher R \'e , Clark Barrett, et al. 2024 b . H2o: Heavy-hitter oracle for efficient generative inference of large language models. Advances in Neural Information Pro...

    2024

  42. [51]

    Guangxiang Zhao, Junyang Lin, Zhiyuan Zhang, Xuancheng Ren, Qi Su, and Xu Sun. 2019. Explicit sparse transformer: Concentrated attention through explicit selection. arXiv preprint arXiv:1912.11637

    2019 arXiv

  43. [52]

    Yiyun Zhou, Wenkang Han, and Jingyuan Chen. 2025 a . Revisiting applicable and comprehensive knowledge tracing in large-scale data. arXiv preprint arXiv:2501.14256

    2025 arXiv

  44. [53]

    Yiyun Zhou, Zheqi Lv, Shengyu Zhang, and Jingyuan Chen. 2024. https://openreview.net/forum?id=UVaPEthRKx Cuff- KT : Tackling learners' real-time learning pattern adjustment via tuning-free knowledge state-guided model updating

    2024

  45. [54]

    Yiyun Zhou, Zheqi Lv, Shengyu Zhang, and Jingyuan Chen. 2025 b . https://arxiv.org/abs/2505.19543 Cuff-kt: Tackling learners' real-time learning pattern adjustment via tuning-free knowledge state guided model updating . Preprint, arXiv:2505.19543

    2025 arXiv

  46. [55]

    Yiyun Zhou, Zheqi Lv, Shengyu Zhang, and Jingyuan Chen. 2025 c . Disentangled knowledge tracing for alleviating cognitive bias. In Proceedings of the ACM on Web Conference 2025, pages 2633--2645

    2025

  47. [56]

    Yiyun Zhou, Chang Yao, and Jingyuan Chen. 2025 d . https://arxiv.org/abs/2505.15471 Cola: Collaborative low-rank adaptation . Preprint, arXiv:2505.15471

    2025 arXiv

  48. [57]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  49. [58]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.