Pith. sign in

REVIEW 2 major objections 5 minor 52 references

Compressing the speech key-value cache inside a Speech LLM to text-level granularity matches or beats the full-resolution baseline while speeding up decoding.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-10 20:28 UTC pith:ZUOGOBDS

load-bearing objection Clean systems result: defer speech compression into the LLM KV cache, keep full-res queries, and you can hit text-level length with no accuracy loss and real speedup on this stack. the 2 major comments →

arxiv 2607.06827 v1 pith:ZUOGOBDS submitted 2026-07-07 eess.AS cs.SD

Compress the Cache, Not the Speech Embedding: KV Compression for Efficient Speech LLMs

Ke-Han Lu , Keqi Deng , Ruchao Fan , Rui Zhao , Jinyu Li This is my paper
classification eess.AS cs.SD
keywords Speech LLMKV cache compressionautomatic speech recognitionlearned poolingmodality adapterdecoding speedupspeech-text alignment
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Speech LLMs turn audio into sequences that are several times longer than the corresponding text, so autoregressive decoding becomes slow and memory-heavy. Most prior work shortens the speech sequence before it enters the language model, which can throw away fine detail that later layers cannot recover. This paper instead keeps the full-resolution speech embeddings and only merges the key and value cache starting at an intermediate transformer layer, using a learned pooling over short windows. On a model trained with 71K hours of ASR data, compressing speech keys and values down to roughly one token per 320 ms still matches or slightly improves the uncompressed baseline on recognition benchmarks, while cutting decoding latency by at least 1.49 imes (and more for longer audio). The practical claim is that the right place to remove temporal redundancy is inside the LLM, after early layers have already begun to aggregate local acoustics, rather than at the adapter.

Core claim

SpeechKV shows that a learned pooling applied only to speech keys and values from an intermediate LLM layer onward can reduce the speech cache to approximately text-level length (R=4) while preserving or slightly improving ASR and entity-recognition accuracy relative to the uncompressed baseline, and while delivering measured decoding speedups of 1.49–2× that grow with audio duration.

What carries the argument

SpeechKV: a lightweight learned gate that, from layer l0 onward, soft-averages every R consecutive speech key/value vectors into one while leaving queries and residual streams at full resolution, so the attention map becomes shorter without early irreversible downsampling.

Load-bearing premise

That by layer 5 of this particular 1.7B model the network has already finished most local acoustic aggregation, so uniform window pooling of keys and values discards only redundant information.

What would settle it

Train the same architecture with compression starting at layer 5 on a different backbone or domain; if entity-error rate or out-of-domain WER then falls well below the uncompressed baseline while adapter-level compression does not, the layer-5 redundancy claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper addresses the long speech-sequence bottleneck in Speech LLMs by deferring temporal compression from the adapter into the LLM. SpeechKV applies a lightweight learned pooling operator to speech keys and values from an intermediate layer l0 onward (default l0=5), while leaving queries and the residual stream at full resolution. On a Qwen3-1.7B backbone trained on 71K hours of ASR data, R=4 compression (text-level granularity) matches or slightly exceeds the uncompressed MLP baseline (relative gains of ~6.6% on in-house entity recognition and ~2.3% on OpenASR) and outperforms adapter-level baselines (Concat-MLP, Window Q-Former) and an in-LLM hidden-state compression alternative. The method yields 1.49–2× end-to-end decoding speedup with vLLM that scales with audio length. Supporting analyses include layer-wise local cosine similarity, l0 ablations, and attention-map diagnostics.

Significance. The efficiency–accuracy trade-off for long speech prompts is a genuine practical problem for Speech LLMs. Deferring compression into the LLM so that full-resolution queries can still attend is a clean and well-motivated design choice, and the head-to-head comparison against both adapter-level and alternative in-LLM compression is carefully constructed. The layer-wise similarity analysis and attention maps provide useful mechanistic evidence that early layers aggregate local acoustics and that KV pooling can act as a structural regularizer. The reported speedups are measured under a realistic serving stack (vLLM). Within the single-backbone, ASR-only setting the empirical case is solid and the method is simple enough to be adopted. The main limitation on significance is that the recommended operating point (l0=5, R=4) is tied to the aggregation dynamics of this particular model and training regime; the paper demonstrates a successful operating point more than a fully architecture-agnostic principle.

major comments (2)
  1. [Section V-B, Fig. 2, Table III] Section V-B, Fig. 2 and Table III: the central operating point (l0=5, R=4) is justified by the local-similarity plateau of the uncompressed Qwen3-1.7B baseline and by the ablation showing that l0=3 and l0=7 degrade in-house ER. That plateau is an empirical property of this backbone, encoder, and 71K-hour regime. The abstract and introduction frame SpeechKV as a general remedy that safely reaches text-level granularity; the manuscript should more explicitly scope the claim to the observed layer-wise dynamics and state how a practitioner would choose l0 on a different LLM (e.g., re-run the similarity diagnostic, or treat l0 as a hyperparameter). Without that framing, the strongest claim over-reaches the evidence.
  2. [Table II, Section V-A] Table II: the headline relative gains (6.6% ER, 2.3% OpenASR at R=4) are modest absolute deltas (e.g., OpenASR 6.12→5.98 WER) and appear to come from single training runs with no reported variance, seeds, or confidence intervals. For a claim of “on par with or even slightly better,” the paper should either report multi-seed statistics or temper the language to “matches the baseline within the observed variation, with a favorable trend on OOD entity recognition.” The ranking versus adapter-level methods is clearer and less sensitive to this issue.
minor comments (5)
  1. [Table II, Table I] Several dataset names in Table II and the text have stray spaces (e.g., “V oiceM.”, “V oxPopuli”, “V oxPopuli” in the data table). Clean these for production.
  2. [Section III-B, Eq. (4)] Equation (4): the gate g is defined on key vectors only; a one-sentence note on why values are not gated independently (or an ablation) would help readers who expect separate K/V importance.
  3. [Figure 3] Figure 3 attention maps are informative but the color scale and the exact meaning of the red dashed speech region could be stated more explicitly in the caption for readers unfamiliar with the visualization convention.
  4. [Section II-B] Related work discusses training-free KV eviction/quantization but the experiments do not include even a simple training-free baseline (e.g., uniform merge or attention-score eviction at inference). A short note on why those are out of scope, or a single comparison, would strengthen the positioning.
  5. [Abstract, Section VI] The conclusion lists extension beyond ASR as future work; given the title “Speech LLMs,” a brief caveat in the abstract or introduction that all quantitative claims are ASR-only would set expectations cleanly.

Circularity Check

0 steps flagged

No circularity: empirical compression method whose reported WER/EER gains and speedups are measured on held-out data after joint training, not forced by construction or self-citation.

full rationale

SpeechKV is a standard empirical systems paper. The compression operator (learned per-window gating of speech keys/values from layer l0 onward, Eq. 4-5), the next-token training objective (Eq. 3), and the evaluation metrics (EER/WER on OpenASR and in-house sets) are independent of one another. The choice l0=5 is motivated by a preliminary cosine-similarity plot on the uncompressed baseline (Fig. 2) and confirmed by ablation (Table III); that plot is an observation about the backbone, not a definition that makes the later accuracy numbers tautological. No free parameter is fitted to a subset of the reported metrics and then re-presented as a prediction. Citations are ordinary related-work references (SnapKV, MiniCache, Q-Former, etc.) and do not supply a uniqueness theorem or ansatz that forces the central claim. The paper therefore contains no self-definitional step, no fitted-input-called-prediction, and no load-bearing self-citation chain. Score 0 is the correct honest finding.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 1 invented entities

The central empirical claim rests on standard transformer mechanics, a publicly available LLM backbone, and a handful of architectural choices (starting layer, window size, gate form) that are either measured or ablated. No new physical entities or free constants are fitted to produce the headline numbers; the free parameters are ordinary hyper-parameters of the method.

free parameters (3)
  • compression starting layer l0 = 5
    Chosen as 5 after inspecting layer-wise local similarity of the uncompressed baseline; ablated but still a free architectural decision that affects the reported gains.
  • compression ratio R = 4 (primary)
    Discrete hyper-parameter (4 or 8) that sets the target speech granularity; R=4 is presented as the practical operating point.
  • LoRA rank = 32
    Fixed at 32 for all trainable LLM adapters; standard but still a free choice.
axioms (3)
  • domain assumption Speech encoder emits frames at a fixed 80 ms rate and a typical English text token spans ~300 ms, yielding an intrinsic ~4× length mismatch.
    Stated in the introduction and used to motivate R=4 as “text-level granularity.”
  • domain assumption Transformer layers of a decoder-only LLM produce increasingly redundant adjacent key/value representations for speech, so that pooling after early layers discards little unique information.
    Supported by the preliminary cosine-similarity analysis (Fig. 2) and used to justify deferring compression into the LLM.
  • domain assumption Next-token prediction loss on ASR transcripts is a sufficient training signal for the learned pooling gates and LoRA adapters.
    Standard supervised ASR objective; no additional losses are introduced.
invented entities (1)
  • SpeechKV learned pooling operator no independent evidence
    purpose: Softly merges every R consecutive speech keys and values via a linear gate while leaving text positions and all queries untouched.
    The operator itself is the technical contribution; it is fully specified by Eq. (4) and has no independent existence outside the method.

pith-pipeline@v1.1.0-grok45 · 18201 in / 2531 out tokens · 30068 ms · 2026-07-10T20:28:42.091420+00:00 · methodology

0 comments
read the original abstract

Speech large language models (Speech LLMs) typically encode speech into sequences far longer than text, creating a major efficiency bottleneck during autoregressive decoding. A common remedy is to compress the speech sequence at the adapter level to remove temporal redundancy before it enters the LLM; however, such early downsampling risks discarding fine-grained information that cannot be recovered. We propose SpeechKV, which applies a learned pooling to the KV cache of speech tokens inside the LLM. This design allows the LLM to fuse speech and text internally while directly accelerating decoding. Trained on 71K hours of speech data, SpeechKV compresses the speech to approximately text-level granularity yet maintains performance on par with or even slightly better than the uncompressed baseline, with relative gains of 6.6% on out-of-domain entity recognition and 2.3% on OpenASR, while delivering at least 1.49 times decoding speedup that scales with audio length.

Figures

Figures reproduced from arXiv: 2607.06827 by Jinyu Li, Ke-Han Lu, Keqi Deng, Ruchao Fan, Rui Zhao.

Figure 1
Figure 1. Figure 1: Overview of our approach. Left: the Speech LLM follows the encoder–adapter–LLM paradigm. We apply the KV cache compression in deeper layers of the LLM. Middle: from layer l0 onward, the learned pooling compresses each window of speech keys and values into a single representation, while text positions remain unchanged. Right: In both training and decoding stage, the full-resolution queries attend to a short… view at source ↗
Figure 2
Figure 2. Figure 2: Layer-wise local similarity (cosine similarity between [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Attention maps of the Baseline and SpeechKV ( [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: End-to-end decoding speedup at each decoding step on [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

52 extracted references · 52 canonical work pages · 15 internal anchors

  1. [1]

    Instruction Tuning with GPT-4

    B. Peng, C. Li, P. He, M. Galley, and J. Gao, “Instruction tuning with gpt-4,”arXiv preprint arXiv:2304.03277, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  2. [2]

    The Llama 3 Herd of Models

    A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Yang, A. Fan,et al., “The llama 3 herd of models,”arXiv preprint arXiv:2407.21783, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  3. [3]

    Qwen3 Technical Report

    A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv,et al., “Qwen3 technical report,”arXiv preprint arXiv:2505.09388, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  4. [4]

    Phi-4 Technical Report

    M. Abdin, J. Aneja, H. Behl, S. Bubeck, R. Eldan, S. Gunasekar, M. Harrison, R. J. Hewett, M. Javaheripi, P. Kauffmann,et al., “Phi-4 technical report,”arXiv preprint arXiv:2412.08905, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  5. [5]

    SALMONN: Towards generic hearing abilities for large language models,

    C. Tang, W. Yu, G. Sun, X. Chen, T. Tan, W. Li, L. Lu, Z. MA, and C. Zhang, “SALMONN: Towards generic hearing abilities for large language models,” inThe Twelfth International Conference on Learning Representations, 2024

    work page 2024

  6. [6]

    Qwen2.5 Technical Report

    A. Yanget al., “Qwen2 Technical Report,”arXiv preprint arXiv:2412.15115, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  7. [7]

    Alignformer: Modality matching can achieve better zero-shot instruction-following speech-llm,

    R. Fan, B. Ren, Y . Hu, R. Zhao, S. Liu, and J. Li, “Alignformer: Modality matching can achieve better zero-shot instruction-following speech-llm,” IEEE Journal of Selected Topics in Signal Processing, vol. 19, no. 7, pp. 1329–1337, 2025

    work page 2025

  8. [8]

    On The Landscape of Spoken Language Models: A Comprehensive Survey

    S. Arora, K.-W. Chang, C.-M. Chien, Y . Peng, H. Wu, Y . Adi, E. Dupoux, H.-Y . Lee, K. Livescu, and S. Watanabe, “On the landscape of spoken language models: A comprehensive survey,”arXiv preprint arXiv:2504.08528, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  9. [9]

    Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs

    A. Abouelenin, A. Ashfaq, A. Atkinson, H. Awadalla, N. Bach, J. Bao, A. Benhaim, M. Cai, V . Chaudhary, C. Chen,et al., “Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture-of-loras,”arXiv preprint arXiv:2503.01743, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  10. [10]

    DeSTA: Enhancing speech language models through descriptive speech-text alignment,

    K.-H. Lu, Z. Chen, S.-W. Fu, H. Huang, B. Ginsburg, Y .-C. F. Wang, and H.-Y . Lee, “DeSTA: Enhancing speech language models through descriptive speech-text alignment,” inInterspeech 2024, pp. 4159–4163, 2024

    work page 2024

  11. [11]

    Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Data,

    K.-H. Luet al., “Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Data,” inIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5, 2025

    work page 2025

  12. [12]

    Desta2.5- audio: Toward general-purpose large audio language model with self- generated cross-modal alignment,

    K.-H. Lu, Z. Chen, S.-W. Fu, C.-H. H. Yang, S.-F. Huang, C.-K. Yang, C.-E. Yu, C.-W. Chen, W.-C. Chen, C.-y. Huang, Y .-C. Lin, Y .-X. Lin, C.-A. Fu, C.-Y . Kuan, W. Ren, X. Chen, W.-P. Huang, E.-P. Hu, T.-Q. Lin, Y .-K. Wu, K.-P. Huang, H.-Y . Huang, H.-C. Chou, K.-W. Chang, C.-H. Chiang, B. Ginsburg, Y .-C. F. Wang, and H.-y. Lee, “Desta2.5- audio: Towa...

    work page 2026

  13. [13]

    Prompting large language models with speech recognition abilities,

    Y . Fathullah, C. Wu, E. Lakomkin, J. Jia, Y . Shangguan, K. Li, J. Guo, W. Xiong, J. Mahadeokar, O. Kalinli, C. Fuegen, and M. Seltzer, “Prompting large language models with speech recognition abilities,” inICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 13351–13355, 2024

    work page 2024

  14. [14]

    WavLLM: Towards Robust and Adaptive Speech Large Language Model

    S. Huet al., “Wavllm: Towards robust and adaptive speech large language model,”arXiv preprint arXiv:2404.00656, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  15. [15]

    An Embarrassingly Simple Approach for LLM with Strong ASR Capacity

    Z. Ma, G. Yang, Y . Yang, Z. Gao, J. Wang, Z. Du, F. Yu, Q. Chen, S. Zheng, S. Zhang,et al., “An embarrassingly simple approach for llm with strong asr capacity,”arXiv preprint arXiv:2402.08846, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  16. [16]

    Wav2Prompt: End-to-end speech prompt learning and task-based fine-tuning for text-based LLMs,

    K. Deng, G. Sun, and P. Woodland, “Wav2Prompt: End-to-end speech prompt learning and task-based fine-tuning for text-based LLMs,” in Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)(L. Chiruzzo, A. Ritter, and L. Wang, eds.), (Alb...

    work page 2025

  17. [17]

    Connecting speech encoder and large language model for asr,

    W. Yu, C. Tang, G. Sun, X. Chen, T. Tan, W. Li, L. Lu, Z. Ma, and C. Zhang, “Connecting speech encoder and large language model for asr,” inICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 12637–12641, 2024

    work page 2024

  18. [18]

    Speech-xl: Towards long-form speech understanding in large speech language models,

    H. Sun, C. Lyu, S. Zhao, X. Ni, X. Kong, L. Wang, W. Luo, and Y . Qin, “Speech-xl: Towards long-form speech understanding in large speech language models,”arXiv preprint arXiv:2602.05373, 2026

    work page arXiv 2026

  19. [19]

    Speech llms are contextual reasoning transcribers,

    K. Deng, R. Fan, B. Ren, Y . Wang, and J. Li, “Speech llms are contextual reasoning transcribers,”arXiv preprint arXiv:2604.00610, 2026

    work page arXiv 2026

  20. [20]

    On decoder-only architecture for speech-to-text and large language model integration,

    J. Wu, Y . Gaur, Z. Chen, L. Zhou, Y . Zhu, T. Wang, J. Li, S. Liu, B. Ren, L. Liu, and Y . Wu, “On decoder-only architecture for speech-to-text and large language model integration,” in2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pp. 1–8, 2023

    work page 2023

  21. [21]

    Speechmap- per: Speech-to-text embedding projector for llms,

    B. Mohapatra, M. Z. Boito, and I. Calapodescu, “Speechmap- per: Speech-to-text embedding projector for llms,”arXiv preprint arXiv:2601.20417, 2026

    work page arXiv 2026

  22. [22]

    Snapkv: Llm knows what you are looking for before generation,

    Y . Li, Y . Huang, B. Yang, B. Venkitesh, A. Locatelli, H. Ye, T. Cai, P. Lewis, and D. Chen, “Snapkv: Llm knows what you are looking for before generation,” inAdvances in Neural Information Processing Systems(A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, eds.), vol. 37, pp. 22947–22970, Curran Associates, Inc., 2024

    work page 2024

  23. [23]

    Minicache: Kv cache compression in depth dimension for large language models,

    A. Liu, J. Liu, Z. Pan, Y . He, G. Haffari, and B. Zhuang, “Minicache: Kv cache compression in depth dimension for large language models,” inProceedings of the 38th International Conference on Neural Infor- mation Processing Systems, NIPS ’24, (Red Hook, NY , USA), Curran Associates Inc., 2024

    work page 2024

  24. [24]

    Open asr leaderboard: Towards reproducible and transparent multilingual and long-form speech recognition evaluation,

    V . Srivastav, S. Zheng, E. Bezzam, E. L. Bihan, N. Koluguri, P. ˙Zelasko, S. Majumdar, A. Moumen, and S. Gandhi, “Open asr leaderboard: Towards reproducible and transparent multilingual and long-form speech recognition evaluation,” 2026

    work page 2026

  25. [25]

    Efficient memory management for large language model serving with pagedattention,

    W. Kwon, Z. Li, S. Zhuang, Y . Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica, “Efficient memory management for large language model serving with pagedattention,” inProceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023

    work page 2023

  26. [26]

    Connection- ist temporal classification: labelling unsegmented sequence data with recurrent neural networks,

    A. Graves, S. Fern ´andez, F. Gomez, and J. Schmidhuber, “Connection- ist temporal classification: labelling unsegmented sequence data with recurrent neural networks,” inProceedings of the 23rd International Conference on Machine Learning, ICML ’06, (New York, NY , USA), p. 369–376, Association for Computing Machinery, 2006

    work page 2006

  27. [27]

    Cjst: Ctc compressor based joint speech and text training for decoder-only asr,

    W. Zhou, J. Jia, L. Sari, J. Mahadeokar, and O. Kalinli, “Cjst: Ctc compressor based joint speech and text training for decoder-only asr,” inICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5, 2025

    work page 2025

  28. [28]

    How to Connect Speech Foundation Models and Large Language Models? What Matters and What Does Not

    F. Verdini, P. Melucci, S. Perna, F. Cariaggi, M. Gaido, S. Papi, S. Mazurek, M. Kasztelnik, L. Bentivogli, S. Brati `eres,et al., “How to connect speech foundation models and large language models? what matters and what does not,”arXiv preprint arXiv:2409.17044, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  29. [29]

    Cif: Continuous integrate-and-fire for end-to- end speech recognition,

    L. Dong and B. Xu, “Cif: Continuous integrate-and-fire for end-to- end speech recognition,” inICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6079–6083, IEEE, 2020

    work page 2020

  30. [30]

    BLIP-2: Bootstrapping language- image pre-training with frozen image encoders and large language mod- els,

    J. Li, D. Li, S. Savarese, and S. Hoi, “BLIP-2: Bootstrapping language- image pre-training with frozen image encoders and large language mod- els,” inProceedings of the 40th International Conference on Machine Learning, vol. 202, pp. 19730–19742, PMLR, 2023

    work page 2023

  31. [31]

    Distilling an End-to-End Voice Assistant Without Instruction Training Data

    W. Held, E. Li, M. Ryan, W. Shi, Y . Zhang, and D. Yang, “Distilling an end-to-end voice assistant without instruction training data,”arXiv preprint arXiv:2410.02678, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  32. [32]

    A survey on large language model acceleration based on KV cache management,

    H. LI, Y . Li, A. Tian, T. Tang, Z. Xu, X. Chen, N. HU, W. Dong, L. Qing, and L. Chen, “A survey on large language model acceleration based on KV cache management,”Transactions on Machine Learning Research, 2025

    work page 2025

  33. [33]

    H2o: Heavy- hitter oracle for efficient generative inference of large language models,

    Z. Zhang, Y . Sheng, T. Zhou, T. Chen, L. Zheng, R. Cai, Z. Song, Y . Tian, C. R ´e, C. Barrett, Z. A. Wang, and B. Chen, “H2o: Heavy- hitter oracle for efficient generative inference of large language models,” inAdvances in Neural Information Processing Systems(A. Oh, T. Nau- mann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, eds.), vol. 36, pp. 346...

    work page 2023

  34. [34]

    Efficient streaming language models with attention sinks,

    G. Xiao, Y . Tian, B. Chen, S. Han, and M. Lewis, “Efficient streaming language models with attention sinks,” inInternational Conference on Learning Representations(B. Kim, Y . Yue, S. Chaudhuri, K. Fragki- adaki, M. Khan, and Y . Sun, eds.), vol. 2024, pp. 21875–21895, 2024

    work page 2024

  35. [35]

    Kvquant: Towards 10 million context length llm inference with kv cache quantization,

    C. Hooper, S. Kim, H. Mohammadzadeh, M. W. Mahoney, Y . S. Shao, K. Keutzer, and A. Gholami, “Kvquant: Towards 10 million context length llm inference with kv cache quantization,” inAdvances in Neural Information Processing Systems(A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, eds.), vol. 37, pp. 1270– 1303, Curran Ass...

    work page 2024

  36. [36]

    Dynamic memory compression: Retrofitting LLMs for accelerated inference,

    P. Nawrot, A. Ła ´ncucki, M. Chochowski, D. Tarjan, and E. Ponti, “Dynamic memory compression: Retrofitting LLMs for accelerated inference,” inForty-first International Conference on Machine Learning, 2024

    work page 2024

  37. [37]

    Model Tells You Where to Merge: Adaptive KV Cache Merging for LLMs on Long-Context Tasks

    Z. Wang, B. Jin, Z. Yu, and M. Zhang, “Model tells you where to merge: Adaptive kv cache merging for llms on long-context tasks,”arXiv preprint arXiv:2407.08454, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  38. [38]

    Adapting language models to compress contexts,

    A. Chevalier, A. Wettig, A. Ajith, and D. Chen, “Adapting language models to compress contexts,” inProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing(H. Bouamor, J. Pino, and K. Bali, eds.), (Singapore), pp. 3829–3846, Association for Computational Linguistics, Dec. 2023

    work page 2023

  39. [39]

    MLS: A Large-Scale Multilingual Dataset for Speech Research

    V . Pratap, Q. Xu, A. Sriram, G. Synnaeve, and R. Collobert, “Mls: A large-scale multilingual dataset for speech research,”ArXiv, vol. abs/2012.03411, 2020

    work page internal anchor Pith review Pith/arXiv arXiv 2012

  40. [40]

    GigaSpeech: An Evolving, Multi-Domain ASR Corpus with 10,000 Hours of Transcribed Audio,

    G. Chen, S. Chai, G.-B. Wang, J. Du, W.-Q. Zhang, C. Weng, D. Su, D. Povey, J. Trmal, J. Zhang, M. Jin, S. Khudanpur, S. Watanabe, S. Zhao, W. Zou, X. Li, X. Yao, Y . Wang, Z. You, and Z. Yan, “GigaSpeech: An Evolving, Multi-Domain ASR Corpus with 10,000 Hours of Transcribed Audio,” inInterspeech 2021, pp. 3670–3674, 2021

    work page 2021

  41. [41]

    SPGISpeech: 5,000 hours of transcribed financial audio for fully formatted end-to-end speech recognition

    P. K. O’Neill, V . Lavrukhin, S. Majumdar, V . Noroozi, Y . Zhang, O. Kuchaiev, J. Balam, Y . Dovzhenko, K. Freyberg, M. D. Shulman, B. Ginsburg, S. Watanabe, and G. Kucsko, “SPGISpeech: 5,000 hours of transcribed financial audio for fully formatted end-to-end speech recognition,”arXiv e-prints, p. arXiv:2104.02014, Apr. 2021

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  42. [42]

    Librispeech: an asr corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: an asr corpus based on public domain audio books,” in2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 5206–5210, IEEE, 2015

    work page 2015

  43. [43]

    Libriheavy: a 50,000 hours asr corpus with punctuation casing and context,

    W. Kang, X. Yang, Z. Yao, F. Kuang, Y . Yang, L. Guo, L. Lin, and D. Povey, “Libriheavy: a 50,000 hours asr corpus with punctuation casing and context,” 2023

    work page 2023

  44. [44]

    Common voice: A massively-multilingual speech corpus,

    R. Ardila, M. Branson, K. Davis, M. Henretty, M. Kohler, J. Meyer, R. Morais, L. Saunders, F. M. Tyers, and G. Weber, “Common voice: A massively-multilingual speech corpus,” inProceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pp. 4211–4215, 2020

    work page 2020

  45. [45]

    V oxPopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation,

    C. Wang, M. Riviere, A. Lee, A. Wu, C. Talnikar, D. Haziza, M. Williamson, J. Pino, and E. Dupoux, “V oxPopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation,” inProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Interna- tional Joint Conf...

    work page 2021

  46. [46]

    Ted-lium 3: Twice as much data and corpus repartition for experiments on speaker adaptation,

    F. Hernandez, V . Nguyen, S. Ghannay, N. Tomashenko, and Y . Esteve, “Ted-lium 3: Twice as much data and corpus repartition for experiments on speaker adaptation,” inSpeech and Computer: 20th International Conference, SPECOM 2018, Leipzig, Germany, September 18–22, 2018, Proceedings 20, pp. 198–208, Springer, 2018

    work page 2018

  47. [47]

    ”earnings- 22: A practical benchmark for accents in the wild

    M. D. Rio, P. Ha, Q. McNamara, C. Miller, and S. Chandra, “”earnings- 22: A practical benchmark for accents in the wild”,” 2022

    work page 2022

  48. [48]

    The ami meeting corpus: A pre-announcement,

    J. Carletta, S. Ashby, S. Bourban, M. Flynn, M. Guillemot, T. Hain, J. Kadlec, V . Karaiskos, W. Kraaij, M. Kronenthal, G. Lathoud, M. Lin- coln, A. Lisowska, I. McCowan, W. Post, D. Reidsma, and P. Wellner, “The ami meeting corpus: A pre-announcement,” inMachine Learning for Multimodal Interaction(S. Renals and S. Bengio, eds.), (Berlin, Heidelberg), pp....

    work page 2006

  49. [49]

    FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech

    A. Conneau, M. Ma, S. Khanuja, Y . Zhang, V . Axelrod, S. Dalmia, J. Riesa, C. Rivera, and A. Bapna, “Fleurs: Few-shot learning evaluation of universal representations of speech,”arXiv preprint arXiv:2205.12446, 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  50. [50]

    Robust speech recognition via large-scale weak super- vision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak super- vision,” inInternational conference on machine learning, PMLR, 2023

    work page 2023

  51. [51]

    Conformer: Convolution- augmented Transformer for Speech Recognition,

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wu, and R. Pang, “Conformer: Convolution- augmented Transformer for Speech Recognition,” inInterspeech 2020, pp. 5036–5040, 2020

    work page 2020

  52. [52]

    LoRA: Low-Rank Adaptation of Large Language Models,

    E. J. Hu, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chen, et al., “LoRA: Low-Rank Adaptation of Large Language Models,” in International Conference on Learning Representations

    work page