Pith. sign in

REVIEW 5 major objections 5 minor 42 references

Context-Driven Dynamic Pruning for Large Speech Foundation Models

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that replacing utterance-level pruning with per-layer, frame-level pruning — conditioned on speaker embeddings, acoustic events, and language vectors — cuts 56.7 GFLOPs from the OWSM-v3.1 encoder while improving…

desk verdict A solid incremental paper on context-conditioned per-layer pruning for speech foundation models, but the headline GFLOPs claim only counts the encoder and omits context-extractor overhead. read the letter →

arxiv 2505.18860 v1 pith:2UDL4QFH submitted 2025-05-24 eess.AS

Masao Someki , Shikhar Bharadwaj , Atharva Anand Joshi , Chyi-Jiunn Lin , Jinchuan Tian , Jee-weon Jung , Markus Müller , Nathan Susanj
show 2 more authors
Jing Liu Shinji Watanabe
This is my paper · ORCID
classification eess.AS
keywords dynamicpruninglocalgatepredictortemporalspeechfoundationmodelsspeakerembeddingstranslationOWSME-Branchformer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Speech foundation models like OWSM-v3.1 are accurate but expensive to run. This paper tries to show that the cost can be reduced dynamically during inference by deciding, at each layer and each frame, which modules to compute, using external context about the speaker, the acoustic scene, and the language. The proposed local gate predictor (localGP) makes these decisions per layer rather than applying one global mask, and it prunes frames rather than whole utterances. On Europarl-ST (German, French, Italian), conditioning encoder pruning on ECAPA speaker embeddings cuts 56.7 GFLOPs from the encoder and improves speech-translation BLEU by a relative 25.7% compared with the fully fine-tuned OWSM-v3.1, while ASR word error rates stay comparable. If this holds, dynamic pruning becomes a way to get both faster and better speech translation from a fixed foundation model.

What carries the argument

The central object is the Local Gate Predictor (localGP), a small cross-attention network inserted at each layer. It uses the layer's input hidden states as queries and the external context vectors as key-value pairs; a linear classifier on the attended output produces the probability that each frame's module should be computed, and a Straight-through Gumbel-softmax estimator binarizes this probability so the mask stays binary and differentiable during training. localGP enables temporal pruning — skipping individual frames within a module — as opposed to utterance-wise pruning that blanks a module for the whole utterance. The cgMLP module is always computed for all frames to avoid infeasible convolution on too few selected frames. This machinery is what lets the pruning decision vary by layer, by frame, and by token history rather than being a single global mask.

What would settle it

Measure total inference cost including ECAPA/BEATs extraction and localGP overhead, or shuffle speaker embeddings across test utterances: if the full-system cost is not below the unpruned encoder's 568.5 GFLOPs, or if shuffled speaker identities leave the BLEU gain unchanged, the central claim that speaker context drives the pruning benefit would be refuted.

Watch

Extended reading notes

Core claim

The central claim is that per-layer, frame-level pruning conditioned on rich external context outperforms both the unpruned model and the previous utterance-level pruning approach. Concretely, localGP generates a separate binary pruning mask for each module in each layer by attending from the layer's current hidden state to a small set of context vectors — an ECAPA speaker embedding, a BEATs acoustic-event embedding, or a URIEL language vector. Because the decision is made per frame, the encoder can skip computation on silence while retaining speech frames, which the paper shows visually as a learned voice-activity-detection pattern concentrated in deeper layers. The headline numbers come from the speaker-embedding condition: 56.7 GFLOPs removed from the 568.5 GFLOP encoder and a 25.7% relative BLEU gain on speech translation, with WER comparable to full fine-tuning. The paper also reports a token-level decoder effect: source-attention modules are used significantly more for tokens that begin a new word, suggesting the gate predictor adapts to linguistic structure.

Load-bearing premise

The efficiency gain rests on the assumption that running the pretrained speaker or acoustic-event extractors plus the per-layer gate predictor costs less than the encoder computation they skip, because the paper's GFLOP savings are measured on the encoder alone and exclude the context extraction overhead.

Editorial extensions

If this is right

  • Encoder pruning with localGP plus speaker embeddings reduces encoder GFLOPs from 568.5 to 511.8 and improves average ST BLEU by a relative 25.7% over full fine-tuning, while ASR WER stays within 0.2 points absolute on German, French, and Italian.
  • Temporal pruning with localGP gives an average relative BLEU improvement of 39.6% over utterance-wise pruning when both use the same frontend context, showing that frame-level decisions matter more than module-level decisions.
  • Adding both speaker and acoustic-event contexts together does not improve over either alone, indicating that a single well-chosen external context is sufficient for the encoder pruning signal.
  • The first encoder layer attends to almost all frames while deeper layers prune silence, so the learned pruning pattern behaves like a voice activity detector, which could be exploited in streaming or low-latency settings.
  • Decoder-side temporal pruning shows source-attention is used significantly more for word-initial tokens (those with a leading space), implying that decoding efficiency can be tailored to token type.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the authors do not spell out: if speaker identity is the operative signal, then pruning decisions could be computed once per speaker segment rather than per frame, amortizing the context-extraction cost further.
  • Because the reported GFLOP savings count only the encoder and exclude ECAPA and BEATs extraction and the gate predictor itself, the net-efficiency claim would be strengthened or weakened by an end-to-end measurement; that measurement is a natural next experiment rather than a result of this paper.
  • The token-initial source-attention effect suggests a testable extension: hard-skip source-attention for non-word-initial tokens during decoding and measure whether BLEU and speed both improve, which the paper does not attempt.
  • The VAD-like layer pattern could be transferred to a lightweight frontend that proposes frames, letting the backbone skip the per-layer gate predictor entirely; this is speculation, not a claim of the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes localGP, a per-layer dynamic pruning mechanism for the speech foundation model OWSM-v3.1. Instead of the utterance-level pruning mask used by the prior globalGP, localGP derives a pruning mask separately for each encoder/decoder layer from a cross-attention over external context (speaker embeddings from ECAPA, acoustic-event embeddings from BEATs, and language vectors) plus the layer's own input. On Europarl-ST German, French, and Italian, the authors fine-tune with a 30% target sparsity and report that encoder temporal pruning with speaker context reduces encoder GFLOPs by 56.7 relative to full fine-tuning while improving average ST BLEU by a relative 25.7%, with ASR WER roughly comparable. The paper also analyzes the learned pruning masks, observing VAD-like frame selection in early encoder layers and token-type-dependent source-attention usage in the decoder.

Significance. If substantiated, the central result is useful: it shows that a lightweight per-layer gate can exploit speaker identity to select informative frames better than utterance-level pruning, and it provides evidence that decoder source-attention is recruited differentially at word boundaries. The paper is readable and mostly transparent about limitations: it explicitly labels the GFLOPs as encoder-only, reports wall-clock overhead relative to globalGP, and notes that combining multiple acoustic contexts is not worth the extra compute. The ablations over context types and the statistical test on decoder token classes are useful. However, the headline efficiency claim is currently not supported by end-to-end computation accounting, and the performance differences are reported without any variance estimate, so the significance as stated is conditional.

major comments (5)
  1. [§4.2, §4.3, Table 1] The 56.7 GFLOPs saving is encoder-only and excludes the ECAPA-TDNN, BEATs, and gate-predictor computation that localGP introduces; the abstract and the contribution bullet omit the 'encoder' qualifier. The paper's own wall-clock line (0.124s for row 6 vs. 0.111s for row 2) shows that the auxiliary machinery adds measurable overhead, so the net computational benefit over globalGP, and even over full fine-tuning, is not established. Please report end-to-end GFLOPs including all context extractors and gate predictors, or full-model latency, for the baseline, globalGP, and localGP rows, and adjust the abstract and conclusion claims accordingly.
  2. [§4.2, Table 1] All WER, BLEU, and GFLOPs numbers come from a single run with no confidence intervals or significance tests; Europarl-ST has only about 20 hours per language, so differences such as row 6 vs. row 7 (13.5 vs. 12.8 BLEU) may be within noise. The central 25.7% relative BLEU improvement and the 56.7 GFLOPs saving need error bars, either from multiple seeds or bootstrap resampling over test utterances, to be load-bearing.
  3. [§4.3] The sentence 'simply applying our proposed method effectively reduces 26.9 GFLOPs ... while achieving a 25.7% relative improvement in BLEU' pairs the GFLOPs reduction of row 5 (from 568.5 to 541.6) with the BLEU improvement of row 6 (from 10.7 to 13.5). Row 5's average BLEU is 12.0, a 12.1% relative gain, not 25.7%; row 6's BLEU gain is about 26.2%, not paired with the 26.9 GFLOPs figure. Please correct this so each claim is tied to the correct row.
  4. [§2.2, Algorithm 1] The inference threshold for converting gate probabilities to binary pruning masks is described only as 'a threshold' without its value or selection rule. Since the measured sparsity, GFLOPs, and WER/BLEU all depend on this threshold, please specify it explicitly or state that it is derived from the target sparsity during training, so that the results are reproducible.
  5. [§4.2] Please clarify whether the row 1 'full fine-tuning (baseline)' was fine-tuned on the same Europarl-ST training splits and with the same ASR+ST objective as the pruned rows. The sentence 'we fine-tuned all models with a pruning objective' is ambiguous, and the fairness of the 25.7% relative BLEU comparison against this baseline depends on that clarification.
minor comments (5)
  1. [Abstract vs. §5] The abstract states a relative BLEU improvement of 25.7% while the conclusion states 25.6%; please reconcile the two numbers.
  2. [§4.4] The text says 'when combining these two feature types in row 9', but the combination of speaker and event features is row 8 in Table 1; row 9 is the decoder with lang2vec. Please correct the row reference.
  3. [§4.2] Please report the parameter counts and inference FLOPs of the ECAPA-TDNN and BEATs context extractors used, so readers can quantify the overhead that is currently excluded from the GFLOPs columns.
  4. [References] Several references are incomplete or inconsistently formatted, for example [18], [19], and [24]; please unify them with the venue's style.
  5. [§4.3] The paper states that decoder-side temporal pruning did not yield inference speed gains because of batch-wise beam search; this qualifier should also appear wherever the method is described as generally reducing inference cost.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: GFLOPs and BLEU gains are measured on held-out Europarl-ST against an external fully fine-tuned OWSM baseline; self-citations to the authors' globalGP work are not load-bearing.

full rationale

The paper's central claims—a 56.7 GFLOPs encoder reduction and a relative 25.7% BLEU improvement at comparable WER—are empirical measurements, not reductions by construction. Pruning masks are produced by the learned Local Gate Predictor (Algorithm 1), trained on Europarl-ST and evaluated on its held-out test set (Sections 4.1-4.3), so no fitted parameter is renamed as a prediction. The Table 1 GFLOPs figures are measured under the learned masks (row 6: 511.8 vs. row 1: 568.5); they are not forced by the 30% target-sparsity hyperparameter, since uniform 70% of the baseline would be about 398 GFLOPs while cgMLP and padding effects yield 511.8. The main comparison is against fully fine-tuned OWSM-v3.1, an external backbone, with external pretrained context extractors (ECAPA, BEATs, lang2vec). Self-citations to the authors' globalGP work appear for the 30% sparsity choice, the SGSE estimator (traced to Jang et al.), and a corroborating decoder observation, but none is load-bearing: globalGP rows are re-measured, and the sparsity choice does not determine the reported improvements. The paper also discloses its own accounting limitation—encoder-only GFLOPs excluding context extractors (Section 4.2)—and notes row 6 wall-clock 0.124s vs. 0.111s for globalGP (Section 4.3). An incomplete computational accounting is a validity concern, not circularity: no equation is defined in terms of the claimed outcome and no result rests solely on prior self-cited work.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical entities. Its free parameters are standard hyperparameters and an unreported threshold; the key unstated premise is that pretrained speaker and acoustic event embeddings reliably indicate which frames to prune.

free parameters (4)
  • target_sparsity = 0.30
    Chosen by hand following [15]; controls how many modules or frames are pruned at training and inference.
  • inference_threshold = unspecified
    Threshold applied to softmax probability to produce binary masks; its value is not reported in the paper, so the exact operating point of pruning is unspecified.
  • context_count_DC = 2 for spk+event
    Number of context sources used; set to 2 when both speaker and event embeddings are used, otherwise 1.
  • BEATs_layer = layer 2
    Acoustic event features are taken from BEATs' second layer, chosen to keep the context extraction module small; the choice is ad hoc.
assumptions (3)
  • domain assumption Pretrained context extractors (ECAPA, BEATs, URIEL) provide informative, fixed features for pruning decisions.
    The whole method rests on the assumption that these embeddings contain signal about which frames and modules are redundant. No ablation without context is provided for the encoder, so this premise is not separately validated.
  • standard math The straight-through Gumbel-softmax estimator yields stable training with binary masks.
    Inherited from [15] and [32]; the paper relies on this for differentiability of the mask sampling.
  • domain assumption Fine-tuning OWSM-v3.1 with a pruning objective does not substantially degrade the backbone's learned representations.
    The fine-tuning is performed on a small (20h per language) dataset; the paper does not check whether the backbone weights drift or collapse.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Context-Driven Dynamic Pruning for Large Speech Foundation Models." pith.science (2026). https://pith.science/paper/2UDL4QFH

@misc{pith2026250518860,
  author       = {Pith},
  title        = {Pith review of: Context-Driven Dynamic Pruning for Large Speech Foundation Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2UDL4QFH}},
  note         = {Machine review of arXiv:2505.18860}
}
read the original abstract

Speech foundation models achieve strong generalization across languages and acoustic conditions, but require significant computational resources for inference. In the context of speech foundation models, pruning techniques have been studied that dynamically optimize model structures based on the target audio leveraging external context. In this work, we extend this line of research and propose context-driven dynamic pruning, a technique that optimizes the model computation depending on the context between different input frames and additional context during inference. We employ the Open Whisper-style Speech Model (OWSM) and incorporate speaker embeddings, acoustic event embeddings, and language information as additional context. By incorporating the speaker embedding, our method achieves a reduction of 56.7 GFLOPs while improving BLEU scores by a relative 25.7% compared to the fully fine-tuned OWSM model.

Figures

Figures reproduced from arXiv: 2505.18860 by the authors.

Figure 1
Figure 1. Log-Mel spectrogram (top) and temporal pruning mask for self-attention modules (bottom). The y-axis of the lower plot represents the layers, with the initial layer at the bot￾tom. The x-axis of both plots represents the time scale. The or￾ange regions in the lower plot indicate activated self-attention modules, while the white regions represent pruned modules [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 37 canonical work pages

  1. [1]

    These speech foundation models demon- strate strong generalization ability and robustness across lan- guages [3], speakers, and acoustic conditions [1]

    Introduction Training large neural networks on large-scale speech datasets has achieved significant success in various speech-related tasks [1–4]. These speech foundation models demon- strate strong generalization ability and robustness across lan- guages [3], speakers, and acoustic conditions [1]. While these large models achieve high performance, the us...

    work page

  2. [2]

    Related Works Pruning techniques for speech processing models have been extensively studied [11, 14, 26, 27]

    Preliminaries and Related Works 2.1. Related Works Pruning techniques for speech processing models have been extensively studied [11, 14, 26, 27]. However, in these ap- proaches, the pruning mask is determined either during or af- ter training and remains fixed during inference. As a result, the sparse pattern isstaticregardless of the input data during i...

    arXiv 2025

  3. [3]

    Local Gate Predictor In localGP, the pruning maskzis computed separately for each layer

    Proposed Method 3.1. Local Gate Predictor In localGP, the pruning maskzis computed separately for each layer. Consequently, Equation 1 is modified as follows: zi enc =t(g(x i−1 enc , ˆC)), z i dec =t(g(x i−1 dec , ˆC)).(3) In globalGP, the contextCrepresents discrete identifiers such as language ID or task ID, often implemented as special tokens. However,...

    work page

  4. [4]

    Experiments 4.1. Models. In this study, we employed version 3.1 of OWSM model [4], an open-source alternative to OpenAI’s Whisper [1]. We chose OWSM-v3.1 for its openness and reproducibility; unlike Whis- per, it is trained entirely on publicly available data. This allows us to ensure that our evaluation set was not included in pretrain- ing, enabling a f...

    work page

  5. [5]

    Conclusion In this study, we proposed localGP, a context-driven dynamic inference optimization method that integrates external context, including speaker embeddings, acoustic events, and linguistic information. Our experiments demonstrated that combining lo- calGP with temporal pruning and speaker embeddings as addi- tional context reduced computation by ...

    work page

  6. [6]

    Acknowledgement Experiments of this work used the Bridges2 system at PSC and Delta system at NCSA through allocations CIS210014 and IRI120008P from the Advanced Cyberinfrastructure Co- ordination Ecosystem: Services & Support (ACCESS) pro- gram, supported by National Science Foundation grants #2138259,#:2138286, #:2138307, #:2137603, and #:2138296

  7. [7]

    Colld: Contrastive layer-to-layer distillation for compressing multilingual pre-trained speech encoders,

    H.-J. Chang, N. Dong, R. Mavlyutov, S. Popuri, and Y .-A. Chung, “Colld: Contrastive layer-to-layer distillation for compressing multilingual pre-trained speech encoders,” inICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024

    work page 2024

  8. [8]

    These findings indicate that multiple audio context features does not necessarily lead to performance improvement, and it is sufficient to select a single model that with the best results. 4.5. Analysis on Decoder result We also focused on the decoder side to examine how pruning is performed for each token. As shown in Figure 2, we found that (a)Token:[Sp...

    work page 2025

Show all 42 references
  1. [9]

    Robust speech recognition via large-scale weak supervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” inInternational conference on machine learning. PMLR, 2023

    2023

  2. [10]

    Less is more: Accu- rate speech recognition & translation without web-scale data,

    K. C. Puvvada, P. ˙Zelasko, H. Huang, O. Hrinchuk, N. R. Koluguri, K. Dhawan, S. Majumdar, E. Rastorgueva, Z. Chen, V . Lavrukhin, J. Balam, and B. Ginsburg, “Less is more: Accu- rate speech recognition & translation without web-scale data,” in Interspeech 2024, 2024

    2024

  3. [11]

    Scal- ing speech technology to 1,000+ languages,

    V . Pratap, A. Tjandra, B. Shi, P. Tomasello, A. Babu, S. Kundu, A. Elkahky, Z. Ni, A. Vyas, M. Fazel-Zarandi, A. Baevski, Y . Adi, X. Zhang, W.-N. Hsu, A. Conneau, and M. Auli, “Scal- ing speech technology to 1,000+ languages,”arXiv preprint, vol. 2305.13516, 2023

    2023 arXiv

  4. [12]

    Owsm v3.1: Better and faster open whisper-style speech models based on e-branchformer,

    Y . Peng, J. Tian, W. Chen, S. Arora, B. Yan, Y . Sudo, M. Shakeel, K. Choi, J. Shi, X. Chang, J. weon Jung, and S. Watanabe, “Owsm v3.1: Better and faster open whisper-style speech models based on e-branchformer,” inInterspeech 2024, 2024

    2024

  5. [13]

    Distil-whisper: Robust knowledge distillation via large-scale pseudo labelling,

    S. Gandhi, P. von Platen, and A. M. Rush, “Distil-whisper: Robust knowledge distillation via large-scale pseudo labelling,” 2023

    2023

  6. [14]

    Distilhubert: Speech representation learning by layer-wise distillation of hidden-unit bert,

    H.-J. Chang, S.-w. Yang, and H.-y. Lee, “Distilhubert: Speech representation learning by layer-wise distillation of hidden-unit bert,” inICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022

    2022

  7. [15]

    4-bit quantization of lstm- based speech recognition models,

    A. Fasoli, C.-Y . Chen, M. Serrano, X. Sun, N. Wang, S. Venkataramani, G. Saon, X. Cui, B. Kingsbury, W. Zhang, Z. T ¨uske, and K. Gopalakrishnan, “4-bit quantization of lstm- based speech recognition models,” inInterspeech 2021, 2021

    2021

  8. [16]

    2-bit conformer quantization for automatic speech recog- nition,

    O. Rybakov, P. Meadowlark, S. Ding, D. Qiu, J. Li, D. Rim, and Y . He, “2-bit conformer quantization for automatic speech recog- nition,” inInterspeech 2023, 2023

    2023

  9. [17]

    Usm-lite: Quan- tization and sparsity aware fine-tuning for speech recognition with universal speech models,

    S. Ding, D. Qiu, D. Rim, Y . He, O. Rybakov, B. Li, R. Prab- havalkar, W. Wang, T. N. Sainath, Z. Hanet al., “Usm-lite: Quan- tization and sparsity aware fine-tuning for speech recognition with universal speech models,” inICASSP 2024-2024 IEEE Interna- tional Conference on Aco...

    2024

  10. [18]

    Learning N: M fine-grained struc- tured sparse neural networks from scratch,

    Z. Aojun, M. Yukun, Z. Junnan, L. Jianbo, Z. Zhijie, Y . Kun, S. Wenxiu, and L. Hongsheng, “Learning N: M fine-grained struc- tured sparse neural networks from scratch,”International Confer- ence on Learning Representations (ICLR), 2021

    2021

  11. [19]

    Learning sparse neu- ral networks throughl 0 regularization,

    C. Louizos, M. Welling, and D. P. Kingma, “Learning sparse neu- ral networks throughl 0 regularization,” inThe International Con- ference on Learning Representations (ICLR), 2018

    2018

  12. [20]

    Parp: Prune, adjust and re-prune for self-supervised speech recognition,

    C.-I. J. Lai, Y . Zhang, A. H. Liu, S. Chang, Y .-L. Liao, Y .-S. Chuang, K. Qian, S. Khurana, D. Cox, and J. Glass, “Parp: Prune, adjust and re-prune for self-supervised speech recognition,”Ad- vances in Neural Information Processing Systems, 2021

    2021

  13. [21]

    Structured pruning of self-supervised pre-trained models for speech recog- nition and understanding,

    Y . Peng, K. Kim, F. Wu, P. Sridhar, and S. Watanabe, “Structured pruning of self-supervised pre-trained models for speech recog- nition and understanding,” inICASSP 2023-2023 IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023

    2023

  14. [22]

    Context-aware dynamic pruning for speech foundation models,

    M. Someki, Y . Peng, S. Arora, M. M ¨uller, A. Mouchtaris, G. Strimel, J. Liu, and S. Watanabe, “Context-aware dynamic pruning for speech foundation models,” inThe Thirteenth Inter- national Conference on Learning Representations, 2025

    2025

  15. [23]

    Dphubert: Joint distillation and pruning of self-supervised speech models,

    Y . Peng, Y . Sudo, M. Shakeel, and S. Watanabe, “Dphubert: Joint distillation and pruning of self-supervised speech models,” in24th Annual Conference of the International Speech Communication Association, Interspeech 2023, Dublin, Ireland, August 20-24, 2023, 2023

    2023

  16. [24]

    Dynamic to- ken pruning in plain vision transformers for semantic segmenta- tion,

    Q. Tang, B. Zhang, J. Liu, F. Liu, and Y . Liu, “Dynamic to- ken pruning in plain vision transformers for semantic segmenta- tion,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023

    2023

  17. [25]

    Trainable dynamic subsampling for end-to-end speech recognition,

    S. Zhang, E. Loweimi, Y . Xu, P. Bell, and S. Renals, “Trainable dynamic subsampling for end-to-end speech recognition,” inIn- terspeech 2019, 2019

    2019

  18. [26]

    Adaptive computation time for recurrent neural net- works,

    A. Graves, “Adaptive computation time for recurrent neural net- works,”arXiv preprint, vol. 1603.08983, 2016

    2016 arXiv

  19. [27]

    Avoid overthinking in self-supervised models for speech recognition,

    D. Berrebbi, B. Yan, and S. Watanabe, “Avoid overthinking in self-supervised models for speech recognition,” inICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing, Proceedings, 2023

    2023

  20. [28]

    Efficient sequence transduction by jointly predicting tokens and durations,

    H. Xu, F. Jia, S. Majumdar, H. Huang, S. Watanabe, and B. Gins- burg, “Efficient sequence transduction by jointly predicting tokens and durations,” inProceedings of the 40th International Confer- ence on Machine Learning, 2023

    2023

  21. [29]

    I3d: Transformer architectures with input-dependent dynamic depth for speech recognition,

    Y . Peng, J. Lee, and S. Watanabe, “I3d: Transformer architectures with input-dependent dynamic depth for speech recognition,” in ICASSP 2023 - 2023 IEEE International Conference on Acous- tics, Speech and Signal Processing (ICASSP), 2023

    2023

  22. [30]

    Ecapa-tdnn: Emphasized channel attention, propagation and aggregation in tdnn based speaker verification,

    B. Desplanques, J. Thienpondt, and K. Demuynck, “Ecapa-tdnn: Emphasized channel attention, propagation and aggregation in tdnn based speaker verification,” inInterspeech 2020, 2020

    2020

  23. [31]

    Beats: audio pre-training with acoustic tok- enizers,

    S. Chen, Y . Wu, C. Wang, S. Liu, D. Tompkins, Z. Chen, W. Che, X. Yu, and F. Wei, “Beats: audio pre-training with acoustic tok- enizers,” inProceedings of the 40th International Conference on Machine Learning. JMLR.org, 2023

    2023

  24. [32]

    Uriel and lang2vec: Representing languages as typo- logical, geographical, and phylogenetic vectors,

    P. Littell, D. R. Mortensen, K. Lin, K. Kairis, C. Turner, and L. Levin, “Uriel and lang2vec: Representing languages as typo- logical, geographical, and phylogenetic vectors,” inProceedings of the 15th Conference of the European Chapter of the Association for Computational Lin...

    2017

  25. [33]

    Re- thinking pruning for accelerating deep inference at the edge,

    D. Gao, X. He, Z. Zhou, Y . Tong, K. Xu, and L. Thiele, “Re- thinking pruning for accelerating deep inference at the edge,” in Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020

    2020

  26. [34]

    Audio lottery: Speech recog- nition made ultra-lightweight, noise-robust, and transferable,

    S. Ding, T. Chen, and Z. Wang, “Audio lottery: Speech recog- nition made ultra-lightweight, noise-robust, and transferable,” in International Conference on Learning Representations, 2021

    2021

  27. [35]

    Compute cost amortized transformer for streaming asr,

    Y . Xie, J. J. Macoskey, M. Radfar, F.-J. Chang, B. King, A. Ras- trow, A. Mouchtaris, and G. Strimel, “Compute cost amortized transformer for streaming asr,” inInterspeech 2022, 2022

    2022

  28. [36]

    Amortized neural networks for low-latency speech recognition,

    J. Macoskey, G. P. Strimel, J. Su, and A. Rastrow, “Amortized neural networks for low-latency speech recognition,” inInter- speech 2021, 2021

    2021

  29. [37]

    A study of the recur- rent neural network encoder-decoder for large vocabulary speech recognition,

    L. Lu, X. Zhang, K. Cho, and S. Renals, “A study of the recur- rent neural network encoder-decoder for large vocabulary speech recognition,” inInterspeech 2015, 2015

    2015

  30. [38]

    Adaptive feature selection for end-to-end speech translation,

    B. Zhang, I. Titov, B. Haddow, and R. Sennrich, “Adaptive feature selection for end-to-end speech translation,” inFindings of the As- sociation for Computational Linguistics: EMNLP 2020, 2020

    2020

  31. [39]

    Categorical reparameterization with gumbel-softmax,

    E. Jang, S. Gu, and B. Poole, “Categorical reparameterization with gumbel-softmax,” inInternational Conference on Learning Representations, ICLR, 2017

    2017

  32. [40]

    MLP-based architecture with variable length input for automatic speech recognition,

    J. Sakuma, T. Komatsu, and R. Scheibler, “MLP-based architecture with variable length input for automatic speech recognition,” 2022. [Online]. Available: https://openreview.net/ forum?id=RA-zVvZLYIy

    2022

  33. [41]

    E-branchformer: Branchformer with enhanced merging for speech recognition,

    K. Kim, F. Wu, Y . Peng, J. Pan, P. Sridhar, K. J. Han, and S. Watanabe, “E-branchformer: Branchformer with enhanced merging for speech recognition,” in2022 IEEE Spoken Language Technology Workshop (SLT), 2023, pp. 84–91

    2023

  34. [42]

    Europarl-st: A multilingual corpus for speech translation of parliamentary de- bates,

    J. Iranzo-S ´anchez, J. A. Silvestre-Cerd `a, J. Jorge, N. Rosell ´o, A. Gim ´enez, A. Sanchis, J. Civera, and A. Juan, “Europarl-st: A multilingual corpus for speech translation of parliamentary de- bates,” inICASSP 2020 - 2020 IEEE International Conference on Acoustics, Spee...

    2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.