Pith. sign in

REVIEW 4 major objections 7 minor 39 references

VietASR: Achieving Industry-level Vietnamese ASR with 50-hour labeled data and Large-Scale Speech Pretraining

T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 68M-parameter VietASR model, pre-trained on 73,000 unlabeled hours and fine-tuned on 50 labeled hours, beats the 1.5B Whisper large-v3 on Vietnamese ASR benchmarks.

desk verdict Plausible 50-hour ASR recipe, but the headline comparison to Whisper hinges on an unverified no-overlap assumption for the YouTube-sourced test set. read the letter →

arxiv 2505.21527 v2 pith:NNVADBP5 submitted 2025-05-23 eess.AS cs.AIcs.CLcs.SD

Jianheng Zhuo , Yifan Yang , Yiwen Shao , Yong Xu , Dong Yu , Kai Yu , Xie Chen This is my paper · ORCID
classification eess.AScs.AIcs.CLcs.SD
keywords VietnameseASRself-supervisedlearninglow-resourcespeechrecognitionHuBERTZipformerASR-biasedcodebookRNN-TFbankfront-end
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

VietASR claims that a 68M-parameter Vietnamese ASR system can reach industry-level accuracy with only 50 hours of transcribed audio, provided it is pre-trained on roughly 73,000 hours of unlabeled YouTube speech. The central idea is to replace HuBERT's purely acoustic clustering targets with a supervised codebook: k-means labels extracted from the encoder of an ASR model trained on the limited labeled data. Iterating this pipeline—train an ASR on 50 hours, extract labels, pre-train a Zipformer encoder on unlabeled audio, fine-tune—improves word error rate at each round. On three public Vietnamese test sets, the third iteration averages 8.31% WER, below the 16.44% of Whisper large-v3 and below commercial systems, with a training budget of roughly 20 days on eight V100 GPUs. The claim matters because it suggests that low-resource languages can get practical ASR without either huge labeled corpora or huge models.

What carries the argument

The load-bearing mechanism is a four-stage iterative pipeline. Stage 1 trains a Zipformer ASR model on 50 hours of labeled audio with pruned RNN-T loss; Stage 2 extracts pre-training labels by k-means clustering (500 clusters) on that encoder's final-layer features; Stage 3 pre-trains a Zipformer encoder on 73,000 hours of unlabeled Fbank audio with a masked cross-entropy loss, masking 8% of frames with span 10; Stage 4 fine-tunes the encoder on the same labeled 50 hours. Each subsequent iteration recycles the Stage 4 encoder as the new label extractor, so the label quality improves without any additional labeled data. The shift from waveform to Fbank input and from the original HuBERT cosine-similarity loss to a simplified cross-entropy loss is what makes pre-training on 73,000 hours affordable on eight V100 GPUs.

What would settle it

Run an audio-fingerprint or near-duplicate search between the GigaSpeech2 Vietnamese test utterances and the 73,000-hour YouTube pretraining set. If a material fraction of test utterances are found in pretraining, the 7.68 GigaSpeech2 WER and the claim of beating Whisper large-v3 and commercial systems are unsupported; if the overlap is zero, the claim stands.

Watch

Extended reading notes

Core claim

The paper discovers that the pre-training target for masked-prediction self-supervised learning does not need to come from unsupervised clustering alone. Training a weak Zipformer ASR model on just 50 hours of labeled audio, clustering its encoder outputs into 500 k-means units, and using those units as HuBERT-style prediction targets yields better and faster-converging representations than k-means on MFCC or hidden states. Repeating the loop—fine-tuning the pre-trained encoder on the same 50 hours, re-clustering its features, and re-pre-training—gives monotonic WER improvements, ending at 7.68 on GigaSpeech2, 11.46 on Common Voice, and 10.96 on FLEURS. The paper states that the resulting 68M model outperforms Whisper large-v3 (1542M) and commercial services on these benchmarks, while also working in a streaming configuration with 640 ms chunks.

Load-bearing premise

The load-bearing assumption is that the GigaSpeech2 test set was not part of the 73,000-hour YouTube pretraining corpus; the paper reports no deduplication between them, and both come from YouTube, so if test audio leaked into pretraining the headline WER comparison is inflated.

Editorial extensions

If this is right

  • A 68M-parameter model, not a 1B+ model, can reach the accuracy of Whisper large-v3 and commercial APIs on Vietnamese, so deployment cost and latency drop.
  • The supervised codebook beats the standard HuBERT codebook at matched compute: average WER 8.68 vs 9.44 at iteration 2, and the gap already appears in iteration 1.
  • The same pre-trained encoder transfers to streaming ASR: with 640 ms chunks, VietASR's streaming model gets a 10.13 average WER, beating the HuBERT-codebook streaming baseline's 11.64.
  • Each pipeline iteration improves WER, so the method offers a predictable scaling path through repeated pre-training instead of requiring more labeled data.
  • Fine-tuning with 50 hours after VietASR pre-training outperforms training from scratch on the same 50 hours by a large margin (8.31 vs 20.54 average WER), showing that the unlabeled corpus, not the labeled data, drives the gain.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The supervised-codebook recipe should generalize to any language with tens of hours of labeled audio, since it does not require a strong ASR model—the paper shows even the first weak model yields better targets than spectrum clustering.
  • Because all gains come from unlabeled audio plus a 50-hour transcript, an obvious extension is to test whether iteration count can be traded against corpus size, such as whether 30,000 hours with more iterations matches 73,000 hours.
  • The GigaSpeech2 result is the strongest but also the riskiest, since both pretraining and test audio come from YouTube; if a future audit finds overlap, the conservative evidence would be the Common Voice and FLEURS WERs, which still beat commercial systems.
  • The same pipeline should be tried on a non-tonal or morphologically complex low-resource language to see whether the ASR-biased codebook retains its advantage across typologically different phoneme inventories.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes VietASR, a multi-iteration HuBERT-style speech pretraining pipeline for Vietnamese ASR built on a Zipformer encoder with an Fbank front-end. The key methodological novelty is replacing the unsupervised k-means codebook with a supervised codebook obtained by clustering the encoder features of an ASR model trained on 50 hours of labeled YouTube audio. Pretraining on 73,000 hours of unlabeled YouTube audio and fine-tuning on the same 50-hour labeled set yields a 68M-parameter RNN-T model that, according to Table 3, achieves an average WER of 8.31 on GigaSpeech2, Common Voice, and FLEURS, compared with 16.44 for Whisper large-v3 and 11.78 for the best commercial system. The paper also reports streaming ASR results and an ablation against a standard HuBERT codebook.

Significance. If the benchmark results are valid, the paper shows that ASR-biased self-supervised learning with a supervised codebook can reduce labeled-data requirements to 50 hours while retaining a lightweight, deployable encoder and moderate compute (about 20 days on 8 V100 GPUs). The detailed training recipe — mask strategy, loss function, optimizer, iteration schedule, and streaming setup — is a useful contribution for low-resource ASR practitioners. The comparison against strong open-source and commercial baselines, including Whisper large-v3, Google USM, Azure Speech, and GigaSpeech 2 models, is valuable. The main risk is a possible overlap between the YouTube-derived pretraining/fine-tuning data and the GigaSpeech2 test set, which, if unresolved, would undermine the headline claim.

major comments (4)
  1. [§4.1, Table 1; §4.2, Table 3] The central claim depends on the assumption that GigaSpeech 2 TEST (11 h, YouTube) is disjoint from the 73,000 h unlabeled YouTube pretraining set and the 50 h YouTube fine-tuning set. No deduplication, utterance-level overlap check, or audio fingerprinting is reported. Because iterations 2 and 3 each run 18 epochs over 573k steps (Table 2), any accidental inclusion of test utterances could be memorized and artificially lower WER. Since GigaSpeech 2 TEST is the largest test set and the reported average is word-count weighted, the headline 8.31 vs. 16.44 comparison is dominated by that column. The authors should perform and report an overlap audit (e.g., exact/near-duplicate audio matching between pretraining/fine-tuning and each test set), re-present results after removing any matched utterances, and report per-set results without relying solely on the weighted average.
  2. [§4.2, Table 3] The conclusion that VietASR "outperforms the Whisper-large-v3 (1.5B) and commercial models" is an overstatement on the present evidence. On Common Voice, VietASR (11.46) is second to Azure (10.21), and on FLEURS, Whisper large-v3 (8.59) beats VietASR (10.96). Only the GigaSpeech 2 column supports the superiority claim. Unless the overlap concern is resolved or the claim is explicitly narrowed to the YouTube domain, the general statement should be revised.
  3. [§3.2, §4.1] The supervised codebook is derived from an ASR model trained on the same 50 h labeled set used for fine-tuning, and subsequent iterations use the fine-tuned encoder from the previous iteration. Thus the SSL targets are partly defined by the downstream labeled data, and the "50-hour labeled data" framing conflates fine-tuning data with pretraining-target supervision. The evaluation on held-out public test sets is not circular in the WER sense, but the method's claim to leverage "merely 50-hour labeled data" needs an explicit acknowledgment that these 50 h are used twice, or an ablation where the codebook extractor is trained on an independent small labeled set.
  4. [Table 3 (also Tables 4–5)] No confidence intervals or significance tests are reported for the WER values, and several comparisons on Common Voice and FLEURS differ by only 1–2 WER absolute (e.g., VietASR 11.46 vs. Azure 11.88 on Common Voice). The authors should report bootstrap confidence intervals or multiple-seed runs so that readers can assess whether observed differences are meaningful beyond the dominant GigaSpeech 2 column.
minor comments (7)
  1. [§3.1.3, Eq. (1)] Equation (1) is malformed: "log exp(ztct )PC i=1 exp(zti)" should be written as -log( exp(z_{t,c_t}) / sum_i exp(z_{t,i}) ). Please fix the notation.
  2. [§3.1.3] The sentence "The loss function is given by" introduces a per-frame expression; the full loss should sum over all masked frames. Please clarify.
  3. [§4.1] Typos: "FunASR V AD" should be "FunASR VAD", and "1000s audio in a single batch" should be "1000 seconds of audio per batch".
  4. [§4.2] "constraint beam search" should be "constrained beam search".
  5. [Table 1] It would help to specify the exact test split selection (e.g., official test splits for Common Voice 17.0 and FLEURS) and to report utterance counts, since the weighted average depends on word counts that are not given.
  6. [§4.2] The statement "29.5% lower than the second-best model" should specify that this is a relative WER reduction; the absolute gap is 3.47 WER (8.31 vs. 11.78).
  7. [References] Reference [24] should be "Vaswani et al." rather than "Waswani et al.".

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the reported WERs come from held-out public test sets, and the pretraining target is an auxiliary cluster codebook rather than the downstream labels themselves.

full rationale

The VietASR derivation chain is: train a weak Zipformer ASR on a 50-hour labeled set; extract k-means cluster labels from that encoder's output; run HuBERT-style masked prediction (Eq. 1) on 73,000 hours of unlabeled YouTube audio with those labels; fine-tune on the same 50 hours; repeat. The supervised codebook is therefore derived from a model that has seen the exact fine-tuning data, so the pretraining objective is shaped by the downstream labeled set. That is a data-reuse design choice, not a circular reduction: the pretraining loss predicts cluster IDs, not transcriptions, and the paper's WER claims are evaluated on GigaSpeech 2 TEST, Common Voice 17.0, and FLEURS, which are distinct from the 50-hour fine-tuning set. No equation in the paper defines the test WER in terms of the pretraining targets by construction, and no fitted parameter is renamed as a prediction. The self-citations ([10] GigaSpeech 2, [23] k2SSL) reference a public benchmark and a related prior system by overlapping authors, but the central improvement claim does not rest on an unverified self-citation: the VietASR-vs-HuBERT codebook comparison in Table 4 is an empirical experiment, and the Whisper/commercial baselines are external. The dominant validity risk is the unaddressed potential overlap between the 73k-hour YouTube pretraining corpus and the GigaSpeech 2 TEST set, which would inflate the headline WER; that is a data-leakage concern, not a circularity of the derivation chain. Score 1 reflects only the minor, non-load-bearing self-citations and the shared-data design, not any by-construction equivalence.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the unlabeled corpus quality, on absence of test-set overlap, on the informativeness of k-means labels from a weak ASR model, and on transfer from masked prediction to RNN-T. These are domain assumptions rather than formal axioms; the paper provides no formal proof and only one language. The free parameters are standard hyperparameters, several of which are not ablated.

free parameters (6)
  • k-means cluster size = 500
    Used for all pre-training label extractions; the granularity of the codebook directly controls the masked-prediction target and the final WER, but no ablation is given.
  • Mask configuration = mask span 10, 8% of Fbank frames as mask starts
    Taken from HuBERT-style config (Section 3.1.2); mask coverage determines the difficulty of the self-supervised task.
  • Softmax temperature tau in Eq. (3)
    Temperature on the projection logits; the paper gives no value or sensitivity analysis, so it is an unstated free parameter.
  • Pre-training epochs per iteration = 9 for iteration 1, 18 for iterations 2 and 3
    The schedule is justified by lower first-iteration label quality, but the exact choice is manual and affects the comparison in Table 4.
  • BPE vocabulary size = 500
    Fine-tuning output units chosen for the RNN-T model; not varied in the paper.
  • Streaming latency schedule = chunk sizes 320/640/1280 ms, context sizes 1280/2560/5120 ms
    Randomly sampled during streaming training and fixed to 640/5120 ms at decoding; the schedule affects the streaming WER reported in Table 5.
assumptions (4)
  • domain assumption The 73,000-hour unlabeled YouTube corpus contains speech that is usable for ASR pre-training after FunASR VAD segmentation.
    Section 4.1 defines the pre-training set this way; no audio filtering, SNR analysis, or language verification is reported, and the whole pipeline relies on this corpus.
  • domain assumption The GigaSpeech2 test utterances are not contained in the pre-training corpus.
    Section 4.1 Table 1 and Section 4.2; both are YouTube audio. The paper does not report deduplication or overlap checks, yet the GigaSpeech2 advantage is the largest.
  • domain assumption K-means labels from a weak 50-hour ASR encoder are informative targets for masked prediction.
    Stage 2 of Section 3.2 assumes these clusters carry speech-text alignment knowledge that transfers; the comparison against the HuBERT codebook supports it, but only for one test configuration.
  • domain assumption Masked-prediction pre-training with an Fbank front-end transfers to the RNN-T fine-tuning objective.
    Section 3.1.3 and Section 4.2; the paper shows pre-training helps versus from-scratch, but does not isolate which part of the objective is responsible.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VietASR: Achieving Industry-level Vietnamese ASR with 50-hour labeled data and Large-Scale Speech Pretraining." pith.science (2026). https://pith.science/paper/NNVADBP5

@misc{pith2026250521527,
  author       = {Pith},
  title        = {Pith review of: VietASR: Achieving Industry-level Vietnamese ASR with 50-hour labeled data and Large-Scale Speech Pretraining},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NNVADBP5}},
  note         = {Machine review of arXiv:2505.21527}
}
read the original abstract

Automatic speech recognition (ASR) has made remarkable progress but heavily relies on large-scale labeled data, which is scarce for low-resource languages like Vietnamese. While existing systems such as Whisper, USM, and MMS achieve promising performance, their efficacy remains inadequate in terms of training costs, latency, and accessibility. To address these issues, we propose VietASR, a novel ASR training pipeline that leverages vast amounts of unlabeled data and a small set of labeled data. Through multi-iteration ASR-biased self-supervised learning on a large-scale unlabeled dataset, VietASR offers a cost-effective and practical solution for enhancing ASR performance. Experiments demonstrate that pre-training on 70,000-hour unlabeled data and fine-tuning on merely 50-hour labeled data yield a lightweight but powerful ASR model. It outperforms Whisper Large-v3 and commercial ASR systems on real-world data. Our code and models will be open-sourced to facilitate research in low-resource ASR.

Figures

Figures reproduced from arXiv: 2505.21527 by the authors.

Figure 1
Figure 1. An overview of the proposed VietASR. The training pipeline consists of four stages: (1) initial ASR training, (2) label extraction, (3) pre-training, and (4) fine-tuning. Components within the dashed box are reused in the next stage. In the first iteration, Stage 2 reuses the encoder trained in Stage 1 for label extraction; in subsequent iterations, Stage 2 reuses the encoder from Stage 4 of the previous iteration, … view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 32 canonical work pages

  1. [1]

    VietASR: Achieving Industry-level Vietnamese ASR with 50-hour labeled data and Large-Scale Speech Pretraining

    Introduction Advances in supervised learning [1–3] have led to significant improvements in the performance of automatic speech recog- nition (ASR). For mainstream languages like Chinese and En- glish, it is feasible to collect large-scale labeled data for the model training, resulting in impressive ASR performance [4–6]. However, for low-resource language...

    work page Pith review arXiv 2025

  2. [2]

    It adopts the model architecture of wav2vec 2.0 [13], comprising a CNN feature extractor to convert waveform into a local feature, and a trans- former as the context network

    HuBERT and Its Variants HuBERT [12], short for Hidden-Unit BERT, is a self-supervised learning (SSL) method capable of leveraging masked predic- tion to learn robust speech representations. It adopts the model architecture of wav2vec 2.0 [13], comprising a CNN feature extractor to convert waveform into a local feature, and a trans- former as the context n...

    work page

  3. [3]

    The design of VietASR focuses on two key aspects: • Adapting HuBERT-style multi-iteration pre-training to Zip- former encoder for better efficiency and practicality

    VietASR To make the most of unlabeled data for building a speech recognition model suitable for real-world applications in low- resource languages, we propose the VietASR training pipeline. The design of VietASR focuses on two key aspects: • Adapting HuBERT-style multi-iteration pre-training to Zip- former encoder for better efficiency and practicality. •...

    work page

  4. [4]

    Experimental Setups DatasetWe collect approximately 70,000 hours of unlabeled audio from YouTube and apply FunASR V AD [29] for auto- matic segmentation

    Experiments 4.1. Experimental Setups DatasetWe collect approximately 70,000 hours of unlabeled audio from YouTube and apply FunASR V AD [29] for auto- matic segmentation. 50-hour audio from YouTube is adopted for fine-tuning, which is manually transcribed by professionals. Model evaluation is conducted on three public Vietnamese test sets, consisting of G...

    work page

  5. [5]

    Conclusion In this paper, we propose VietASR, a training pipeline designed for low-resource ASR, supporting both online and offline sce- narios. By adapting the HuBERT architecture to Zipformer and incorporating multi-iteration ASR-biased self-supervised learn- ing, VietASR provides a cost-effective and practical solution for improving ASR performance usi...

    work page

  6. [6]

    62206171 and No

    Acknowledgement This work was supported by the National Natural Science Foun- dation of China (No. 62206171 and No. U23B2018), Shanghai Municipal Science and Technology Major Project under Grant 2021SHZDZX0102, and the Tencent AI Lab Rhino-Bird Fo- cused Research Program

    work page

  7. [7]

    Con- nectionist temporal classification: labelling unsegmented se- quence data with recurrent neural networks,

    A. Graves, S. Fern ´andez, F. Gomez, and J. Schmidhuber, “Con- nectionist temporal classification: labelling unsegmented se- quence data with recurrent neural networks,” inProc. ICML, 2006

    2006

  8. [8]

    Speech recognition with deep recurrent neural networks,

    A. Graves, A. Mohamed, and G. Hinton, “Speech recognition with deep recurrent neural networks,” inProc. ICASSP, 2013

    work page 2013

Show all 39 references
  1. [9]

    Recent advances in end-to-end automatic speech recognition,

    J. Liet al., “Recent advances in end-to-end automatic speech recognition,”APSIPA Transactions on Signal and Information Processing, 2022

    2022

  2. [10]

    Robust speech recognition via large-scale weak supervision,

    A. Radford, J. W. Kim, T. Xuet al., “Robust speech recognition via large-scale weak supervision,” inProc. ICML, 2023

    2023

  3. [11]

    Less is more: Ac- curate speech recognition & translation without web-scale data,

    K. C. Puvvada, P. ˙Zelasko, H. Huanget al., “Less is more: Ac- curate speech recognition & translation without web-scale data,” arXiv preprint arXiv:2406.19674, 2024

    2024 arXiv

  4. [12]

    Anatomy of industrial scale multilingual ASR,

    F. M. Ramirez, L. Chkhetiani, A. Ehrenberget al., “Anatomy of industrial scale multilingual ASR,”arXiv preprint arXiv:2404.09841, 2024

    2024 arXiv

  5. [13]

    XLS-R: Self-supervised cross-lingual speech representation learning at scale,

    A. Babu, C. Wang, A. Tjandraet al., “XLS-R: Self-supervised cross-lingual speech representation learning at scale,” inProc. IN- TERSPEECH, 2022

    2022

  6. [14]

    Google USM: Scaling auto- matic speech recognition beyond 100 languages,

    Y . Zhang, W. Han, J. Qinet al., “Google USM: Scaling auto- matic speech recognition beyond 100 languages,”arXiv preprint arXiv:2303.01037, 2023

    2023 arXiv

  7. [15]

    Scaling speech technology to 1,000+ languages,

    V . Pratap, A. Tjandra, B. Shiet al., “Scaling speech technology to 1,000+ languages,”Journal of Machine Learning Research, 2024

    2024

  8. [16]

    GigaSpeech 2: An evolving, large-scale and multi-domain ASR corpus for low-resource lan- guages with automated crawling, transcription and refinement,

    Y . Yang, Z. Song, J. Zhuoet al., “GigaSpeech 2: An evolving, large-scale and multi-domain ASR corpus for low-resource lan- guages with automated crawling, transcription and refinement,” arXiv preprint arXiv:2406.11546, 2024

    2024 arXiv

  9. [17]

    Zipformer: A faster and better encoder for automatic speech recognition,

    Z. Yao, L. Guo, X. Yanget al., “Zipformer: A faster and better encoder for automatic speech recognition,” inProc. ICLR, 2023

    2023

  10. [18]

    HuBERT: Self- supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsaiet al., “HuBERT: Self- supervised speech representation learning by masked prediction of hidden units,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021

    2021

  11. [19]

    wav2vec 2.0: A framework for self-supervised learning of speech representa- tions,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representa- tions,” inProc. NeurIPS, 2020

    2020

  12. [20]

    Supervision-guided codebooks for masked prediction in speech pre-training,

    C. Wang, Y . Wang, Y . Wuet al., “Supervision-guided codebooks for masked prediction in speech pre-training,” inProc. INTER- SPEECH, 2022

    2022

  13. [21]

    Pushing the limits of unsu- pervised unit discovery for SSL speech representation,

    Z. Ma, Z. Zheng, G. Yanget al., “Pushing the limits of unsu- pervised unit discovery for SSL speech representation,” inProc. INTERSPEECH, 2023

    2023

  14. [22]

    Speech pre-training with acoustic piece,

    S. Ren, S. Liu, Y . Wu, L. Zhou, and F. Wei, “Speech pre-training with acoustic piece,” inProc. INTERSPEECH, 2022

    2022

  15. [23]

    Reducing barriers to self- supervised learning: HuBERT pre-training with academic com- pute,

    W. Chen, X. Chang, Y . Penget al., “Reducing barriers to self- supervised learning: HuBERT pre-training with academic com- pute,” inProc. INTERSPEECH, 2023

    2023

  16. [24]

    ASBERT: ASR-specific self-supervised learning with self-training,

    H. Y . Kim, B.-Y . Kim, S. W. Yooet al., “ASBERT: ASR-specific self-supervised learning with self-training,” inProc. SLT, 2023

    2023

  17. [25]

    Biased self-supervised learn- ing for ASR,

    F. L. Kreyssig, Y . Shi, J. Guoet al., “Biased self-supervised learn- ing for ASR,” inProc. INTERSPEECH, 2023

    2023

  18. [26]

    CTCBERT: Advancing hidden-unit BERT with CTC objectives,

    R. Fan, Y . Wang, Y . Gaur, and J. Li, “CTCBERT: Advancing hidden-unit BERT with CTC objectives,” inProc. ICASSP, 2023

    2023

  19. [27]

    MelHuBERT: A simplified hubert on mel spectrograms,

    T.-Q. Lin, H.-y. Lee, and H. Tang, “MelHuBERT: A simplified hubert on mel spectrograms,” inProc. ASRU, 2023

    2023

  20. [28]

    Fast-HuBERT: an efficient train- ing framework for self-supervised speech representation learn- ing,

    G. Yang, Z. Ma, Z. Zhenget al., “Fast-HuBERT: an efficient train- ing framework for self-supervised speech representation learn- ing,” inProc. ASRU, 2023

    2023

  21. [29]

    k2SSL: A faster and better frame- work for self-supervised speech representation learning,

    Y . Yang, J. Zhuo, Z. Jinet al., “k2SSL: A faster and better frame- work for self-supervised speech representation learning,”arXiv preprint arXiv:2411.17100, 2024

    2024 arXiv

  22. [30]

    Attention is all you need,

    A. Waswani, N. Shazeer, N. Parmaret al., “Attention is all you need,” inProc. NeurIPS, 2017

    2017

  23. [31]

    Conformer: Convolution- augmented transformer for speech recognition,

    A. Gulati, J. Qin, C.-C. Chiuet al., “Conformer: Convolution- augmented transformer for speech recognition,” inProc. INTER- SPEECH, 2020

    2020

  24. [32]

    RNN-Transducer with state- less prediction network,

    M. Ghodsi, X. Liu, J. Apfelet al., “RNN-Transducer with state- less prediction network,” inProc. ICASSP, 2020

    2020

  25. [33]

    Sequence transduction with recurrent neural net- works,

    A. Graves, “Sequence transduction with recurrent neural net- works,”arXiv preprint arXiv:1211.3711, 2012

    2012 arXiv

  26. [34]

    Pruned RNN-T for fast, memory-efficient ASR training,

    F. Kuang, L. Guo, W. Kanget al., “Pruned RNN-T for fast, memory-efficient ASR training,” inProc. INTERSPEECH, 2022

    2022

  27. [35]

    FunASR: A fundamental End-to- End speech recognition toolkit,

    Z. Gao, Z. Li, J. Wanget al., “FunASR: A fundamental End-to- End speech recognition toolkit,” inProc. INTERSPEECH, 2023

    2023

  28. [36]

    Common V oice: A massively-multilingual speech corpus,

    R. Ardila, M. Branson, K. Daviset al., “Common V oice: A massively-multilingual speech corpus,” inProc. ACL, 2020

    2020

  29. [37]

    FLEURS: Few-shot learn- ing evaluation of universal representations of speech,

    A. Conneau, M. Ma, S. Khanujaet al., “FLEURS: Few-shot learn- ing evaluation of universal representations of speech,” inProc. SLT, 2023

    2023

  30. [38]

    Neural machine transla- tion of rare words with subword units,

    R. Sennrich, B. Haddow, and A. Birch, “Neural machine transla- tion of rare words with subword units,” inProc. ACL, 2016

    2016

  31. [39]

    Fast and parallel decoding for transducer,

    W. Kang, L. Guo, F. Kuanget al., “Fast and parallel decoding for transducer,” inProc. ICASSP, 2023

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.