Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Delayed-KD: Delayed Knowledge Distillation based CTC for Low-Latency Streaming ASR

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a streaming speech recognizer processing 40 ms chunks can be trained, through delayed knowledge distillation, to match the character error rate of a baseline that waits 320 ms.

desk verdict A genuinely useful training recipe for low-latency streaming CTC ASR, with consistent empirical gains, but the mechanism is under-analyzed and missing baselines/error bars temper the headline. read the letter →

arxiv 2505.22069 v1 pith:O3T23OM5 submitted 2025-05-28 cs.SD eess.AS

Longhao Li , Yangze Li , Hongfei Xue , Jie Liu , Shuai Fang , Kai Wang , Lei Xie This is my paper · ORCID
classification cs.SDeess.AS
keywords streamingspeechrecognitionknowledgedistillationconnectionisttemporalclassificationCTClow-latencyASRtokenemissiondelayAlignmentBufferMandarin
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes Delayed-KD, a training method that lets a very-low-latency streaming speech recognizer learn from a slower, more accurate non-streaming model. The key idea is that the streaming model's CTC outputs lag behind the teacher's, so instead of forcing frame-by-frame imitation, the student is allowed, for every frame, to pick the delay within a bounded window, the Temporal Alignment Buffer (TAB), that best matches the teacher's frame-level posterior distribution. On the AISHELL-1 corpus this yields a 5.42% character error rate at 40 ms latency, matching the U2++ baseline at 320 ms latency and beating it by 9.34% relative at the same 40 ms latency; the same recipe transfers to the 10,000-hour WenetSpeech corpus. If the claim holds, tiny-chunk streaming CTC models can match the accuracy of far higher-latency systems, and the TAB size becomes a fine-grained knob on the latency-accuracy trade-off.

What carries the argument

The load-bearing mechanism is the Temporal Alignment Buffer (TAB), a training-time window $\mathcal{D} = \{0,1,\ldots,d\}$ of allowed delays between the streaming student's frame-level CTC outputs and the non-streaming teacher's. In the delayed knowledge distillation loss, each (batch, frame) pair independently selects the delay $\tau \in \mathcal{D}$ with the minimum KL divergence between the student's shifted posterior and the teacher's posterior, so the student is never forced to imitate a teacher frame it has not yet heard enough context for; this loss is added to the ordinary CTC and attention losses in joint training. The TAB is what turns rigid frame-by-frame distillation (TAB $= 0$, which the paper shows gives worse CER) into a flexible alignment that also pulls CTC spikes earlier and thereby reduces token emission delay without an explicit latency penalty in the loss.

What would settle it

Record the delay $\tau$ chosen by the min-KL rule for every frame during training: under the paper's account the chosen delays should cluster around the true CTC spike lag and form a roughly monotonic alignment, whereas scattered choices across the window on blank-heavy frames would show the gain is not coming from coherent alignment. A second check is to replace the per-frame minimum with the average over the TAB window; if the CER stays the same, the min-selection is not the load-bearing part of the loss.

Watch

Extended reading notes

Core claim

Delayed-KD claims that the accuracy collapse of CTC-based streaming ASR at tiny chunk sizes can be largely repaired during training by distilling CTC posterior probabilities from a frozen non-streaming teacher under a per-frame delay. The distillation loss is $\mathcal{L}_{\mathrm{distill}} = \frac{1}{BT}\sum_{b,t} \min_{\tau \in \mathcal{D}} \sum_c \mathrm{KL}(p^{\mathrm{student}}_{b,t+\tau,c} \| p^{\mathrm{teacher}}_{b,t,c})$ with $\mathcal{D} = \{0,1,\ldots,d\}$ set by the Temporal Alignment Buffer: for each teacher frame, the student's output shifted by the delay that minimizes KL is used. With an 80 ms TAB at 40 ms latency, the method reports 5.42% CER in rescoring mode on AISHELL-1, comparable to U2++ at 320 ms, and a 21.26% relative CER reduction over U2++ at the same 40 ms latency in streaming mode; the same hyperparameters give results on WenetSpeech comparable to U2++ at 320 ms and clearly better than U2++ at 40 ms. The paper also shows that TAB size tunes the emission-delay/accuracy trade-off monotonically and that a distillation weight of $\alpha = 100$ is optimal, with larger weights degrading accuracy by overfitting to the teacher's posteriors.

Load-bearing premise

The whole accuracy gain rests on the assumption that letting each frame independently choose the delay that best matches the teacher teaches a coherent alignment, rather than letting the student lower the loss by mimicking teacher blank outputs or arbitrary shifted posteriors at different delays on different frames.

Editorial extensions

If this is right

  • A tiny-chunk streaming CTC model can reach the CER of a system running at eight times the latency: 5.42% at 40 ms versus 5.44% at 320 ms on AISHELL-1.
  • TAB size acts as a fine-grained control: enlarging the buffer from 0 to 80 ms cuts streaming CER from 7.02% to 6.11% and rescoring CER from 5.77% to 5.42%, at a small, predictable cost in first- and last-token emission delay.
  • The method needs no re-tuning to transfer: on the 10,000-hour WenetSpeech corpus the same settings give 40 ms results comparable to a 320 ms U2++ and far better than a 40 ms U2++.
  • The CTC branch itself improves, not just the rescoring path, since streaming (frame-level) decoding shows the larger relative gain (21.26% over U2++ at 40 ms on AISHELL-1).
  • The distillation signal is a genuine contributor: lowering the weight below $\alpha = 100$ under-uses the teacher, while raising it above 100 overfits to the teacher's CTC posteriors and degrades CER.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The per-frame min over the delay window is a form of soft alignment that could transfer to other misalignment settings, such as distilling across models with different frame rates or streaming-versus-non-streaming encoders for RNN-T; nothing in the mechanism is CTC-specific.
  • If the mechanism works by aligning spiking behavior, the selected delays should cluster around the true teacher-student lag; a training-time histogram of chosen $\tau$ values would show whether TAB teaches 'when to spike' or merely relaxes the distillation target.
  • A testable extension is to constrain the chosen delays to form a monotonic alignment path across frames; if accuracy holds or improves, the gain comes from the delay window, and if it degrades, the per-frame freedom itself matters.
  • Read as a deployment tool, TAB trades a modest, predictable latency increase for a substantial CER gain, so training with a mid-size buffer and decoding with a smaller chunk could squeeze both accuracy and latency from one checkpoint.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The manuscript proposes Delayed-KD, a training method for streaming CTC-based ASR that distills frame-level CTC posteriors from a frozen non-streaming teacher into a streaming student. The key component is a Temporal Alignment Buffer (TAB): for each frame, the distillation loss selects the delay tau in {0,...,d} that minimizes the KL divergence between the student posterior at time t+tau and the teacher posterior at time t. The student is trained jointly with the delayed distillation loss, a CTC loss, and an attention decoder loss. Experiments on AISHELL-1 and WenetSpeech report CER and token-emission latencies, with the headline result that Delayed-KD at 40 ms chunk size reaches 5.42% CER on AISHELL-1 rescoring, comparable to U2++ at 320 ms.

Significance. If the reported results are reproducible, the method is practically valuable: a tiny-chunk streaming CTC model matching a much higher-latency baseline would be a meaningful step for low-latency ASR. The method is specified precisely enough to reimplement, the experiments cover two datasets including the 10,000-hour WenetSpeech benchmark, and the baselines (U2++, Fast-U2++, CUSIDE-T) are strong and relevant. The main open question is whether the per-frame min over delays actually performs the alignment that the paper credits it with; the current manuscript does not supply evidence that the loss mechanism behaves as intended.

major comments (3)
  1. [Sec. 2.2.1, Eq. (1)] The per-frame minimization over tau is the load-bearing mechanism of the paper, but the manuscript provides no evidence about which delays are actually selected. Since the min is taken independently for each time step t, the induced 'alignment' is not constrained to be monotone or smooth. For a typical CTC teacher whose posterior is blank-dominated for most frames, the student can reduce the distillation loss by matching the teacher's blank posterior at different offsets per frame, without learning a coherent non-blank alignment. Under such a degenerate solution, the character-level signal would come from L_CTC and L_AED in Eq. (4), and the reported CER gains would not validate the TAB alignment mechanism. Please report the empirical distribution of selected tau values (e.g., per-frame histograms), check monotonicity or smoothness of the selected delays, and include an ablation with a single utterance-level offset or with a monotone/soft alignment. Without this diagnostic, the central attribution of the accuracy gains to delayed alignment is not established.
  2. [Tables 1 and 2] The headline comparisons rely on very small CER differences: Delayed-KD at 40 ms (5.42%) versus U2++ at 320 ms (5.44%), and TAB size 80 ms versus 120 ms (5.42% versus 5.47%). The manuscript does not report run-to-run variance, number of seeds, or confidence intervals, and it does not state whether the best TAB size and distillation weight alpha=100 were selected on a held-out dev set or on the test set. Since these optimal values are then transferred to WenetSpeech, the risk of selection on test-set noise is material. Please provide dev-set tuning curves, multi-seed results, or at least error bars for the key entries in Tables 1 and 2.
  3. [Sec. 3.5, Table 3] The WenetSpeech results do not uniformly support the claim that Delayed-KD at 40 ms is comparable to U2++ at 320 ms. In rescoring mode, Delayed-KD is worse than U2++ at 320 ms on both Test Meeting (16.96% versus 16.76%) and Test Net (12.75% versus 12.23%). The advantage over the 320 ms U2++ is specific to streaming-mode decoding. The text should quantify which comparisons support the claim of 'consistent superiority' and which do not, and the conclusions should be qualified accordingly.
minor comments (5)
  1. [Eq. (1)] The indices t+tau may exceed the sequence length T near the end of an utterance; the paper should specify how the boundary is handled (padding, cropping, or masking).
  2. [Sec. 3.4 heading] The heading 'Results on ASIHELL-1' contains a typo; it should read 'AISHELL-1'.
  3. [Table 1] The row layout for the '0 ms' TAB condition is confusing: it is not clear that '0 ms' denotes TAB size zero and that '1 40' denotes the chunk-size and latency columns. Please separate the TAB size, chunk size, and latency columns clearly.
  4. [Figure 2] Figure 2 is qualitative; since the paper already reports FTD and LTD, consider adding a quantitative distribution of CTC spike shifts or median spike timing differences for the test set.
  5. [Reference [29]] The text says the timestamps are obtained using a GMM/HMM model trained with the Kaldi toolkit, but reference [29] is the PyTorch-Kaldi wrapper; citing the original Kaldi toolkit would be more appropriate.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Delayed-KD's CER/latency claims are empirically evaluated against external baselines, and the TAB delay bound is definitional but not used as a predicted result.

full rationale

The claimed result—that Delayed-KD improves CER at 40 ms latency—is evaluated against external baselines (U2++, Fast-U2++, CUSIDE-T) on public datasets (AISHELL-1 and WenetSpeech). Equation (1) defines a new training objective, and the paper measures its downstream CER; the loss is not fitted to the reported CER values. Hyperparameter sweeps over TAB size and alpha are presented as comparisons, and the WenetSpeech experiments transfer AISHELL-1 settings, which is an external transfer test rather than a fitted prediction. Self-citations to U2++, WeNet, and WenetSpeech are references to baseline models, the toolkit, and the dataset; none carries a load-bearing uniqueness or correctness claim for Delayed-KD. The statement that TAB controls emission delay is a design property of D={0,...,d}, but the FTD/LTD numbers and the CER trade-off are empirically measured, not consequences of the definition alone. The skeptical concern about the per-frame min-KL degenerating to blank matching is a mechanism/robustness question and does not make the derivation circular.

Assumptions & free parameters 3 free parameters · 3 assumptions · 1 invented entities

The central result depends on two tuned hyperparameters (α and TAB size d) and on the assumption that min-KL delay selection yields a coherent alignment signal. No code or analysis of selected delays is provided, so these are the main unverified inputs.

free parameters (3)
  • Distillation weight α = 100 (AISHELL-1); used for WenetSpeech
    Tuned via Table 2; performance degrades for α=200 and α=400.
  • TAB size d (max delay in alignment window) = 80 ms (2 chunks) on AISHELL-1; transferred to WenetSpeech
    Best CER in Table 1; larger TAB increases emission latency.
  • CTC/AED loss mixing weight λ = 0.3
    Inherited from U2++ baseline and set in §3.2; not analyzed.
assumptions (3)
  • domain assumption The frozen non-streaming teacher's CTC posteriors provide a beneficial distillation target for the streaming student.
    Invoked in §2.2.1; the teacher's training and calibration are not described.
  • domain assumption The per-frame min over delay τ in Eq. (1) yields a temporally coherent alignment signal.
    No analysis of selected τ or comparison with regularized alignment is provided.
  • domain assumption A bounded delay window D is sufficient to align streaming student spikes with non-streaming teacher spikes.
    Motivates TAB via Fig. 2; the bound d is a hyperparameter and no evidence about delays outside D is given.
invented entities (1)
  • Temporal Alignment Buffer (TAB)
    purpose: Defines the allowed delay range D={0,...,d} and selects the minimum-KL delay for distilling CTC posteriors; used to control emission delay.
    TAB is a named hyperparameter of the training objective, not an externally falsifiable entity; its size is tuned on AISHELL-1 and transferred to WenetSpeech.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Delayed-KD: Delayed Knowledge Distillation based CTC for Low-Latency Streaming ASR." pith.science (2026). https://pith.science/paper/O3T23OM5

@misc{pith2026250522069,
  author       = {Pith},
  title        = {Pith review of: Delayed-KD: Delayed Knowledge Distillation based CTC for Low-Latency Streaming ASR},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O3T23OM5}},
  note         = {Machine review of arXiv:2505.22069}
}
read the original abstract

CTC-based streaming ASR has gained significant attention in real-world applications but faces two main challenges: accuracy degradation in small chunks and token emission latency. To mitigate these challenges, we propose Delayed-KD, which applies delayed knowledge distillation on CTC posterior probabilities from a non-streaming to a streaming model. Specifically, with a tiny chunk size, we introduce a Temporal Alignment Buffer (TAB) that defines a relative delay range compared to the non-streaming teacher model to align CTC outputs and mitigate non-blank token mismatches. Additionally, TAB enables fine-grained control over token emission delay. Experiments on 178-hour AISHELL-1 and 10,000-hour WenetSpeech Mandarin datasets show consistent superiority of Delayed-KD. Impressively, Delayed-KD at 40 ms latency achieves a lower character error rate (CER) of 5.42% on AISHELL-1, comparable to the competitive U2++ model running at 320 ms latency.

Figures

Figures reproduced from arXiv: 2505.22069 by the authors.

Figure 1
Figure 1. Model architecture of our proposed Delayed-KD streaming teacher model, the CTC branch of the streaming stu￾dent model, and the attention branch of the streaming student model. Notably, the CTC branches of both the teacher and stu￾dent models share an identical structure, each consisting of a shared encoder for modeling the context of acoustic features and a CTC decoder for aligning frames and tokens. The at￾tention … view at source ↗
Figure 2
Figure 2. Comparison of CTC spike distributions between U2++ and Delayed-KD. Colored lines represent CTC spikes, and dashed lines align time axis positions. 3.3. Emission Latency Metrics To evaluate character-level recognition latency, we calculate each token’s latency by subtracting the corresponding token’s ground-truth end time from the model output timestamp. For chunk-based streaming models, each token’s timestamp corre￾… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. WenetSpeech-Yue: A Large-scale Cantonese Speech Corpus with Multi-dimensional Annotation

    cs.SD 2025-09 conditional novelty 6.0 of 10

    The authors built and released the largest open-source Cantonese speech corpus (21,800 hours, 10 domains, rich metadata), and show that models trained on it match or beat existing speech recognition and synthesis systems.

Reference graph

Works this paper leans on

34 extracted references · 22 canonical work pages · cited by 1 Pith paper

  1. [1]

    Introduction Streaming automatic speech recognition (ASR) has attracted significant attention in real-world applications, aiming to en- sure recognition accuracy with low latency. Currently, the predominant end-to-end ASR models include Connectionist Temporal Classification (CTC) [1], Recurrent Neural Network Transducer (RNN-T) [2] and Attention-based Enc...

    work page

  2. [2]

    Method 2.1. Model architecture As shown in Figure 1, the proposed model architecture, com- prises three main components: the CTC branch of the non- arXiv:2505.22069v1 [cs.SD] 28 May 2025 Attention Decoder Non-streaming Encoder (teacher) Streaming Encoder (student) Teacher CTC Decoder CTC Posterior Probabilities CTC Posterior Probabilities … … Delayed know...

    arXiv 2025

  3. [3]

    Dataset We conduct experiments on two Mandarin Chinese datasets: AISHELL-1 [22] (178 hours) and the large-scale, multi-domain WenetSpeech [24] (10,000 hours)

    Experiments 3.1. Dataset We conduct experiments on two Mandarin Chinese datasets: AISHELL-1 [22] (178 hours) and the large-scale, multi-domain WenetSpeech [24] (10,000 hours). For AISHELL-1, the test set contains 7,176 utterances, while WenetSpeech provides two test sets,Test MeetingandTest Net, which collectively contain approximately 33,100 utterances. ...

    work page

  4. [4]

    Conclusion In this paper, we propose Delayed-KD, a novel delayed knowl- edge distillation method that distills the CTC posterior proba- bilities from a non-streaming teacher model. By introducing a Temporal Alignment Buffer (TAB) during training, Delayed- KD aligns the CTC outputs of streaming and non-streaming models, effectively reducing mismatches in t...

    work page

  5. [5]

    Reducing the gap between streaming and non- streaming transducer-based asr by adaptive two-stage knowledge distillation,

    H. Tang, Y . Fu, L. Sun, J. Xue, D. Liu, Y . Li, Z. Ma, M. Wu, J. Pan, G. Wanet al., “Reducing the gap between streaming and non- streaming transducer-based asr by adaptive two-stage knowledge distillation,” inProc. ICASSP. IEEE, 2023, pp. 1–5

    work page 2023

  6. [6]

    Con- nectionist temporal classification: labelling unsegmented se- quence data with recurrent neural networks,

    A. Graves, S. Fern ´andez, F. Gomez, and J. Schmidhuber, “Con- nectionist temporal classification: labelling unsegmented se- quence data with recurrent neural networks,” inProc. ICML, 2006, pp. 369–376

    2006

  7. [7]

    Sequence transduction with recurrent neural net- works,

    A. Graves, “Sequence transduction with recurrent neural net- works,”arXiv preprint arXiv:1211.3711, 2012

    arXiv 2012

  8. [8]

    Listen, attend and spell,

    W. Chan, N. Jaitly, Q. V . Le, and O. Vinyals, “Listen, attend and spell,”arXiv preprint arXiv:1508.01211, 2015

    arXiv 2015

Show all 34 references
  1. [9]

    Attention-based models for speech recognition,

    J. K. Chorowski, D. Bahdanau, D. Serdyuk, K. Cho, and Y . Ben- gio, “Attention-based models for speech recognition,”Advances in neural information processing systems, vol. 28, 2015

    2015

  2. [10]

    Fastemit: Low- latency streaming asr with sequence-level emission regulariza- tion,

    J. Yu, C.-C. Chiu, B. Li, S.-y. Chang, T. N. Sainath, Y . He, A. Narayanan, W. Han, A. Gulati, Y . Wuet al., “Fastemit: Low- latency streaming asr with sequence-level emission regulariza- tion,” inProc. ICASSP. IEEE, 2021, pp. 6004–6008

    2021

  3. [11]

    Xlsr- transducer: Streaming asr for self-supervised pretrained models,

    S. Kumar, S. Madikeri, J. Zuluaga-Gomez, E. Villatoro-Tello, I. Thorbecke, P. Motlicek, A. Ganapathirajuet al., “Xlsr- transducer: Streaming asr for self-supervised pretrained models,” arXiv preprint arXiv:2407.04439, 2024

    2024 arXiv

  4. [12]

    Mamba for streaming asr combined with uni- modal aggregation,

    Y . Fang and X. Li, “Mamba for streaming asr combined with uni- modal aggregation,”arXiv preprint arXiv:2410.00070, 2024

    2024 arXiv

  5. [13]

    Fast-u2++: Fast and accurate end-to-end speech recognition in joint ctc/attention frames,

    C. Liang, X.-L. Zhang, B. Zhang, D. Wu, S. Li, X. Song, Z. Peng, and F. Pan, “Fast-u2++: Fast and accurate end-to-end speech recognition in joint ctc/attention frames,” inProc. ICASSP. IEEE, 2023, pp. 1–5

    2023

  6. [14]

    Cuside-t: Chunking, simu- lating future and decoding for transducer based streaming asr,

    W. Zhao, Z. Li, C. Yu, and Z. Ou, “Cuside-t: Chunking, simu- lating future and decoding for transducer based streaming asr,” in Proc. ISCSLP. IEEE, 2024, pp. 11–15

    2024

  7. [15]

    Knowledge distilla- tion from non-streaming to streaming asr encoder using auxiliary non-streaming layer,

    K. Shim, J. Lee, S. Chang, and K. Hwang, “Knowledge distilla- tion from non-streaming to streaming asr encoder using auxiliary non-streaming layer,”arXiv preprint arXiv:2308.16415, 2023

    2023 arXiv

  8. [16]

    Peak-first ctc: reducing the peak latency of ctc models by applying peak-first regularization,

    Z. Tian, H. Xiang, M. Li, F. Lin, K. Ding, and G. Wan, “Peak-first ctc: reducing the peak latency of ctc models by applying peak-first regularization,” inProc. ICASSP. IEEE, 2023, pp. 1–5

    2023

  9. [17]

    Delay-penalized transducer for low- latency streaming asr,

    W. Kang, Z. Yao, F. Kuang, L. Guo, X. Yang, L. Lin, P. ˙Zelasko, and D. Povey, “Delay-penalized transducer for low- latency streaming asr,” inProc. ICASSP. IEEE, 2023, pp. 1–5

    2023

  10. [18]

    Knowledge distillation for streaming transformer-transducer

    A. Kojima, H. Hermansky, H. Cernock `y, L. Burget, L. Lamel, and O. Scharenborg, “Knowledge distillation for streaming transformer-transducer.” inInterspeech, 2021, pp. 2841–2845

    2021

  11. [19]

    Distilling the knowledge in a neural network,

    G. Hinton, “Distilling the knowledge in a neural network,”arXiv preprint arXiv:1503.02531, 2015

    2015 arXiv

  12. [20]

    Bayes risk ctc: Controllable ctc alignment in sequence-to-sequence tasks,

    J. Tian, B. Yan, J. Yu, C. Weng, D. Yu, and S. Watanabe, “Bayes risk ctc: Controllable ctc alignment in sequence-to-sequence tasks,”arXiv preprint arXiv:2210.07499, 2022

    2022 arXiv

  13. [21]

    Stable distilla- tion: Regularizing continued pre-training for low-resource auto- matic speech recognition,

    A. Seth, S. Ghosh, S. Umesh, and D. Manocha, “Stable distilla- tion: Regularizing continued pre-training for low-resource auto- matic speech recognition,” inProc. ICASSP. IEEE, 2024, pp. 10 821–10 825

    2024

  14. [22]

    Dual-mode asr: Unify and improve streaming asr with full-context modeling,

    J. Yu, W. Han, A. Gulati, C.-C. Chiu, B. Li, T. N. Sainath, Y . Wu, and R. Pang, “Dual-mode asr: Unify and improve streaming asr with full-context modeling,”arXiv preprint arXiv:2010.06030, 2020

    2010 arXiv

  15. [23]

    Alignment knowledge distilla- tion for online streaming attention-based speech recognition,

    H. Inaguma and T. Kawahara, “Alignment knowledge distilla- tion for online streaming attention-based speech recognition,” IEEE/ACM Transactions on Audio, Speech, and Language Pro- cessing, vol. 31, pp. 1371–1385, 2021

    2021

  16. [24]

    Guiding ctc posterior spike timings for improved posterior fusion and knowledge distillation,

    G. Kurata and K. Audhkhasi, “Guiding ctc posterior spike timings for improved posterior fusion and knowledge distillation,”arXiv preprint arXiv:1904.08311, 2019

    1904 arXiv

  17. [25]

    layers, while the CTC decoder is composed of a linear layer followed by a log softmax layer. This design serves a dual purpose: firstly, it applies the CTC loss function to the soft- max output during training, and secondly, it facilitates delayed knowledge distillation of fra...

  18. [26]

    Guiding frame-level ctc alignments using self-knowledge distillation,

    E. Kim, H. Kim, and K. Lee, “Guiding frame-level ctc alignments using self-knowledge distillation,”arXiv preprint arXiv:2406.07909, 2024

    2024 arXiv

  19. [27]

    Aishell-1: An open- source mandarin speech corpus and a speech recognition base- line,

    H. Bu, J. Du, X. Na, B. Wu, and H. Zheng, “Aishell-1: An open- source mandarin speech corpus and a speech recognition base- line,” in2017 20th conference of the oriental chapter of the inter- national coordinating committee on speech databases and speech I/O systems and assess...

    2017

  20. [28]

    U2++: Unified two-pass bidirectional end-to-end model for speech recognition,

    D. Wu, B. Zhang, C. Yang, Z. Peng, W. Xia, X. Chen, and X. Lei, “U2++: Unified two-pass bidirectional end-to-end model for speech recognition,”arXiv preprint arXiv:2106.05642, 2021

    2021 arXiv

  21. [29]

    Wenetspeech: A 10000+ hours multi-domain mandarin corpus for speech recognition,

    B. Zhang, H. Lv, P. Guo, Q. Shao, C. Yang, L. Xie, X. Xu, H. Bu, X. Chen, C. Zenget al., “Wenetspeech: A 10000+ hours multi-domain mandarin corpus for speech recognition,” inProc. ICASSP. IEEE, 2022, pp. 6182–6186

    2022

  22. [30]

    Conformer: Convolution- augmented transformer for speech recognition,

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wuet al., “Conformer: Convolution- augmented transformer for speech recognition,”arXiv preprint arXiv:2005.08100, 2020

    2005 arXiv

  23. [31]

    Wenet 2.0: More productive end-to- end speech recognition toolkit,

    B. Zhang, D. Wu, Z. Peng, X. Song, Z. Yao, H. Lv, L. Xie, C. Yang, F. Pan, and J. Niu, “Wenet 2.0: More productive end-to- end speech recognition toolkit,”arXiv preprint arXiv:2203.15455, 2022

    2022 arXiv

  24. [32]

    Low latency speech recognition using end-to-end prefetching

    S.-Y . Chang, B. Li, D. Rybach, Y . He, W. Li, T. N. Sainath, and T. Strohman, “Low latency speech recognition using end-to-end prefetching.” inInterspeech, 2020, pp. 1962–1966

    2020

  25. [33]

    Dissect- ing user-perceived latency of on-device e2e speech recognition,

    Y . Shangguan, R. Prabhavalkar, H. Su, J. Mahadeokar, Y . Shi, J. Zhou, C. Wu, D. Le, O. Kalinli, C. Fuegenet al., “Dissect- ing user-perceived latency of on-device e2e speech recognition,” arXiv preprint arXiv:2104.02207, 2021

    2021 arXiv

  26. [34]

    The pytorch-kaldi speech recognition toolkit,

    M. Ravanelli, T. Parcollet, and Y . Bengio, “The pytorch-kaldi speech recognition toolkit,” inProc. ICASSP. IEEE, 2019, pp. 6465–6469

    2019

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.