Pith. sign in

REVIEW 3 major objections 4 minor 56 references

Aligner-Encoders: Self-Attention Transformers Can Be Self-Transducers

T0 review · 3 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A Transformer encoder can perform audio-to-text alignment itself during the forward pass, enabling simple cross-entropy training and O(U) decoding with accuracy close to RNN-T.

desk verdict A real and surprising ASR result under a hard-coded diagonal alignment, with a serious length-generalization gap that the paper leaves undiagnosed. read the letter →

arxiv 2502.05232 v1 pith:54IYG2P7 submitted 2025-02-06 cs.SD cs.AIcs.LGeess.AS

Adam Stooke , Rohit Prabhavalkar , Khe Chai Sim , Pedro Moreno Mengibar This is my paper · ORCID
classification cs.SDcs.AIcs.LGeess.AS
keywords automaticspeechrecognitionAligner-EncoderRNN-Transducercross-entropylossself-attentionalignmentself-transductionsequencetransductionConformer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a Transformer encoder for speech can do the alignment between audio and text itself, during its forward pass, before any decoding begins. That would let an ASR model be trained with the simple frame-wise cross-entropy loss of attention-based encoder-decoder models, while the decoder is a light text-only recurrence that scans encoder frames in order and emits one token per frame. The authors build this Aligner-Encoder and show its word error rate is close to RNN-Transducer on LibriSpeech and Voice Search, with decoder inference complexity O(U) instead of O(T+U) or O(U*T). They also find the alignment appears explicitly in self-attention weights of a specific layer, and that the encoder can learn reversed alignments, suggesting the idea may extend beyond monotonic ASR.

What carries the argument

The load-bearing object is the Aligner-Encoder's one-to-one coupling equation, $P(y_i \mid x, y_{<i}) = f_{\mathrm{joint}}(h_i, g_i)$, which forces the encoder output frame $i$ to carry the information for token $i$ and ignores all encoder frames beyond the label length $U$. This turns the alignment problem into a constraint on the encoder: the encoder must move acoustic evidence into the correct front-aligned position through self-attention alone. The paper identifies a specific phenomenon in the self-attention weights—audio-to-text alignment becoming visible in roughly two layers of the Conformer—as the mechanism that makes the constraint satisfiable.

What would settle it

Train or test the Aligner on utterances longer than its maximum alignable length (more text tokens than encoder frames) and measure deletions at the tail; the paper itself reports base-model WER rising to 28.0% on Test-Clean utterances over 21 seconds, and the same failure appears when the alignment layer's self-attention only covers part of the utterance. A direct check would be to construct an utterance where $U > T'$ and observe whether the model can ever output the surplus tokens.

Watch

Extended reading notes

Core claim

The central claim is that modern Transformer-based ASR encoders are capable of 'self-transduction': they can reorder and relocate speech information within the embedding sequence so that, by the time the encoder is done, the i-th encoder frame holds the content for the i-th output token. The paper argues this capability makes dynamic programming unnecessary: instead of marginalizing over alignments as RNN-T does, one can couple frame i with token i directly in the loss and train with ordinary cross-entropy. On LibriSpeech the Aligner reaches 2.3% WER on Test-Clean versus 2.1% for RNN-T and 2.4% for AED, and on the Voice Search main set it gets 3.7% versus 3.6% for RNN-T. The authors report a 2x total inference speedup over RNN-T and 16x over AED in a representative setting.

Load-bearing premise

The model assumes every output token can be assigned to exactly one encoder frame in order, and that there are always at least as many encoder frames as text tokens, so any utterance whose text needs more frames than the encoder has cannot be transcribed without deletions.

Editorial extensions

If this is right

  • ASR training can drop the dynamic-programming lattice of RNN-T; the loss is a plain sum of per-frame cross-entropy terms, and the paper measures roughly a 10x reduction in decoder-plus-loss training time for its LibriSpeech setup.
  • Auto-regressive decoding becomes O(U) with a small constant: one LSTM step per output token, with no blank emissions and no cross-attention to the full encoder sequence.
  • Beam search no longer needs RNN-T-style path merging, because a text hypothesis corresponds to a single decoder path.
  • Long-form recognition can be done by chunking inside the model with prediction-network reset and state-priming, recovering near-blind-segmenter WER on YouTube without extra training.
  • Because the encoder can learn reversed alignments, the same mechanism may apply to non-monotonic sequence transduction tasks such as speech translation, though output-longer-than-input tasks like text translation would need modification.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The one-frame-per-token constraint is a hard capacity bound: any practical deployment must guarantee the encoder's frame rate (after subsampling) never drops below the expected token count, or deletions are structural, not just a training artifact.
  • The visible alignment in self-attention could be turned into a free forced-alignment output, replacing the RNN-T lattice probability as a way to get token timestamps without modifying the model.
  • A natural next experiment is allowing multiple tokens per frame, either by predicting several labels at one position or by letting the decoder advance only when the encoder marks a boundary; this would soften the $U \le T'$ constraint and could extend Aligners to higher-compression frame rates.
  • If the alignment layer generalizes across domains, the technique could simplify streaming ASR by running the encoder on fixed-size chunks and carrying only the LSTM state, but that requires the encoder to align per-chunk, which the paper has not demonstrated.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes the Aligner-Encoder, an ASR model that combines a Conformer encoder with a text-only LSTM prediction network and a joint network. The model is trained with frame-wise cross-entropy loss in which encoder frame i is paired with text token i, and decoding scans the encoder embedding frames in order, emitting one token per frame until an end-of-message token is produced. The authors report WER close to an RNN-T baseline on LibriSpeech, Voice Search, and YouTube test sets, and faster inference than RNN-T and AED. They further analyze self-attention weights to argue that a single encoder layer performs the alignment, which they call self-transduction, and they propose an inference-time chunking scheme for long-form audio.

Significance. If the results hold, the paper shows that a transformer encoder can be trained with a simple frame-wise cross-entropy loss and a lightweight text-only decoder to approach RNN-T accuracy while reducing decoder complexity to O(U). The strengths are the clear model formulation, the multi-dataset empirical evaluation, the concrete timing measurements, and the clever diagnostic in which an RNN-T is trained on top of frozen Aligner layers. The main limitation is that the alignment is prescribed by the diagonal loss, so the 'self-transduction' evidence reflects learning to implement a fixed routing rather than autonomously discovering an alignment. The long-form capability is also fragile: without concatenated training the base model's WER rises sharply on utterances longer than 21 seconds, and the successful chunking configuration was tuned on the test set. These caveats do not erase the empirical contribution, but they require the claims to be rephrased and supported.

major comments (3)
  1. [§2.1, Eqs. (3)–(4)] Equation (3) trains P(y_i | x, y_<i) = f_joint(h_i, g_i), coupling output index i to encoder frame i, and Equation (4) applies the loss only for i ≤ U; frames beyond U receive no training signal. The alignment is therefore prescribed by the objective, not inferred by the model. The self-attention visualizations in §4.5.1 show that the encoder learns to implement this prescribed diagonal routing, not that it autonomously discovers an audio-to-text alignment. The central conceptual claim should be rephrased: the contribution is that a frame-wise cross-entropy loss with a fixed one-to-one mapping is sufficient to train an efficient recognizer, not that the architecture alone discovers alignments. This does not invalidate the empirical WER and efficiency results, but it changes what the title and abstract claim.
  2. [§4.5.3, Tables 7–8] The base Aligner degrades from 2.4% on Test-Clean utterances under 17 seconds to 7.0% on 17–21 second utterances and 28.0% on utterances over 21 seconds, while RNN-T degrades only mildly (2.1% to 2.8%). The paper never reports T' (the encoder frame count after subsampling) and U for the failing utterances, so it cannot distinguish between the hard capacity limit U ≤ T' and a failure of attention or positional length extrapolation. This distinction is central to the claimed capability: if the failures are mostly capacity-driven, the model has a rigid one-frame-per-token limit; if they are extrapolation-driven, a different remedy may exist. The long-form recognition claim is also weakened by the fact that the chunking configuration (14 second period, 10-token priming, resetting the prediction network) was selected on the YouTube test set; without a validation-based tuning protocol, the reported 7.3% WER overstates the model's off-the-shelf long-form ability.
  3. [§4.3, Table 3] The text states that the paper reports 'the best score from a small number of runs and checkpoints,' but no variance, number of runs, or checkpoint selection criterion is given. On LibriSpeech, the differences between Aligner and RNN-T are 0.2–0.5 WER absolute (2.3 vs 2.1 on Test-Clean, 5.1 vs 4.6 on Test-Other), which is within the range of typical run-to-run variation at this scale. The 'remarkably close' claim needs at least a mean and standard deviation over runs, or a statement of how many checkpoints were evaluated, to be supported.
minor comments (4)
  1. [§2.1] The sentence around Equation (4) says 'The loss only applies to encoder frames within the length of the label, T′ ≤ U; all remaining frames (T′ > U) are ignored.' The inequality is reversed: the required condition is U ≤ T′, consistent with the later statement that Aligners cannot downsample the encoder to fewer frames than the text sequence. Please correct the inequality and the surrounding wording.
  2. [§4.3, Tables 3 and 7] It is unclear whether the Aligner row in Table 3 corresponds to the base model or the concatenation-trained model, since Table 7 lists both 'ALIGNER' and 'ALIGNER-CONCAT' and the text says random concatenation was used for the LibriSpeech training set. Please state explicitly which configuration each table reports.
  3. [§4.5.1] The paper says that all attention heads showed the same alignment operation in layers 14 and 15 'for every input example we observed,' but it does not state how many utterances or heads were inspected. Please provide a quantitative count or a small table so the claim is verifiable rather than anecdotal.
  4. [Table 5, footnote 5] The paper notes that path merging was disabled for RNN-T in the timing comparison. Please clarify whether the RNN-T WER numbers throughout the paper also used this approximate beam search, and whether enabling path merging changes the WER comparison.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's empirical claims are self-contained; the diagonal alignment in Eq. (3) is a training target, not a derived prediction.

full rationale

The paper is an empirical systems paper. Its central claims—that an Aligner-Encoder trained with frame-wise cross-entropy achieves WER close to RNN-T on LibriSpeech, Voice Search, and YouTube; that decoding is faster; and that long-form performance degrades without chunking—are supported by direct experiments and comparisons, not by a derivation from the model definition. Equation (3) does prescribe a one-to-one mapping between encoder frame i and output token i, so the alignment is a training objective rather than an emergent discovery. But the paper does not pretend otherwise: it states that it 'enforce[s] the alignment at the encoder output' by restricting the model to use the acoustic and text embedding frames one-to-one, and then empirically verifies that the encoder learns to satisfy the objective and that self-attention weights exhibit the corresponding pattern. The attention visualization is therefore evidence of compliance with the loss, not a circular prediction of a quantity already fitted. The reported length-generalization limitation (Table 7) is honestly disclosed and is a robustness/correctness issue, not a circularity. No load-bearing argument reduces to a self-citation: the cited prior works by the authors (e.g., frame-rate reduction) are background and are not used to justify the Aligner-Encoder's capability or accuracy. Hence no step in the paper's derivation chain is equivalent to its inputs by construction.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The model reduces to RNN-T's lattice diagonal (Section 2.2), so the free parameters are the hyperparameters that were chosen by hand or tuned on the target test sets, most importantly the long-form inference settings. The axioms are the monotonic one-frame-per-token routing assumption and the CE-on-diagonal training signal; both are load-bearing and both are acknowledged as limitations in the paper's length generalization experiments.

free parameters (6)
  • label smoothing weight = 0.1
    Applied to the CE loss for all Aligner models; reported as always used and beneficial, not tuned per dataset.
  • beam size = 6
    Used for all WER evaluations, chosen as a 'modest' beam with 'small but consistent improvement'.
  • label smoothing debiasing parameter = 2
    Threshold 2/V in beam search post-processing; chosen because 'relatively large... is helpful'.
  • LibriSpeech random concatenation fraction = 15%
    Randomly concatenated subset of batches to create examples up to 36s, affecting long-utterance WER; no sweep reported.
  • YouTube chunk period = 14 s (176 embedding frames)
    Inference-only setting selected by experimentation on the YouTube test scenario to reduce boundary errors.
  • state-priming history length = 10 tokens
    Inference-only setting: prediction network reset at chunk boundary and primed with 10 tokens, giving 7.3% vs 7.5% without priming.
assumptions (5)
  • domain assumption Every output token can be represented by a single encoder frame, with U <= T'
    Eq. (3) forces one-to-one frame-label pairing and the loss ignores frames beyond U; if audio needs multiple frames per token, the model cannot represent the utterance.
  • domain assumption ASR alignment is monotonic and order-preserving
    The diagonal pairing relies on audio-to-text alignment being monotonic; the paper states this and tests only a completely reversed ordering as the extreme case.
  • ad hoc to paper Frame-wise CE on the diagonal is a sufficient training signal to induce internal alignment
    The central design choice (Section 2.1) is not derived from theory; the empirical success is the evidence, so the whole claim rests on this assumption.
  • domain assumption RNN-T lattice alignment is a valid reference for probing Aligner alignment
    Section 4.5.2 trains an RNN-T on frozen Aligner layers and treats its forward-backward probabilities as the alignment yardstick for the self-attention visualization.
  • ad hoc to paper Disabling RNN-T path merging does not materially change the comparison
    Table 5's speed comparison is made with path merging disabled for RNN-T; if this degrades RNN-T beam search quality, the efficiency comparison is incomplete.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Aligner-Encoders: Self-Attention Transformers Can Be Self-Transducers." pith.science (2026). https://pith.science/paper/54IYG2P7

@misc{pith2026250205232,
  author       = {Pith},
  title        = {Pith review of: Aligner-Encoders: Self-Attention Transformers Can Be Self-Transducers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/54IYG2P7}},
  note         = {Machine review of arXiv:2502.05232}
}
read the original abstract

Modern systems for automatic speech recognition, including the RNN-Transducer and Attention-based Encoder-Decoder (AED), are designed so that the encoder is not required to alter the time-position of information from the audio sequence into the embedding; alignment to the final text output is processed during decoding. We discover that the transformer-based encoder adopted in recent years is actually capable of performing the alignment internally during the forward pass, prior to decoding. This new phenomenon enables a simpler and more efficient model, the "Aligner-Encoder". To train it, we discard the dynamic programming of RNN-T in favor of the frame-wise cross-entropy loss of AED, while the decoder employs the lighter text-only recurrence of RNN-T without learned cross-attention -- it simply scans embedding frames in order from the beginning, producing one token each until predicting the end-of-message. We conduct experiments demonstrating performance remarkably close to the state of the art, including a special inference configuration enabling long-form recognition. In a representative comparison, we measure the total inference time for our model to be 2x faster than RNN-T and 16x faster than AED. Lastly, we find that the audio-text alignment is clearly visible in the self-attention weights of a certain layer, which could be said to perform "self-transduction".

Figures

Figures reproduced from arXiv: 2502.05232 by the authors.

Figure 1
Figure 1. Information flow through an Aligner-Encoder versus traditional audio encoders. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Self-attention probabilities from a single head [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Decoding lattice probabilities (U vs T) from RNN￾T-on-Aligner and self-attention weights exhibiting successful alignment, within a specific layer. Using the same diagnostic technique, we can examine the failure mode of poor length generalization [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Decoding lattice probabilities (U vs T) from RNN￾T-on-Aligner and self-attention weights exhibiting a failure mode for an utterance 1.5x longer than trained. We conducted one final experiment to demonstrate the likely applicability of Aligner-Encoders to tasks other th…
Figure 5
Figure 5. Figure 5: Self-attention weights in an Aligner-Encoder (from a single head) trained on reversed audio; [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 43 canonical work pages

  1. [1]

    Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks

    Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd international conference on Machine learning , pages 369–376, 2006

    2006

  2. [2]

    Sequence transduction with recurrent neural networks

    Alex Graves. Sequence transduction with recurrent neural networks. arXiv preprint arXiv:1211.3711, 2012

    arXiv 2012

  3. [3]

    End-to-end con- tinuous speech recognition using attention-based recurrent nn: First results

    Jan Chorowski, Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. End-to-end con- tinuous speech recognition using attention-based recurrent nn: First results. arXiv preprint arXiv:1412.1602, 2014

    arXiv 2014

  4. [4]

    Attention-based models for speech recognition

    Jan K Chorowski, Dzmitry Bahdanau, Dmitriy Serdyuk, Kyunghyun Cho, and Yoshua Bengio. Attention-based models for speech recognition. Advances in neural information processing systems, 28, 2015

    work page 2015

  5. [5]

    Listen, attend and spell

    William Chan, Navdeep Jaitly, Quoc V Le, and Oriol Vinyals. Listen, attend and spell. arXiv preprint arXiv:1508.01211, 2015

    arXiv 2015

  6. [6]

    End-to-end attention-based large vocabulary speech recognition

    Dzmitry Bahdanau, Jan Chorowski, Dmitriy Serdyuk, Philémon Brakel, and Yoshua Bengio. End-to-end attention-based large vocabulary speech recognition. In 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 4945–4949, 2016

    work page 2016

  7. [7]

    Recent advances in end-to-end automatic speech recognition

    Jinyu Li. Recent advances in end-to-end automatic speech recognition. APSIPA Transactions on Signal and Information Processing , 11(1), 2022

    work page 2022

  8. [8]

    Sainath, Ralf Schlüter, and Shinji Watanabe

    Rohit Prabhavalkar, Takaaki Hori, Tara N. Sainath, Ralf Schlüter, and Shinji Watanabe. End-to- end speech recognition: A survey. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 32:325–351, 2024. doi: 10.1109/TASLP.2023.3328283

    work page arXiv 2024

Show all 56 references
  1. [9]

    Efficient implementation of recurrent neural network transducer in tensorflow

    Tom Bagby, Kanishka Rao, and Khe Chai Sim. Efficient implementation of recurrent neural network transducer in tensorflow. In 2018 IEEE Spoken Language Technology Workshop (SLT), pages 506–512, 2018. doi: 10.1109/SLT.2018.8639690

    2018

  2. [10]

    Sainath, and Michiel Bacchiani

    Khe Chai Sim, Arun Narayanan, Tom Bagby, Tara N. Sainath, and Michiel Bacchiani. Improving the efficiency of forward-backward algorithm using batched computation in tensorflow. In 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) , pages 258–264,

    2017

  3. [11]

    Jay Mahadeokar, Yuan Shangguan, Duc Le, Gil Keren, Hang Su, Thong Le, Ching-Feng Yeh, Christian Fuegen, and Michael L. Seltzer. Alignment restricted streaming recurrent neural network transducer. In 2021 IEEE Spoken Language Technology Workshop (SLT), pages 52–59, 2021

    2021

  4. [12]

    Pruned rnn-t for fast, memory-efficient asr training, 2022

    Fangjun Kuang, Liyong Guo, Wei Kang, Long Lin, Mingshuang Luo, Zengwei Yao, and Daniel Povey. Pruned rnn-t for fast, memory-efficient asr training, 2022

    2022

  5. [13]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. 11

    2017

  6. [14]

    Speech-transformer: A no-recurrence sequence-to- sequence model for speech recognition

    Linhao Dong, Shuang Xu, and Bo Xu. Speech-transformer: A no-recurrence sequence-to- sequence model for speech recognition. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 5884–5888, 2018

    2018

  7. [15]

    A comparative study on transformer vs rnn in speech applications

    Shigeki Karita, Nanxin Chen, Tomoki Hayashi, Takaaki Hori, Hirofumi Inaguma, Ziyan Jiang, Masao Someki, Nelson Enrique Yalta Soplin, Ryuichi Yamamoto, Xiaofei Wang, Shinji Watanabe, Takenori Yoshimura, and Wangyou Zhang. A comparative study on transformer vs rnn in speech appl...

    2019

  8. [16]

    Self-attention transducers for end-to-end speech recognition

    Zhengkun Tian, Jiangyan Yi, Jianhua Tao, Ye Bai, and Zhengqi Wen. Self-attention transducers for end-to-end speech recognition. In Interspeech, pages 2019–2023, 2019

    2019

  9. [17]

    Transformer-transducer: End-to- end speech recognition with self-attention

    Ching-Feng Yeh, Jay Mahadeokar, Kaustubh Kalgaonkar, Yongqiang Wang, Duc Le, Mahaveer Jain, Kjell Schubert, Christian Fuegen, and Michael L Seltzer. Transformer-transducer: End-to- end speech recognition with self-attention. arXiv preprint arXiv:1910.12977, 2019

    1910 arXiv

  10. [18]

    Transformer-based acoustic modeling for hybrid speech recognition

    Yongqiang Wang, Abdelrahman Mohamed, Due Le, Chunxi Liu, Alex Xiao, Jay Mahadeokar, Hongzhao Huang, Andros Tjandra, Xiaohui Zhang, Frank Zhang, et al. Transformer-based acoustic modeling for hybrid speech recognition. In ICASSP 2020-2020 IEEE International Conference on Acoust...

    2020

  11. [19]

    Transformer transducer: A streamable speech recognition model with transformer encoders and rnn-t loss

    Qian Zhang, Han Lu, Hasim Sak, Anshuman Tripathi, Erik McDermott, Stephen Koo, and Shankar Kumar. Transformer transducer: A streamable speech recognition model with transformer encoders and rnn-t loss. In ICASSP 2020 - 2020 IEEE International Confer- ence on Acoustics, Speech ...

    2020

  12. [20]

    Conformer: Convolution-augmented transformer for speech recognition

    Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, et al. Conformer: Convolution-augmented transformer for speech recognition. arXiv preprint arXiv:2005.08100, 2020

    2005 arXiv

  13. [21]

    Long short-term memory

    Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8): 1735–1780, 1997

    1997

  14. [22]

    Learning to forget: Continual prediction with lstm

    Felix A Gers, Jürgen Schmidhuber, and Fred Cummins. Learning to forget: Continual prediction with lstm. Neural computation, 12(10):2451–2471, 2000

    2000

  15. [23]

    Re- thinking the inception architecture for computer vision

    Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Re- thinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 2818–2826, 2016

    2016

  16. [24]

    Towards better decoding and language model integration in sequence to sequence models

    Jan Chorowski and Navdeep Jaitly. Towards better decoding and language model integration in sequence to sequence models. arXiv preprint arXiv:1612.02695, 2016

    2016 arXiv

  17. [25]

    Sainath, Yonghui Wu, Rohit Prabhavalkar, Patrick Nguyen, Zhifeng Chen, Anjuli Kannan, Ron J

    Chung-Cheng Chiu, Tara N. Sainath, Yonghui Wu, Rohit Prabhavalkar, Patrick Nguyen, Zhifeng Chen, Anjuli Kannan, Ron J. Weiss, Kanishka Rao, Ekaterina Gonina, Navdeep Jaitly, Bo Li, Jan Chorowski, and Michiel Bacchiani. State-of-the-art speech recognition with sequence-to- sequ...

    2018

  18. [26]

    Exploring architectures, data and units for streaming end-to-end speech recognition with rnn-transducer

    Kanishka Rao, Ha¸ sim Sak, and Rohit Prabhavalkar. Exploring architectures, data and units for streaming end-to-end speech recognition with rnn-transducer. In 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 193–199. IEEE, 2017

    2017

  19. [27]

    Hybrid connectionist models for continuous speech recognition

    Hervé Bourlard and Nelson Morgan. Hybrid connectionist models for continuous speech recognition. In Automatic Speech and Speaker Recognition: Advanced Topics , pages 259–283. Springer, 1996

    1996

  20. [28]

    Monotonic recurrent neural network transducer and decoding strategies

    Anshuman Tripathi, Han Lu, Hasim Sak, and Hagen Soltau. Monotonic recurrent neural network transducer and decoding strategies. In 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 944–948, 2019. doi: 10.1109/ASRU46091.2019. 9003822. 12

    2019

  21. [29]

    Hybrid autoregressive transducer (hat)

    Ehsan Variani, David Rybach, Cyril Allauzen, and Michael Riley. Hybrid autoregressive transducer (hat). In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6139–6143. IEEE, 2020

    2020

  22. [30]

    Factorized neural transducer for efficient language model adaptation

    Xie Chen, Zhong Meng, Sarangarajan Parthasarathy, and Jinyu Li. Factorized neural transducer for efficient language model adaptation. In ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 8132–8136, 2022

    2022

  23. [31]

    Modular hybrid autoregressive transducer

    Zhong Meng, Tongzhou Chen, Rohit Prabhavalkar, Yu Zhang, Gary Wang, Kartik Audhkhasi, Jesse Emond, Trevor Strohman, Bhuvana Ramabhadran, W Ronny Huang, et al. Modular hybrid autoregressive transducer. In 2022 IEEE Spoken Language Technology Workshop (SLT), pages 197–204. IEEE, 2023

    2022

  24. [32]

    Liu, Ron J

    Colin Raffel, Minh-Thang Luong, Peter J. Liu, Ron J. Weiss, and Douglas Eck. Online and linear-time attention by enforcing monotonic alignments, 2017

    2017

  25. [33]

    Cif: Continuous integrate-and-fire for end-to-end speech recognition, 2020

    Linhao Dong and Bo Xu. Cif: Continuous integrate-and-fire for end-to-end speech recognition, 2020

    2020

  26. [34]

    Label-synchronous neural transducer for end-to-end asr

    Keqi Deng and Philip C Woodland. Label-synchronous neural transducer for end-to-end asr. arXiv preprint arXiv:2307.03088, 2023

    2023 arXiv

  27. [35]

    Cif-t: A novel cif-based transducer architecture for automatic speech recognition

    Tian-Hao Zhang, Dinghao Zhou, Guiping Zhong, Jiaming Zhou, and Baoxiang Li. Cif-t: A novel cif-based transducer architecture for automatic speech recognition. In ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 10531–...

    2024

  28. [36]

    Hybrid ctc/attention architecture for end-to-end speech recognition

    Shinji Watanabe, Takaaki Hori, Suyoun Kim, John R Hershey, and Tomoki Hayashi. Hybrid ctc/attention architecture for end-to-end speech recognition. IEEE Journal of Selected Topics in Signal Processing, 11(8):1240–1253, 2017

    2017

  29. [37]

    Hybrid transducer and attention based encoder-decoder modeling for speech-to- text tasks

    Yun Tang, Anna Sun, Hirofumi Inaguma, Xinyue Chen, Ning Dong, Xutai Ma, Paden Tomasello, and Juan Pino. Hybrid transducer and attention based encoder-decoder modeling for speech-to- text tasks. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors, Proceedings of the...

    2023

  30. [38]

    Bayes risk ctc: Controllable ctc alignment in sequence-to-sequence tasks

    Jinchuan Tian, Brian Yan, Jianwei Yu, Chao Weng, Dong Yu, and Shinji Watanabe. Bayes risk ctc: Controllable ctc alignment in sequence-to-sequence tasks. arXiv preprint arXiv:2210.07499, 2022

    2022 arXiv

  31. [39]

    Bayes risk transducer: Transducer with controllable alignment prediction

    Jinchuan Tian, Jianwei Yu, Hangting Chen, Brian Yan, Chao Weng, Dong Yu, and Shinji Watanabe. Bayes risk transducer: Transducer with controllable alignment prediction. arXiv preprint arXiv:2308.10107, 2023

    2023 arXiv

  32. [40]

    Massive end-to-end models for short search queries, 2023

    Weiran Wang, Rohit Prabhavalkar, Dongseong Hwang, Qiujia Li, Khe Chai Sim, Bo Li, James Qin, Xingyu Cai, Adam Stooke, Zhong Meng, CJ Zheng, Yanzhang He, Tara Sainath, and Pedro Moreno Mengibar. Massive end-to-end models for short search queries, 2023

    2023

  33. [41]

    Extreme encoder output frame rate reduction: Improving computational latencies of large end-to-end models

    Rohit Prabhavalkar, Zhong Meng, Weiran Wang, Adam Stooke, Xingyu Cai, Yanzhang He, Arun Narayanan, Dongseong Hwang, Tara N Sainath, and Pedro J Moreno. Extreme encoder output frame rate reduction: Improving computational latencies of large end-to-end models. In ICASSP 2024-202...

    2024

  34. [42]

    Librispeech: an asr corpus based on public domain audio books

    Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. Librispeech: an asr corpus based on public domain audio books. In 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP) , pages 5206–5210. IEEE, 2015

    2015

  35. [43]

    Pseudo label is better than human label

    Dongseong Hwang, Khe Chai Sim, Zhouyuan Huo, and Trevor Strohman. Pseudo label is better than human label. In Hanseok Ko and John H. L. Hansen, editors,Interspeech 2022, 23rd Annual Conference of the International Speech Communication Association, Incheon, Korea, 18-22 Septemb...

    2022 doi

  36. [44]

    Google’s neural machine translation system: Bridging the gap between human and machine translation

    Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144, 2016

    2016 arXiv

  37. [45]

    The implicit length bias of label smoothing on beam search decoding

    Bowen Liang, Pidong Wang, and Yuan Cao. The implicit length bias of label smoothing on beam search decoding. arXiv preprint arXiv:2205.00659, 2022

    2022 arXiv

  38. [46]

    Transformer-xl: Attentive language models beyond a fixed-length context

    Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V Le, and Ruslan Salakhutdinov. Transformer-xl: Attentive language models beyond a fixed-length context. arXiv preprint arXiv:1901.02860, 2019

    1901 arXiv

  39. [47]

    Roformer: Enhanced transformer with rotary position embedding

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063, 2024

    2024

  40. [48]

    Sainath, Patrick Nguyen, Liangliang Cao, and Yonghui Wu

    Chung-Cheng Chiu, Arun Narayanan, Wei Han, Rohit Prabhavalkar, Yu Zhang, Navdeep Jaitly, Ruoming Pang, Tara N. Sainath, Patrick Nguyen, Liangliang Cao, and Yonghui Wu. Rnn-t models fail to generalize to out-of-domain audio: Causes and solutions. In 2021 IEEE Spoken Language Te...

    2021

  41. [49]

    Partially overlapped inference for long-form speech recognition

    Tae Gyoon Kang, Ho-Gyeong Kim, Min-Joong Lee, Jihyun Lee, and Hoshik Lee. Partially overlapped inference for long-form speech recognition. In ICASSP 2021-2021 IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 5989–5993. IEEE, 2021

    2021

  42. [50]

    Neural machine translation by jointly learning to align and translate

    Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In International Conference on Learning Representations , 2015

    2015

  43. [51]

    Listen and translate: A proof of concept for end-to-end speech-to-text translation

    Alexandre Berard, Olivier Pietquin, Christophe Servan, and Laurent Besacier. Listen and translate: A proof of concept for end-to-end speech-to-text translation. In NIPS Workshop on End-to-End Learning for Speech and Audio Processing , 2016

    2016

  44. [52]

    Cross attention augmented transducer networks for simultaneous translation

    Dan Liu, Mengge Du, Xiaoxi Li, Ya Li, and Enhong Chen. Cross attention augmented transducer networks for simultaneous translation. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors,Proceedings of the 2021 Conference on Empirical Methods in N...

    2021

  45. [53]

    Investigating the reordering capability in CTC-based non-autoregressive end-to-end speech translation

    Shun-Po Chuang, Yung-Sung Chuang, Chih-Chiang Chang, and Hung-yi Lee. Investigating the reordering capability in CTC-based non-autoregressive end-to-end speech translation. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli, editors, Findings of the Association for Com...

    2021

  46. [54]

    Large-Scale Streaming End-to-End Speech Translation with Neural Transducers

    Jian Xue, Peidong Wang, Jinyu Li, Matt Post, and Yashesh Gaur. Large-Scale Streaming End-to-End Speech Translation with Neural Transducers. In Proc. Interspeech 2022, pages 3263–3267, 2022

    2022

  47. [55]

    E-branchformer: Branchformer with enhanced merging for speech recognition

    Kwangyoun Kim, Felix Wu, Yifan Peng, Jing Pan, Prashant Sridhar, Kyu J Han, and Shinji Watanabe. E-branchformer: Branchformer with enhanced merging for speech recognition. In 2022 IEEE Spoken Language Technology Workshop (SLT), pages 84–91. IEEE, 2023. 14 A Appendix / suppleme...

    2022

  48. [2017]

    doi: 10.1109/ASRU.2017.8268944

    2017

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.