Delayed Fusion: Integrating Large Language Models into First-Pass Decoding in End-to-end Speech Recognition

Adnan Haider; Erik McDermott; Martin Kocour; Takaaki Hori; Xiaodan Zhuang

REVIEW 3 major objections 5 minor 35 references

Delayed Fusion: Integrating Large Language Models into First-Pass Decoding in End-to-end Speech Recognition

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Delayed fusion lets LLMs cut word error rate 4-13% in decoding.

desk verdict Delayed fusion is a real and useful extension of shallow fusion for LLM-ASR, with credible WER/RTF gains, but the paper never verifies the tokenization-uniqueness assumption that its score approximation rests on. read the letter →

arxiv 2501.09258 v1 pith:U2LX4YDG submitted 2025-01-16 cs.CL cs.SDeess.AS

Takaaki Hori , Martin Kocour , Adnan Haider , Erik McDermott , Xiaodan Zhuang This is my paper · ORCID

classification cs.CLcs.SDeess.AS

keywords delayedfusionlargelanguagemodelspeechrecognitionshallowN-bestrescoringCTCbeamsearchtokenizationmismatchLLM

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces delayed fusion, a way to add a pretrained large language model's scores to a speech recognizer's first-pass beam search without retraining either model. Instead of scoring every partial hypothesis at every step, the decoder waits until after pruning and until a hypothesis has reached a word boundary, then re-tokenizes that boundary prefix and scores all surviving hypotheses in one batched LLM call. This keeps LLM inference calls proportional to the length of the shortest hypothesis rather than to the number of frames, and it lets the ASR and the LLM keep different token vocabularies. On the LibriHeavy corpus, delayed fusion with OpenLLaMA 3B, OpenLLaMA 7B, and Mistral 7B reports word error rates below the baseline and below N-best rescoring, at real-time factors around 0.12–0.17. The practical point: an off-the-shelf LLM can be evaluated or used in a streaming ASR pipeline without retraining.

What carries the argument

The load-bearing mechanism is the fusion-condition gate FUSABLE called inside the beam-search loop, together with the re-tokenization step at word boundaries. After the decoder extends and prunes the hypothesis list to the top K, FUSABLE decides whether to invoke the LLM: shortest-hypothesis fusion fires whenever the shortest re-tokenized prefix among surviving hypotheses grows, and fixed-interval fusion fires every I frames provided at least one hypothesis changed. At each LLM call, the longest prefix ending at a word boundary is re-tokenized with the LLM's SentencePiece tokenizer, and the LLM scores the newly appended tokens in one batched forward pass using a key-value cache from the previous call. This keeps the number of LLM calls at most one per token of the shortest hypothesis and makes N-best rescoring a special case when the gate never fires.

What would settle it

Take a set of decoded hypotheses, and for each word-boundary prefix that triggered delayed fusion, compute the LLM log-probability of that prefix alone and again with the following word included before re-tokenization; if the two log-probabilities differ by more than a small threshold on a substantial fraction of prefixes, the cached-prefix scores used for pruning are not the true conditional LM scores, and the WER comparisons would be affected.

Watch

Extended reading notes

Core claim

Delayed fusion makes the LLM a late but cheap participant in first-pass beam search. The decoder extends and prunes hypotheses as usual; only afterward does a fusion condition decide whether to call the LLM. The shortest-hypothesis trigger fires when the shortest re-tokenized prefix among surviving hypotheses has grown, i.e. $FUSABLE(H_{0:t},t)$ is true when $\varphi(\bar{H}_{t-1}) < \varphi(\bar{H}_t)$, where $\varphi$ is the length of the shortest sequence in the re-tokenized hypothesis list. At each trigger, every surviving hypothesis is re-tokenized at the longest prefix ending in a word-end token, and the LLM logscores the newly added tokens in one batched forward pass, reusing a key-value cache from the previous call: $S_{LM}(h) = \log P_{LM}(\text{prefix}) + \log P_{LM}(\text{new tokens} \mid \text{prefix})$. Because scoring happens after pruning, the number of LLM calls is bounded by the length of the shortest hypothesis, not the number of frames. On LibriHeavy, delayed fusion with OpenLLaMA 3B, OpenLLaMA 7B, and Mistral 7B reports 3.05%/5.68% WER on lh-clean/lh-other at a 0.115 real-time factor for CTC-prefix search, and the paper concludes it gives lower WERs than N-best LLM rescoring and standard NLM fusion.

Load-bearing premise

The method assumes that the way the LLM tokenizer splits a word into pieces does not depend on the words around it, and that the LLM score cached for the shortest prefix stays valid for every beam hypothesis that shares that prefix.

Editorial extensions

If this is right

Delayed fusion can run in streaming first-pass decoding, so live-captioning and other latency-sensitive applications can use LLM scores where N-best rescoring cannot.
The same ASR system can be evaluated against many off-the-shelf LLMs without retraining or fine-tuning, turning LLM comparison in ASR into a drop-in scoring change.
The fusion interval I sets a continuous trade-off between decoding speed and accuracy; at large I the method approaches N-best rescoring, at small I it approaches shallow fusion.
Combining delayed LLM fusion with a small in-domain NLM shallow fusion reduces pruning errors, at the cost of extra NLM computation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the word-boundary uniqueness claim is specific to unigram SentencePiece tokenizers, porting delayed fusion to BPE or WordPiece tokenizers may require a stricter fusion condition to avoid context-dependent tokenizations.
The same gating idea could be applied to RNN-T decoding, where the predictor emits word pieces, allowing LLM fusion at emitted word boundaries without frame-level triggers.
The reported benefit should be interpreted primarily as a pruning-error reduction: with a far larger beam, delayed fusion's advantage over N-best rescoring may shrink, since the first-pass search would already retain the correct hypothesis.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Worth reading if you work on E2E-ASR decoding or LLM fusion. The idea is simple and practical: score partial hypotheses with the LLM only after pruning, at word boundaries, and reuse cached KV states for common prefixes. That reduces LLM calls and handles vocabulary mismatch through re-tokenization. This is a legitimate extension of word-triggered shallow fusion; the controllable fusion interval and batched KV-cached LLM scoring are new enough to count.

What impresses me: the paper does the right comparison. They train a CTC-AED model with the LLM tokenizer and compare delayed fusion against real LLM shallow fusion (Table III). They also compare against N-best rescoring over beam sizes and report consistent WER gains: 4-13% relative over no LM and a smaller but consistent improvement over NLM shallow fusion. The experiments use three public LLMs including Mistral, and the RTF numbers on a V100 are plausible. The authors also honestly note that their WERs cannot be strictly compared to LibriHeavy baselines, and that NLM shallow fusion followed by N-best rescoring is competitive.

Soft spots, in order of importance. First, the central approximation in Section III-B relies on the claim that with a standard SentencePiece tokenizer, 'each word is tokenized into a unique token sequence.' That is not generally guaranteed and is not verified for OpenLLaMA or Mistral. If a word's tokenization depends on neighboring words, the cached prefix score is not the probability of the actual hypothesis, and beam pruning could be biased. The paper should either verify the property for the tokenizers used or quantify the approximation error. This is the one thing that could change the WER conclusions. Second, there are no error bars or significance tests; differences like 3.05 vs 3.08 are treated as meaningful. Third, the streaming claim is not backed by an experiment; all decoding uses full utterance context. Fourth, the RTF comparisons are a little tangled: N-best rescoring with small N can be faster than delayed fusion, so the speed claim should not be phrased unconditionally. Fifth, no code or configs are released, and results come from a single corpus. These are standard weaknesses for a systems paper, not fatal ones.

The citation pattern is fine; the self-citations are to the authors' prior decoding work and are relevant. There is no sign of circular fitting or invented entities.

My take: this deserves a serious referee. I would ask for the tokenization analysis and error bars before accepting, but the method and experiments are solid enough to publish after revision. I would cite it in my own work if I were writing on LLM-ASR fusion.

Referee Report

3 major / 5 minor

Summary. The paper proposes 'delayed fusion,' a first-pass decoding method for end-to-end ASR that integrates a pre-trained LLM by scoring partial hypotheses after beam pruning and at word boundaries. This permits using LLMs whose tokenizer/vocabulary differs from the ASR model without retraining. The method is evaluated on the LibriHeavy corpus with two decoding modes (CTC prefix beam search and joint CTC-attention decoding) and three LLMs (OpenLLaMA 3B/7B, Mistral 7B). The authors report WER and RTF values showing improvements over NLM shallow fusion and N-best LLM rescoring in several configurations, and they argue the approach is suitable for streaming applications.

Significance. If the claims hold, delayed fusion is a practically useful technique: it avoids retraining the ASR model or LLM to reconcile vocabularies, reuses a standard KV cache for efficient LLM scoring, and can operate in streaming-compatible first-pass decoding. The paper is clearly written and the experimental setup is substantial: three public LLMs, two decoding modes, and a 50k-hour corpus. The algorithm is presented in enough detail to be reimplemented, and the authors are transparent about the WER-RTF trade-off in fixed-interval fusion. The main risk is an unverified tokenization-assumption that underwrites the score computation, and the speed comparison with N-best rescoring is less clean than the abstract suggests.

major comments (3)

[III-B] The validity of the approximate LLM score in Section III-B depends on the claim that 'each word is tokenized into a unique token sequence' when a standard SentencePiece tokenizer is used. This property is not guaranteed for the LLMs employed in the experiments (OpenLLaMA and Mistral), especially for byte-level BPE tokenizers where merges can cross whitespace boundaries. If tokenization of a word depends on neighboring words, the concatenation of per-word tokenizations need not equal the tokenization of the full hypothesis prefix, so the cached prefix score log P(v1..v_{\hat M_j}) is the probability of a different string. Because these scores feed directly into pruning (Algorithm 1, line 6) and final hypothesis selection (line 13), an unfaithful SLM can bias the search and compromise the reported WER comparisons. The paper itself acknowledges that 'tokenization may be incorrect for incomplete hypotheses' but does not verify that the word-boundary re-tokenization resolves this for the three LLMs. I ask the authors to empirically measure the mismatch rate between per-word concatenated tokenizations and full-sequence tokenizations on the dev set, and to quantify how often the cached prefix is not a true prefix; if the mismatch is non-negligible, the scoring procedure should be revised.
[IV-B (Table II, Fig. 3) and Abstract] The abstract and the contributions list state that delayed fusion provides 'improved decoding speed ... compared to N-best rescoring.' Table II does not support this as stated: with beam size 10, N-best rescoring (N=10) has RTF 0.029, while fixed-interval delayed fusion ranges from 0.041 to 0.063 and shortest-hypothesis delayed fusion is 0.115. The only evidence for a speed advantage is Figure 3, but neither the text nor the caption reports the underlying numbers or defines how the N-best list size N scales with the beam size. To support the headline speed claim, please provide the full WER-RTF data for the beam-size sweeps in Figure 3 as a table, and qualify the claim to the configurations where delayed fusion is actually faster.
[V] The conclusion claims 'significant WER reduction' and 'lower WERs than N-best LLM rescoring and standard NLM fusion.' The differences in Table II and Figure 3 are small (e.g., 1–3% relative WER in the Fig. 3 comparison), and the paper reports single point estimates without error bars, confidence intervals, or significance tests. With results from a single corpus (LibriHeavy) and a single ASR model family (CTC-AED), it is not possible to assess whether the improvements are stable. Please add utterance-level confidence intervals or a significance test for the main delayed-fusion vs. N-best-rescoring comparisons, or explicitly frame the observed differences as tentative rather than statistically established.

minor comments (5)

[Abstract] In the abstract, 're-tokenizion' is a typo for 're-tokenization'.
[Algorithm 1] The score combination in Algorithm 1 and the selection rule in line 13 are written as SE2E + SLM, but Section IV-A says fusion weights are tuned on the dev set for each LM. Please define the weighted score explicitly (e.g., SE2E + λ SLM) and state how λ is used in the algorithm.
[III-B] In the approximate score equation, \hat M_j is used as the split point, but the text uses j both for the decoding-frame index and the LLM-call index. Please clarify the indexing convention, e.g., denote the LLM-call index by k and define \hat M_k as the shortest sequence length at call k.
[IV-A] The description of the SentencePiece tokenizer for the ASR model does not state whether the unigram or BPE model is used or whether pretokenization is enabled. This is directly relevant to the uniqueness assumption in Section III-B, so please specify.
[Fig. 3] The caption lists curves (c)–(f), but the main text does not explain what each curve represents or give the N-best list sizes used. Please add a description or a table with the plotted values.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity found: delayed fusion's WER claims are empirical comparisons against external benchmarks, not derived from its inputs.

full rationale

The central claim — that delayed fusion gives lower WERs than N-best rescoring and standard NLM fusion — is an experimental result measured on LibriHeavy test sets. The method combines ASR scores from a 101M-parameter CTC-AED model, an in-domain NLM, and three public LLMs without fine-tuning, so no reported accuracy number is produced by fitting an equation to those WERs. The only tunable quantities are LM fusion weights, which are explicitly tuned on the LibriHeavy dev set and then evaluated on test-clean and test-other; this is a standard protocol and does not make the comparison circular. The incremental LLM score formula in Section III-B is a chain-rule decomposition of a joint log-probability using cached prefix scores, not a self-definitional identity or a fitted parameter renamed as a prediction. Self-citations such as joint CTC/attention decoding [32,34], on-the-fly rescoring [19], and word-based RNN LM fusion [22] provide standard background algorithms; the paper's claimed advantage is verified against baselines rather than imported from those citations. The re-tokenization uniqueness assumption for SentencePiece is an external property of a tokenizer; even if it fails for some LLM tokenizers, that would be a correctness or robustness limitation, not an input-output identity or a fitted-input-as-prediction reduction. No circular step can be quoted from the paper, so the appropriate finding is no circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper's central claim depends on tuned hyperparameters and on assumptions about tokenizer uniqueness and cached prefix scores; there are no invented entities.

free parameters (4)

LM fusion weight = Not reported; tuned on LibriHeavy dev for each LM
Shallow and delayed fusion results depend on a scalar weight balancing E2E and LM scores; the paper tunes it on dev but never reports the chosen values.
Fixed fusion interval I = 16, 32, and 64 encoder frames
For fixed-interval delayed fusion, I sets the LLM call cadence and is a user-chosen trade-off knob; WER and RTF are reported at these three values.
Beam size K = 10 for main results; 5, 10, 15, 20 in Figure 3
Beam size is fixed for the headline comparison and swept only in one analysis; accuracy and speed depend on it.
N-best list size N in rescoring baseline = 10
The N-best LLM rescoring baseline is evaluated with N=10; this choice bounds how strong that baseline can be.

assumptions (4)

standard math CTC prefix beam search and joint CTC-attention decoding are correctly used as background algorithms.
Algorithm 1 is stated abstractly and relies on established CTC and attention decoding rules [21,32,34] without re-deriving them.
domain assumption A standard SentencePiece tokenizer gives each word a unique, context-independent token sequence.
Section III-B states that with SentencePiece each word is tokenized into a unique token sequence, which justifies re-tokenizing only the word-length prefix during decoding. If tokenization varies with context, the LLM score assigned to a partial hypothesis may be wrong.
domain assumption LLM scores computed with a KV cache over the shortest hypothesis prefix remain a valid approximation for all hypotheses in the beam.
The incremental scoring equation in Section III-B reuses cached scores from a previous LLM call; this assumes the cached prefix scores stay aligned with each hypothesis's actual history after pruning.
domain assumption General-text public LLM word probabilities improve in-domain ASR accuracy on LibriHeavy.
The entire experimental program assumes that LLM text priors help speech recognition; this is only verified empirically on one corpus and three LLMs.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Delayed Fusion: Integrating Large Language Models into First-Pass Decoding in End-to-end Speech Recognition." pith.science (2026). https://pith.science/paper/U2LX4YDG

@misc{pith2026250109258,
  author       = {Pith},
  title        = {Pith review of: Delayed Fusion: Integrating Large Language Models into First-Pass Decoding in End-to-end Speech Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U2LX4YDG}},
  note         = {Machine review of arXiv:2501.09258}
}

read the original abstract

This paper presents an efficient decoding approach for end-to-end automatic speech recognition (E2E-ASR) with large language models (LLMs). Although shallow fusion is the most common approach to incorporate language models into E2E-ASR decoding, we face two practical problems with LLMs. (1) LLM inference is computationally costly. (2) There may be a vocabulary mismatch between the ASR model and the LLM. To resolve this mismatch, we need to retrain the ASR model and/or the LLM, which is at best time-consuming and in many cases not feasible. We propose "delayed fusion," which applies LLM scores to ASR hypotheses with a delay during decoding and enables easier use of pre-trained LLMs in ASR tasks. This method can reduce not only the number of hypotheses scored by the LLM but also the number of LLM inference calls. It also allows re-tokenizion of ASR hypotheses during decoding if ASR and LLM employ different tokenizations. We demonstrate that delayed fusion provides improved decoding speed and accuracy compared to shallow fusion and N-best rescoring using the LibriHeavy ASR corpus and three public LLMs, OpenLLaMA 3B & 7B and Mistral 7B.

Figures

Figures reproduced from arXiv: 2501.09258 by the authors.

**Figure 1.** Re-tokenization for delayed LLM fusion. With delayed fusion, we can call the LLM at any time during decoding. For efficient LLM computation, we propose (1) shortest-hypothesis fusion and (2) fixed-interval fusion. The shortesthypothesis fusion calls the LLM only when the length of the shortest re-tokenized sequence has increased. The fusion condition is defined as FUSABLE(H0:t, t) = TRUE if φ(H¯t−1) < φ(H¯t) FAL… view at source ↗

**Figure 2.** LLM score computation in decoding. a key-value cache to take full advantage of GPU acceleration [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** compares delayed fusion and N-best rescoring performance for different beam sizes in CTC prefix beam search. The results indicate that delayed fusion, (e) & (f), achieves lower WERs and RTFs than N-best rescoring, (c) & (d) [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 26 canonical work pages

[1]

GPT-4 technical report,

J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al. , “GPT-4 technical report,” arXiv preprint arXiv:2303.08774 , 2023

arXiv 2023
[2]

LLaMA: Open and efficient foundation language models,

H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar et al. , “LLaMA: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023

arXiv 2023
[3]

PaLM: Scal- ing language modeling with pathways,

A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann et al. , “PaLM: Scal- ing language modeling with pathways,” Journal of Machine Learning Research, vol. 24, no. 240, pp. 1–113, 2023

work page 2023
[4]

Mistral 7b,

A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. d. l. Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier et al. , “Mistral 7b,” arXiv preprint arXiv:2310.06825 , 2023

arXiv 2023
[5]

Large language models: A survey,

S. Minaee, T. Mikolov, N. Nikzad, M. Chenaghlu, R. Socher, X. Am- atriain, and J. Gao, “Large language models: A survey,” arXiv preprint arXiv:2402.06196, 2024

arXiv 2024
[6]

Attention is all you need,

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in Proc. NIPS, Los Angeles, CA, Dec. 2017, pp. 5998–6008

work page 2017
[7]

AudioPaLM: A large language model that can speak and listen,

P. K. Rubenstein, C. Asawaroengchai, D. D. Nguyen, A. Bapna, Z. Bor- sos, F. d. C. Quitry, P. Chen, D. E. Badawy, W. Han, E. Kharitonov et al., “AudioPaLM: A large language model that can speak and listen,” arXiv preprint arXiv:2306.12925 , 2023

arXiv 2023
[8]

On decoder-only architecture for speech-to-text and large language model integration,

J. Wu, Y . Gaur, Z. Chen, L. Zhou, Y . Zhu, T. Wang, J. Li, S. Liu, B. Ren, L. Liu et al., “On decoder-only architecture for speech-to-text and large language model integration,” in Proc. IEEE ASRU , 2023, pp. 1–8

work page 2023

Show all 35 references

[9]

SALSA: Speedy asr- llm synchronous aggregation,

A. Mittal, D. Prabhu, S. Sarawagi, and P. Jyothi, “SALSA: Speedy asr- llm synchronous aggregation,” in Proc. Interspeech, Kos, Greece, Sep. 2024, pp. 3485–3489

2024
[10]

LLaMA 2: Open foundation and fine-tuned chat models,

H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale et al. , “LLaMA 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288, 2023

2023 arXiv
[11]

OpenLLaMA: An open reproduction of LLaMA,

X. Geng and H. Liu, “OpenLLaMA: An open reproduction of LLaMA,” May 2023. [Online]. Available: https://github.com/openlm-research/ open llama

2023
[12]

End-to-end speech recognition: A survey,

R. Prabhavalkar, T. Hori, T. N. Sainath, R. Schl ¨uter, and S. Watanabe, “End-to-end speech recognition: A survey,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , 2023

2023
[13]

Advances in joint CTC- attention based end-to-end speech recognition with a deep CNN encoder and RNN-LM,

T. Hori, S. Watanabe, Y . Zhang, and W. Chan, “Advances in joint CTC- attention based end-to-end speech recognition with a deep CNN encoder and RNN-LM,” in Proc. Interspeech, Aug. 2017

2017
[14]

A comparison of techniques for language model integration in encoder-decoder speech recognition,

S. Toshniwal, A. Kannan, C.-C. Chiu, Y . Wu, T. N. Sainath, and K. Livescu, “A comparison of techniques for language model integration in encoder-decoder speech recognition,” in Proc. IEEE SLT , Athens, Greece, Dec. 2018, pp. 369–375

2018
[15]

On language model integration for RNN transducer based speech recognition,

W. Zhou, Z. Zheng, R. Schl ¨uter, and H. Ney, “On language model integration for RNN transducer based speech recognition,” in Proc. IEEE ICASSP, Singapore, May 2022, pp. 8407–8411

2022
[16]

Cold fusion: Training seq2seq models together with language models,

A. Sriram, H. Jun, S. Satheesh, and A. Coates, “Cold fusion: Training seq2seq models together with language models,” in Proc. Interspeech , Hyderabad, India, Sep. 2018, pp. 387–391

2018
[17]

Simple fusion: Return of the language model,

F. Stahlberg, J. Cross, and V . Stoyanov, “Simple fusion: Return of the language model,” in WMT 2018, Belgium, Brussels, Oct. 2018, pp. 204– 211

2018
[18]

Internal language model estimation for domain-adaptive end-to-end speech recognition,

Z. Meng, S. Parthasarathy, E. Sun, Y . Gaur, N. Kanda, L. Lu, X. Chen, R. Zhao, J. Li, and Y . Gong, “Internal language model estimation for domain-adaptive end-to-end speech recognition,” in Proc. IEEE SLT , Shenzhen, China, Dec. 2020, pp. 243–250

2020
[19]

Efficient WFST-based one-pass decoding with on-the-fly hypothesis rescoring in extremely large vocabulary continuous speech recognition,

T. Hori, C. Hori, Y . Minami, and A. Nakamura, “Efficient WFST-based one-pass decoding with on-the-fly hypothesis rescoring in extremely large vocabulary continuous speech recognition,” IEEE Transactions on audio, speech, and language processing , vol. 15, no. 4, pp. 1352–1365, 2007

2007
[20]

On-the-fly lattice rescoring for real- time automatic speech recognition,

H. Sak, M. Saraclar, and T. G ¨ung¨or, “On-the-fly lattice rescoring for real- time automatic speech recognition,” in Proc. Interspeech, Sep. 2010, pp. 2450–2453

2010
[21]

First-pass large vocabulary continuous speech recognition using bi-directional recurrent DNNs,

A. Y . Hannun, A. L. Maas, D. Jurafsky, and A. Y . Ng, “First-pass large vocabulary continuous speech recognition using bi-directional recurrent DNNs,” Dec. 2014, arXiv:1408.2873

2014 arXiv
[22]

End-to-end speech recognition with word-based RNN language models,

T. Hori, J. Cho, and S. Watanabe, “End-to-end speech recognition with word-based RNN language models,” in Proc. IEEE SLT, Athens, Greece, Dec. 2018, pp. 389–396

2018
[23]

An empirical study of efficient ASR rescoring with transformers,

H. Huang and F. Peng, “An empirical study of efficient ASR rescoring with transformers,” arXiv preprint arXiv:1910.11450 , 2019

1910 arXiv
[24]

RescoreBERT: Discriminative speech recognition rescoring with BERT,

L. Xu, Y . Gu, J. Kolehmainen, H. Khan, A. Gandhe, A. Rastrow, A. Stol- cke, and I. Bulyko, “RescoreBERT: Discriminative speech recognition rescoring with BERT,” in Proc. IEEE ICASSP , Singapore, May 2022, pp. 6117–6121

2022
[25]

Multilingual and fully non- autoregressive asr with large language model fusion: A comprehensive study,

W. R. Huang, C. Allauzen, T. Chen, K. Gupta, K. Hu, J. Qin, Y . Zhang, Y . Wang, S.-Y . Chang, and T. N. Sainath, “Multilingual and fully non- autoregressive asr with large language model fusion: A comprehensive study,” in Proc. IEEE ICASSP . IEEE, 2024, pp. 13 306–13 310

2024
[26]

Effect and analysis of large-scale language model rescoring on competitive asr systems,

T. Udagawa, M. Suzuki, G. Kurata, N. Itoh, and G. Saon, “Effect and analysis of large-scale language model rescoring on competitive asr systems,” in Proc. Interspeech , Incheon, Korea, Sep. 2022, pp. 3919– 3923

2022
[27]

Generative speech recognition error correction with large language models and task-activating prompting,

C.-H. H. Yang, Y . Gu, Y .-C. Liu, S. Ghosh, I. Bulyko, and A. Stolcke, “Generative speech recognition error correction with large language models and task-activating prompting,” in Proc. IEEE ASRU , Taipei, Dec. 2023, pp. 1–8

2023
[28]

N-best T5: Robust asr error correction using multiple input hypotheses and constrained decoding space,

R. Ma, M. J. Gales, K. M. Knill, and M. Qian, “N-best T5: Robust asr error correction using multiple input hypotheses and constrained decoding space,” in Proc. Interspeech, Aug. 2023

2023
[29]

Connectionist temporal classification: Labelling unsegmented sequence data with re- current neural networks,

A. Graves, S. Fern ´andez, F. Gomez, and J. Schmidhuber, “Connectionist temporal classification: Labelling unsegmented sequence data with re- current neural networks,” in Proc. ICML, Pittsburgh, PA, Jun. 2006, pp. 369–376

2006
[30]

Sentencepiece: A simple and language inde- pendent subword tokenizer and detokenizer for neural text processing,

T. Kudo and J. Richardson, “Sentencepiece: A simple and language inde- pendent subword tokenizer and detokenizer for neural text processing,” Proc. EMNLP, p. 66, 2018

2018
[31]

Libriheavy: a 50,000 hours asr corpus with punctuation casing and context,

W. Kang, X. Yang, Z. Yao, F. Kuang, Y . Yang, L. Guo, L. Lin, and D. Povey, “Libriheavy: a 50,000 hours asr corpus with punctuation casing and context,” in Proc. IEEE ICASSP , 2024, pp. 10 991–10 995

2024
[32]

Hybrid CTC/attention architecture for end-to-end speech recognition,

S. Watanabe, T. Hori, S. Kim, J. R. Hershey, and T. Hayashi, “Hybrid CTC/attention architecture for end-to-end speech recognition,” IEEE Journal of Selected Topics in Signal Processing , vol. 11, no. 8, pp. 1240– 1253, 2017

2017
[33]

WeNet: Production oriented streaming and non- streaming end-to-end speech recognition toolkit,

Z. Yao, D. Wu, X. Wang, B. Zhang, F. Yu, C. Yang, Z. Peng, X. Chen, L. Xie, and X. Lei, “WeNet: Production oriented streaming and non- streaming end-to-end speech recognition toolkit,” in Proc. Interspeech, Brno, Czechia, Sep. 2021, pp. 4054–4058

2021
[34]

Joint CTC/attention decoding for end-to-end speech recognition,

T. Hori, S. Watanabe, and J. Hershey, “Joint CTC/attention decoding for end-to-end speech recognition,” in Proc. ACL, Vancouver, BC, Canada, Jul. 2017, pp. 518–529

2017
[35]

Sequence transduction with recurrent neural networks,

A. Graves, “Sequence transduction with recurrent neural networks,” in Proc. ICML, Edinburgh, Scotland, Jun. 2012

2012

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

GPT-4 technical report,

J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al. , “GPT-4 technical report,” arXiv preprint arXiv:2303.08774 , 2023

arXiv 2023

[2] [2]

LLaMA: Open and efficient foundation language models,

H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar et al. , “LLaMA: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023

arXiv 2023

[3] [3]

PaLM: Scal- ing language modeling with pathways,

A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann et al. , “PaLM: Scal- ing language modeling with pathways,” Journal of Machine Learning Research, vol. 24, no. 240, pp. 1–113, 2023

work page 2023

[4] [4]

Mistral 7b,

A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. d. l. Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier et al. , “Mistral 7b,” arXiv preprint arXiv:2310.06825 , 2023

arXiv 2023

[5] [5]

Large language models: A survey,

S. Minaee, T. Mikolov, N. Nikzad, M. Chenaghlu, R. Socher, X. Am- atriain, and J. Gao, “Large language models: A survey,” arXiv preprint arXiv:2402.06196, 2024

arXiv 2024

[6] [6]

Attention is all you need,

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in Proc. NIPS, Los Angeles, CA, Dec. 2017, pp. 5998–6008

work page 2017

[7] [7]

AudioPaLM: A large language model that can speak and listen,

P. K. Rubenstein, C. Asawaroengchai, D. D. Nguyen, A. Bapna, Z. Bor- sos, F. d. C. Quitry, P. Chen, D. E. Badawy, W. Han, E. Kharitonov et al., “AudioPaLM: A large language model that can speak and listen,” arXiv preprint arXiv:2306.12925 , 2023

arXiv 2023

[8] [8]

On decoder-only architecture for speech-to-text and large language model integration,

J. Wu, Y . Gaur, Z. Chen, L. Zhou, Y . Zhu, T. Wang, J. Li, S. Liu, B. Ren, L. Liu et al., “On decoder-only architecture for speech-to-text and large language model integration,” in Proc. IEEE ASRU , 2023, pp. 1–8

work page 2023