Pith. sign in

REVIEW 3 major objections 7 minor 1 cited by

Serialized Output Prompting for Large Language Model-based Multi-Talker Speech Recognition

T0 review · 3 major / 7 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A rough per-speaker transcript cuts 3-talker speech errors by a quarter

desk verdict Useful and consistent gains, but the headline effect is confounded by an extra adaptation stage, so the prompt's own contribution is not actually isolated. read the letter →

arxiv 2509.04488 v1 pith:SBEHIO4J submitted 2025-09-01 cs.CL cs.AIcs.SDeess.AS

Hao Shi , Yusuke Fujita , Tomoya Mizumoto , Lianbo Liu , Atsushi Kojima , Yui Sudo This is my paper · ORCID
classification cs.CLcs.AIcs.SDeess.AS
keywords multi-talkerASRserializedoutputtrainingconnectionisttemporalclassificationpromptinglargelanguagemodelsLibriMixspeechrecognitionLLMdecoderadaptation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a method called serialized output prompting (SOP) for large language model-based multi-talker speech recognition. Instead of giving the LLM decoder only a static task prompt, the system uses per-talker CTC decoders to write a rough serialized transcript of who said what, in speaking order, and feeds that text into the LLM as a guiding prompt. Three training stages are used: SOT fine-tuning, separator and CTC extraction, then SOP adaptation with a fresh LoRA adapter. On LibriMix, the full system improves word error rate over the SOT baseline under both two-talker and three-talker conditions, with the largest relative gains on three-talker clean speech. Prompting is a cheap and largely unexplored lever for making LLM decoders handle overlapping speech.

What carries the argument

The central object is the serialized output prompt (SOP). An LSTM Separator and per-talker CTC heads are inserted after the speech encoder; each CTC branch is aligned to a talker in first-speaking-first-out order, and greedy decoding produces a rough per-talker transcript. These transcripts are concatenated with speaker-change tokens into SOP, embedded, and concatenated to the speech features to form the LLM input. The claimed mechanism is that SOP supplies explicit, acoustically grounded serialization cues, so the LLM does not have to infer overlap structure from acoustic features alone.

What would settle it

Run the same three-stage recipe but feed a fixed task prompt, or no prompt, during stage 3 while keeping the new LoRA adaptation; if WER matches the SOP result, the prompt content is not the cause. Alternatively, feed SOP into the stage-1 model without the extra LoRA adaptation and check whether WER still drops.

Watch

Extended reading notes

Core claim

This paper claims that an LLM-based multi-talker ASR system can be improved by prompting the decoder with a serialized output prompt (SOP): a rough text draft of each talker's speech in speaking order, produced by greedy decoding per-talker CTC branches. The authors argue that the serialized CTC outputs provide complete, well-aligned speech content for each talker (Figures 2 and 3), and that explicitly providing these guiding cues helps LLM decoding (Section IV-C). Their evidence is the WER comparison in Table I: on the 3B decoder model, SOP-MT-ASR reduces Libri3Mix clean evaluation WER from 22.0% to 16.5% and noisy evaluation WER from 31.7% to 28.1%, with smaller but consistent gains on Lib

Load-bearing premise

The reported gains are credited to the SOP prompt, but the main comparison adds both the prompt and a new LoRA adaptation stage at once; the paper never runs a stage-3 control with the adaptation but without the SOP to prove the prompt itself is the cause.

Editorial extensions

If this is right

  • Three-talker recognition, the setting where a plain SOT-trained LLM degrades, is where SOP gives the largest relative gain: clean Libri3Mix evaluation WER drops from 22.0% to 16.5%.
  • SOP is cheap at inference: it is produced by greedy CTC decoding from a side branch, so the heavy LLM is simply being fed better context.
  • The three-stage recipe is load-bearing: training everything jointly (ID-0) or stopping after SOT-CTC produces worse WER than the SOP stage.
  • Removing the mixed speech encoding while keeping only the SOP text is not viable; the acoustic path stays necessary.
  • The improvement is consistent across 1B, 3B, and 8B LLM decoders, though the 3B model gives the best absolute results in these experiments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • I infer the SOP is a general prompting recipe: a lighter auxiliary ASR that writes a per-speaker draft could be bolted onto any promptable LLM decoder, not just this SOT-CTC architecture.
  • The paper leaves open whether the stage-3 LoRA adaptation alone, without the SOP text, would account for some of the gain; a no-prompt control would test this directly.
  • Because the paper shows errors concentrate in high-overlap and speaker-switching frames, improving the CTC draft in those frames should directly enlarge the prompt's benefit.
  • The 8B model's underperformance relative to 3B suggests data-limited adaptation; more SOT data or a different adapter scheme might let larger decoders win.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. This paper proposes SOP-MT-ASR, a three-stage training framework for LLM-based multi-talker ASR. A WavLM encoder is followed by a Separator and S serialized CTC branches; greedy decoding of these branches yields a Serialized Output Prompt (SOP) that is concatenated with the projected mixture encoding and text embedding as the LLM decoder input. Stage 1 fine-tunes the SOT baseline, Stage 2 trains the encoder/separator/CTC branches with a CTC+CE loss, and Stage 3 adapts a fresh LoRA module while freezing the rest. On Libri2Mix/Libri3Mix with LLaMA-3.2-1B/3B/LLaMA-3.1-8B, the reported Stage-3 system improves WER over the Stage-1 SOT baseline under both noisy and clean conditions. The paper concludes that SOP assists LLM decoding. The main empirical evidence is a comparison of Table I's Stage-1/2 rows with Stage-3 SOP rows.

Significance. If the effect were isolated, SOP would be a low-cost way to inject serialized acoustic-text hypotheses into an LLM decoder and could be useful for LLM-based MT-ASR. Strengths: systematic evaluation across three LLM sizes, two overlap conditions, and noisy/clean inputs; explicit architecture and loss equations; honest discussion of CTC branch errors. However, the central claim is currently underdetermined: the reported gain is not attributable to the SOP text because the Stage-3 condition also introduces new LoRA adaptation. The missing control is simple, and the paper would be much stronger with it. Reproducibility is also handicapped by absent training hyperparameters.

major comments (3)
  1. [Section III-B3 / Table I] The central claim in Sec. IV.C ('SOP assists LLM decoding') is based on comparisons such as Table I row 6 (SOT-CTC, input [Hp;Et]) vs row 7 (SOP, input [Esop;Hp;Et]) for 3B Libri3Mix (eval 30.7 -> 28.1) and row 5 vs row 7. These conditions differ in two ways: the SOP token sequence Esop is added, and Stage 3 trains an additional set of LoRA adapters (Sec. III-B3). Since Stage 2 is reported to degrade several conditions (e.g., 3B clean Libri3Mix eval 22.0 -> 23.4), the Stage-3 improvement may reflect recovery from that degradation through additional adaptation, independent of prompt content. A minimal control is a Stage-3 LoRA-adapted model with input [Hp;Et] and no Esop. Without this control the improvement cannot be assigned to SOP.
  2. [Section IV.C, Table I rows 4/8/12] The ablation labeled '– Mixed speech encoding' removes Hp, not Esop, so it changes the acoustic input and leaves the model with only text-like prompt embeddings; the catastrophic WER (e.g., row 8, Libri3Mix eval 154.0) is expected and says nothing about whether Esop contributes. To show the prompt content matters, the paper should compare identical Stage-3 adaptation with and without Esop and, ideally, with a mismatched or random SOP sequence. The current design cannot rule out that any extra token sequence plus adaptation yields the gain.
  3. [Section IV.B / IV.C] The experimental section omits nearly all training hyperparameters (optimizer, learning rate, batch size, epochs, LoRA rank/alpha, number of CTC branches, and the value of alpha in Eq. (10)). The 'underlined p<0.05' results are not accompanied by a description of the significance test, the unit of analysis, or the number of utterances. The three-speaker offset files are withheld until after review. These omissions make the experiments non-reproducible and prevent assessment of the variability behind the claimed gains.
minor comments (7)
  1. [Abstract / Introduction] The claim that there is 'no prior work exploring the design of prompts' is contradicted by reference [3], which uses versatile instructions. Recommend rephrasing to 'no prior work using decoded serialized output as a prompt'.
  2. [Section III.A] The number of talkers S is assumed known and fixed, since the separator and CTC branches are built for a predetermined S. This limitation should be stated explicitly, especially because multi-talker ASR is often evaluated with variable numbers of speakers.
  3. [Section III.B] Stage 1 says LoRA weights are merged into the LLM, but Stage 3 introduces another LoRA. Clarify whether the Stage-1 adapters remain merged and how the Stage-3 adapters are initialized and combined with the merged weights.
  4. [Section III.B2] The sentence 'The training loss is defined as in Eqn. (10), which is applied not only to the CTC branch, but also to the LLM output' is ambiguous: L_CTC-EncSep uses H_s_sep and T_s, while L_SOT uses Te. Spell out which gradients flow through the LLM in Stage 2 and whether the LLM is frozen.
  5. [Section IV.C / Figs. 2-3] The examples in Figs. 2 and 3 are only two cases; the statement that 'the overall output quality ... provides complete and well-aligned speech content' needs a quantitative CTC-branch WER or accuracy metric, especially because the shown examples contain errors in high-overlap frames.
  6. [Table II] On clean Libri3Mix, the proposed method remains worse than the non-LLM GEncSep baseline (3B eval 16.5 vs 13.1). The conclusion should carefully phrase the contribution as an improvement over the SOT-LLM baseline, not over prior MT-ASR systems.
  7. [Notation / References] Hcon is used for the concatenated separated embeddings in Eq. (11) and again for the decoder input in Eq. (22), which is actually Hsop; please avoid the overloaded notation. Also, 'Fig. 1–(Stage 3)' should be written as 'Fig. 1 (Stage 3)'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: SOP is an auxiliary CTC-derived prompt, and the reported WER gains are empirical; the main concern (Stage-3 LoRA confound) is experimental attribution, not a definitional reduction.

full rationale

The paper's derivation chain is not circular. Nothing is predicted from a fitted parameter or defined in terms of its own output: the SOP is generated by greedy decoding of serialized CTC branches (Eqs. 18-20), embedded, and concatenated with speech and text embeddings as the decoder input (Eq. 22). The claim that 'SOP assists LLM decoding' is an empirical comparison on held-out LibriMix sets (Table I), not a reduction. The CTC branches share the same reference transcripts as the LLM decoder (Eqs. 9 and 6), but that is a design choice about auxiliary supervision, not circularity: the prompt is not an oracle, is produced from the test audio, and the ablation that removes the mixed speech encoding (rows 4/8/12) collapses performance, showing the main recognition information still comes from the speech encoder. The most serious issue is an experimental confound, not circularity: rows 6 vs 7 differ both by adding Esop and by adding a Stage-3 LoRA adaptation, and there is no Stage-3 no-prompt control, so the gain cannot be unambiguously attributed to the prompt. That is a validity concern; it does not reduce an equation to its input. Self-citations to [2] supply the Separator architecture, but the current results are new experiments and the citation is not invoked to forbid alternatives or to carry a uniqueness proof. Therefore the circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No physical or external entities are introduced. The SOP is an engineered text prompt, and the Separator is inherited from ref [2], so there are no new postulated entities requiring independent evidence. The central claim rests on empirical setup rather than derivation, on pretrained components, on a fixed talker count, and on a training recipe whose hyperparameters are largely unreported.

free parameters (4)
  • alpha (loss weighting) = not reported
    Explicit tunable hyperparameter in Eq. 10 balancing CTC-EncSep and SOT losses; no value is given, so it is a free parameter.
  • number of CTC branches = 2 or 3
    Set equal to the known number of talkers; the architecture cannot handle a variable talker count, and the evaluation never tests that.
  • LoRA or adaptation configuration = not reported
    Stage 3 LoRA rank, learning rate, epochs, and adapter placement are not specified; WER depends on these choices.
  • downsampling and separator input choice = three 2x CNN layers, separator on H(2)
    Hand-selected architecture decisions stated in Section III.A with no ablation; they affect prompt quality and compute.
assumptions (4)
  • domain assumption Pretrained WavLM-Large and LLaMA models provide suitable speech and text representations for this task.
    The system is built on these backbones; their suitability for LibriMix mixtures is assumed from pretraining rather than demonstrated here.
  • ad hoc to paper The number of talkers S is known and fixed.
    The architecture allocates exactly S CTC branches; no mechanism handles unknown or varying talker counts.
  • domain assumption The serialized CTC branches recover speaker order and content well enough to serve as a useful prompt.
    The paper shows the CTC hypotheses contain errors in overlapping frames (Figs. 2/3), yet assumes the prompt still guides the LLM beneficially.
  • ad hoc to paper The three-stage training strategy is necessary and the CTC-induced degradation is resolved in stage 3.
    This is an empirical claim in Section III.B/IV.C; no comparison with alternative training orders or joint training is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Serialized Output Prompting for Large Language Model-based Multi-Talker Speech Recognition." pith.science (2026). https://pith.science/paper/SBEHIO4J

@misc{pith2026250904488,
  author       = {Pith},
  title        = {Pith review of: Serialized Output Prompting for Large Language Model-based Multi-Talker Speech Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SBEHIO4J}},
  note         = {Machine review of arXiv:2509.04488}
}
read the original abstract

Prompts are crucial for task definition and for improving the performance of large language models (LLM)-based systems. However, existing LLM-based multi-talker (MT) automatic speech recognition (ASR) systems either omit prompts or rely on simple task-definition prompts, with no prior work exploring the design of prompts to enhance performance. In this paper, we propose extracting serialized output prompts (SOP) and explicitly guiding the LLM using structured prompts to improve system performance (SOP-MT-ASR). A Separator and serialized Connectionist Temporal Classification (CTC) layers are inserted after the speech encoder to separate and extract MT content from the mixed speech encoding in a first-speaking-first-out manner. Subsequently, the SOP, which serves as a prompt for LLMs, is obtained by decoding the serialized CTC outputs using greedy search. To train the model effectively, we design a three-stage training strategy, consisting of serialized output training (SOT) fine-tuning, serialized speech information extraction, and SOP-based adaptation. Experimental results on the LibriMix dataset show that, although the LLM-based SOT model performs well in the two-talker scenario, it fails to fully leverage LLMs under more complex conditions, such as the three-talker scenario. The proposed SOP approach significantly improved performance under both two- and three-talker conditions.

Figures

Figures reproduced from arXiv: 2509.04488 by the authors.

Figure 1
Figure 1. The flowchart of the proposed SOP for LLM-based MT-ASR. It contains three training stages: (1) fine-tuning with SOT, (2) serialized speech [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. One example of the SOP content extracted using serialized CTC layers under the two-talker condition. The [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. One example of the SOP content extracted using serialized CTC layers under the three-talker condition. The [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Beyond Acoustic Prefixes: Persistent Grounding in Serialized Acoustic Memory for LLM-Based Multi-Talker Speech Recognition

    cs.SD 2026-03 accept novelty 6.0 of 10

    Persistent gated residual cross-attention over onset-ordered talker acoustic memory, refined with LoRA, substantially improves LLM-SOT multi-talker ASR especially on three-talker mixtures.

Reference graph

Works this paper leans on

43 extracted references · 36 canonical work pages · cited by 1 Pith paper

  1. [3]

    Large language model can transcribe speech in multi-talker scenarios with versatile instructions,

    L. Meng, S. Hu, J. Kang, Z. Li, Y . Wang, W. Wu, X. Wu, X. Liu, and H. Meng, “Large language model can transcribe speech in multi-talker scenarios with versatile instructions,” in ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2025, pp. 1–5

    work page 2025

  2. [1]

    Advancing multi-talker ASR performance with large lan- guage models,

    M. Shi, Z. Jin, Y . Xu, Y . Xu, S.-X. Zhang, K. Wei, Y . Shao, C. Zhang, and D. Yu, “Advancing multi-talker ASR performance with large lan- guage models,” in 2024 IEEE Spoken Language Technology Workshop (SLT), 2024, pp. 14–21

    work page 2024

  3. [2]

    Serialized speech informa- tion guidance with overlapped encoding separation for multi-speaker automatic speech recognition,

    H. Shi, Y . Gao, Z. Ni, and T. Kawahara, “Serialized speech informa- tion guidance with overlapped encoding separation for multi-speaker automatic speech recognition,” in IEEE Spoken Language Technology Workshop (SLT), 2024, pp. 198–204

    work page 2024

  4. [4]

    End-to-end monaural multi-speaker ASR system without pretraining,

    X. Chang, Y . Qian, K. Yu, and S. Watanabe, “End-to-end monaural multi-speaker ASR system without pretraining,” in ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019, pp. 6256–6260

    work page 2019

  5. [5]

    End- to-end multi-speaker speech recognition,

    S. Settle, J. L. Roux, T. Hori, S. Watanabe, and J. R. Hershey, “End- to-end multi-speaker speech recognition,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018, pp. 4819–4823

    work page 2018

  6. [6]

    Adaptive attention network with domain adversarial training for multi- accent speech recognition,

    Y . Yang, H. Shi, Y . Lin, M. Ge, L. Wang, Q. Hou, and J. Dang, “Adaptive attention network with domain adversarial training for multi- accent speech recognition,” in 2022 13th International Symposium on Chinese Spoken Language Processing (ISCSLP) , 2022, pp. 6–10

    work page 2022

  7. [7]

    Waveform-domain speech enhancement using spectrogram encoding for robust speech recognition,

    H. Shi, M. Mimura, and T. Kawahara, “Waveform-domain speech enhancement using spectrogram encoding for robust speech recognition,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 32, pp. 3049–3060, 2024

    work page 2024

  8. [8]

    Spectrograms fusion-based end-to-end robust automatic speech recognition,

    H. Shi, L. Wang, S. Li, C. Fan, J. Dang, and T. Kawahara, “Spectrograms fusion-based end-to-end robust automatic speech recognition,” in 2021 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) , 2021, pp. 438–442

    work page 2021

Show all 43 references
  1. [9]

    Large language model based generative error correc- tion: A challenge and baselines for speech recognition, speaker tagging, and emotion recognition,

    C.-H. H. Yang, T. Park, Y . Gong, Y . Li, Z. Chen, Y .-T. Lin, C. Chen, Y . Hu, K. Dhawan, P.˙Zelasko, C. Zhang, Y .-N. Chen, Y . Tsao, J. Balam, B. Ginsburg, S. M. Siniscalchi, E. S. Chng, P. Bell, C. Lai, S. Watanabe, and A. Stolcke, “Large language model based generative er...

    2024

  2. [10]

    Error correction by paying attention to both acoustic and confidence references for automatic speech recognition,

    Y . Shu, B. Hu, Y . He, H. Shi, L. Wang, and J. Dang, “Error correction by paying attention to both acoustic and confidence references for automatic speech recognition,” in Interspeech 2024, 2024, pp. 3500–3504

    2024

  3. [11]

    Language-specific Characteristic Assistance for Code-switching Speech Recognition,

    T. Song, Q. Xu, M. Ge, L. Wang, H. Shi, Y . Lv, Y . Lin, and J. Dang, “Language-specific Characteristic Assistance for Code-switching Speech Recognition,” in Interspeech 2022, 2022, pp. 3924–3928

    2022

  4. [12]

    Adapting whisper for code-switching through encoding refining and language-aware decoding,

    J. Zhao, H. Shi, C. Cui, T. Wang, H. Liu, Z. Ni, L. Ye, and L. Wang, “Adapting whisper for code-switching through encoding refining and language-aware decoding,” in ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2025, pp. 1–5

    2025

  5. [13]

    Investigation of speech separation as a front-end for noise robust speech recognition,

    A. Narayanan and D. Wang, “Investigation of speech separation as a front-end for noise robust speech recognition,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 22, no. 4, pp. 826– 835, 2014

    2014

  6. [14]

    A separation priority pipeline for single-channel speech separation in noisy environments,

    S. Dang, T. Matsumoto, Y . Takeuchi, and H. Kudo, “A separation priority pipeline for single-channel speech separation in noisy environments,” in ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2024, pp. 12 511–12 515

    2024

  7. [15]

    Supervised speech separation based on deep learning: An overview,

    D. Wang and J. Chen, “Supervised speech separation based on deep learning: An overview,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 26, no. 10, pp. 1702–1726, 2018

    2018

  8. [16]

    Multitalker speech separation with utterance-level permutation invariant training of deep recurrent neural networks,

    M. Kolbæk, D. Yu, Z.-H. Tan, and J. Jensen, “Multitalker speech separation with utterance-level permutation invariant training of deep recurrent neural networks,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 25, no. 10, pp. 1901–1913, 2017

    1901

  9. [17]

    End-to-end multi-talker overlapping speech recognition,

    A. Tripathi, H. Lu, and H. Sak, “End-to-end multi-talker overlapping speech recognition,” in ICASSP 2020 - 2020 IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP) , 2020, pp. 6129–6133

    2020

  10. [18]

    Permutation invariant training of deep models for speaker-independent multi-talker speech sep- aration,

    D. Yu, M. Kolbæk, Z.-H. Tan, and J. Jensen, “Permutation invariant training of deep models for speaker-independent multi-talker speech sep- aration,” in 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2017, pp. 241–245

    2017

  11. [19]

    Serialized Output Training for End-to-End Overlapped Speech Recognition,

    N. Kanda, Y . Gaur, X. Wang, Z. Meng, and T. Yoshioka, “Serialized Output Training for End-to-End Overlapped Speech Recognition,” in Interspeech 2020, 2020, pp. 2797–2801

    2020

  12. [20]

    Roberta: A robustly optimized bert pretraining approach,

    Y . Liu, “Roberta: A robustly optimized bert pretraining approach,” arXiv preprint arXiv:1907.11692, vol. 364, 2019

    1907 arXiv

  13. [21]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in North American Association for Computational Linguistics (NAACL) , 2019

    2019

  14. [22]

    Improving language understanding with unsupervised learning,

    A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever, “Improving language understanding with unsupervised learning,” Technical report, OpenAI, 2018

    2018

  15. [23]

    BART: Denoising sequence-to- sequence pre-training for natural language generation, translation, and comprehension,

    M. Lewis, Y . Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V . Stoyanov, and L. Zettlemoyer, “BART: Denoising sequence-to- sequence pre-training for natural language generation, translation, and comprehension,” in Proceedings of the 58th Annual Meeting of the Associat...

    2020

  16. [24]

    Xlnet: Generalized autoregressive pretraining for language understanding,

    Z. Yang, “Xlnet: Generalized autoregressive pretraining for language understanding,” arXiv preprint arXiv:1906.08237 , 2019

    1906 arXiv

  17. [25]

    WavLLM: Towards robust and adaptive speech large language model,

    S. Hu, L. Zhou, S. Liu, S. Chen, L. Meng, H. Hao, J. Pan, X. Liu, J. Li, S. Sivasankaran, L. Liu, and F. Wei, “WavLLM: Towards robust and adaptive speech large language model,” in Findings of the Association for Computational Linguistics: EMNLP 2024 , 2024, pp. 4552–4572

    2024

  18. [26]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar et al. , “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  19. [27]

    Language mod- els are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., “Language mod- els are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020

    1901

  20. [28]

    End-to-end continuous speech recognition using attention-based recurrent NN: First results,

    J. Chorowski, D. Bahdanau, K. Cho, and Y . Bengio, “End-to-end continuous speech recognition using attention-based recurrent NN: First results,” in NIPS 2014 Workshop on Deep Learning , 2014

    2014

  21. [29]

    Listen, attend and spell: A neural network for large vocabulary conversational speech recognition,

    W. Chan, N. Jaitly, Q. Le, and O. Vinyals, “Listen, attend and spell: A neural network for large vocabulary conversational speech recognition,” in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2016, pp. 4960–4964

    2016

  22. [30]

    Lora: Low-rank adaptation of large language models

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chen et al., “Lora: Low-rank adaptation of large language models.” ICLR, vol. 1, no. 2, p. 3, 2022

    2022

  23. [31]

    Wavlm: Large-scale self-supervised pre- training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao, J. Wu, L. Zhou, S. Ren, Y . Qian, Y . Qian, J. Wu, M. Zeng, X. Yu, and F. Wei, “Wavlm: Large-scale self-supervised pre- training for full stack speech processing,” IEEE Journal of Select...

    2022

  24. [32]

    Cocktail hubert: Generalized self- supervised pre-training for mixture and single-source speech,

    M. Fazel-Zarandi and W.-N. Hsu, “Cocktail hubert: Generalized self- supervised pre-training for mixture and single-source speech,” in ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2023, pp. 1–5

    2023

  25. [33]

    Robust speech recognition via large-scale weak supervi- sion,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervi- sion,” in International conference on machine learning . PMLR, 2023, pp. 28 492–28 518

    2023

  26. [34]

    Exploration of adapter for noise robust automatic speech recognition,

    H. Shi and T. Kawahara, “Exploration of adapter for noise robust automatic speech recognition,” arXiv preprint arXiv:2402.18275 , 2024

    2024 arXiv

  27. [35]

    Dual-path adaptation of pretrained feature extraction module for robust automatic speech recognition,

    ——, “Dual-path adaptation of pretrained feature extraction module for robust automatic speech recognition,” in Interspeech 2024 , 2024, pp. 2850–2854

    2024

  28. [36]

    Long short-term memory,

    A. Graves and A. Graves, “Long short-term memory,” Supervised sequence labelling with recurrent neural networks , pp. 37–45, 2012

    2012

  29. [37]

    Librimix: An open-source dataset for generalizable speech separation,

    J. Cosentino, M. Pariente, S. Cornell, A. Deleforge, and E. Vincent, “Librimix: An open-source dataset for generalizable speech separation,” arXiv preprint arXiv:2005.11262 , 2020

    2005 arXiv

  30. [38]

    Librispeech: An ASR corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: An ASR corpus based on public domain audio books,” in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015, pp. 5206–5210

    2015

  31. [39]

    Wham!: Extending speech separation to noisy environments,

    G. Wichern, J. Antognini, M. Flynn, L. R. Zhu, E. McQuinn, D. Crow, E. Manilow, and J. Le Roux, “Wham!: Extending speech separation to noisy environments,” in Proc. Interspeech, Sep. 2019

    2019

  32. [40]

    Multi-Speaker ASR Com- bining Non-Autoregressive Conformer CTC and Conditional Speaker Chain,

    P. Guo, X. Chang, S. Watanabe, and L. Xie, “Multi-Speaker ASR Com- bining Non-Autoregressive Conformer CTC and Conditional Speaker Chain,” in Proc. Interspeech, 2020, pp. 3720–3724

    2020

  33. [41]

    Exploring time-frequency domain target speaker extraction for causal and non-causal processing,

    W. Zhang, L. Yang, and Y . Qian, “Exploring time-frequency domain target speaker extraction for causal and non-causal processing,” in Proc. ASRU, 2023, pp. 1–6

    2023

  34. [42]

    A sidecar separator can convert a single-talker speech recognition system to a multi-talker one,

    L. Meng, J. Kang, M. Cui, Y . Wang, X. Wu, and H. Meng, “A sidecar separator can convert a single-talker speech recognition system to a multi-talker one,” in Proc. ICASSP, 2023, pp. 1–5

    2023

  35. [43]

    Adapting self- supervised models to multi-talker speech recognition using speaker embeddings,

    Z. Huang, D. Raj, P. García, and S. Khudanpur, “Adapting self- supervised models to multi-talker speech recognition using speaker embeddings,” in Proc. ICASSP, 2023, pp. 1–5

    2023

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.