Pith. sign in

REVIEW 3 major objections 6 minor 34 references

Selective Invocation for Multilingual ASR: A Cost-effective Approach Adapting to Speech Recognition Difficulty

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SIMA, a spoken large language model that estimates its own recognition difficulty, can route only hard speech to a stronger ASR model, cutting word error by 18.7% relative to the base SLLM and halving invocation cost relative to…

desk verdict Solid engineering idea with sloppy headline math and an unvalidated label proxy; worth refereeing but needs a fix. read the letter →

arxiv 2505.16168 v1 pith:Z3UAUO5B submitted 2025-05-22 cs.SD eess.AS

Hongfei Xue , Yufeng Tang , Jun Zhang , Xuelong Geng , Lei Xie This is my paper · ORCID
classification cs.SDeess.AS
keywords multilingualASRselectiveinvocationspokenlargelanguagemodelspeechrecognitiondifficultycost-effectiveroutingworderrorrateidentificationconfidencefusion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Multilingual ASR systems that route by predicted language pay to send every utterance to a commercial SOTA model and fail when language identification is wrong. SIMA instead asks the system itself how hard each utterance is: a spoken large language model (SLLM) produces a transcription and an invocation label, and only labels such as 'Yes' or 'Uncertain' route the audio to a stronger ASR model. On MLS, VoxPopuli, and FLEURS, the paper reports that this selective invocation reduces word error rate by 18.7% relative to the base SLLM and cuts invocation cost by about 51% relative to language-identification-based routing, with routing accuracy near 70%. If correct, this offers a cost-quality trade-off for multilingual ASR that does not depend on reliable language tags and turns model self-assessment into a routing signal.

What carries the argument

Key machinery is the three-way invocation header learned on top of the base SLLM, together with the WER-threshold data pipeline that generates its supervision and the fusion confidence strategy that resolves the middle case. In the multitask training format, the model emits a language tag, a language confidence, transcription tokens, and a special token 'Invocation No', 'Invocation Yes', or 'Invocation Uncertain'; the special tokens carry the routing decision. The data pipeline turns the base SLLM's own WER into labels—[0,2] No, >10 Yes, (2,10] Uncertain—so no human routing annotations are needed. For Uncertain inputs, a fusion rule combines average maximum posterior probability, entropy, and a four-level transcription-confidence score produced by the model itself; threshold values P=0.96, E=0.0015, T=level B decide whether the utterance goes to the SOTA model. The 'Uncertain' class is what absorbs mid-difficulty speech, and the ablations show it is needed to keep both invocation rate and WER low.

What would settle it

Take a held-out language or noisy domain and label every utterance by the oracle rule: invoke whenever the SOTA model's WER is lower than the base SLLM's WER. If SIMA's invocation accuracy against these oracle labels is near chance, or if SIMA's end-to-end WER at a fixed invocation rate is no better than a random-invocation policy at the same rate, the claim that the SLLM can assess recognition difficulty is falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that an SLLM can judge whether its own transcription will be accurate enough and should therefore decide, for each utterance, whether to transcribe directly or invoke a specialized SOTA ASR model. SIMA is a fine-tuned SLLM that outputs an 'Invocation No', 'Invocation Yes', or 'Invocation Uncertain' token. Training labels are generated automatically by running the base SLLM on a large pool of speech, computing word error rates against reference texts, and assigning 'No' to WER in [0,2], 'Yes' to WER above 10, and 'Uncertain' to WER in (2,10]. Uncertain cases are resolved by a fusion of posterior probability, entropy, and an LLM-scored transcription confidence level; if the posterior is below 0.96, entropy above 0.0015, and confidence below level B, the utterance is routed to the SOTA model. Compared with a language-identification router that invokes the SOTA model for 100% of utterances, SIMA invokes only 45.5–57.6% of utterances while achieving lower WER than the base model on all three test sets and staying close to the LID-top oracle WER. The paper concludes that SLLMs can judge speech complexity well enough to make selective invocation a scalable and cost-effective alternative to language-based routing.

Load-bearing premise

Everything rests on the base SLLM's word error rate being a trustworthy measure of whether a stronger SOTA model would actually improve the transcription; if that calibration fails for a language or domain, the automatically generated invocation labels and the learned routing policy inherit the mistake.

Editorial extensions

If this is right

  • At the same invocation rate, SIMA beats a random-invocation policy on all three test sets (WER improvements of 6.6% on MLS, 4.2% on VoxPopuli, and 16.8% on FLEURS), showing the routing decision itself, not just the SOTA model, drives the gain.
  • Deploying SIMA with a paid ASR API should roughly halve per-utterance invocation cost while keeping WER close to an LID-based system that sends every utterance to the API.
  • Because uncertainty is modeled as a third output category with a fusion confidence check, the system can avoid both wasteful invocations on clean speech and risky direct transcription on hard speech; ablations removing either component raise WER or lower efficiency.
  • The same automatic WER-threshold labeling pipeline can bootstrap invocation supervision for new languages without manual annotation, needing only reference transcripts and a base SLLM.
  • Language-specific invocation intervals, set around each language's SOTA WER, reduce invocation rate further and improve invocation efficiency from 2.5 to 2.9 on MLS.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A stronger base SLLM than the Ideal-LLM used here would presumably shift the whole cost-quality frontier; testing SIMA on Whisper-class bases is the natural next experiment, and the authors point to it in future work.
  • Oracle labels computed from the actual SOTA model's WER, rather than the base model's WER, could reveal how much headroom remains when the base model misjudges its own difficulty—especially on languages where base and SOTA quality diverge.
  • The same gated-invocation pattern could transfer to other sequence tasks (machine translation, summarization) where a cheap model can output a self-confidence token and decide when to escalate to an expensive model.
  • Per-language or per-domain threshold adaptation of P, E, and T is likely necessary in deployment; the paper's language-specific interval results suggest a single global rule leaves efficiency on the table.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes SIMA, a selective invocation mechanism for multilingual ASR. SIMA is built on a spoken large language model (SLLM) and decides for each input whether to transcribe it directly or invoke a more capable SOTA ASR model, with a three-way decision output ('Invocation No', 'Invocation Yes', 'Invocation Uncertain') and a fusion confidence strategy for the uncertain cases. Training data are generated by labeling utterances according to the base SLLM's own WER intervals: [0,2] is 'No', (2,10] is 'Uncertain', and >10 is 'Yes'. Experiments on MLS, VoxPopuli, and FLEURS report that SIMA lowers WER relative to the base SLLM and reduces invocation cost relative to LID-based routing. The abstract claims an 18.7% WER reduction and roughly halved invocation costs.

Significance. The idea of routing on recognition difficulty rather than only on language is interesting and practically motivated, and the paper includes several good experimental controls: a random-invocation baseline at matched invocation rate, an ablation of the confidence strategy, an ablation removing the 'Uncertain' class, and an out-of-domain FLEURS evaluation. If the central claims hold, the work offers a cost-effective alternative to unconditional LID-based routing. However, the headline quantitative claim is not reproducible from the reported tables, one analysis leaks test-set information into threshold selection, and the label proxy underlying the routing decisions is not validated. These issues currently prevent the paper from being accepted as is.

major comments (3)
  1. [Abstract; Section 3.3; Table 2] The headline '18.7% WER reduction' is not reproducible from Table 2. The text reports per-dataset reductions of 18.6%, 9.3%, and 28.2% relative to the base model on MLS, VoxPopuli, and FLEURS, respectively. However, Table 2 gives FLEURS base WER 10.76 and SIMA WER 6.43, which is a 40.2% reduction, not 28.2%. The average of the three stated reductions happens to be 18.7%, which appears to be the source of the abstract's number; using the correct FLEURS value gives an average relative reduction of about 22.7%. Please correct the arithmetic and recompute the headline claim, or report exactly which aggregation (e.g., pooled word error count) yields 18.7%.
  2. [Section 3.4; Table 4] The language-specific invocation strategy uses the test-set LID-Top WER i per language to set the 'Uncertain' interval as (i-2.5, i+2.5]. This leaks test-set statistics into the threshold selection, so the comparison between language-agnostic and language-specific strategies in Table 4 does not measure a deployable procedure. Please derive i from a held-out development set, or explicitly label Table 4 as an oracle analysis.
  3. [Section 2.3; Section 3.3] The invocation labels are defined by the base SLLM's WER intervals ([0,2], (2,10], >10), not by whether the SOTA model would actually improve the transcript. This proxy is never validated: the paper does not report the distribution of SOTA WER conditional on each label, nor the fraction of 'Yes' utterances for which the SOTA model beats the base model. Because the random-invocation baseline already recovers a substantial part of the WER gains (e.g., FLEURS: base 10.76, random 7.73, SIMA 6.43), the marginal benefit of the learned routing decisions should be quantified and shown to stem from genuine difficulty assessment rather than the label construction. Please add per-label SOTA WER statistics or an analysis comparing base versus SOTA WER per utterance.
minor comments (6)
  1. [Section 3.2] The random invocation baseline is described as making decisions randomly at the same overall invocation rate as SIMA, but the number of random seeds and the variance across draws are not reported; please specify how many seeds were used and whether the reported WER is averaged.
  2. [Section 2.2] The transcription confidence levels A-D are not precisely defined; please describe how the model produces the score and how the threshold T = level B is applied numerically, so that the fusion strategy is reproducible.
  3. [Table 3] The metrics SIMA-Cost and SIMA-Invoke-Errors are not defined precisely; please state the cost model (e.g., whether it includes base-model inference for all inputs) and the exact definition of an invocation error.
  4. [Section 2.3] The WER interval boundaries for the invocation labels ([0,2], (2,10], >10) are presented without justification; please provide evidence or a sensitivity analysis for these thresholds.
  5. [Section 3.3] The FLEURS reduction value 28.2% conflicts with Table 2; after correcting the arithmetic, please verify all derived numbers in the text and abstract.
  6. [Section 2.3; Tables 2 and 3] There are typographical errors, including 'Data Pipline' in the Section 2.3 heading and 'V oxPopuli' in Tables 2 and 3; these should be corrected.

Circularity Check

2 steps flagged · score 4.0 of 10

Invocation labels are self-defined from the base SLLM's own WER, and the language-specific thresholds are fitted to test-set WER; the main cost/WER result remains externally benchmarked.

  1. self definitional [Section 2.3, Data Pipeline]
    "We employ a multilingual SLLM [29] as the base model for generating training data. As illustrated in Figure 3, the SLLM processes the input speech to produce pseudo text labels, from which we compute the corresponding WER. Based on the WER, we assign invocation labels for all languages as follows: • Invocation No: WER in the interval [0, 2]. • Invocation Yes: WER exceeding 10. • Invocation Uncertain: WER in the interval (2, 10]."

    The ground-truth invocation labels are generated by thresholding the base SLLM's own WER, and SIMA is initialized from that same SLLM and trained to emit these labels. The paper's central notion of 'speech recognition difficulty' is therefore, by construction, the base model's self-measured difficulty, not an independently validated property of the speech or of SOTA-model improvement. The claimed conclusion that 'SLLMs can ... assess whether they can transcribe the speech accurately' is a supervised imitation of a thresholded version of the model's own error rate, so the evidence for the hypothesis is partly self-referential. The final WER comparison against external baselines remains independent, which limits the severity.

  2. fitted input called prediction [Section 3.4, Language-Specific Invocation Strategies]
    "In contrast, the language-specific strategy customizes this interval: for a given language with a LID-Top model test result of i, the 'Uncertain' interval is defined as (i−2.5, i+2.5]."

    The parameter i is the LID-Top model's WER computed on the MLS test set, and Table 4 reports the WER, invocation rate, and efficiency of this language-specific strategy on that same test set. The routing interval is therefore fitted to the evaluation set before the strategy is evaluated on it, so the reported efficiency improvement of the language-specific strategy is not an out-of-sample prediction; it is an in-sample fit presented as a result. This is a 'fitted input called prediction' pattern for the language-specific analysis, though it does not affect the main language-agnostic results.

full rationale

The paper's main contribution is externally benchmarked: SIMA's WER is compared with the base SLLM, random invocation, LID-Top, and Whisper on MLS, VoxPopuli, and FLEURS, so the headline 18.7% relative WER reduction and halved invocation cost are not forced by the training-label definition alone. The random-invocation baseline even shows that most of the WER gain comes from the invocation mechanism rather than from selective routing. However, the training labels for the invocation decision are computed by thresholding the base SLLM's own WER (Section 2.3), which makes the system's 'difficulty' concept self-referential: the model is trained to predict a binned version of its own error rate, and the paper then presents this as evidence that SLLMs can assess their transcription accuracy. Additionally, the language-specific strategy in Section 3.4 centers the 'Uncertain' interval on the test-set LID-Top WER, so the reported efficiency gains for that strategy are obtained by fitting the router to the evaluation set. These are genuine but partial circularities: the central cost/quality claim retains independent content because it is measured against external systems and because SIMA's label accuracy is only about 70%, so the routing is imperfect. There is no load-bearing self-citation or uniqueness argument; citation [29] merely supplies the base architecture. Overall score 4 reflects moderate circularity in the difficulty signal and one test-set-fitted secondary result, without the main derivation collapsing into its inputs.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim depends on a small set of hand-chosen WER cutoffs and fusion thresholds, plus the domain assumption that an SLLM can assess its own transcription difficulty. No new physical entities are introduced. The language-specific threshold tuning in Section 3.4 is a free parameter fitted to the test set, which is a red flag.

free parameters (3)
  • WER interval thresholds for invocation labels = No: [0,2], Uncertain: (2,10], Yes: >10
    Chosen by hand in Section 2.3 to define difficulty tiers; the central routing decision depends on these cutoffs.
  • Fusion confidence thresholds = P=0.96, E=0.0015, T=level B
    Set on the validation set in Section 3.2 and used to resolve 'Uncertain' cases.
  • Language-specific Uncertain interval offset = ±2.5 WER points around LID-Top test WER per language
    Defined in Section 3.4 from the test-set WER of the LID-Top model, then evaluated on the same test set; this is a fitted-to-test-set parameter.
assumptions (4)
  • domain assumption SLLMs can assess whether they can transcribe input speech accurately.
    Stated as a hypothesis in Section 1 and Section 3.3; the feasibility of the method rests on this.
  • domain assumption Base SLLM WER is a valid proxy for recognition difficulty and for the benefit of invoking a SOTA model.
    Section 2.3 labels training data by WER intervals; no oracle validation that SOTA actually improves on 'Yes' samples.
  • domain assumption The commercial/open SOTA models selected as LID-Top are appropriate references and their routing errors are negligible.
    Section 3.2 excludes LID prediction errors for LID-Top, assuming multilingual SOTA models are robust to routing errors.
  • domain assumption The LLM-based transcription confidence score levels A-D correlate with transcription quality.
    Section 2.3 uses Phi-3 to score pseudo-labels; no demonstration of reliability is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Selective Invocation for Multilingual ASR: A Cost-effective Approach Adapting to Speech Recognition Difficulty." pith.science (2026). https://pith.science/paper/Z3UAUO5B

@misc{pith2026250516168,
  author       = {Pith},
  title        = {Pith review of: Selective Invocation for Multilingual ASR: A Cost-effective Approach Adapting to Speech Recognition Difficulty},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z3UAUO5B}},
  note         = {Machine review of arXiv:2505.16168}
}
read the original abstract

Although multilingual automatic speech recognition (ASR) systems have significantly advanced, enabling a single model to handle multiple languages, inherent linguistic differences and data imbalances challenge SOTA performance across all languages. While language identification (LID) models can route speech to the appropriate ASR model, they incur high costs from invoking SOTA commercial models and suffer from inaccuracies due to misclassification. To overcome these, we propose SIMA, a selective invocation for multilingual ASR that adapts to the difficulty level of the input speech. Built on a spoken large language model (SLLM), SIMA evaluates whether the input is simple enough for direct transcription or requires the invocation of a SOTA ASR model. Our approach reduces word error rates by 18.7% compared to the SLLM and halves invocation costs compared to LID-based methods. Tests on three datasets show that SIMA is a scalable, cost-effective solution for multilingual ASR applications.

Figures

Figures reproduced from arXiv: 2505.16168 by the authors.

Figure 1
Figure 1. Three systems for multilingual ASR. (a) A single mul￾tilingual model, such as Whisper, which recognizes multiple languages with one model. (b) A language identification (LID)- based system that identifies the language and invokes the cor￾responding SOTA model. (c) Selective invocation for multilin￾gual ASR (SIMA) that directly transcribes simpler speech and invokes SOTA models for more complex inputs. model, further… view at source ↗
Figure 2
Figure 2. The multitask training format of the SIMA model. Multilingual SLLM Model Speech Pseudo Text Pseudo LID LLM Transcription Confidence Language Confidence [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Data pipeline of the SIMA dataset. integrate three complementary confidence evaluation methods: Posterior Probability: We extract the maximum probability from the softmax output of the final layer, where y ∈ R T ×C , with T denoting the number of frames and C representing the number of label classes. The average posterior probability is computed as follows: Probability = 1 T XT t=1 max c∈{1,...,C} yt,c (1) Entropy: … view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 22 canonical work pages

  1. [1]

    Selective Invocation for Multilingual ASR: A Cost-effective Approach Adapting to Speech Recognition Difficulty

    Introduction Multilingual automatic speech recognition (ASR) models have gained significant attention for their ability to recognize mul- tiple languages using a single model [1, 2, 3, 4], as illustrated in Figure 1(a). Recent advances have led to impressive perfor- mance in various languages through large-scale supervised or self-supervised pre-training ...

    work page Pith review arXiv 2025

  2. [2]

    Invoca- tion No

    Method As illustrated in Figure 1(c), our proposed method comprises two main modules. The first module is the SIMA model, which evaluates the difficulty of the input speech and either directly produces a transcription or outputs an invocation label. The sec- ond module is a library of SOTA ASR models. When the SIMA model yields the invocation label, the i...

    work page

  3. [3]

    Uncertain

    Experiments 3.1. Datasets To ensure domain diversity and improve robustness, we utilize three datasets: Multilingual LibriSpeech (MLS) [26], V oxPop- uli [27], and FLEURS [28]. We select the languages common to these datasets, namely English (en), German (de), Dutch (nl), French (fr), Spanish (es), Italian (it), and Polish (pl). For train- ing, we generat...

    work page

  4. [4]

    Leveraging a base spoken large lan- guage model, SIMA dynamically determines whether to tran- scribe speech directly or invoke specialized SOTA models

    Conclusion This paper introduces SIMA, a novel selective invocation strat- egy for multilingual ASR. Leveraging a base spoken large lan- guage model, SIMA dynamically determines whether to tran- scribe speech directly or invoke specialized SOTA models. Ex- tensive experiments on three benchmark datasets demonstrate that SIMA reduces the word error rate by...

    work page

  5. [5]

    Multilingual speech recognition with a single end-to-end model,

    S. Toshniwal, T. N. Sainath, R. J. Weiss, B. Li, P. J. Moreno, E. Weinstein, and K. Rao, “Multilingual speech recognition with a single end-to-end model,” inInternational Conference on Acous- tics, Speech and Signal Processing (ICASSP). IEEE, 2018, pp. 4904–4908

    work page 2018

  6. [6]

    Cross-lingual language model pre- training,

    A. Conneau and G. Lample, “Cross-lingual language model pre- training,” inConference on Neural Information Processing Sys- tems (NeurIPS), 2019, pp. 7057–7067

    work page 2019

  7. [7]

    Unsupervised cross-lingual representation learning for speech recognition,

    A. Conneau, A. Baevski, R. Collobert, A. Mohamed, and M. Auli, “Unsupervised cross-lingual representation learning for speech recognition,” inConference of the International Speech Commu- nication Association (Interspeech). ISCA, 2021, pp. 2426–2430

    work page 2021

  8. [8]

    ML- SUPERB: multilingual speech universal performance bench- mark,

    J. Shi, D. Berrebbi, W. Chen, E. Hu, W. Huang, H. Chung, X. Chang, S. Li, A. Mohamed, H. Lee, and S. Watanabe, “ML- SUPERB: multilingual speech universal performance bench- mark,” inConference of the International Speech Communication Association (Interspeech). ISCA, 2023, pp. 884–888

    work page 2023

Show all 34 references
  1. [9]

    Joint unsupervised and supervised training for multilingual ASR,

    J. Bai, B. Li, Y . Zhang, A. Bapna, N. Siddhartha, K. C. Sim, and T. N. Sainath, “Joint unsupervised and supervised training for multilingual ASR,” inInternational Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 6402– 6406

    2022

  2. [10]

    Robust speech recognition via large-scale weak supervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” inInternational Conferenceon Machine Learning (ICML), vol. 202, 2023, pp. 28 492–28 518

    2023

  3. [11]

    TranUSR: Phoneme-to-word Transcoder Based Unified Speech Representa- tion Learning for Cross-lingual Speech Recognition,

    H. Xue, Q. Shao, P. Chen, P. Guo, L. Xie, and J. Liu, “TranUSR: Phoneme-to-word Transcoder Based Unified Speech Representa- tion Learning for Cross-lingual Speech Recognition,” inConfer- ence of the International Speech Communication Association (In- terspeech). ISCA, 2023

    2023

  4. [12]

    Scaling speech technology to 1, 000+ languages,

    V . Pratap, A. Tjandra, B. Shi, P. Tomasello, A. Babu, S. Kundu, A. Elkahky, Z. Ni, A. Vyas, M. Fazel-Zarandi, A. Baevski, Y . Adi, X. Zhang, W. Hsu, A. Conneau, and M. Auli, “Scaling speech technology to 1, 000+ languages,”J. Mach. Learn. Res., vol. 25, pp. 97:1–97:52, 2024

    2024

  5. [13]

    Google usm: Scaling auto- matic speech recognition beyond 100 languages,

    Y . Zhang, W. Han, J. Qin, Y . Wang, A. Bapna, Z. Chen, N. Chen, B. Li, V . Axelrod, G. Wanget al., “Google usm: Scaling auto- matic speech recognition beyond 100 languages,”arXiv preprint arXiv:2303.01037, 2023

    2023 arXiv

  6. [14]

    Seamlessm4t-massively multilingual & multimodal ma- chine translation,

    L. Barrault, Y .-A. Chung, M. C. Meglioli, D. Dale, N. Dong, P.- A. Duquenne, H. Elsahar, H. Gong, K. Heffernan, J. Hoffman et al., “Seamlessm4t-massively multilingual & multimodal ma- chine translation,”arXiv preprint arXiv:2308.11596, 2023

    2023 arXiv

  7. [15]

    SSHR: leveraging self-supervised hierarchical representations for multi- lingual automatic speech recognition,

    H. Xue, Q. Shao, K. Huang, P. Chen, J. Liu, and L. Xie, “SSHR: leveraging self-supervised hierarchical representations for multi- lingual automatic speech recognition,” inInternational Confer- ence on Multimedia and Expo (ICME). IEEE, 2024, pp. 1–6

    2024

  8. [16]

    Anatomy of industrial scale multilingual ASR,

    F. M. Ramirez, L. Chkhetiani, A. Ehrenberg, R. McHardy, R. Botros, Y . Khare, A. Vanzo, T. Peyash, G. Oexle, M. Liang, I. Sklyar, E. Fakhan, A. Etefy, D. McCrystal, S. Flamini, D. Do- nato, and T. Yoshioka, “Anatomy of industrial scale multilingual ASR,”CoRR, vol. abs/2404.09841, 2024

    2024 arXiv

  9. [17]

    Improving multilingual asr robustness to errors in language in- put,

    B. Houston, O. Sadjadi, Z. Hou, S. Vishnubhotla, and K. Han, “Improving multilingual asr robustness to errors in language in- put,” inConference of the International Speech Communication Association (Interspeech). ISCA, 2024, pp. 1250–1254

    2024

  10. [18]

    Recurrent neural networks for noise reduction in robust ASR,

    A. L. Maas, Q. V . Le, T. M. O’Neil, O. Vinyals, P. Nguyen, and A. Y . Ng, “Recurrent neural networks for noise reduction in robust ASR,” inConference of the International Speech Communication Association (Interspeech). ISCA, 2012, pp. 22–25

    2012

  11. [19]

    MUSAN: A music, speech, and noise corpus,

    D. Snyder, G. Chen, and D. Povey, “MUSAN: A music, speech, and noise corpus,”CoRR, vol. abs/1510.08484, 2015

    2015 arXiv

  12. [20]

    Wavlm: Large- scale self-supervised pre-training for full stack speech process- ing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao, J. Wu, L. Zhou, S. Ren, Y . Qian, Y . Qian, J. Wu, M. Zeng, X. Yu, and F. Wei, “Wavlm: Large- scale self-supervised pre-training for full stack speech process- ing,”IEEE J. Sel. Top. Sig...

    2022

  13. [21]

    Noise robust automatic speech recognition: review and analysis,

    M. Dua, Akanksha, and S. Dua, “Noise robust automatic speech recognition: review and analysis,”Int. J. Speech Technol., vol. 26, no. on, pp. 475–519, 2023

    2023

  14. [22]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample, “Llama: Open and efficient foundation language models,”arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  15. [23]

    Qwen technical report,

    J. Bai, S. Bai, Y . Chu, Z. Cui, K. Dang, X. Deng, Y . Fan, W. Ge, Y . Han, F. Huanget al., “Qwen technical report,”arXiv preprint arXiv:2309.16609, 2023

    2023 arXiv

  16. [24]

    Gpt-4 technical report,

    OpenAI, “Gpt-4 technical report,”arXiv preprint arXiv:2308.11276, 2023

    2023 arXiv

  17. [25]

    On decoder-only architecture for speech-to-text and large language model integration,

    J. Wu, Y . Gaur, Z. Chen, L. Zhou, Y . Zhu, T. Wang, J. Li, S. Liu, B. Ren, L. Liu, and Y . Wu, “On decoder-only architecture for speech-to-text and large language model integration,” inAuto- matic Speech Recognition and Understanding Workshop (ASRU). IEEE, 2023, pp. 1–8

    2023

  18. [26]

    Qwen-audio: Advancing universal audio understand- ing via unified large-scale audio-language models,

    Y . Chu, J. Xu, X. Zhou, Q. Yang, S. Zhang, Z. Yan, C. Zhou, and J. Zhou, “Qwen-audio: Advancing universal audio understand- ing via unified large-scale audio-language models,”arXiv preprint arXiv:2311.07919, 2023

    2023 arXiv

  19. [27]

    Wavllm: Towards robust and adaptive speech large language model,

    S. Hu, L. Zhou, S. Liu, S. Chen, L. Meng, H. Hao, J. Pan, X. Liu, J. Li, S. Sivasankaran, L. Liu, and F. Wei, “Wavllm: Towards robust and adaptive speech large language model,” inEMNLP. Association for Computational Linguistics, 2024, pp. 4552–4572

    2024

  20. [28]

    Qwen2-audio technical re- port,

    Y . Chu, J. Xu, Q. Yang, H. Wei, X. Wei, Z. Guo, Y . Leng, Y . Lv, J. He, J. Lin, C. Zhou, and J. Zhou, “Qwen2-audio technical re- port,”arXiv preprint arXiv:2407.10759, 2024

    2024 arXiv

  21. [29]

    E- chat: Emotion-sensitive spoken dialogue system with large lan- guage models,

    H. Xue, Y . Liang, B. Mu, S. Zhang, Q. Chen, and L. Xie, “E- chat: Emotion-sensitive spoken dialogue system with large lan- guage models,” inISCSLP. IEEE, 2024

    2024

  22. [30]

    MLS: A large-scale multilingual dataset for speech research,

    V . Pratap, Q. Xu, A. Sriram, G. Synnaeve, and R. Collobert, “MLS: A large-scale multilingual dataset for speech research,” in Conference of the International Speech Communication Associa- tion (Interspeech). ISCA, 2020, pp. 2757–2761

    2020

  23. [31]

    V oxpopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation,

    C. Wang, M. Rivi `ere, A. Lee, A. Wu, C. Talnikar, D. Haz- iza, M. Williamson, J. M. Pino, and E. Dupoux, “V oxpopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation,” inACL/IJCNLP. Association for Computationa...

    2021

  24. [32]

    FLEURS: few-shot learning evaluation of universal representations of speech,

    A. Conneau, M. Ma, S. Khanuja, Y . Zhang, V . Axelrod, S. Dalmia, J. Riesa, C. Rivera, and A. Bapna, “FLEURS: few-shot learning evaluation of universal representations of speech,” inSpoken Lan- guage Technology Workshop (SLT). IEEE, 2022, pp. 798–805

    2022

  25. [33]

    Ideal-LLM: Integrating dual encoders and language-adapted llm for multilingual speech-to-text,

    H. Xue, W. Ren, X. Geng, K. Wei, L. Li, Q. Shao, L. Yang, K. Diao, and L. Xie, “Ideal-LLM: Integrating dual encoders and language-adapted llm for multilingual speech-to-text,”arXiv preprint arXiv:2409.11214, 2024

    2024 arXiv

  26. [34]

    Phi-3 technical report: A highly capable language model locally on your phone,

    M. Abdin, S. A. Jacobs, A. A. Awan, J. Aneja, A. Awadallah, H. Awadalla, N. Bach, A. Bahree, A. Bakhtiari, H. Behlet al., “Phi-3 technical report: A highly capable language model locally on your phone,”arXiv preprint arXiv:2404.14219, 2024

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.