Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Adapting Whisper for Code-Switching through Encoding Refining and Language-Aware Decoding

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Adapting Whisper with an LSTM encoder refiner and two language-aware decoder streams reduces code-switching error and beats previous state of the art.

desk verdict Useful adaptation recipe with a real but fixable inconsistency about whether CTC is actually used. read the letter →

arxiv 2412.16507 v3 pith:VXMDFXQ6 submitted 2024-12-21 cs.CL cs.SDeess.AS

Jiahui Zhao , Hao Shi , Chenrui Cui , Tianrui Wang , Hexin Liu , Zhaoheng Ni , Lingxuan Ye , Longbiao Wang This is my paper · ORCID
classification cs.CLcs.SDeess.AS
keywords code-switchingASRWhisperadaptationencoderrefinerlanguage-awaredecodingadapter-basedfine-tuningCTCguidanceSEAMElanguageprompt
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that Whisper, a large multilingual speech recognizer, can be adapted to Mandarin–English code-switched speech more effectively by acting on both sides of the model. On the encoder side it adds an LSTM refiner trained with a CTC auxiliary loss to sharpen temporal modeling of intra-sentence switches. On the decoder side it runs two parallel adapter streams, one conditioned on a Mandarin prompt and one on an English prompt, then fuses their outputs with a learned softmax weighting. On the SEAME benchmark the combined method reduces mix error rate by 4.1% relative on the Mandarin-dominant test set and 7.2% on the English-dominant test set over an adapter-only baseline, and edges past the previous best adapted model on both sets. The practical question the paper addresses is whether parameter-efficient adaptation can give large pre-trained multilingual models a working ability to track language switches within a sentence.

What carries the argument

The encoder refiner is the first load-bearing piece: two LSTM layers placed on Whisper's encoder output, with a CTC loss attached during training to encourage the refined frames to align with text tokens across language switches. The second piece is language-aware decoding: every decoder layer has two adapter branches, one using the Mandarin prompt embedding and one the English prompt embedding, and a two-linear-layer softmax fusion module combines the two language-specific outputs at the end. Together these pieces let the model keep one shared Whisper backbone while splitting the language-specific processing into parameter-efficient add-on paths.

What would settle it

Re-run the reported recipe with λ set to 0.9 or 0.7 so the CTC term is nonzero, and compare the resulting MER on dev_man and dev_sge against the λ=1.0 model; if CTC supervision does not improve the numbers, the paper's central attribution of the encoder refiner's gain to CTC guidance is unsupported.

Watch

Extended reading notes

Core claim

The paper's central discovery is that two complementary adaptation modules together let Whisper separate the two languages in code-switched speech better than either module alone. The encoder refiner, an LSTM stack supervised by CTC during training, improves encoding of non-native segments; the language-aware decoding, with separate Mandarin- and English-prompted adapter paths and a fusion layer, improves the decoder's language-specific output. The paper reports that the combined system achieves 14.0 MER on dev_man and 20.6 MER on dev_sge, beating the previous state-of-the-art AG Whisper (14.2 and 20.8) and the adapter-only baseline (14.6 and 22.2). It also reports large relative gains on the non-native side of each test set—about 20% on Mandarin in the English-dominant set—which supports the interpretation that the method reduces language confusion rather than just fitting the training domain.

Load-bearing premise

The claimed benefit of CTC-guided encoder refining depends on the CTC loss actually being used when the final model is trained, but the paper sets λ=1.0, which removes the CTC term from the final loss.

Editorial extensions

If this is right

  • If the encoder refiner's gains come from better temporal modeling, then replacing LSTM with other sequence models of comparable capacity should preserve or improve the reported MER reductions.
  • Language-aware decoding can be applied to Whisper models of other sizes without changing the training objective, so the method is a drop-in recipe for any Whisper-based code-switching ASR system.
  • The method gives larger gains on the non-native side of each test set, so it directly targets language confusion rather than overall acoustic mismatch.
  • The fusion module provides a learned per-token weighting between language-specific decoder outputs, which could be reused as a soft language-segmentation signal.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, if the CTC loss is truly absent under the reported λ=1.0 setting, the encoder refiner's improvement is attributable mainly to the LSTM's temporal modeling rather than to CTC guidance; a clean comparison with λ<1.0 would settle which component matters.
  • Beyond the paper, the learned fusion weights could be visualized against actual code-switch points in SEAME to test whether the model is genuinely tracking intra-sentence language switches.
  • Beyond the paper, the dual-prompt adapter design could extend naturally to three or more languages in multilingual communities, where the number of prompt-specific branches would grow linearly with the language inventory.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes adapting Whisper-small to Mandarin-English code-switching ASR by adding an LSTM-based encoder refiner supervised by CTC, dual language-aware adapters in decoder layers, and a decoder fusion module. On SEAME, it reports MER reductions of 4.1% (dev_man) and 7.2% (dev_sge) relative to an encoder+decoder adapter baseline (ID-3), and slightly better MER than the prior AG Whisper system. The manuscript includes a p-value for comparisons against ID-3 and analyzes language-specific gains, particularly for non-native language regions.

Significance. If the reported improvements are reproducible, the work is a useful contribution to parameter-efficient adaptation of multilingual ASR for code-switching. The paper provides a systematic ablation over adapter placement and CTC use, compares against a strong recent baseline, and reports significance for the main within-paper comparisons. Its main limitations are the unresolved λ=1.0 issue, the unspecified relationship between the two loss formulations, the nonstandard cross-entropy term in Eq. (11), and the lack of code/data-split release, which prevent full verification of the mechanism.

major comments (4)
  1. [§IV.A and Eq. (12)] The setting λ=1.0 in §IV.A makes the CTC term in Eq. (12) vanish, reducing L_final to L_dec. Yet §III.A and §IV.B.1 repeatedly attribute improvements to 'training with CTC' and to the encoder refiner's CTC guidance. The paper does not describe a two-stage schedule in which Eq. (5) with α=0.7 is optimized first; as written, ID-5, ID-6, and ID-8 are trained without any CTC loss. Please state the actual training schedule and, if λ=1.0 is intended, revise the attribution of gains to CTC; if it is a typo, give the intended value.
  2. [§III.A and Eq. (12)] The relationship between Eq. (5) and Eq. (12) is unspecified. L_att in Eq. (5) and L_dec in Eq. (11) appear to overlap in content, but the paper never states whether the encoder refiner is trained jointly with the decoder from the start, whether Eq. (5) is pre-training, or how the two losses are combined. This is load-bearing because the relative contribution of the CTC loss—the claimed mechanism—cannot be determined from the reported configuration.
  3. [§III.B, Eq. (11)] Eq. (11) defines CE(h''_dec_zh, pzh) and CE(h''_dec_en, pen), but cross-entropy is defined between a predicted distribution and a target token sequence, not between a hidden embedding and a prompt embedding. The loss formulation is therefore not well-defined as written. Please specify the exact tensors (e.g., logits after an output projection) and the target labels used for the language-aware loss.
  4. [§IV.B, Table I-II] The headline claim of surpassing AG Whisper rests on a difference of 0.2 MER on both test sets (14.0 vs. 14.2 and 20.6 vs. 20.8). The reported p-values are only against ID-3 within Table I, and no significance test or variance estimate is given for the comparison in Table II. Please report whether the difference against AG Whisper is statistically significant or present it as a non-significant improvement.
minor comments (5)
  1. [Abstract and Section I] The abstract and Section I contain the typo 'swithching', which should be corrected to 'switching'.
  2. [§IV.A] The paper does not provide code, data splits, or a description of the SEAME split used; the experimental section should at least state whether the official SEAME split is adopted.
  3. [Table II] Table II omits model sizes, trainable parameter counts, and runtimes, which are relevant for an adaptation method claiming efficiency.
  4. [§V Conclusions] The conclusion states that the encoder refiner 'can adopt various structures,' but only the LSTM variant is evaluated; please either remove this claim or add evidence.
  5. [§III.A and §IV.A] The hyperparameters α and λ are reported without sensitivity analysis; at minimum justify the chosen values, especially λ=1.0 in light of the CTC motivation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity found; the CTC lambda inconsistency is a correctness concern, not a circularity.

full rationale

The paper's central claim is an empirical comparison on the SEAME benchmark, with the proposed encoder refiner and language-aware decoding evaluated against adapter baselines and prior work on held-out dev_man and dev_sge sets. No equation in the derivation reduces another equation to its own inputs: the encoder refiner loss (Eq. 5) and the combined loss (Eq. 12) are objective functions, not fitted parameters used to produce the reported MER numbers, and the model is selected by validation loss rather than tuned to reproduce test-set numbers. The only self-citations (e.g., [20], [29]) are motivational and not load-bearing. The paper does contain an internal inconsistency: Section IV.A sets λ = 1.0 in Eq. 12, which zeroes the (1 − λ)∗L_CTC term, yet Section IV.B.1 attributes the ID-6 improvement to 'training with CTC'; this is a missing-support/correctness issue, not a circular derivation, because the reported empirical gain would still be an independent measurement even if the stated mechanism is unsupported. Standard model selection on the same dataset is neither a fitted-input-called-prediction nor a self-definitional step. Therefore no circularity step can be exhibited under the required evidence standard.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or conceptual entities. It relies on standard free hyperparameters and domain assumptions about the effectiveness of LSTM/CTC refinement and prompt-conditioned adapters. The main accounting issue is the ambiguous role of λ=1.0, which effectively removes CTC from the final loss and complicates the interpretation of the training objective.

free parameters (6)
  • α = 0.7
    Loss weight in Eq. 5 balancing attention loss and CTC loss in the encoder refiner. Chosen by hand/validation; not derived.
  • λ = 1.0
    Loss weight in Eq. 12. With λ=1.0 the CTC term becomes zero, which is inconsistent with the paper's emphasis on CTC guidance. This hand-set value affects training substantially.
  • learning rate = 1e-4
    Adam learning rate; set without a reported sweep.
  • number of epochs = 8
    Training epochs; early stopping based on validation loss.
  • LSTM hidden size = 512
    Hidden dimension of the encoder refiner LSTM, chosen by hand.
  • LSTM layers = 2
    Number of LSTM layers in the encoder refiner, chosen by hand.
assumptions (4)
  • domain assumption Whisper-small's encoder output features are compatible with LSTM temporal modeling for code-switching refinement.
    The encoder refiner passes the last encoder hidden states through LSTM layers; the paper does not analyze whether this preserves the acoustic and linguistic information needed for ASR.
  • domain assumption The SEAME dataset is a representative benchmark for Mandarin-English code-switching ASR.
    The evaluation is on SEAME; the paper assumes its conclusions generalize to other code-switching settings, with no experiments on other corpora.
  • domain assumption CTC loss can effectively guide the encoder refiner to learn language-switching information.
    The paper relies on CTC supervision (Eq. 4) as a central mechanism, but provides no analysis of why CTC specifically helps language switching beyond a general 'misalignment' statement.
  • standard math Standard training assumptions in deep learning (optimizer convergence, adapter stability, no catastrophic forgetting).
    Implicit in all empirical deep-learning papers; not stated or verified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adapting Whisper for Code-Switching through Encoding Refining and Language-Aware Decoding." pith.science (2026). https://pith.science/paper/VXMDFXQ6

@misc{pith2026241216507,
  author       = {Pith},
  title        = {Pith review of: Adapting Whisper for Code-Switching through Encoding Refining and Language-Aware Decoding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VXMDFXQ6}},
  note         = {Machine review of arXiv:2412.16507}
}
read the original abstract

Code-switching (CS) automatic speech recognition (ASR) faces challenges due to the language confusion resulting from accents, auditory similarity, and seamless language switches. Adaptation on the pre-trained multi-lingual model has shown promising performance for CS-ASR. In this paper, we adapt Whisper, which is a large-scale multilingual pre-trained speech recognition model, to CS from both encoder and decoder parts. First, we propose an encoder refiner to enhance the encoder's capacity of intra-sentence swithching. Second, we propose using two sets of language-aware adapters with different language prompt embeddings to achieve language-specific decoding information in each decoder layer. Then, a fusion module is added to fuse the language-aware decoding. The experimental results using the SEAME dataset show that, compared with the baseline model, the proposed approach achieves a relative MER reduction of 4.1% and 7.2% on the dev_man and dev_sge test sets, respectively, surpassing state-of-the-art methods. Through experiments, we found that the proposed method significantly improves the performance on non-native language in CS speech, indicating that our approach enables Whisper to better distinguish between the two languages.

Figures

Figures reproduced from arXiv: 2412.16507 by the authors.

Figure 1
Figure 1. The encoder refiner and the language-aware adaptation [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Code-Switching in End-to-End Automatic Speech Recognition: A Systematic Literature Review

    cs.CL 2025-07 conditional novelty 4.0 of 10

    A systematic review of 127 papers shows code-switching ASR research is concentrated in a few language pairs and fragmented across datasets, metrics, and non-reproducible methods.

Reference graph

Works this paper leans on

44 extracted references · 32 canonical work pages · cited by 1 Pith paper

  1. [1]

    Un- supervised cross-lingual representation learning for speech recognition,

    A. Conneau, A. Baevski, R. Collobert, A. Mohamed, and M. Auli, “Un- supervised cross-lingual representation learning for speech recognition,” in Proc. Interspeech, 2021, pp. 2426–2430

    2021

  2. [2]

    Robust speech recognition via large-scale weak supervision,

    A. Radford, J.-W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” in Proc. ICML, 2023, pp. 28492–28518

    work page 2023

  3. [3]

    Scaling speech technol- ogy to 1,000+ languages,

    V . Pratap, A. Tjandra, B. Shi, P. Tomasello, A. Babu, S. Kundu, A. Elkahky, Z. Ni, A. Vyas, M. Fazel-Zarandi, A. Baevski, Y . Adi, X. Zhang, W.-N. Hsu, A. Conneau, and M. Auli, “Scaling speech technol- ogy to 1,000+ languages,” Journal of Machine Learning Research, vol. 25, no. 97, pp. 1–52, 2024

    work page 2024

  4. [4]

    Mamba in Speech: Towards an Alternative to Self-Attention,

    X. Zhang, Q. Zhang, H. Liu, T. Xiao, X. Qian, B. Ahmed, E. Ambikaira- jah, H. Li, and J. Epps, “Mamba in Speech: Towards an Alternative to Self-Attention,” arXiv preprint arXiv:2405.12609, 2024

    arXiv 2024

  5. [5]

    H. Liu, L. P. Garcia, X. Zhang, A. W. H. Khong and S. Khudanpur, ”En- hancing Code-Switching Speech Recognition With Interactive Language Biases,” in Proc. ICASSP, 2024, pp. 10886-10890

    work page 2024

  6. [6]

    SUPERB: Speech processing universal performance benchmark,

    S.-w. Yang, P.-H. Chi, Y .-S. Chuang, et al, “SUPERB: Speech processing universal performance benchmark,” in Proc. Interspeech, 2021, pp. 1194- 1198

    work page 2021

  7. [7]

    H. Sun, S. Zhao, X. Wang, W. Zeng, Y . Chen and Y . Qin, ”Fine- Grained Disentangled Representation Learning For Multimodal Emotion Recognition,” in Proc. ICASSP, 2024, pp. 11051-11055

    work page 2024

  8. [8]

    Google USM: Scaling automatic speech recognition beyond 100 languages,

    Y . Zhang, W. Han, J. Qin, et al, “Google USM: Scaling automatic speech recognition beyond 100 languages,” arXiv preprint arXiv:2303.01037, 2023

    arXiv 2023

Show all 44 references
  1. [9]

    Waveform-domain speech enhancement using spectrogram encoding for robust speech recognition,

    H. Shi, M. Mimura, and T. Kawahara, “Waveform-domain speech enhancement using spectrogram encoding for robust speech recognition,” IEEE/ACM Trans. on Audio, Speech, and Language Processing, vol. 32, pp. 3049–3060, 2024

    2024

  2. [10]

    Rethinking mamba in speech processing by self-supervised models,

    X. Zhang, J. Ma, M. Shahin, B. Ahmed, and J. Epps, “Rethinking mamba in speech processing by self-supervised models,” arXiv preprint arXiv:2409.07273, 2024

    2024 arXiv

  3. [11]

    Iterative prototype refinement for ambiguous speech emotion recognition,

    H. Sun, S. Zhao, X. Kong, X. Wang, H. Wang, J. Zhou, and Y . Qin, “Iterative prototype refinement for ambiguous speech emotion recognition,” in Interspeech, 2024, pp. 3200–3204

    2024

  4. [12]

    On the effectiveness of asr representations in real-world noisy speech emotion recognition,

    X. Shi, J. He, X. Li, and T. Toda, “On the effectiveness of asr representations in real-world noisy speech emotion recognition,” arXiv preprint arXiv:2311.07093

  5. [13]

    Developing vocal system impaired patient-aimed voice quality assessment approach using ASR representation-included multiple features

    S. Dang, T. Matsumoto, Y . Takeuchi, T. Tsuboi, Y . Tanaka, D. Nakat- subo, S. Maesawa, R. Saito, M. Katsuno, and H. Kudo, “Developing vocal system impaired patient-aimed voice quality assessment approach using ASR representation-included multiple features” in Proc. Inter- spe...

    2024

  6. [14]

    Multimodal Fusion of Music Theory- Inspired and Self-Supervised Representations for Improved Emotion Recognition,

    X. Shi, X. Li, and T. Toda, “Multimodal Fusion of Music Theory- Inspired and Self-Supervised Representations for Improved Emotion Recognition,” in Proc. Interspeech, 2024, pp.2024–2350

    2024

  7. [15]

    S. Dang, T. Matsumoto, Y . Takeuchi, H. Kudo, T. Tsuboi, Y . Tanaka, and M. Katsuno, ”Using Self-learning Representations for Objective Assessment of Patient V oice in Dysphonia,” in Proc. APSIPA ASC, 2022, pp. 359-363

    2022

  8. [16]

    Representation learning with contrastive predictive coding,

    A. v. d. Oord, Y . Li, and O. Vinyals, “Representation learning with contrastive predictive coding,” arXiv preprint arXiv:1807.03748, 2018

    2018 arXiv

  9. [17]

    MERLIon CCS Challenge: A English-Mandarin code-switching child- directed speech corpus for language identification and diarization,

    V . Y . H. Chua, H. Liu, L. P. Garcia, F. T. Woon, J. Wong, X. Zhang, S. Khudanpur, A. W. H. Khong, J. Dauwels, and S. J. Styles, “MERLIon CCS Challenge: A English-Mandarin code-switching child- directed speech corpus for language identification and diarization,” in Proc. Inte...

    2023

  10. [18]

    Monolingual recognizers fusion for code-switching speech recognition,

    T. Song, Q. Xu, H. Lu, L. Wang, H. Shi, Y . Lin, Y . Yang, and J. Dang, “Monolingual recognizers fusion for code-switching speech recognition,” arXiv preprint arXiv:2211.01046, 2022

    2022 arXiv

  11. [19]

    Language-specific characteristic assistance for code-switching speech recognition,

    T. Song, Q. Xu, M. Ge, L. Wang, H. Shi, Y . Lv, Y . Lin, and J. Dang, “Language-specific characteristic assistance for code-switching speech recognition,” in Proc. Interspeech, 2022, pp. 3924–3928

    2022

  12. [20]

    Exploration of adapter for noise robust automatic speech recognition,

    H. Shi and T. Kawahara, “Exploration of adapter for noise robust automatic speech recognition,” arXiv preprint arXiv:2402.18275, 2024

    2024 arXiv

  13. [21]

    Speech emotion recogni- tion with multi-level acoustic and semantic information extraction and interaction,

    Y . Gao, H. Shi, C. Chu, and T. Kawahara, “Speech emotion recogni- tion with multi-level acoustic and semantic information extraction and interaction,” in Proc. Interspeech, 2024, pp. 1060–1064

    2024

  14. [22]

    Adapting the adapters for code-switching in multilingual asr,

    A. Kulkarni, A. Kulkarni, M. Couceiro, and H. Aldarmaki, “Adapting the adapters for code-switching in multilingual asr,” arXiv preprint arXiv:2310.07423, 2023

    2023 arXiv

  15. [23]

    Bi-encoder transformer network for mandarin-english code-switching speech recognition using mixture of experts,

    Y . Lu, M. Huang, H. Li, J. Guo, and Y . Qian, “Bi-encoder transformer network for mandarin-english code-switching speech recognition using mixture of experts,” in Proc. Interspeech, 2020, pp. 4766–4770

    2020

  16. [24]

    LAE: Language-aware encoder for monolingual and multilingual asr,

    J. Tian, J. Yu, C. Zhang, C. Weng, Y . Zou, and D. Yu, “LAE: Language-aware encoder for monolingual and multilingual asr,” in Proc. Interspeech, 2022, pp. 3178–3182

    2022

  17. [25]

    LAE-ST-MOE: Boosted language-aware encoder using speech translation auxiliary task for E2E code-switching ASR,

    G. Ma, W. Wang, Y . Li, Y . Yang, B. Du, and H. Fu, “LAE-ST-MOE: Boosted language-aware encoder using speech translation auxiliary task for E2E code-switching ASR,” in Proc. ASRU, 2023, pp. 1–8

    2023

  18. [26]

    wav2vec 2.0: A framework for self-supervised learning of speech representations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” Advances in neural information processing systems, vol. 33, pp. 12449– 12460, 2020

    2020

  19. [27]

    Mandarin-english code-switching speech recognition with self-supervised speech represen- tation models,

    L.-H. Tseng, Y .-K. Fu, H.-J. Chang, and H.-y. Lee, “Mandarin-english code-switching speech recognition with self-supervised speech represen- tation models,” arXiv preprint arXiv:2110.03504, 2021

    2021 arXiv

  20. [28]

    Attention-guided adaptation for code-switching speech recognition,

    B. Aditya, M. Rohmatillah, L.-H. Tai, and J.-T. Chien, “Attention-guided adaptation for code-switching speech recognition,” in Proc. ICASSP, 2024, pp. 10256–10260

    2024

  21. [29]

    Serialized speech information guidance with overlapped encoding separation for multi-speaker auto- matic speech recognition,

    H. Shi, Y . Gao, Z. Ni, and T. Kawahara, “Serialized speech information guidance with overlapped encoding separation for multi-speaker auto- matic speech recognition,” arXiv preprint arXiv:2409.00815, 2024

    2024 arXiv

  22. [30]

    Prompting the hidden talent of web-scale speech models for zero-shot task generalization,

    P. Peng, B. Yan, S. Watanabe, and D. Harwath, “Prompting the hidden talent of web-scale speech models for zero-shot task generalization,” in Proc. Interspeech, 2023, pp. 396–400

    2023

  23. [31]

    Adapting openai’s whisper for speech recognition on code-switch mandarin- english seame and asru2019 datasets,

    Y . Yang, Y . Peng, X. Zhong, H. Huang, and E.-S. Chng, “Adapting openai’s whisper for speech recognition on code-switch mandarin- english seame and asru2019 datasets,” arXiv preprint arXiv:2311.17382, 2023

    2023 arXiv

  24. [32]

    Parameter-efficient transfer learning for nlp,

    N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. de Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-efficient transfer learning for nlp,” in Proc. ICML, 2019, pp. 2790–2799

    2019

  25. [33]

    Exploration of whisper fine-tuning strategies for low-resource asr,

    Y . Liu, X. Yang, and D. Qu, “Exploration of whisper fine-tuning strategies for low-resource asr,” EURASIP Journal on Audio, Speech, and Music Processing, vol. 2024, no. 1, pp. 29, 2024

    2024

  26. [34]

    An adapter based multi-label pre-training for speech separation and enhancement,

    T. Wang, X. Chen, Z. Chen, S. Yu, and W. Zhu, “An adapter based multi-label pre-training for speech separation and enhancement,” in Proc. ICASSP, 2023, pp. 1–5

    2023

  27. [35]

    Adapter Integration: Mitigating catastrophic forgetting in multi-language and multi-accent whisper asr model fine-tuning,

    Z. Huang, H. Xing, and M. Liu, “Adapter Integration: Mitigating catastrophic forgetting in multi-language and multi-accent whisper asr model fine-tuning,”

  28. [36]

    LoRA- Whisper: Parameter-efficient and extensible multilingual asr,

    Z. Song, J. Zhuo, Y . Yang, Z. Ma, S. Zhang, and X. Chen, “LoRA- Whisper: Parameter-efficient and extensible multilingual asr,” arXiv preprint arXiv:2406.06619, 2024

    2024 arXiv

  29. [37]

    Mandarin–english code- switching speech corpus in south-east asia: Seame,

    D.-C. Lyu, T.-P. Tan, E.-S. Chng, and H. Li, “Mandarin–english code- switching speech corpus in south-east asia: Seame,” Language Resources and Evaluation, vol. 49, pp. 581–600, 2015

    2015

  30. [38]

    Internal language model estimation based language model fusion for cross-domain code-switching speech recognition,

    Y . Peng, Y . Liu, J. Zhang, H. Xu, Y . He, H. Huang, and E. S. Chng, “Internal language model estimation based language model fusion for cross-domain code-switching speech recognition,” arXiv preprint arXiv:2207.04176, 2022

    2022 arXiv

  31. [39]

    Language-specific acoustic boundary learning for mandarin-english code-switching speech recognition,

    Z. Fan, L. Dong, C. Shen, Z. Liang, J. Zhang, L. Lu, and Z. Ma, “Language-specific acoustic boundary learning for mandarin-english code-switching speech recognition,” arXiv preprint arXiv:2306.05279, 2023

    2023 arXiv

  32. [40]

    Reducing language confusion for code-switching speech recognition with token-level language diarization,

    H. Liu, H. Xu, L. P. Garcia, A. WH Khong, Y . He, and S. Khudanpur, “Reducing language confusion for code-switching speech recognition with token-level language diarization,” in Proc. ICASSP, 2023, pp. 1–5

    2023

  33. [41]

    Cyclic transfer learning for mandarin-english code-switching speech recognition,

    C. H. Nga, D.-Q. Vu, H. H. Luong, C.-L. Huang, and J.-C. Wang, “Cyclic transfer learning for mandarin-english code-switching speech recognition,” IEEE Signal Processing Letters, 2023

    2023

  34. [42]

    Aligning speech to languages to enhance code-switching speech recognition,

    H. Liu, X. Zhang, L. P. Garcia, A. WH Khong, E. S. Chng, and S. Watanabe, “Aligning speech to languages to enhance code-switching speech recognition,” arXiv preprint arXiv:2403.05887, 2024

    2024

  35. [43]

    An effective mixture-of-experts approach for code-switching speech recog- nition leveraging encoder disentanglement,

    T.-T. Yang, H.-W. Wang, Y .-C. Wang, C.-H. Lin, and B. Chen, “An effective mixture-of-experts approach for code-switching speech recog- nition leveraging encoder disentanglement,” in Proc. ICASSP, 2024, pp. 11226–11230

    2024

  36. [44]

    Intermediate-layer output regularization for attention-based speech recognition with shared decoder,

    J. Zhang, Y . Peng, H. Xu, Y . He, E. S. Chng, and H. Huang, “Intermediate-layer output regularization for attention-based speech recognition with shared decoder,” arXiv preprint arXiv:2207.04177, 2022

    2022 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.