Pith. sign in

REVIEW 4 major objections 4 minor 27 references

Fine-tuning on adversarial text embeddings—injected into an aligned speech-text embedding space—improves speech translation robustness to inflectional errors by over 3 BLEU points across four language pairs, with no adversarial speech neede

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 23:56 UTC pith:XW7BOJHQ

load-bearing objection Solid empirical paper with a genuinely new robustness-transfer idea; the 3+ BLEU gain is consistent, but the mechanism is under-validated and the evaluation lacks variance and a non-adversarial control. the 4 major comments →

arxiv 2602.11933 v2 pith:XW7BOJHQ submitted 2026-02-12 cs.CL

Cross-Modal Robustness Transfer (CMRT): Training Robust Speech Translation Models Using Adversarial Text

Abderrahmane Issam , Yusuf Can Semerci , Jan Scholtes , Gerasimos Spanakis This is my paper
classification cs.CL
keywords cross-modal robustness transferspeech translationadversarial robustnessmorphological inflectioncontrastive learningmixup trainingSpeech-MORPHEUSrobustness-accuracy trade-off
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that acoustic robustness—here, resilience to inflectional errors typical of non-native or dialectal speech—can be transferred from the text modality to the speech modality without ever generating adversarial audio. The method first trains a speech-to-text translation model with word-level contrastive learning and mixup training so that speech and text embeddings of the same word occupy a shared semantic space. Then it fine-tunes the model by swapping in adversarial text embeddings for inflected words, keeping the speech encoder frozen. Across English-to-German/Catalan/Arabic and French-to-English, this raises BLEU on the adversarial spoken test set by more than 3 points over a strong baseline, reaching within about 1.4 BLEU of direct TTS-side adversarial fine-tuning while losing far less on clean speech. The paper's own plateau analysis shows the transfer is partial, and the limitations section acknowledges a slight clean-set drop on English-to-X directions and the extra training stage—so the robustness gain is not free.

Core claim

On the paper's own terms, the central discovery is that morphological robustness transfers across modalities through an aligned embedding space. Using a combination of word-aligned contrastive learning, mixup training, and symmetric KL regularisation, the authors construct a model (CMRT-TR) whose speech and text representations for the same word are close. Fine-tuning this model (CMRT-FN) with adversarial text embeddings—keeping the speech encoder frozen and using an asymmetric KL to keep the adversarial mixup's output distribution close to that of clean inputs—produces a model that, on the Speech-MORPHEUS adversarial test set, gains an average of 3.4 BLEU over HuBERT-Transformer and 3.1 ove

What carries the argument

The load-bearing mechanism is the aligned speech-text embedding space built by CMRT-TR, combined with an adversarial-mixup fine-tuning step (CMRT-FN). Word-aligned Contrastive Learning (WACO) pulls speech and text embeddings of the same word together; mixup training and symmetric KL-divergence familiarise the translation encoder-decoder with mixed representations. In the robustness stage, the frozen speech encoder's output for attacked words is replaced by adversarial text embeddings (the inflected form), and an asymmetric KL loss pushes the model's distribution on that adversarial mixup toward its distributions on clean speech and text. The paper also constructs the evaluation tool, Speech-

Load-bearing premise

The whole method rests on the assumption that the frozen speech encoder, at test time, produces embeddings for genuinely inflected speech that lie close enough to the adversarial text embeddings used in fine-tuning that the text substitution is equivalent to training on the real acoustic perturbation; the paper's own plateau result shows this holds only partially, and if the gap is large the measured gains reflect matching to the MORPHEUS text distribution rather than true cr

What would settle it

Measure the cosine distance between the adversarial text embeddings used in CMRT-FN and the actual speech-encoder embeddings of the same inflected words produced by the TTS system. If the distance is systematically large, or if fine-tuning with random (non-adversarial) text embeddings yields the same BLEU gain on Speech-MORPHEUS, the claimed cross-modal transfer mechanism is not what delivers the robustness.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Speech translation systems can be hardened against a realistic class of morphological variation at a fraction of the compute cost of synthetic adversarial audio.
  • The robustness–accuracy trade-off is partially decoupled: CMRT-FN's clean-set loss (~0.6 BLEU) is much smaller than the 3.6 BLEU loss of TTS-side adversarial fine-tuning.
  • The transfer mechanism scales to massively pre-trained translation backbones (NLLB-initialised models), where CMRT-FN gains ~4.4 BLEU on adversarial speech, suggesting the method is not tied to a small architecture.
  • Word-level alignment quality becomes a measurable driver of adversarial robustness, giving representation-learning objectives a concrete safety payoff.
  • Because the method requires no TTS at training time, it can be applied to language pairs where high-fidelity TTS does not exist.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the mechanism is genuinely cross-modal, the same trick should transfer other text-side corruptions (typos, dialectal lexicon) to speech without audio-side data — a testable extension beyond inflectional morphology.
  • The plateau below TTS-side fine-tuning (§5.3) suggests that contrastive alignment leaves residue of modality-specific information; methods that more aggressively minimise the modality gap might close the remaining ~1.4 BLEU.
  • A control experiment replacing adversarial text embeddings with arbitrary inflected text embeddings would determine whether the gains come from adversarial targeting or merely from mixup regularisation; the paper does not report this control.
  • Since Speech-MORPHEUS is TTS-generated, part of the measured robustness may be an artefact of TTS normalisation; the 'Clean' column of Table 1 provides a partial baseline for this confound.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes Cross-Modal Robustness Transfer (CMRT), a two-stage method for improving the robustness of end-to-end speech translation (E2E-ST) models to inflectional morphological perturbations without using adversarial speech during training. In the first stage (CMRT-TR), the model is trained to align speech and text representations via word-aligned contrastive learning (WACO), mixup training, and symmetric KL divergence. In the second stage (CMRT-FN), the speech encoder is frozen and the translation encoder-decoder is fine-tuned on "adversarial mixup" sequences in which the embeddings of words attacked by the MORPHEUS inflection generator are replaced by their adversarial text embeddings, with asymmetric KL losses to clean speech/text outputs. The paper also introduces Speech-MORPHEUS, a TTS-based adaptation of MORPHEUS, and reports experiments on CoVoST 2 (En-De, En-Ca, En-Ar, Fr-En) showing average gains of 3.4 BLEU over HuBERT-Transformer and 3.1 over CMRT-TR on the adversarial test set, while falling 1.4 BLEU short of TTS-side adversarial fine-tuning, and a smaller clean-set accuracy drop. An appendix extends the results to NLLB-initialized models with an average gain of 4.4 BLEU.

Significance. If the central claim holds, the paper would make a useful practical contribution: it addresses a real and under-studied failure mode of E2E-ST models (morphological robustness), and it proposes a comparatively cheap method that avoids large-scale adversarial TTS generation. The empirical scope is reasonable—four language pairs, a strong TTS-side adversarial training baseline, a clean-TTS control, and an appendix with a massively pre-trained initialization. The paper is also honest about the robustness-accuracy trade-off and about the plateau relative to speech-side fine-tuning in Section 5.3. However, the current evidence does not yet establish the proposed mechanism. The training and test perturbations come from the same MORPHEUS generator, the central embedding-gap assumption is never directly measured, and the design lacks a control that isolates adversarial text substitution from the regularizing effect of the KL losses. These issues are fixable with additional experiments and analysis, but they are load-bearing for the paper's key claims.

major comments (4)
  1. [§5.1, Table 1] The headline results are point estimates from single runs, with no variance, confidence intervals, or significance testing. For example, the En-Ar Morpheus column reports 14.5 for CMRT-FN versus 12.7 for CMRT-TR, and the Fr-En difference is 24.6 versus 21.9; these differences may fall within normal test-set noise for ST BLEU. Since the main contribution is quantitative (average +3.4/+3.1, and +4.4 in Appendix B), please report multiple seeds with means and standard deviations, or at least paired bootstrap confidence intervals over test segments, and state whether the per-direction and average gains are statistically significant.
  2. [§3.3, §5.1] There is a distribution-matching concern: the training-time adversarial text is produced by the MORPHEUS generator, and the Speech-MORPHEUS test set is produced by the same generator's inflections rendered with XTTS-v2. The measured gains could therefore reflect adaptation to the specific inflection inventory and selection behavior of MORPHEUS rather than general morphological robustness. To support the claim of robustness transfer, the evaluation should include held-out perturbation types (e.g., a different inflector, a different POS inventory, or a held-out subset of inflections) and TTS voices/speakers not used in development. This is a load-bearing issue because the paper's stated goal is robustness to real non-native/dialectal morphological variation, not just one attack generator.
  3. [§3.2.3, Eq. (14), §5.3] The core mechanism is unverified. The paper claims that injecting adversarial text embeddings into the speech manifold simulates adversarial speech perturbations, which requires that the frozen speech encoder's test-time embeddings of TTS-rendered inflected words lie close to the text embeddings used during CMRT-FN. This is never directly measured. Figure 3 is correlational (alignment strength is confounded with training objective), and the CKA analysis in §5.4 uses whole-sentence mean representations, which dilutes the effect of attacked words. Moreover, §5.3 explicitly reports a plateau below TTS-Morpheus-FN and attributes it to 'modality specific information that limits cross-modal transfer'—an admission that the assumption holds only partially. Please measure the cosine distance between text embeddings of inflected forms and speech-encoder embeddings of the corresponding XTTS audio (
  4. [Eqs. (15)-(18), §5.2] The gains attributed to adversarial text substitution could instead come from the asymmetric KL losses (Eqs. 16-17), which resemble VAT-style regularization, or from additional fine-tuning on the clean ST/MT objectives. The paper lacks a control that isolates the adversarial substitution component. Please add ablations in which (a) the same loss is used but clean text embeddings are inserted at attacked positions, (b) random non-adversarial text substitutions are used at attacked positions, and (c) CMRT-TR is fine-tuned with only the clean ST/MT and KL losses. If (a) or (b) recovers most of the gain, then the claim that adversarial text embeddings are the operative factor is not supported.
minor comments (4)
  1. [Abstract] Typo: 'highly vulnerable it' should be 'highly vulnerable to it'.
  2. [Table 1] The difference between 'CMRT-FN (50K)' and 'CMRT-FN' is not explained in the text. Please clarify whether both use 50K adversarial text examples, and if not, what dataset size the second variant uses and why it is reported alongside the 50K variant.
  3. [Figure 3] Please add axis labels, specify which BLEU score is plotted (adversarial test set), include error bars or a confidence interval for the cosine similarity, and report the correlation coefficient and its significance. The current caption is too underspecified for the strong qualitative claim.
  4. [Appendix B] Typo: 'On overage' should be 'On average'.

Circularity Check

0 steps flagged

No derivation-level circularity: CMRT's text-to-speech robustness transfer is an empirically tested cross-modal hypothesis, not an identity by construction; the only self-citation is motivational.

full rationale

The claimed derivation chain is: (1) align speech and text representations with WACO and mixup (Eqs. 1-13), (2) fine-tune by injecting MORPHEUS adversarial text embeddings into the frozen speech encoder's output (Eqs. 14-18), and (3) evaluate on Speech-MORPHEUS (Sec. 3.3). None of these equations equates the training input with the test input. Eq. (14) defines an adversarial mixup sequence from text embeddings of inflected words, but the test-time input is TTS audio of inflected transcriptions passed through the same frozen HuBERT/mHuBERT encoder; whether those speech encoder outputs are close to the injected text embeddings is the empirical hypothesis tested in Sec. 5.1, not a definition. The 3+ BLEU gain is measured on audio, so it is not forced by construction. The self-citation to Issam et al. (2025) in Sec. 3.2.1 is used only to motivate why textual syntactic features are relevant to NMT robustness; the paper's own WACO/mixup ablation (Fig. 3) and the TTS-Morpheus-FN comparison provide direct, non-self-referential evidence, so the citation is not load-bearing. The strongest concern is that CMRT-FN's text perturbations and the Speech-MORPHEUS test set are generated by the same MORPHEUS procedure (Sec. 4.1: 'we use Speech-MORPHEUS ... to generate an adversarial copy of 50k examples sampled from the train set, and a copy of the dev and test sets'). This restricts the robustness claim to the defended attack family, but it is a scope limitation rather than a circular reduction: the fine-tuning signal is text embeddings and the evaluation is audio, and Sec. 5.3 explicitly concedes imperfect transfer ('CMRT-FN plateaus below TTS-Morpheus-FN suggests that the model retains modality specific information that limits cross-modal transfer'). No uniqueness theorem, imported ansatz, or renamed known result appears. The derivation is therefore self-contained against the stated benchmarks; any remaining doubts concern external validity and mechanism verification, not circularity.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The central claim rests on the alignment assumption (text perturbations approximate speech perturbations), the validity of TTS-rendered MORPHEUS as a proxy for non-native speech, and forced-alignment accuracy — all domain assumptions inherited from prior work rather than established here. Four hyperparameters/choices affect the headline numbers but are conventional tunings. No invented entities; CMRT and Speech-MORPHEUS are procedures, not entities with independent falsifiable handles.

free parameters (4)
  • KL divergence weight λkl = 2.0 (CMRT-TR), 5.0 (CMRT-FN)
    Tuned on the dev set; §5.3 shows it directly trades adversarial BLEU against original-test BLEU, so the reported headline numbers depend on this choice.
  • mixup probability p* = 0.812 (CMRT-TR), 0.80 (CMRT-FN)
    Set by hand; footnote 12 reports that decreasing it hurts original CoVoST 2 performance without improving alignment.
  • adversarial fine-tuning data size = 50k examples
    Chosen as 'a reasonable amount' (§5.1 footnote 3); results are reported with and without it, and the 50k run is consistently 0.3–0.5 BLEU lower, indicating data-scale sensitivity.
  • contrastive temperature τ and weight λctr = τ=0.2, λctr=1.0
    Standard hyperparameters inherited from WACO (§3.2.2, Appendix A); part of the alignment recipe, not the central claim.
axioms (5)
  • domain assumption Strong word-level alignment between speech and text embeddings is achievable and sufficient for text-side perturbations to approximate speech-side perturbations (the embedding-gap assumption).
    Invoked in §1 hypothesis and §3.2.3, where the frozen speech encoder is bypassed by injecting adversarial text embeddings into the mixup sequence; §5.3 admits the model 'retains modality specific information that limits cross-modal transfer,' so the assumption holds only partially.
  • domain assumption MORPHEUS-generated inflectional substitutions are meaning-preserving and representative of non-native/dialectal speech errors.
    Adopted from Tan et al. 2020 (§3.3), whose human-evaluation result was for written English; the paper extends this to spoken Catalan, Arabic, and French TTS without a new human study.
  • domain assumption TTS (XTTS-v2) renderings of inflected transcriptions preserve the morphological error acoustically, and the eSpeak-based homophone filter guarantees audibility (French).
    §3.3 and Appendix A; the test sets are entirely TTS-based, so the validity of the robustness numbers depends on this.
  • domain assumption NeMo Forced Aligner word boundaries are accurate enough to support contrastive pooling and mixup segment selection.
    Appendix A preprocessing; alignment errors would misassign positive pairs and mixup segments, degrading the claimed alignment.
  • standard math Standard ML background (InfoNCE-style contrastive loss, symmetric/asymmetric KL divergence, mixup interpolation) behaves as published.
    Equations 3, 11–13, 16–18; no new mathematical claims are made.

pith-pipeline@v1.3.0-alltime-deepseek · 14759 in / 18988 out tokens · 161464 ms · 2026-08-02T23:56:11.447799+00:00 · methodology

0 comments
read the original abstract

End-to-End Speech Translation (E2E-ST) has seen significant advancements, yet current models are primarily benchmarked on curated, "clean" datasets. This overlooks critical real-world challenges, such as morphological robustness to inflectional variations common in non-native or dialectal speech. In this work, we adapt a text-based adversarial attack targeting inflectional morphology to the speech domain and demonstrate that state-of-the-art E2E-ST models are highly vulnerable it. While adversarial training effectively mitigates such risks in text-based tasks, generating high-quality adversarial speech data remains computationally expensive and technically challenging. To address this, we propose Cross-Modal Robustness Transfer (CMRT), a framework that transfers adversarial robustness from the text modality to the speech modality. Our method eliminates the requirement for adversarial speech data during training. Extensive experiments across four language pairs demonstrate that CMRT improves adversarial robustness by an average of more than 3 BLEU points, establishing a new baseline for robust E2E-ST without the overhead of generating adversarial speech.

Figures

Figures reproduced from arXiv: 2602.11933 by Abderrahmane Issam, Gerasimos Spanakis, Jan Scholtes, Yusuf Can Semerci.

Figure 1
Figure 1. Figure 1: CMRT aligns speech and text semantic spaces [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: An overview illustration of our proposed [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The figure shows the correlation between the [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: We compare the CKA similarity of repre￾sentations of adversarial sentences of CMRT-TR and CMRT-FN models against TTS-Morpheus-FN. CMRT￾FN model is more aligned with TTS-Morpheus-FN be￾cause both are trained to handle Morpheus errors. the sequence representations of each sentence and use Centered Kernel Alignment (CKA) (Kornblith et al., 2019) to compute the similarity of CMRT-FN and CMRT-TR with TTS-Morphe… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

27 extracted references · 5 linked inside Pith

  1. [6]

    MuST-C: a Multilingual Speech Translation Corpus. InProceed- ings of the 2019 Conference of the North American Chapter of the Association for Computational Lin- guistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2012–2017, Min- neapolis, Minnesota. Association for Computational Linguistics. Qianqian Dong, Mingxuan Wang, Hao Zho...

    2019

  2. [9]

    Spoken and written produc- tion of inflectional morphology among l1 mandarin speakers of english.Journal of Memory and Lan- guage, 120:104250. U. Gut. 2009.Non-native Speech: A Corpus-based Analysis of Phonological and Phonetic Properties of L2 English and German. English corpus linguistics. Peter Lang. Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, ...

    2009

  3. [10]

    InProceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, pages 1872–1881, Online

    Source and target bidirectional knowledge distillation for end-to-end speech trans- lation. InProceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, pages 1872–1881, Online. Association for Computational Linguistics. Sathish Indurthi, Shamil Chollampatt, Ravi Agrawa...

    2021

  4. [11]

    InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9049–9056, Singapore

    CLAD-ST: Contrastive learning with adversarial data for robust speech trans- lation. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9049–9056, Singapore. Association for Com- putational Linguistics. Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes, and Gerasimos Spanakis

    2023

  5. [13]

    Similarity of neural network representations revisited. InProceedings of the 36th International Conference on Machine Learn- ing, ICML 2019, 9-15 June 2019, Long Beach, Cali- fornia, USA, volume 97 ofProceedings of Machine Learning Research, pages 3519–3529. PMLR. Xiang Li, Haiyang Xue, Wei Chen, Yang Liu, Yang Feng, and Qun Liu

    2019

  6. [14]

    Weixin Liang, Yuhui Zhang, Yongchan Kwon, Serena Yeung, and James Zou

    Improving the robustness of speech translation.Preprint, arXiv:1811.00728. Weixin Liang, Yuhui Zhang, Yongchan Kwon, Serena Yeung, and James Zou

    Pith/arXiv arXiv

  7. [16]

    Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli

    No language left behind: Scal- ing human-centered machine translation.Preprint, arXiv:2207.04672. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli

    Pith/arXiv arXiv

  8. [17]

    InProceedings of the 2019 Con- ference of the North American Chapter of the Associa- tion for Computational Linguistics (Demonstrations), pages 48–53, Minneapolis, Minnesota

    fairseq: A fast, extensible toolkit for sequence modeling. InProceedings of the 2019 Con- ference of the North American Chapter of the Associa- tion for Computational Linguistics (Demonstrations), pages 48–53, Minneapolis, Minnesota. Association for Computational Linguistics. Siqi Ouyang, Rong Ye, and Lei Li

    2019

  9. [18]

    InProceedings of the 18th International Conference on Spoken Lan- guage Translation (IWSLT 2021), pages 236–244, Bangkok, Thailand (online)

    Inverted projec- tion for robust speech translation. InProceedings of the 18th International Conference on Spoken Lan- guage Translation (IWSLT 2021), pages 236–244, Bangkok, Thailand (online). Association for Compu- tational Linguistics. Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu

    2021

  10. [19]

    InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7175–7188, Singapore

    Accented speech recognition with accent-specific codebooks. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7175–7188, Singapore. Association for Computational Linguistics. Philippe Prévost and Lydia White

    2023

  11. [20]

    Nemo forced aligner and its application to word align- ment for subtitle generation. InProc. Interspeech 2023, pages 5257–5258. Mirco Ravanelli, Jianyuan Zhong, Santiago Pascual, Pawel Swietojanski, Joao Monteiro, Jan Trmal, and Yoshua Bengio

    2023

  12. [21]

    InICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6989–6993

    Multi-task self-supervised learning for robust speech recognition. InICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6989–6993. Andrew Rosenberg, Yu Zhang, Bhuvana Ramabhad- ran, Ye Jia, Pedro Moreno, Yonghui Wu, and Zelin Wu

    2020

  13. [22]

    InInterna- tional Conference on Learning Representations

    Ro- bustness may be at odds with accuracy. InInterna- tional Conference on Learning Representations. Changhan Wang, Anne Wu, and Juan Pino. 2020a. Cov- ost 2: A massively multilingual speech-to-text trans- lation corpus.Preprint, arXiv:2007.10310. Chengyi Wang, Yu Wu, Shujie Liu, Zhenglu Yang, and Ming Zhou. 2020b. Bridging the gap between pre- training a...

    Pith/arXiv arXiv 2007

  14. [23]

    Cross- modal contrastive learning for speech translation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 5099–5113, Seattle, United States. Association for Computational Linguistics. Marcely Zanon Boito, Varun Iyer, Nathanaël Lagos, Laurent Besacier...

    2022

  15. [25]

    InICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Process- ing (ICASSP), pages 11041–11045

    Soft align- ment of modality space for end-to-end speech trans- lation. InICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Process- ing (ICASSP), pages 11041–11045. Yan Zhou, Qingkai Fang, and Yang Feng

    2024

  16. [26]

    InProceedings of the 61st Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7873–7887, Toronto, Canada

    CMOT: Cross-modal mixup via optimal transport for speech translation. InProceedings of the 61st Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7873–7887, Toronto, Canada. Association for Computational Linguistics. A Experimental Setup Details Speech-MorpheusMorpheus was introduced ini- tially for English o...

    2021

  17. [27]

    B CMRT Improves Robustness of Massively Pre-trained Models Table 2 shows the results of using No Language Left Behind (NLLB) (NLLB Team et al.,

    to compute detokenized case-sensitive BLEU score (Papineni et al., 2002). B CMRT Improves Robustness of Massively Pre-trained Models Table 2 shows the results of using No Language Left Behind (NLLB) (NLLB Team et al.,

    2002

  18. [2011]

    English, chinglish or china en- glish?: Analysing chinglish, chinese english and china english.English Today, 27(4):64–70. David M. Eberhard, Gary F. Simons, and Charles D. Fennig, editors. 2025.Ethnologue: Languages of the World, twenty-eighth edition. SIL International, Dallas, Texas. Qingkai Fang, Rong Ye, Lei Li, Yang Feng, and Mingxuan Wang

    2025

  19. [2016]

    Learning aligned cross-modal representations from weakly aligned data.2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2940–

    2016

  20. [2018]

    Data augmentation improves recognition of foreign accented speech. In Proc. Interspeech 2018, pages 2409–2413. Qingyuan Gardner, Holly P. Branigan, and Vasiliki Chondrogianni

    2018

  21. [2019]

    Neural machine translation of text from non-native speakers. InProceedings of the 2019 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3070–3080, Minneapolis, Min- nesota. Association for Computational Linguistics. 5https://voxreality.eu/ Rosa...

    2019

  22. [2020]

    Giuseppe Martucci, Mauro Cettolo, Matteo Negri, and Marco Turchi

    Bridging the modality gap for speech- to-text translation.ArXiv, abs/2010.14920. Giuseppe Martucci, Mauro Cettolo, Matteo Negri, and Marco Turchi

    Pith/arXiv arXiv 2010

  23. [2021]

    InProceedings of the 18th International Conference on Spoken Language Trans- lation (IWSLT 2021), pages 1–29, Bangkok, Thailand (online)

    FINDINGS OF THE IWSLT 2021 EV AL- UATION CAMPAIGN. InProceedings of the 18th International Conference on Spoken Language Trans- lation (IWSLT 2021), pages 1–29, Bangkok, Thailand (online). Association for Computational Linguistics. Antonios Anastasopoulos, Alison Lui, Toan Q. Nguyen, and David Chiang

    2021

  24. [2022]

    InICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Process- ing (ICASSP), pages 4298–4302

    Noise-robust speech recognition with 10 minutes unparalleled in-domain data. InICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Process- ing (ICASSP), pages 4298–4302. Seamless Communication, Loïc Barrault, Yu-An Chung, Mariano Coria Meglioli, David Dale, Ning Dong, Mark Duppenthaler, Paul-Ambroise Duquenne, Brian Ellis, Ha...

    2022

  25. [2023]

    Preprint, arXiv:2312.05187

    Seamless: Multi- lingual expressive and streaming speech translation. Preprint, arXiv:2312.05187. Mattia A. Di Gangi, Roldano Cattoni, Luisa Bentivogli, Matteo Negri, and Marco Turchi

    Pith/arXiv arXiv

  26. [2024]

    mhubert-147: A compact multilingual hubert model. InProc. Interspeech 2024, pages 3939–3943. Chengwei Zhang, Yue Zhou, Rui Zhao, Yidong Chen, and Xiaodong Shi

    2024

  27. [2025]

    InFindings of the Association for Com- putational Linguistics: ACL 2025, pages 8579–8601, Vienna, Austria

    A representation level analysis of NMT model robustness to grammat- ical errors. InFindings of the Association for Com- putational Linguistics: ACL 2025, pages 8579–8601, Vienna, Austria. Association for Computational Lin- guistics. Sai Muralidhar Jayanthi and Adithya Pratapa

    2025