Optimal Multi-Task Learning at Regularization Horizon for Speech Translation Task

Jungho Jung; Junhyun Lee

REVIEW 4 major objections 4 minor 41 references

Optimal Multi-Task Learning at Regularization Horizon for Speech Translation Task

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Four tuning knobs collapse into one scalar in speech translation

desk verdict Useful empirical sweep of consistency losses for ST/MT MTL, but the 'regularization horizon' is a heuristic rather than a demonstrated collapse onto one scalar. read the letter →

arxiv 2509.09701 v1 pith:GLS7R4AH submitted 2025-09-04 cs.CL

JungHo Jung , Junhyun Lee This is my paper · ORCID

classification cs.CL

keywords speechtranslationmulti-tasklearningconsistencyregularizationR-droptotalhorizonMuST-CBLEU

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Speech-to-text translation suffers from scarce paired speech-to-text data, so models are commonly trained jointly with a text machine-translation task. This paper claims that in such a setup, every major tuning knob—the weight of cross-modal consistency loss, the weight of R-drop on the same speech input, the weight of the auxiliary MT loss, and the dropout rate—is really one knob called total regularization. The authors estimate total regularization R as a linear combination of the four hyperparameters from BLEU scores measured in the over-regularized regime, then show that all their measured BLEU scores collapse onto a single curve when plotted against R. The optimum of that curve, the "regularization horizon," is where performance peaks; choosing hyperparameters near it yields near state-of-the-art BLEU on MuST-C across four languages. The reader should care because it suggests that grid-searching four independent hyperparameters can be replaced by pursuing one scalar target, and it gives a concrete explanation of why lowering the MT-loss weight behaves like raising dropout or consistency strength.

What carries the argument

The machinery is the "total regularization" scalar R from Eq. (7), a linear combination of the four regularization-related hyperparameters, together with the regression BLEU = βR·R + βB from Eq. (8). The authors fix βR = −1 for scale, identify βf by setting f(0,0,1,0)=0, and fit the remaining coefficients only on the over-regularized (post-peak) points of the dropout sweeps. The resulting collapse of all points onto a single BLEU-versus-R curve is the argument that R is a valid control variable; the "regularization horizon" is the peak region of that curve, the optimal surface separating under-regularized from over-regularized hyperparameter combinations. The linear form is explicitly an app

What would settle it

Hold R constant along an iso-R contour computed from the fitted coefficients, e.g., trade a large increase in αcr against a compensating drop in dropout, and measure BLEU. If the collapse is real, all points on the contour give the same BLEU within bootstrap noise; if points drift by more than about 1 BLEU, the linear additive model fails. A second test: use the same coefficients to predict BLEU on a held-out language pair and check whether the under-regularized points (those with R below the horizon) still lie on the fitted curve, since they were not used in the regression.

Watch

Extended reading notes

Core claim

The central claim is that a multi-task speech-translation model is controlled by a single scalar, total regularization R, not by four independent loss and dropout hyperparameters. R is defined as βcr·αcr + βrd·αrd + βt·αt + βdo·(d/o) + βf, with αcr, αrd, αt the weights of consistency regularization, R-drop, and the MT loss, and d/o the dropout rate. After fitting the β coefficients on post-peak BLEU points, every measured configuration—including under-regularized points absent from the fit—collapses onto one BLEU-versus-R curve. The peak of this curve is the regularization horizon; hyperparameter points on it match or beat several published systems on MuST-C across four languages. The paper

Load-bearing premise

The load-bearing premise is that αcr, αrd, αt, and dropout add linearly into one total-regularization scalar R, with the fit made only on post-peak points; if the sources interact nonlinearly (as the αcr–αrd saturation suggests), the single-curve collapse may not hold outside the fitted region.

Editorial extensions

If this is right

Hyperparameter search can be reduced from four dimensions to one: set a target R at the horizon, then pick any combination of αcr, αrd, αt, and dropout that realizes it via Eq. (7).
Consistency losses should be placed at the softmax output with KL divergence; this choice beat the encoder, cross-attention, last-decoder-state, and logits placements for both consistency regularization and R-drop.
Lowering αt is a genuine regularization lever, interchangeable in direction with raising αcr, αrd, or dropout; the peak shifts accordingly when more regularization is already present.
Adding consistency regularization on top of R-drop saturates rather than compounds, behaving like m-time R-drop, so beyond the horizon extra regularization loss is wasted or harmful.
Horizon-tuned configurations carry across languages: the same αcr=5 or αrd=8 with dropout 0.05 settings that peak on en-de also give uniform gains on en-es, en-fr, and en-it.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the linear-collapse claim transfers, the same regression procedure could be run at a different data scale and the horizon should shift to smaller R; this predicts a concrete scaling relation testable without architectural changes.
The paper leaves the saturation between αcr and αrd unresolved in Eq. (7); a natural extension is to add an interaction term βcr·rd·αcr·αrd and check whether the BLEU-versus-R collapse improves or whether the residual is exactly the saturation zone.
Conceptually, R can be read as a perturbation budget: speech and text inputs are two corrupted views of one semantic content, and regularization enforces output consistency across corruption level; this unifies Cutoff-style augmentation, consistency regularization, and dropout under one budget and predicts that optimal R depends on training data size.
The fitted curve uses post-peak points only; a stricter test is to hold R fixed along an iso-R contour and verify BLEU stays flat within bootstrap noise, which would confirm that the horizon is a true ridge rather than a plateau artifact.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: the paper earns its keep on the empirical side. The systematic comparison of where to enforce consistency (encoder, cross-attention, decoder states, logits, softmax) and which distance metric to use (MSE, cosine, KL) is more thorough than prior work, and the conclusion that softmax-level KL works best is believable. The low-data advantage of R-drop over consistency regularization, and the narrowing of that gap when external MT data is available, is a concrete and useful observation. The saturation when combining the two losses (Table 2) is real, and the m-time R-drop analogy is a reasonable way to think about it. The multilingual check across four languages adds confidence. The writing is clear, and the authors are upfront that their linear total-regularization model is an oversimplification.

The soft spot is the central claim. Equation 7 assumes linear additivity of four regularization sources, but the paper's own saturation result says α_cr and α_rd do not add linearly. The regression is fit only on points visually selected as 'after the peak' in Figure 2, and the collapse in Figure 3 is judged by eye, with no error bars or goodness-of-fit measure. Because each hyperparameter is varied one at a time from a single default, the data lie on rays along coordinate axes; a linear model can always fit those rays. There is no experiment with simultaneous large α_cr and α_rd, which is exactly where an interaction would show up. So the single-scalar 'total regularization' is plausible as a local approximation, but it is not established. The Horizon is a mnemonic, not a law.

Who this is for: practitioners in speech translation who want guidance on where to apply consistency losses and how to think about tuning α_t. The layer/metric comparison and the saturation result are worth taking at face value. The horizon concept should be treated more cautiously.

Recommendation: this deserves a serious referee, because the empirical sweep is a legitimate contribution and the framing could be useful if properly qualified. But I would not accept it in current form. The authors need to either reframe the horizon as a conjecture or support it with interaction experiments, error bars, and a quantitative out-of-sample fit. The citation pattern looks appropriate; no red flags there.

Referee Report

4 major / 4 minor

Summary. The paper studies multi-task learning for end-to-end speech translation (ST) with an MT auxiliary task, viewing several MTL ingredients as sources of regularization: consistency regularization between speech and text modalities, R-drop within the speech modality, the MT loss weight α_t, and dropout rate. Experiments on MuST-C show that softmax-level KL consistency is the most effective, that R-drop is stronger in the low-data regime, and that combining the two consistency losses saturates. The authors then define a scalar 'total regularization' R as a linear function of α_cr, α_rd, α_t, and dropout (Eq. 7), fit a linear BLEU-on-R model (Eq. 8) to over-regularized points, and claim that all measured points collapse onto a single curve when plotted against R (Fig. 3). This collapse is used to define a 'regularization horizon,' the optimal region in hyperparameter space, where tuning yields near-state-of-the-art results across four languages (Table 4).

Significance. If the single-scalar collapse were rigorously established, it would provide a practically useful conceptual tool for navigating MTL hyperparameters in ST: instead of tuning four interacting knobs separately, one could aim for a target regularization level. The paper also contains solid empirical contributions: the systematic comparison of embedding layers and distance metrics (Table 1), the demonstration that α_t behaves as a regularization knob (Table 3), and the saturation observation when compounding consistency losses (Table 2) are valuable and reproducible. However, the central quantitative claim—that total regularization R as defined in Eq. (7) controls performance across all four parameters—is not yet established because the linear additive model is contradicted by the paper's own saturation result and because the regression is fit and evaluated in a partly circular, visually-based manner. The paper is honest about the linear approximation being an oversimplification, but the load-bearing evidence for the collapse needs additional quantitative support.

major comments (4)

[Section 4.2, Eq. (7)] Eq. (7) assumes total regularization is linear and additive in α_cr, α_rd, α_t, and dropout, with no interaction terms. This is contradicted by the paper's own Table 2: in the base setting, softmax-KL consistency alone gives 27.32 and softmax-KL R-drop alone gives 28.40, but adding consistency on top of R-drop gives 28.37, essentially no gain. If R were linear, the combined effect would not saturate as observed. The text in Section 4.2 itself acknowledges that 'the effect of having both αcr and αrd saturates,' yet Eq. (7) has no term to express that saturation. This is a load-bearing inconsistency: the single-scalar collapse in Fig. 3 depends on the validity of Eq. (7). The authors should either add an interaction/saturation term (e.g., a product or logarithmic term) and refit, or explicitly restrict the claim to a region where additivity is demonstrated, with data in that region.
[Section 4.2, Fig. 3] The regression is fit only to points visually selected as 'after the peak' in Fig. 2, and the collapse of the under-regularized points is asserted visually without a quantitative out-of-sample check. This is a circularity risk: the fitted coefficients are computed from over-regularized points by construction, so those points lie exactly on the fitted line. The under-regularized points are independent evidence, but the paper reports no error bars, confidence intervals, or goodness-of-fit measures (e.g., RMSE or R² on held-out under-regularized points). The coefficients in footnote 3 are reported without uncertainty. I request a quantitative evaluation of the collapse: define an explicit, non-visual criterion for 'after the peak,' fit on a subset, and report prediction errors for the points not used in the fit. Without this, the central claim that a single scalar controls performance is no
[Section 4.2, experimental grid] The grid search varies α_cr, α_rd, α_t, and dropout one at a time from the default (α_cr=1.0, α_rd=5.0, α_t=1.0, d/o varied). This means all data lie along coordinate axes. A linear model can always fit such data, and the 'collapse' in Fig. 3 may simply reflect the chosen grid trajectory rather than a genuine single-scalar law. The data provide no test of additivity at off-axis points, such as simultaneously large α_cr and α_rd. Please add experiments with joint variations (e.g., α_cr=5 with α_rd=8, and α_cr=5 with α_t=0.1, etc.) and test whether the same R values give the same BLEU regardless of how they are composed. This directly targets the validity of Eq. (7).
[Section 4.2, regularization horizon] The paper calls the optimal region a 'regularization horizon' and describes it in the abstract as 'the optimal regularization contour in the high-dimensional space.' However, the experiments only identify a few points on this surface (Table 4 uses two configurations, α_cr=5 and α_rd=8 with d/o=0.05). No surface is actually characterized, and Fig. 3 shows a one-dimensional curve, not a 3D surface in 4D parameter space. The concept is interesting, but the claim is overstated relative to the evidence. The paper should either temper the terminology or provide a more explicit characterization of the horizon, e.g., by fitting a model of the peak location as a function of the four parameters and verifying it on unseen configurations.

minor comments (4)

[Eq. (3)] In Eq. (3), the two forward passes for R-drop are written as D_j(f^i_s(x_s), f^i_s(x_s)); as written, this is the distance between identical vectors and is trivially zero. Use different superscripts (e.g., f^{i,(1)}_s and f^{i,(2)}_s) to denote two dropout-perturbed forward passes.
[Figure 2] The figure caption does not state which panel corresponds to which α. Please label the panels (e.g., 'varying α_cr', 'varying α_rd', 'varying α_t') and add a legend for the dropout-rate color scheme.
[Tables 1-4] The table captions use 'Base' and 'Expand' without defining them. Define these terms in the caption or in the text before the first table. Also, the significance dagger (†) is mentioned in Section 2.2 but Table 4 appears to have no daggers or significance annotation; please clarify whether all Table 4 differences are significant.
[Section 4.2, footnote 3] The regression coefficients are given without standard errors or p-values. Since the paper makes a quantitative claim about the collapse, reporting at least standard errors or bootstrap intervals would greatly improve confidence in the result.

Circularity Check

1 steps flagged · score 4.0 of 10

Partial circularity: the BLEU–R collapse is partially guaranteed by regressing R on BLEU; only the held-out under-regularized points provide independent evidence.

fitted input called prediction [Section 4.2 (Eqs. 7–8, Fig. 3)]
"As (8) is only valid in the over-regularization regime, we select the points after the peak in Fig. 2 for the regression. ... We assign R values to each experiment from the regression coefficients and plot the BLEU score as a function of R for all data in Fig. 3. ... they collapse to a single curve, even for the points not included in the regression."

Eq. 7 defines R as a linear combination of αcr, αrd, αt, and d/o, and the coefficients are estimated by regressing BLEU on those same variables over exactly the over-regularized points. Consequently, for those fitted points, the plot of BLEU against the fitted R is a least-squares fitted line; their 'collapse' is guaranteed by construction, not an empirical finding. The only out-of-sample evidence is the non-fitted, under-regularized points, which the paper assesses visually without a quantitative goodness-of-fit. The selection of 'after the peak' also uses the target BLEU values to choose the training subset, adding selection bias. This is partial circularity because the under-regularized points are held out and do provide some independent support.

full rationale

The central derivation chain is Eq. 7 (linear additive total regularization) plus Eq. 8 (linear BLEU decrease in the over-regularized regime). The coefficients are fit to points chosen as 'after the peak' in Fig. 2, and then the same fitted R is used to display a collapse in Fig. 3. For the fitted points, the collapse is tautological; the paper's admission that 'only the points in the over-regularized regime were used in the regression' makes this explicit. The independent content is the claim that the under-regularized, non-fitted points also lie on the same curve, which would be genuine evidence if quantified. The paper also acknowledges Eq. 7 is 'evidently an oversimplification,' which is a correctness risk (nonlinear interactions, e.g., the saturation in Table 2) rather than circularity. There are no load-bearing self-citations; references to prior consistency-training work are external. Overall, one important evidential step is partially circular, but the central claim is not fully forced, so a moderate score is appropriate.

Assumptions & free parameters 2 free parameters · 4 assumptions · 1 invented entities

The central construction is the scalar R, entirely determined by regression coefficients fitted to the same experiments, plus the visual selection of the over-regularized subset. The linear form and the linear BLEU-R relation are explicit approximations, not derived from first principles.

free parameters (2)

beta_cr, beta_rd, beta_t, beta_do, beta_f, beta_B (linear total regularization coefficients) = beta_cr=0.245, beta_rd=0.159, beta_t=-0.814, beta_do=13.8, beta_f=0.814, beta_B=32.6
Fit by linear regression of BLEU on alpha_cr, alpha_rd, alpha_t, d/o for points selected as over-regularized (Section 4.2). These coefficients define the scalar R that is the paper's central construct.
Peak selection thresholds = visual, per panel
Which points count as 'after the peak' in Fig. 2 is chosen by eye, affecting which data fit the regression.

assumptions (4)

ad hoc to paper R is a linear function of alpha_cr, alpha_rd, alpha_t, and d/o
Equation (7), stated as 'evidently an oversimplification' in Section 4.2. The entire collapse argument rests on this additive linear form.
ad hoc to paper BLEU decreases linearly with R in the over-regularized regime
Equation (8), justified as 'reasonable approximation within our parameter range of interest'.
standard math Softmax-KL loss roughly scales as (x-y) dot (e^x - e^y)
Used in Section 3.1 to explain why softmax-KL outperforms logits-MSE. This is an informal approximation, not a rigorous derivation.
domain assumption Text input can be viewed as a corrupted version of speech input
Motivates the correspondence between consistency regularization and R-drop in Section 4.1; used to interpret alpha_t as a perturbation strength.

invented entities (1)

Total regularization R
purpose: A scalar variable claimed to subsume the effect of four hyperparameters (alpha_cr, alpha_rd, alpha_t, d/o) on model performance.
R is defined via coefficients fit to the paper's own experiments. It has no independent falsifiable handle outside the reported data.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Optimal Multi-Task Learning at Regularization Horizon for Speech Translation Task." pith.science (2026). https://pith.science/paper/GLS7R4AH

@misc{pith2026250909701,
  author       = {Pith},
  title        = {Pith review of: Optimal Multi-Task Learning at Regularization Horizon for Speech Translation Task},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GLS7R4AH}},
  note         = {Machine review of arXiv:2509.09701}
}

read the original abstract

End-to-end speech-to-text translation typically suffers from the scarcity of paired speech-text data. One way to overcome this shortcoming is to utilize the bitext data from the Machine Translation (MT) task and perform Multi-Task Learning (MTL). In this paper, we formulate MTL from a regularization perspective and explore how sequences can be regularized within and across modalities. By thoroughly investigating the effect of consistency regularization (different modality) and R-drop (same modality), we show how they respectively contribute to the total regularization. We also demonstrate that the coefficient of MT loss serves as another source of regularization in the MTL setting. With these three sources of regularization, we introduce the optimal regularization contour in the high-dimensional space, called the regularization horizon. Experiments show that tuning the hyperparameters within the regularization horizon achieves near state-of-the-art performance on the MuST-C dataset.

Figures

Figures reproduced from arXiv: 2509.09701 by the authors.

**Figure 2.** The effect of dropout rate on the ST BLEU score, when varying the loss rate coefficients. The form of the [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** We plot the BLEU score against the total [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 33 canonical work pages

[1]

Antonios Anastasopoulos and David Chiang. 2018. Tied multitask learning for neural speech translation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 82--91

work page 2018
[2]

Lo \" c Barrault, Yu-An Chung, Mariano Coria Meglioli, David Dale, Ning Dong, Mark Duppenthaler, Paul-Ambroise Duquenne, Brian Ellis, Hady Elsahar, Justin Haaheim, and 1 others. 2023. Seamless: Multilingual expressive and streaming speech translation. arXiv preprint arXiv:2312.05187

arXiv 2023
[3]

Luisa Bentivogli, Mauro Cettolo, Marco Gaido, Alina Karakanta, Alberto Martinelli, Matteo Negri, and Marco Turchi. 2021. Cascade versus direct speech translation: Do the differences still make a difference? In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Lang...

work page 2021
[4]

Alexandre B \'e rard, Laurent Besacier, Ali Can Kocabiyikoglu, and Olivier Pietquin. 2018. End-to-end automatic speech translation of audiobooks. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6224--6228. IEEE

work page 2018
[5]

Alexandre B \'e rard, Olivier Pietquin, Laurent Besacier, and Christophe Servan. 2016. Listen and translate: A proof of concept for end-to-end speech-to-text translation. In NIPS Workshop on end-to-end learning for speech and audio processing

work page 2016
[6]

Christian Buck and Philipp Koehn. 2016. Findings of the wmt 2016 bilingual document alignment shared task. In Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers, pages 554--563

work page 2016
[7]

Rich Caruana. 1997. Multitask learning. Machine learning, 28:41--75

work page 1997
[8]

Xuxin Cheng, Qianqian Dong, Fengpeng Yue, Tom Ko, Mingxuan Wang, and Yuexian Zou. 2023. M 3 st: Mix at three levels for speech translation. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1--5. IEEE

work page 2023

Show all 41 references

[9]

Mattia A Di Gangi, Roldano Cattoni, Luisa Bentivogli, Matteo Negri, and Marco Turchi. 2019. Must-c: a multilingual speech translation corpus. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Te...

2019
[10]

Qingkai Fang and Yang Feng. 2023. https://doi.org/10.18653/v1/2023.acl-long.884 Understanding and bridging the modality gap for speech translation . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15864--15881

2023 doi
[11]

Qingkai Fang, Rong Ye, Lei Li, Yang Feng, and Mingxuan Wang. 2022. Stemm: Self-learning with speech-text manifold mixup for speech translation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7050--7062

2022
[12]

Pengzhi Gao, Ruiqing Zhang, Zhongjun He, Hua Wu, and Haifeng Wang. 2024. An empirical study of consistency regularization for end-to-end speech-to-text translation. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguis...

2024
[13]

Chi Han, Mingxuan Wang, Heng Ji, and Lei Li. 2021. Learning shared semantic space for speech-to-text translation. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2214--2225

2021
[14]

Yuchen Han, Chen Xu, Tong Xiao, and Jingbo Zhu. 2023. https://doi.org/10.18653/v1/2023.acl-short.115 Modality adaption or regularization? a case study on end-to-end speech translation . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics ...

2023 doi
[15]

Kingma and Jimmy Ba

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In ICLR (Poster)

2015
[16]

Philipp Koehn. 2004. Statistical significance tests for machine translation evaluation. In Proceedings of the 2004 conference on empirical methods in natural language processing, pages 388--395

2004
[17]

Taku Kudo and John Richardson. 2018. Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 66--71

2018
[18]

Hojin Lee, Changmin Lee, and Seung-won Hwang. 2023. Consistency is key: On data-efficient modality transfer in speech translation. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 13572--13581

2023
[19]

Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computationa...

2019
[20]

Siqi Ouyang, Rong Ye, and Lei Li. 2023. https://doi.org/10.18653/v1/2023.acl-long.216 WACO : Word-aligned contrastive learning for speech translation . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3891--3907

2023 doi
[21]

Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015. Librispeech: an asr corpus based on public domain audio books. In 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 5206--5210. IEEE

2015
[22]

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311--318

2002
[23]

Ngoc-Quan Pham, Jan Niehues, Thanh-Le Ha, and Alex Waibel. 2019. Improving zero-shot translation with language-independent constraints. In Proceedings of the Fourth Conference on Machine Translation (Volume 1: Research Papers), pages 13--23

2019
[24]

Matt Post. 2018. A call for clarity in reporting bleu scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, page 186. Association for Computational Linguistics

2018
[25]

Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In International conference on machine learning, pages 28492--28518. PMLR

2023
[26]

Dinghan Shen, Mingzhi Zheng, Yelong Shen, Yanru Qu, and Weizhu Chen. 2020. A simple but tough-to-beat data augmentation approach for natural language understanding and generation. arXiv preprint arXiv:2009.13818

2020 arXiv
[27]

Yun Tang, Juan Pino, Xian Li, Changhan Wang, and Dmitriy Genzel. 2021. Improving speech translation by understanding and learning from the auxiliary text translation task. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th I...

2021
[28]

Ioannis Tsiamas, Gerard I G \'a llego, Jos \'e AR Fonollosa, and Marta R Costa-juss \`a . 2023. Efficient speech translation with dynamic latent perceivers. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1--5. IEEE

2023
[29]

Anne Wu, Changhan Wang, Juan Pino, and Jiatao Gu. 2020. Self-supervised representations improve end-to-end speech translation. In Proc. Interspeech 2020, pages 1491--1495

2020
[30]

Lijun Wu, Juntao Li, Yue Wang, Qi Meng, Tao Qin, Wei Chen, Min Zhang, Tie-Yan Liu, and 1 others. 2021. R-drop: Regularized dropout for neural networks. Advances in Neural Information Processing Systems, 34:10890--10905

2021
[31]

Qizhe Xie, Zihang Dai, Eduard Hovy, Thang Luong, and Quoc Le. 2020. Unsupervised data augmentation for consistency training. Advances in neural information processing systems, 33:6256--6268

2020
[32]

Chen Xu, Bojie Hu, Yanyang Li, Yuhao Zhang, Shen Huang, Qi Ju, Tong Xiao, and Jingbo Zhu. 2021. Stacked acoustic-and-textual encoding: Integrating the pre-trained models into speech translation encoders. In Proceedings of the 59th Annual Meeting of the Association for Computat...

2021
[33]

Brian Yan, Xuankai Chang, Antonios Anastasopoulos, Yuya Fujita, and Shinji Watanabe. 2024. Cross-modal multi-tasking for speech-to-text translation via hard parameter sharing. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)...

2024
[34]

Rong Ye, Mingxuan Wang, and Lei Li. 2021. End-to-end speech translation via cross-modal progressive training. In Proc. of INTERSPEECH

2021
[35]

Rong Ye, Mingxuan Wang, and Lei Li. 2022. Cross-modal contrastive learning for speech translation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5099--5113

2022
[36]

Biao Zhang, Philip Williams, Ivan Titov, and Rico Sennrich. 2020. Improving massively multilingual neural machine translation and zero-shot translation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1628--1639

2020
[37]

Han Zhang, Zizhao Zhang, Augustus Odena, and Honglak Lee. 2019. Consistency regularization for generative adversarial networks. In International Conference on Learning Representations

2019
[38]

Hao Zhang, Nianwen Si, Yaqi Chen, Wenlin Zhang, Xukui Yang, Dan Qu, and Wei-Qiang Zhang. 2023 a . Improving speech translation by cross-modal multi-grained contrastive learning. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 31:1075--1086

2023
[39]

Linlin Zhang, Kai Fan, Boxing Chen, and Luo Si. 2023 b . A simple concatenation can effectively improve speech translation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 1793--1802

2023
[40]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...
[41]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

Antonios Anastasopoulos and David Chiang. 2018. Tied multitask learning for neural speech translation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 82--91

work page 2018

[2] [2]

Lo \" c Barrault, Yu-An Chung, Mariano Coria Meglioli, David Dale, Ning Dong, Mark Duppenthaler, Paul-Ambroise Duquenne, Brian Ellis, Hady Elsahar, Justin Haaheim, and 1 others. 2023. Seamless: Multilingual expressive and streaming speech translation. arXiv preprint arXiv:2312.05187

arXiv 2023

[3] [3]

Luisa Bentivogli, Mauro Cettolo, Marco Gaido, Alina Karakanta, Alberto Martinelli, Matteo Negri, and Marco Turchi. 2021. Cascade versus direct speech translation: Do the differences still make a difference? In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Lang...

work page 2021

[4] [4]

Alexandre B \'e rard, Laurent Besacier, Ali Can Kocabiyikoglu, and Olivier Pietquin. 2018. End-to-end automatic speech translation of audiobooks. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6224--6228. IEEE

work page 2018

[5] [5]

Alexandre B \'e rard, Olivier Pietquin, Laurent Besacier, and Christophe Servan. 2016. Listen and translate: A proof of concept for end-to-end speech-to-text translation. In NIPS Workshop on end-to-end learning for speech and audio processing

work page 2016

[6] [6]

Christian Buck and Philipp Koehn. 2016. Findings of the wmt 2016 bilingual document alignment shared task. In Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers, pages 554--563

work page 2016

[7] [7]

Rich Caruana. 1997. Multitask learning. Machine learning, 28:41--75

work page 1997

[8] [8]

Xuxin Cheng, Qianqian Dong, Fengpeng Yue, Tom Ko, Mingxuan Wang, and Yuexian Zou. 2023. M 3 st: Mix at three levels for speech translation. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1--5. IEEE

work page 2023