Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

When End-to-End is Overkill: Rethinking Cascaded Speech-to-Text Translation

T0 review · 4 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a cascaded speech-to-text translation system fed five aligned ASR candidates plus self-supervised speech features matches a strong end-to-end model on GigaST English-to-Chinese.

desk verdict A useful empirical result on multi-candidate cascaded ST, but the causal mechanism is asserted, not shown. read the letter →

arxiv 2502.00377 v1 pith:B3MSYYNU submitted 2025-02-01 cs.CL cs.AIcs.MMcs.SDeess.AS

Anna Min , Chenxu Hu , Yi Ren , Hang Zhao This is my paper · ORCID
classification cs.CLcs.AIcs.MMcs.SDeess.AS
keywords speech-to-texttranslationcascadedsystemmulti-candidateASRattentionaveragingself-supervisedspeechfeaturesmachineerrorpropagationGigaST
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that cascaded speech-to-text translation remains competitive with end-to-end models when the machine translation stage is allowed to see several plausible ASR outputs at once rather than only the top one. On GigaST English-to-Chinese, the proposed system reaches 38.1 BLEU, against 36.8 for the plain cascade and 38.0 for the strong end-to-end SSL-Transformer baseline, and it does so with no additional parameters and roughly one hour of fine-tuning. The paper identifies the root of cascading errors as the divergence between acoustically similar speech samples once they are mapped to text: ASR often picks the wrong homophone or elided form, while the correct word survives in a lower-ranked candidate. A translator with rich text patterns can, the authors argue, choose the right words from aligned candidates and preserve acoustic clues through self-supervised speech features. The practical stake is that high-quality speech translation may not require costly paired speech, transcript, and target data at end-to-end scale.

What carries the argument

The mechanism is a multi-candidate, attention-averaging MT decoder built on mBART. Five ASR outputs from WeNet's beam search are aligned to the top candidate through iterative longest-common-substring matching, padding non-common parts with 'unk' tokens, so that corresponding words sit at the same positions. During decoding, the model computes the mean attention $A'(Q)=\frac{1}{n}\sum_{i=1}^n A_{\mathrm{candidate}_i}(Q)$ over candidates once, just before the final layer normalization, and uses that pooled attention for beam search. A second encoder stream supplies self-supervised speech units from HuBERT's 11th layer, quantized by K-means into a 1000-token vocabulary, which is fused with the text encoder's output in cross-attention. The alignment makes the attention average meaningful; the speech stream preserves phonetic information that text alone loses.

What would settle it

Translate each of the five ASR candidates independently with the same mBART and select outputs by a confidence score; if that oracle ensemble matches 38.1 BLEU, attention averaging and 'unk' alignment are not the operative cause. Alternatively, log the decoder's attention over correct versus erroneous candidate words and check whether correct words receive systematically higher weights.

Watch

Extended reading notes

Core claim

The central discovery claimed is that the conventional weakness of cascaded systems, error propagation from ASR to MT, can be largely neutralized by giving the MT model the top five ASR candidates instead of only the first, aligning those candidates by longest common substrings with 'unk' padding, and averaging the decoder's attention over the candidates before the final layer normalization. This alone raises BLEU from 36.8 to 37.8. Adding a parallel stream of HuBERT discrete units, the 11th layer's K-means quantized tokens, as a second encoder source raises the score to 38.1, matching the end-to-end SSL-Transformer. The paper also reports that the best ASR candidate by word error rate is the best translation candidate only 45.35% of the time, so the top-1 ASR decision is a poor proxy for translation quality. On that basis the authors claim that cascaded systems are not inherently inferior; they are just underfed.

Load-bearing premise

The paper's explanation of why multi-candidate inputs help depends on the translation model's attention actually favoring correct words among aligned candidates, yet no attention-weight measurements are reported, so the gain could come from ensembling or from the speech feature stream alone.

Editorial extensions

If this is right

  • Cascaded systems can close the gap to end-to-end models on large-scale speech translation benchmarks without adding parameters or retraining ASR.
  • Because the technique is compatible with any attention-based MT backbone, future gains in pre-trained ASR and MT models should transfer directly to speech-to-text translation.
  • Production systems can exploit the finding that only 45.35% of top-ranked ASR hypotheses yield the best translation by carrying multiple hypotheses instead of committing to the first.
  • One hour of fine-tuning on the MT stage reaches 37.3 BLEU, suggesting the approach is practical when end-to-end paired data are scarce or costly.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test this, but its account implies that feeding n-best hypotheses could improve other noisy-source-to-text cascades, such as speech summarization or speech-to-speech translation, wherever the first stage returns a ranked list.
  • A testable extension the authors do not run is replacing the longest-common-substring alignment with learned or edit-distance alignment; if the gain survives, providing alternatives rather than exact positional alignment is the active ingredient.
  • A caution the paper leaves open: if the BLEU gain is mostly ensembling, the attention-averaging and 'unk' padding are not doing the causal work, so logging the decoder's attention on correct versus incorrect candidate words would test the paper's explanation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper argues that cascaded speech-to-text translation remains competitive with end-to-end models when the ASR front end supplies multiple n-best candidates and self-supervised speech features to the MT model. On GigaST English-to-Chinese, the proposed multi-candidate cascaded system with averaged attention and HuBERT-derived discrete speech units reaches BLEU 38.1, outperforming the plain Wenet+mBART cascade (36.8) and matching the SSL-Transformer end-to-end baseline (38.0), with no additional parameters and only one hour of fine-tuning. The authors attribute the improvement to a novel causal analysis: the primary source of cascading errors is the divergence between acoustically similar but semantically distant ASR hypotheses, and the MT model's attention mechanism can select correct words from aligned candidates.

Significance. If the empirical result holds, the paper makes a practically useful contribution by showing that existing ASR and MT checkpoints can be adapted for speech translation with a lightweight, parameter-free inference change, which is of interest to practitioners who cannot train end-to-end ST models from scratch. The paper also ships a concrete external-benchmark evaluation against published baselines, and the method itself is simple and reproducible in principle. However, the central causal claim about attention-based word selection is currently unmeasured, and the absence of comparison against prior n-best/lattice cascaded systems weakens the novelty and generality claims. The significance is therefore conditional on a more rigorous validation of the mechanism and on statistical support for the reported gains.

major comments (4)
  1. [§II.A, Table II] The text immediately after Table II states: "It indicates that the top-ranked ASR candidate always performs the best translation result." This directly contradicts the preceding sentence and the table, which show that the top-ranked candidate yields the best BLEU only 45.35% of the time. The sentence should be corrected to "does not always perform the best translation result"; as written, the internal contradiction undermines the stated motivation for the multi-candidate approach and must be fixed in revision.
  2. [§III.A.2 and §IV.C.3] The paper asserts that the MT model's attention mechanism selects correct words among aligned candidates, but no attention weights are ever measured or reported. The ablation comparison of settings (5) and (6) shows that alignment matters (36.9 vs. 37.8), yet alignment also replaces divergent words with "unk" tokens, which may simply denoise the input by producing a consensus sequence rather than enabling attention-based selection. The authors should provide either attention-weight statistics (e.g., whether correct candidates receive higher averaged attention than incorrect ones) or a control experiment that removes the selection mechanism (e.g., replacing all non-overlapping words with "unk" in a single candidate). Without this, the causal narrative in the abstract and Section III.A is unsupported, and the justification for the specific alignment procedure is not established.
  3. [§IV.C.1 and references [7]–[9]] The Introduction cites prior work on integrating ASR lattices and n-best lists into MT ([7]–[9]) and later claims the proposed method is advantageous because it requires no additional parameters and no further training. However, the experiments include no comparison against any of these existing multi-hypothesis cascaded systems. Since the novelty claim is relative to that prior work, the authors should include at least one representative baseline from [7]–[9] (or a reimplementation) in Table III, or explicitly state why such a comparison is infeasible. Without it, the claimed advantage over previous n-best/lattice approaches is unsubstantiated.
  4. [§IV.C, Table III] All BLEU scores in Table III are point estimates with no error bars, significance tests, or multiple-run variance. The key comparisons are small in absolute terms: setting (7) improves over setting (4) by 1.3 BLEU and over setting (3) by only 0.1 BLEU. The authors should report bootstrap confidence intervals, paired significance tests (e.g., paired bootstrap on sentences), or at least multiple fine-tuning runs to establish that these differences are not noise. This is particularly important for the headline claim that the method matches the end-to-end SSL-Transformer.
minor comments (6)
  1. [§I] The phrase "data or argumentation [2]–[6]" appears to be a typographical error; the context suggests "data augmentation" or "data curation." Please clarify.
  2. [§II.A] The sentence "We then analyze the index and percentage of the highest BLEU score and find that only 45.35% of the candidates with the highest BLEU score corresponded to the candidate with the lowest word error rate" is confusing because it conflates two different comparisons (best BLEU vs. lowest WER and versus rank index). Please rewrite to state separately the percentage of cases where the top-ranked ASR candidate has the best BLEU and the percentage where the lowest-WER candidate has the best BLEU.
  3. [§III.A.1] Step 3 of the alignment algorithm refers to "the largest length of uncommon substrings" without defining how this is computed or how substrings are connected. A precise algorithmic description or a simple pseudocode example would improve reproducibility.
  4. [§III.B and §IV.B.1] There is an inconsistency about the training data for the K-means model: Section III.B says "K-means model trained on English speech," while Section IV.B.1 says the mHuBERT and K-means models are learned from VoxPopuli English, Spanish, and French. Please clarify which speech data the K-means vocabulary was actually trained on, since this affects reproducibility.
  5. [§IV.C.3] The case study quote "the candidate with a BLEU score of 100 is ranked fifth" is not supported by any table or quantitative evidence. If this is a BLEU score of the ASR transcript against the reference, please state the metric and provide the actual candidate sentences or a table.
  6. [Appendix/Notation] The notation in Eq. (1) and Eq. (2) uses A'(Q) for averaged attention but does not define Q or distinguish it across candidates. Please define all symbols explicitly and specify the dimension over which averaging occurs (heads, layers, tokens).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central BLEU result is an external-benchmark comparison, and the proposed mechanism, while under-measured, is not defined into the result.

full rationale

The paper's main claim is an empirical result on the GigaST test set (human-translated) compared against external baselines: setting (7) reaches 38.1 BLEU versus 36.8 for Wenet+mBART cascaded and 38.0 for SSL-Transformer (Table III). This is a genuine external evaluation, not a quantity fitted to the test set and then reported as a prediction. The proposed components — multi-candidate ASR inputs, substring alignment, attention averaging, and HuBERT discrete units — are defined in Section III using external references ([20], [22], [23], [24], [32]) and are not derived from or defined in terms of the target BLEU scores. Equations (1) and (2) merely define the attention-averaging procedure; they do not encode the claimed outcome. The preliminary analysis in Section II.A (lexical overlap and candidate BLEU statistics) motivates the method but is not used as a training signal or as the evaluation metric, so it does not make the final result circular. The self-citations are minimal: [29] (UWSpeech, co-authored by Yi Ren) is cited only to contrast VAE-based representations with HuBERT and is not load-bearing for the main claim. The case study in Section IV.C.3 asserts that the MT attention mechanism weights correct candidates more heavily, but no attention weights are measured; this is an evidentiary gap about the causal mechanism, not a circular reduction of the result to its inputs. Therefore, under the stated hard rules, no specific circular step can be exhibited, and the paper is assessed as having no significant circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central result rests on a small set of design choices (candidate count, attention averaging position, speech unit layer and vocabulary size) and on assumptions about the transferability of mHuBERT units and about attention as a selection mechanism. There are no invented entities. The main unquantified premise is the causal explanation of error propagation, which is asserted rather than measured.

free parameters (4)
  • Number of ASR candidates n = 5
    Selected for experiments after Table I reports overlaps for n=1,5,10,20; no ablation on this choice is given, and it directly controls the input size for attention averaging.
  • Attention averaging location = after all decoder layers, just before final layer normalization
    Chosen because the authors 'find that calculating only once at the final layer yields the best results'; no ablation is shown.
  • HuBERT layer index = 11th layer
    Adopted from reference [23] without re-tuning; the layer choice may affect the quality of discrete units.
  • K-means vocabulary size for speech units = 1000
    Adopted from reference [23]; the vocabulary size bounds the resolution of the self-supervised speech features.
assumptions (5)
  • standard math Dynamic programming for longest common substrings correctly aligns the n-best transcripts for attention averaging.
    Assumed in Section III.A.1; ref [22] provides fast algorithms, but the alignment's robustness to different candidate segmentations is not analyzed.
  • domain assumption The GigaST human-translated test set and its BLEU scores are a valid measure of translation quality for the conclusions.
    The paper relies on GigaST pseudo-labeled training and human-translated test set without discussing translation quality variance or test set size.
  • domain assumption mHuBERT discrete units trained on VoxPopuli (English, Spanish, French) transfer to English speech in GigaST.
    Section IV.B states the models are learned from VoxPopuli and used to encode English speech only; no adaptation or quality check is reported.
  • ad hoc to paper Attention averaging over aligned candidates allows the MT model to select correct words, i.e., the decoder weights reflect semantic correctness.
    This is the mechanism proposed in Section III.A.2, but no attention analysis or causal test is provided beyond case examples.
  • ad hoc to paper The primary cause of cascading errors is increased divergence between similar samples in the speech domain when mapped to text.
    Stated in the abstract and Section II.B as a hypothesis supported only by homophone examples; no quantitative divergence measure is computed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of When End-to-End is Overkill: Rethinking Cascaded Speech-to-Text Translation." pith.science (2026). https://pith.science/paper/B3MSYYNU

@misc{pith2026250200377,
  author       = {Pith},
  title        = {Pith review of: When End-to-End is Overkill: Rethinking Cascaded Speech-to-Text Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/B3MSYYNU}},
  note         = {Machine review of arXiv:2502.00377}
}
read the original abstract

Though end-to-end speech-to-text translation has been a great success, we argue that the cascaded speech-to-text translation model still has its place, which is usually criticized for the error propagation between automatic speech recognition (ASR) and machine translation (MT) models. In this paper, we explore the benefits of incorporating multiple candidates from ASR and self-supervised speech features into MT. Our analysis reveals that the primary cause of cascading errors stems from the increased divergence between similar samples in the speech domain when mapped to the text domain. By including multiple candidates and self-supervised speech features, our approach allows the machine translation model to choose the right words and ensure precise translation using various speech samples. This strategy minimizes error spread and takes advantage of large ASR and MT datasets, along with pre-trained ASR/MT models, while addressing associated issues.

Figures

Figures reproduced from arXiv: 2502.00377 by the authors.

Figure 1
Figure 1. The correct words are scattered among various candidates, while [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of our proposed MC-sslS system III. METHOD To leverage the powerful capability of machine translation in capturing semantic patterns, we propose utilizing multi￾candidate ASR inputs and averaging attention computation in the MT model. Furthermore, to address the issue of error propagation in ASR caused by homophones, we employ self￾supervised speech representations to enhance accuracy. The combination of me… view at source ↗
Figure 3
Figure 3. Above is how the 3 rd process is calculated. After finding the longest common subsequences, candidates are aligned and padded. The orange circles denote “unk” tokens. 2) Average attention among candidates: Thus, we pro￾pose an innovative approach, incorporating multiple ASR candidates into a single attention-based machine translation model. Here we use mBART as the backbone architecture. The multi-candidate model ca… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. It's Not a Walk in the Park! Challenges of Idiom Translation in Speech-to-text Systems

    cs.CL 2025-06 conditional novelty 6.0 of 10

    End-to-end speech translation systems translate idioms worse than text-based systems, frequently producing literal or incorrect outputs, across German and Russian to English.

Reference graph

Works this paper leans on

32 extracted references · 18 canonical work pages · cited by 1 Pith paper

  1. [7]

    On the integration of speech recognition and statistical machine translation,

    E. Matusov, S. Kanthak, and H. Ney, “On the integration of speech recognition and statistical machine translation,” in Ninth European Conference on Speech Communication and Technology , 2005

    work page 2005

  2. [9]

    Neural lattice search for speech recognition,

    R. Ma, H. Li, Q. Liu, L. Chen, and K. Yu, “Neural lattice search for speech recognition,” in ICASSP 2020-2020 IEEE International Confer- ence on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2020, pp. 7794–7798

    work page 2020

  3. [1]

    Listen and translate: A proof of concept for end-to-end speech-to-text translation,

    A. B ´erard, O. Pietquin, C. Servan, and L. Besacier, “Listen and translate: A proof of concept for end-to-end speech-to-text translation,” arXiv preprint arXiv:1612.01744, 2016

    arXiv 2016

  4. [2]

    Enhanced direct speech-to-speech translation using self-supervised pre-training and data augmentation,

    S. Popuri, P.-J. Chen, C. Wang, J. Pino, Y . Adi, J. Gu, W.-N. Hsu, and A. Lee, “Enhanced direct speech-to-speech translation using self-supervised pre-training and data augmentation,” arXiv preprint arXiv:2204.02967, 2022

    arXiv 2022

  5. [3]

    Leveraging weakly supervised data to improve end-to-end speech-to-text translation,

    Y . Jia, M. Johnson, W. Macherey, R. J. Weiss, Y . Cao, C.-C. Chiu, N. Ari, S. Laurenzo, and Y . Wu, “Leveraging weakly supervised data to improve end-to-end speech-to-text translation,” in ICASSP 2019- 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, pp. 7180–7184

    work page 2019

  6. [4]

    Analyzing asr pretraining for low-resource speech-to-text translation,

    M. C. Stoian, S. Bansal, and S. Goldwater, “Analyzing asr pretraining for low-resource speech-to-text translation,” in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, pp. 7909–7913

    work page 2020

  7. [5]

    Self-Training for End-to-End Speech Translation

    J. Pino, Q. Xu, X. Ma, M. J. Dousti, and Y . Tang, “Self-training for end-to-end speech translation,” arXiv preprint arXiv:2006.02490 , 2020

    work page Pith review arXiv 2006

  8. [6]

    Large-Scale Self- and Semi-Supervised Learning for Speech Translation

    C. Wang, A. Wu, J. Pino, A. Baevski, M. Auli, and A. Conneau, “Large- scale self-and semi-supervised learning for speech translation,” arXiv preprint arXiv:2104.06678, 2021

    work page Pith review arXiv 2021

Show all 32 references
  1. [8]

    Integrated n-best re-ranking for spoken language translation

    V . H. Quan, M. Federico, and M. Cettolo, “Integrated n-best re-ranking for spoken language translation.” in Interspeech, 2005, pp. 3181–3184

    2005

  2. [10]

    A new decoder for spoken language trans- lation based on confusion networks,

    N. Bertoldi and M. Federico, “A new decoder for spoken language trans- lation based on confusion networks,” in IEEE Workshop on Automatic Speech Recognition and Understanding, 2005. IEEE, 2005, pp. 86–91

    2005

  3. [11]

    Neural speech translation using lattice transformations and graph networks,

    D. Beck, T. Cohn, and G. Haffari, “Neural speech translation using lattice transformations and graph networks,” in Proceedings of the Thirteenth Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-13), 2019, pp. 26–31

    2019

  4. [12]

    Self-attentional models for lattice inputs,

    M. Sperber, G. Neubig, N.-Q. Pham, and A. Waibel, “Self-attentional models for lattice inputs,” arXiv preprint arXiv:1906.01617 , 2019

    1906 arXiv

  5. [13]

    Spoken language translation using automatically transcribed text in training,

    S. Peitz, S. Wiesler, M. Nußbaum-Thom, and H. Ney, “Spoken language translation using automatically transcribed text in training,” in Proceed- ings of the 9th International Workshop on Spoken Language Translation: Papers, 2012

    2012

  6. [14]

    Breaking the data barrier: Towards robust speech translation via adversarial stability training,

    Q. Cheng, M. Fang, Y . Han, J. Huang, and Y . Duan, “Breaking the data barrier: Towards robust speech translation via adversarial stability training,” arXiv preprint arXiv:1909.11430 , 2019

    1909 arXiv

  7. [15]

    Robust neural machine translation for clean and noisy speech transcripts,

    M. A. Di Gangi, R. Enyedi, A. Brusadin, and M. Federico, “Robust neural machine translation for clean and noisy speech transcripts,” arXiv preprint arXiv:1910.10238, 2019

    1910 arXiv

  8. [16]

    Searchable hidden intermediates for end-to-end models of decomposable sequence tasks,

    S. Dalmia, B. Yan, V . Raunak, F. Metze, and S. Watanabe, “Searchable hidden intermediates for end-to-end models of decomposable sequence tasks,” arXiv preprint arXiv:2105.00573 , 2021

    2021 arXiv

  9. [17]

    Fast-md: Fast multi-decoder end-to-end speech translation with non-autoregressive hidden intermediates,

    H. Inaguma, S. Dalmia, B. Yan, and S. Watanabe, “Fast-md: Fast multi-decoder end-to-end speech translation with non-autoregressive hidden intermediates,” in 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, 2021, pp. 922–929

    2021

  10. [18]

    On the opportunities and risks of foundation models,

    R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill et al. , “On the opportunities and risks of foundation models,” arXiv preprint arXiv:2108.07258, 2021

    2021 arXiv

  11. [19]

    Pre-trained models: Past, present and future,

    X. Han, Z. Zhang, N. Ding, Y . Gu, X. Liu, Y . Huo, J. Qiu, Y . Yao, A. Zhang, L. Zhang et al., “Pre-trained models: Past, present and future,” AI Open, vol. 2, pp. 225–250, 2021

    2021

  12. [20]

    Wenet: Production oriented streaming and non-streaming end-to-end speech recognition toolkit,

    Z. Yao, D. Wu, X. Wang, B. Zhang, F. Yu, C. Yang, Z. Peng, X. Chen, L. Xie, and X. Lei, “Wenet: Production oriented streaming and non-streaming end-to-end speech recognition toolkit,”arXiv preprint arXiv:2102.01547, 2021

    2021 arXiv

  13. [21]

    Gi- gast: A 10,000-hour pseudo speech translation corpus,

    R. Ye, C. Zhao, T. Ko, C. Meng, T. Wang, M. Wang, and J. Cao, “Gi- gast: A 10,000-hour pseudo speech translation corpus,” arXiv preprint arXiv:2204.03939, 2022

    2022 arXiv

  14. [22]

    Faster Algorithms for Longest Common Substring,

    P. Charalampopoulos, T. Kociumaka, S. P. Pissis, and J. Radoszewski, “Faster Algorithms for Longest Common Substring,” in 29th Annual European Symposium on Algorithms (ESA 2021) , ser. Leibniz International Proceedings in Informatics (LIPIcs), P. Mutzel, R. Pagh, and G. Herman...

    2021

  15. [23]

    Direct speech-to-speech translation with discrete units,

    A. Lee, P.-J. Chen, C. Wang, J. Gu, S. Popuri, X. Ma, A. Polyak, Y . Adi, Q. He, Y . Tanget al., “Direct speech-to-speech translation with discrete units,” arXiv preprint arXiv:2107.05604 , 2021

    2021 arXiv

  16. [24]

    Hubert: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 3451–3460, 2021

    2021

  17. [25]

    Superb: Speech processing universal performance benchmark,

    S.-w. Yang, P.-H. Chi, Y .-S. Chuang, C.-I. J. Lai, K. Lakhotia, Y . Y . Lin, A. T. Liu, J. Shi, X. Chang, G.-T. Lin et al., “Superb: Speech processing universal performance benchmark,” arXiv preprint arXiv:2105.01051 , 2021

    2021 arXiv

  18. [26]

    On gener- ative spoken language modeling from raw audio,

    K. Lakhotia, E. Kharitonov, W.-N. Hsu, Y . Adi, A. Polyak, B. Bolte, T.-A. Nguyen, J. Copet, A. Baevski, A. Mohamed et al. , “On gener- ative spoken language modeling from raw audio,” Transactions of the Association for Computational Linguistics, vol. 9, pp. 1336–1354, 2021

    2021

  19. [27]

    Speech resynthesis from discrete disentangled self-supervised representations,

    A. Polyak, Y . Adi, J. Copet, E. Kharitonov, K. Lakhotia, W.- N. Hsu, A. Mohamed, and E. Dupoux, “Speech resynthesis from discrete disentangled self-supervised representations,” arXiv preprint arXiv:2104.00355, 2021

    2021 arXiv

  20. [28]

    Speech-to-speech translation between untranscribed unknown languages,

    A. Tjandra, S. Sakti, and S. Nakamura, “Speech-to-speech translation between untranscribed unknown languages,” in 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, 2019, pp. 593–600

    2019

  21. [29]

    Uwspeech: Speech to speech translation for unwritten languages,

    C. Zhang, X. Tan, Y . Ren, T. Qin, K. Zhang, and T.-Y . Liu, “Uwspeech: Speech to speech translation for unwritten languages,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 35, no. 16, 2021, pp. 14 319–14 327

    2021

  22. [30]

    Gigaspeech: An evolving, multi- domain asr corpus with 10,000 hours of transcribed audio,

    G. Chen, S. Chai, G. Wang, J. Du, W.-Q. Zhang, C. Weng, D. Su, D. Povey, J. Trmal, J. Zhang et al. , “Gigaspeech: An evolving, multi- domain asr corpus with 10,000 hours of transcribed audio,” arXiv preprint arXiv:2106.06909, 2021

    2021 arXiv

  23. [31]

    V oxpopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation,

    C. Wang, M. Riviere, A. Lee, A. Wu, C. Talnikar, D. Haziza, M. Williamson, J. Pino, and E. Dupoux, “V oxpopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation,” arXiv preprint arXiv:2101.00390 , 2021

    2021 arXiv

  24. [32]

    Multilingual denoising pre-training for neural machine translation,

    Y . Liu, J. Gu, N. Goyal, X. Li, S. Edunov, M. Ghazvininejad, M. Lewis, and L. Zettlemoyer, “Multilingual denoising pre-training for neural machine translation,” Transactions of the Association for Computational Linguistics, vol. 8, pp. 726–742, 2020

    2020

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.