Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Aligning Pre-trained Models for Spoken Language Translation

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A small trainable connector between frozen pre-trained ASR and MT models is a viable, scalable, and domain-adaptive route to end-to-end speech translation, with the best system reaching 48.9 BLEU on the How2 English-Portuguese test set.

desk verdict Useful empirical study of connector-based speech translation, but the headline 'universally improves' claim is contradicted by the paper's own Section 10 numbers. read the letter →

arxiv 2411.18294 v1 pith:ADYV7GFH submitted 2024-11-27 cs.CL cs.AIcs.LG

Šimon Sedláček , Santosh Kesiraju , Alexander Polok , Jan Černocký This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords speechtranslationmodelalignmentconnectornetworkSTEQ-Formerfrozenpre-trainedmodelsdomainadaptationHow2
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that end-to-end speech translation can be built by keeping large pre-trained ASR and MT models frozen and training only a small connector module that maps ASR encoder outputs into the MT model's representation space. The authors test this on the How2 English-Portuguese benchmark and report that the connector, under 5% of the larger models' parameter count, scales cleanly: swapping in larger ASR and MT models improves translation everywhere. In the best configuration, aligned Whisper-small with a T5 decoder reaches 48.9 BLEU, beating a cascade baseline and the trained end-to-end baseline at 45.2 BLEU and exceeding the T5 MT model's own How2 score by over 9 points. The paper reads the latter gain as evidence that connectors act as domain adapters, and takes the overall results as showing that the alignment approach is a viable, cheaper alternative to training end-to-end ST from scratch.

What carries the argument

The load-bearing component is the connector module, specifically the paper's STE (Subsampler-Transformer Encoder): a 2-layer stack of 1D convolutions reduces the ASR embedding sequence by a factor of 4 and projects it to the connector's hidden size, followed by transformer encoder blocks and a final linear projection into the MT model's dimension. Unlike the Q-Former baseline, which compresses a variable-length speech embedding sequence into a fixed number of learned queries, the STE's output length stays proportional to the input length, which the paper argues is why it performs better on longer utterances and needs no query-count tuning. The two alignment architectures specify where the connector plugs in: ECD feeds the connector's output directly to the MT decoder's cross-attention (replacing the MT encoder), while ECED injects the output into the MT encoder's input space, optionally with a prepended task prompt.

What would settle it

Embed the gold English transcript with the T5 encoder and pass those embeddings through the same STE connector into the T5 decoder; if BLEU stays near 38.8 instead of 48.9, the improvement depends on the Whisper audio representation, not on the connector as a domain adapter. A matched same-scale end-to-end model trained on the same foundation models would further settle whether the alignment, rather than model choice, is what scales.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that a small trainable connector between frozen speech and text models is a viable and scalable way to do end-to-end speech translation. The evidence is a series of alignment experiments on How2: with the connector held at a fixed small size, every increase in foundation-model scale improves BLEU, and the best system (Whisper-small encoder + T5 decoder + STE connector in the ECD architecture) reaches 48.9 BLEU on the test set, compared with 45.2 for the trained end-to-end baseline and 40.4 for the cascade. The same experiments show that the connector can act as a domain adapter: aligning the out-of-domain T5 MT model with Whisper improves its How2 BLEU from 38.8 to 48.9, an improvement the paper attributes to the connector's learned mapping rather than to simply discarding the T5 encoder, since ECED runs that keep the T5 encoder nearly match ECD results.

Load-bearing premise

The domain-adapter claim assumes that the roughly 10-point BLEU gain over the T5 MT model is caused by the connector's learned mapping, not by the aligned architecture bypassing the out-of-domain T5 encoder or by Whisper's audio representations being easier to translate than text inputs.

Editorial extensions

If this is right

  • With connector size fixed, ST quality scales with the foundation ASR and MT models, implying the approach can inherit future improvements in speech recognition and translation without retraining the whole stack.
  • Because only the connector is trained, an aligned ST system can be built with roughly a tenth of the tunable parameters of a comparable end-to-end system, and the reported runs complete in about 10 hours on a single GPU.
  • Connectors transfer out-of-domain MT models to a new speech domain, as shown by the over-9-point BLEU gain over the T5 model's base How2 score, suggesting frozen MT decoders can be adapted without fine-tuning.
  • The STE connector's variable-length mapping makes it preferable to the Q-Former for speech inputs, where determining the right number of queries is dataset- and model-dependent.
  • In simulated low-resource settings, stronger foundation models still help the aligned system, for instance 45.4 versus 43.3 BLEU on the 153-hour split, indicating data-efficient gains from model scaling.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If connectors genuinely act as domain adapters, one could train a single small connector per target domain while keeping one frozen MT model, turning domain adaptation for speech translation into a cheap per-domain add-on rather than a full fine-tune.
  • The observed ceiling with the 1024-dimension OPUS MT model (47.3 BLEU) suggests the fixed 256-dimension connector will eventually become the bottleneck; a testable prediction is that scaling connector width with foundation-model width restores the scaling trend.
  • The Q-Former query-count sweet spot may reflect a general information-bottleneck problem for fixed-length connectors on variable-length speech, so long-form speech benchmarks would likely widen the gap between STE and Q-Former.
  • A direct comparison with same-scale end-to-end models trained from the same foundation-model initializations would tell whether the alignment approach's advantage is architectural or simply a result of using stronger pre-trained components.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a framework for end-to-end speech translation that connects a frozen ASR encoder to a frozen MT decoder via a small trainable connector module (Q-Former or the proposed STE connector). The connector is the only part optimized, under cross-entropy loss. Experiments on How2 English-Portuguese compare two architectures (ECD, ECED), two connectors, several foundation-model combinations, and low-resource data splits. The main reported findings are that STE outperforms Q-Former, that scaling the frozen ASR/MT models improves ST results while keeping the connector small, and that the connector acts as a domain adapter, yielding 48.9 BLEU on How2 test with Whisper-small + T5, versus 38.8 BLEU for the T5 MT model alone.

Significance. If the central claims hold, this is a practically attractive route to ST: it avoids fine-tuning large models, trains only a small connector, and can reuse strong off-the-shelf ASR/MT components. The paper's strengths include its use of public models and data, transparent reporting of hyperparameters, a clean architectural comparison between Q-Former and STE, and an initial low-resource study. However, the headline 'universally improves' scaling claim is contradicted by the paper's own Section 10 result, the domain-adapter interpretation is not fully isolated with a matched text-input control, and all comparisons rest on single runs without variance or significance testing. These issues are central to the paper's conclusions and require revision.

major comments (4)
  1. [Section 7 vs. Section 10] The abstract and Section 7 claim that increasing the size and capability of the foundation ASR and MT models 'universally improves translation results', but Section 10 reports the opposite at the high end: the stronger OPUS-MT En-Pt model (58.2 BLEU on How2 text) aligned with Whisper yields 47.3 BLEU, below the 48.9 BLEU obtained with the weaker T5 model under the same connector. The paper attributes this to connector capacity, which is a reasonable hypothesis, but it invalidates the unqualified word 'universally' in the abstract, Section 7, and Section 9. This is a load-bearing claim and must be qualified with the observed non-monotonicity, or the experiment must be extended with a larger connector to test the capacity explanation.
  2. [Section 7.1 / Table 5] The domain-adapter claim rests on a 9-point BLEU gain over the base T5 MT score. The ECED runs (lines 7-8 of Table 5) provide a useful control by keeping the T5 encoder, but there is no matched text-input condition: feeding the same ASR transcript tokens through a connector, or applying a standard domain-adaptation method such as LoRA to the frozen T5, would be needed to attribute the gain to the connector's cross-modal mapping rather than to properties of Whisper's audio representations. Without such a control, the statement that 'the connector network is able to serve as a domain adapter' is an interpretation rather than a demonstrated mechanism.
  3. [Tables 4, 5, 6, 8] All experimental results are single runs with no variance, confidence intervals, or significance tests. Several headline comparisons are close (e.g., 44.8 vs. 45.2 BLEU between the best aligned STE system and the E2E baseline in Table 4; 48.9 vs. 47.3 in Section 10), and the paper repeatedly uses terms like 'significant margin' and 'outperforms' (Section 6.1, Section 7). For an empirical paper whose central claims are quantitative, at least a few repeated-seed runs with standard deviations, or a significance test over utterance-level scores, are necessary to support the conclusions.
  4. [Section 7] The scaling conclusion is not benchmarked against an end-to-end ST system trained with the same foundation models. The E2E baseline in Table 3 uses the small E-Branchformer and MarianMT, while the best aligned system uses Whisper-small and T5. Without a same-foundation-model E2E baseline (e.g., Whisper-small encoder with T5 decoder trained end-to-end), the claim that the alignment approach 'scales' and is 'viable' relative to the strongest end-to-end alternative is not established. This is distinct from the non-monotonicity issue and affects the paper's 'scalable' conclusion.
minor comments (5)
  1. [Section 2.2] The sentence beginning 'a similar approach was explored' starts with a lowercase 'a' after a period; it should be capitalized.
  2. [Section 6.2] The text contains 'theval and test sets' with a missing space; it should read 'the val and test sets'.
  3. [Table 1] The table header 'dmodeltest' is missing a space; it should split into 'dmodel' and 'test'.
  4. [Figure 1 caption] The caption uses the string '*‘' for the frozen marker, which appears to be a typographical artifact; use a consistent marker such as '*' or 'frozen'.
  5. [Section 7.1] The phrase 'both T5 models with either ASR encoder almost match the performances of their ECD counterparts' is unclear because Table 5 lists four ECED configurations involving T5; please clarify which two systems are being compared.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claims are empirical evaluations on a held-out test set, with no fitted parameter or self-cited theorem doing load-bearing work.

full rationale

This is an empirical paper, not a derivation, and no step reduces to its own inputs by construction. The connector is trained on How2 training splits and evaluated on the held-out How2 test split; this is standard supervised evaluation, not a fitted quantity renamed as a prediction. The domain-adapter claim in Section 7.1 is an observed BLEU gain after training the connector on How2 ST data, which is exactly the mechanism the claim describes rather than a circular redefinition. The scaling claim ('universally improves') is internally weakened by the paper's own Section 10 OPUS-MT result, and the authors explicitly attribute that outcome to connector capacity; that is a correctness/internal-consistency concern, not circularity. The self-citations (Kesiraju et al. 2023; Vydana et al. 2021) are contextual references for ST training strategies and benchmark usage, and they are not load-bearing for the alignment results. There are no imported uniqueness theorems, no ansatz smuggled through self-citation, and no renaming of a known result as organization.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claims rest on manual connector hyperparameters (query count, subsampling factor, hidden size) and on the domain assumption that a cross-entropy-trained connector can align frozen ASR and MT latent spaces. No novel physical or conceptual entities are introduced; the STE connector is a neural architectural component, not an invented entity in the sense of a new particle, force, or conserved quantity.

free parameters (4)
  • Q-Former query count nq = 40, 60, 80, 100, 128
    The number of learnable queries is a hyperparameter with a data-dependent sweet spot; Section 6.3 reports performance degradation outside it, and Table 5 uses 128 for E-Branchformer medium but 100 for Whisper.
  • Connector layer count = 2, 4, or 6 layers
    Layer depth is swept in Table 8 and fixed at 6 for the scaling experiments; performance degrades for fewer layers.
  • STE subsampling factor = 4
    The two-layer 1D convolutional subsampler reduces sequence length by a factor of 4 (Eq. 1), chosen by hand to match MT token granularity.
  • Connector hidden size dmodel = 256
    Kept constant across experiments; Section 10 reports that this fixed capacity hurts alignment when the MT model has dmodel 1024, showing it is a manual capacity choice that bounds results.
assumptions (4)
  • domain assumption Cross-entropy loss at the MT decoder is a sufficient training signal for connector alignment.
    Training in Section 2 uses only standard cross-entropy at the decoder output, with no auxiliary alignment, contrastive, or reconstruction loss.
  • domain assumption Frozen ASR and MT models retain enough latent information for translation without parameter updates.
    The method freezes both foundation models (Sections 1, 2); if the embeddings are too modality-specific, the connector cannot recover the missing transfer.
  • domain assumption How2 English-Portuguese is representative enough for the scaling and domain-adaptation conclusions.
    All experiments run on a single dataset and language pair (Section 4.1), yet the abstract makes a general 'universally improves' claim.
  • standard math Standard transformer training and backpropagation behave as assumed.
    The results rely on standard deep learning optimization with no formal guarantees.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Aligning Pre-trained Models for Spoken Language Translation." pith.science (2026). https://pith.science/paper/ADYV7GFH

@misc{pith2026241118294,
  author       = {Pith},
  title        = {Pith review of: Aligning Pre-trained Models for Spoken Language Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ADYV7GFH}},
  note         = {Machine review of arXiv:2411.18294}
}
read the original abstract

This paper investigates a novel approach to end-to-end speech translation (ST) based on aligning frozen pre-trained automatic speech recognition (ASR) and machine translation (MT) models via a small connector module (Q-Former, our Subsampler-Transformer Encoder). This connector bridges the gap between the speech and text modalities, transforming ASR encoder embeddings into the latent representation space of the MT encoder while being the only part of the system optimized during training. Experiments are conducted on the How2 English-Portuguese dataset as we investigate the alignment approach in a small-scale scenario focusing on ST. While keeping the size of the connector module constant and small in comparison ( < 5% of the size of the larger aligned models), increasing the size and capability of the foundation ASR and MT models universally improves translation results. We also find that the connectors can serve as domain adapters for the foundation MT models, significantly improving translation performance in the aligned ST setting. We conclude that this approach represents a viable and scalable approach to training end-to-end ST systems.

Figures

Figures reproduced from arXiv: 2411.18294 by the authors.

Figure 1
Figure 1. Diagram of the ECD (A) and the ECED (B) alignment architectures for ST. Modules annotated with the [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Diagram of the two connector modules. Vari [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Performance study of the STE connector and [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Approaching Dialogue State Tracking via Aligning Speech Encoders and LLMs

    eess.AS 2025-06 conditional novelty 5.0 of 10

    An open-source WavLM-plus-connector-plus-LLM pipeline achieves state-of-the-art spoken dialogue state tracking on SpokenWOZ test (34.66% JGA with OLMo-1B, 42.17% with Gemma-2-9B), with detailed ablations.

Reference graph

Works this paper leans on

42 extracted references · 12 canonical work pages · cited by 1 Pith paper

  1. [1]

    Menick, Sebastian Borgeaud, Andy Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikolaj Binkowski, Ricardo Barreira, Oriol Vinyals, Andrew Zisserman, and Kar \' e n Simonyan

    Jean - Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob L. Menick, Sebastian Borgeaud, Andy Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikolaj Bin...

    work page 2022

  2. [2]

    Rosana Ardila, Megan Branson, Kelly Davis, Michael Kohler, Josh Meyer, Michael Henretty, Reuben Morais, Lindsay Saunders, Francis Tyers, and Gregor Weber. 2020. https://aclanthology.org/2020.lrec-1.520 Common Voice : A Massively - Multilingual Speech Corpus . In Proceedings of the Twelfth Language Resources and Evaluation Conference , pages 4218--4222, Ma...

    work page 2020

  3. [3]

    Diedre Carmo, Marcos Piau, Israel Campiotti, Rodrigo Nogueira, and Roberto Lotufo. 2020. Ptt5: Pretraining and validating the t5 model on brazilian portuguese data. arXiv preprint arXiv:2008.09144

    arXiv 2020

  4. [4]

    Feilong Chen, Minglun Han, Haozhi Zhao, Qingyang Zhang, Jing Shi, Shuang Xu, and Bo Xu. 2023 a . https://arxiv.org/abs/2305.04160 X-llm: Bootstrapping advanced large language models by treating multi-modalities as foreign languages . Preprint, arXiv:2305.04160

    arXiv 2023

  5. [5]

    Guo Chen, Yin-Dong Zheng, Jiahao Wang, Jilan Xu, Yifei Huang, Junting Pan, Yi Wang, Yali Wang, Yu Qiao, Tong Lu, and Limin Wang. 2023 b . https://arxiv.org/abs/2305.13292 Videollm: Modeling video sequence with large language models . Preprint, arXiv:2305.13292

    arXiv 2023

  6. [6]

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven C. H. Hoi. 2023. http://papers.nips.cc/paper\_files/paper/2023/hash/9a6a435e75419a836fe47ab6793623e6-Abstract-Conference.html Instructblip: Towards general-purpose vision-language models with instruction tuning . In Advances in Neura...

    2023

  7. [7]

    Miquel Espl \`a , Mikel Forcada, Gema Ram \' rez-S \'a nchez, and Hieu Hoang. 2019. https://aclanthology.org/W19-6721 P ara C rawl: Web-scale parallel corpora for the languages of the EU . In Proceedings of Machine Translation Summit XVII: Translator, Project and User Tracks, pages 118--119, Dublin, Ireland. European Association for Machine Translation

    work page 2019

  8. [8]

    Philip Gage. 1994. A new algorithm for data compression. C Users J., 12(2):23–38

    work page 1994

Show all 42 references
  1. [9]

    Godfrey, E.C

    J.J. Godfrey, E.C. Holliman, and J. McDaniel. 1992. https://doi.org/10.1109/ICASSP.1992.225858 SWITCHBOARD : telephone speech corpus for research and development . In [ Proceedings ] ICASSP -92: 1992 IEEE International Conference on Acoustics , Speech , and Signal Processing ,...

    1992

  2. [10]

    Francois Hernandez, Vincent Nguyen, Sahar Ghannay, Natalia Tomashenko, and Yannick Estève. 2018. https://doi.org/10.1007/978-3-319-99579-3_21 TED - LIUM 3: Twice as Much Data and Corpus Repartition for Experiments on Speaker Adaptation : 20th International Conference , SPECOM ...

    2018 doi

  3. [11]

    Yukiya Hono, Koh Mitsuda, Tianyu Zhao, Kentaro Mitsui, Toshiaki Wakatsuki, and Kei Sawada. 2023. https://arxiv.org/abs/2312.03668 An integration of pre-trained speech and language models for end-to-end speech recognition . Preprint, arXiv:2312.03668

    2023 arXiv

  4. [12]

    Hirofumi Inaguma, Shun Kiyono, Kevin Duh, Shigeki Karita, Nelson Yalta, Tomoki Hayashi, and Shinji Watanabe. 2020. https://doi.org/10.18653/v1/2020.acl-demos.34 ESP net- ST : All-in-one speech translation toolkit . In Proceedings of the 58th Annual Meeting of the Association f...

    2020 doi

  5. [13]

    Marcin Junczys-Dowmunt, Roman Grundkiewicz, Tomasz Dwojak, Hieu Hoang, Kenneth Heafield, Tom Neckermann, Frank Seide, Ulrich Germann, Alham Fikri Aji, Nikolay Bogoychev, Andr \'e F. T. Martins, and Alexandra Birch. 2018. https://doi.org/10.18653/v1/P18-4020 M arian: Fast neura...

    2018 doi

  6. [14]

    Shigeki Karita, Nelson Enrique Yalta Soplin, Shinji Watanabe, Marc Delcroix, Atsunori Ogawa, and Tomohiro Nakatani. 2019. https://doi.org/10.21437/Interspeech.2019-1938 Improving Transformer-Based End-to-End Speech Recognition with Connectionist Temporal Classification and Lan...

    2019 doi

  7. [15]

    Santosh Kesiraju, Marek Sarvaš, Tomáš Pavlíček, Cécile Macaire, and Alejandro Ciuba. 2023. https://doi.org/10.21437/interspeech.2023-2506 Strategies for improving low resource speech to text translation relying on pre-trained asr models . In INTERSPEECH 2023. ISCA

    2023 doi

  8. [16]

    Kwangyoun Kim, Felix Wu, Yifan Peng, Jing Pan, Prashant Sridhar, Kyu Jeong Han, and Shinji Watanabe. 2022. https://doi.org/10.1109/SLT54892.2023.10022656 E-branchformer: Branchformer with enhanced merging for speech recognition . In IEEE Spoken Language Technology Workshop, SL...

    2022

  9. [17]

    Taku Kudo. 2018. https://doi.org/10.18653/v1/P18-1007 Subword regularization: Improving neural network translation models with multiple subword candidates . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), page...

    2018 doi

  10. [18]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven C. H. Hoi. 2023. https://proceedings.mlr.press/v202/li23q.html BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models . In International Conference on Machine Learning, ICML 2023,...

    2023

  11. [19]

    Alexandre Lopes, Rodrigo Nogueira, Roberto Lotufo, and Helio Pedrini. 2020. https://www.aclweb.org/anthology/2020.wmt-1.90 Lite training strategies for P ortuguese- E nglish and E nglish- P ortuguese translation . In Proceedings of the Fifth Conference on Machine Translation, ...

    2020

  12. [20]

    Chenyang Lyu, Minghao Wu, Longyue Wang, Xinting Huang, Bingshuai Liu, Zefeng Du, Shuming Shi, and Zhaopeng Tu. 2023. https://arxiv.org/abs/2306.09093 Macaw-llm: Multi-modal language modeling with image, audio, video, and text integration . Preprint, arXiv:2306.09093

    2023 arXiv

  13. [21]

    Jan Niehues, Rolando Cattoni, Sebastian St \"u ker, Matteo Negri, Marco Turchi, Thanh-Le Ha, Elizabeth Salesky, Ramon Sanabria, Loic Barrault, Lucia Specia, and Marcello Federico. 2019. https://aclanthology.org/2019.iwslt-1.1 The IWSLT 2019 evaluation campaign . In Proceedings...

    2019

  14. [22]

    Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015. https://doi.org/10.1109/ICASSP.2015.7178964 Librispeech: An ASR corpus based on public domain audio books . In 2015 IEEE International Conference on Acoustics , Speech and Signal Processing ( ICASSP ) , ...

    2015

  15. [23]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. https://doi.org/10.3115/1073083.1073135 B leu: a method for automatic evaluation of machine translation . In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311--3...

    2002

  16. [24]

    Park, William Chan, Yu Zhang, Chung-Cheng Chiu, Barret Zoph, Ekin D

    Daniel S. Park, William Chan, Yu Zhang, Chung-Cheng Chiu, Barret Zoph, Ekin D. Cubuk, and Quoc V. Le. 2019. https://doi.org/10.21437/interspeech.2019-2680 Specaugment: A simple data augmentation method for automatic speech recognition . In Interspeech 2019. ISCA

    2019 doi

  17. [25]

    Paul and Janet M

    Douglas B. Paul and Janet M. Baker. 1992. https://aclanthology.org/H92-1073 The Design for the Wall Street Journal -based CSR Corpus . In Speech and Natural Language : Proceedings of a Workshop Held at Harriman , New York , February 23-26, 1992

    1992

  18. [26]

    Maja Popovi \'c . 2015. https://doi.org/10.18653/v1/W15-3049 chr F : character n-gram F -score for automatic MT evaluation . In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392--395, Lisbon, Portugal. Association for Computational Linguistics

    2015 doi

  19. [27]

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. https://proceedings.mlr.press/v202/radford23a.html Robust speech recognition via large-scale weak supervision . In International Conference on Machine Learning, ICML 2023, 23-29 J...

    2023

  20. [28]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. http://jmlr.org/papers/v21/20-074.html Exploring the limits of transfer learning with a unified text-to-text transformer . J. Mach. Learn. Res., ...

    2020

  21. [29]

    Ramon Sanabria, Ozan Caglayan, Shruti Palaskar, Desmond Elliott, Lo\"ic Barrault, Lucia Specia, and Florian Metze. 2018. http://arxiv.org/abs/1811.00347 How2: a large-scale dataset for multimodal language understanding . In Proceedings of the Workshop on Visually Grounded Inte...

    2018 arXiv

  22. [30]

    Yixuan Su, Tian Lan, Huayang Li, Jialu Xu, Yan Wang, and Deng Cai. 2023. https://aclanthology.org/2023.tllm-1.2 P anda GPT : One model to instruction-follow them all . In Proceedings of the 1st Workshop on Taming Large Language Models: Controllability in the era of Interactive...

    2023

  23. [31]

    J \"o rg Tiedemann. 2020. https://aclanthology.org/2020.wmt-1.139 The tatoeba translation challenge -- realistic data sets for low resource and multilingual MT . In Proceedings of the Fifth Conference on Machine Translation, pages 1174--1182, Online. Association for Computatio...

    2020

  24. [32]

    J \"o rg Tiedemann and Santhosh Thottingal. 2020. https://aclanthology.org/2020.eamt-1.61 OPUS - MT -- building open translation services for the world . In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, pages 479--480, Lisboa, P...

    2020

  25. [33]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf Attention is all you need . In Advances in Ne...

    2017

  26. [34]

    Hari Vydana, Martin Karafi\' a t, Kate r ina Z mol\' i kov\' a , Luk\' a s Burget, and Jan C ernock\' y

    K. Hari Vydana, Martin Karafi\' a t, Kate r ina Z mol\' i kov\' a , Luk\' a s Burget, and Jan C ernock\' y . 2021. https://doi.org/10.1109/ICASSP39728.2021.9414159 Jointly trained transformers models for spoken language translation . In ICASSP 2021 - 2021 IEEE International Co...

    2021

  27. [35]

    Changhan Wang, Morgane Riviere, Ann Lee, Anne Wu, Chaitanya Talnikar, Daniel Haziza, Mary Williamson, Juan Pino, and Emmanuel Dupoux. 2021. https://doi.org/10.18653/v1/2021.acl-long.80 VoxPopuli : A Large - Scale Multilingual Speech Corpus for Representation Learning , Semi - ...

    2021 doi

  28. [36]

    Changhan Wang, Yun Tang, Xutai Ma, Anne Wu, Dmytro Okhonko, and Juan Pino. 2020. https://aclanthology.org/2020.aacl-demo.6 Fairseq S 2 T : Fast speech-to-text modeling with fairseq . In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Comput...

    2020

  29. [37]

    Rubenstein, Lukas Zilka, Dian Yu, Golan Pundak, Nikhil Siddhartha, Johan Schalkwyk, and Yonghui Wu

    Mingqiu Wang, Wei Han, Izhak Shafran, Zelin Wu, Chung - Cheng Chiu, Yuan Cao, Nanxin Chen, Yu Zhang, Hagen Soltau, Paul K. Rubenstein, Lukas Zilka, Dian Yu, Golan Pundak, Nikhil Siddhartha, Johan Schalkwyk, and Yonghui Wu. 2023. https://doi.org/10.1109/ASRU57964.2023.10389703 ...

    2023

  30. [38]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020 doi

  31. [39]

    Wenyi Yu, Changli Tang, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun Ma, and Chao Zhang. 2023. https://arxiv.org/abs/2309.13963 Connecting speech encoder and large language model for asr . Preprint, arXiv:2309.13963

    2023 arXiv

  32. [40]

    Hang Zhang, Xin Li, and Lidong Bing. 2023. https://doi.org/10.18653/v1/2023.emnlp-demo.49 Video- LL a MA : An instruction-tuned audio-visual language model for video understanding . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Syst...

    2023 doi

  33. [41]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  34. [42]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.