Pith. sign in

REVIEW 1 major objections 4 minor 1 cited by

Leveraging Broadcast Media Subtitle Transcripts for Automatic Speech Recognition and Subtitling

T0 review · 1 major / 4 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read TV subtitle transcripts, though inexact and misaligned, can be used as weakly supervised training data that improves both verbatim speech recognition and automatic subtitle generation in a single end-to-end model.

desk verdict Solid architecture for using TV subtitles to improve verbatim ASR, but the flagship '50% WER reduction' overstates what the same-baseline numbers support. read the letter →

arxiv 2502.03212 v1 pith:V4MJ4BNZ submitted 2025-02-05 eess.AS cs.SD

Jakob Poncelet , Hugo Van hamme This is my paper · ORCID
classification eess.AScs.SD
keywords AutomaticSpeechRecognitionWeakSupervisionEnd-to-EndModellingSubtitlesBroadcastMediaDataFlemishDutchCascadedEncodersMultitaskLearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Many languages have abundant broadcast television with manually written intralingual subtitles, but subtitles are not exact transcripts: subtitlers shorten, rephrase, and normalise dialect into standard written language, and their timestamps are often misaligned. This paper tries to use such weakly supervised subtitle data anyway, in combination with a small manually transcribed verbatim corpus, to improve both exact speech-to-text and automatic subtitling for Flemish Dutch. The key move is to treat verbatim transcripts and subtitles as two different output languages and to train a single end-to-end model with a shared acoustic encoder, a small cascade encoder that translates acoustic features into subtitle features, and two separate decoders. On 14,000 hours of subtitles the model reports large verbatim error reductions, from 10.61% to 6.49% WER on one long-form test set and from 14.17% to 8.63% on another, while simultaneously producing subtitle text with a BLEU score of 61.76. If this transfers to other medium-resource languages, subtitle archives could become a practical substitute for expensive manual transcription.

What carries the argument

The load-bearing mechanism is the cascaded dual-feature architecture. A 12-layer Conformer acoustic encoder produces verbatim-oriented features and is regularised with CTC on the verbatim data; a smaller Transformer subtitle encoder is stacked on its outputs and trained, through the subtitle decoder's loss, to rewrite those features into standard written subtitle text. Each decoder is a Multi-Transformer decoder with two cross-attention layers per block, one attending to the ASR encoder and one to the subtitle encoder, so the verbatim decoder can also exploit the subtitle representation and the subtitle decoder can retain fine-grained temporal information. The joint loss mixes verbatim ASR loss and subtitle cross-entropy, with optional subtitle CTC, and batches balance verbatim and subtitle utterances. This design is what lets gradients from subtitle-only data flow back into the shared acoustic encoder without corrupting the verbatim CTC path.

What would settle it

Randomly permute the audio–subtitle pairing in the 14,000-hour training set so each subtitle belongs to a different utterance, train the same cascaded dual-feature model, and check verbatim WER on the long-form test sets; if WER still drops by the same margin, the subtitle content is not the cause of the reported improvement.

Watch

Extended reading notes

Core claim

The paper's central claim is that a joint model with cascaded encoders and separate decoders can learn from both domains without parallel data, and that the subtitle stream actively improves the verbatim branch. The authors argue that naively mixing subtitles into verbatim training is harmful, but that a dedicated subtitle encoder cascaded onto the ASR encoder absorbs the differences between spoken and written language, so the shared ASR encoder can be optimised for both objectives. The best variant, cascaded encoder with dual features, conditions both decoders on both encoder outputs through double cross-attention. The reported result is that, compared with a strong Conformer-based baseline trained only on the verbatim corpus, adding weakly supervised subtitles yields large relative WER improvements on long-form test sets in the large-scale experiment, with the paper describing up to 50% relative reduction and concrete long-form numbers going from 10.61% to 6.49% and from 14.17% to 8.63%. The same model produces subtitles that score 61.76 BLEU on a held-out 14,000-hour sample.

Load-bearing premise

The approach rests on the assumption that one acoustic encoder can learn a feature space that simultaneously supports exact verbatim decoding and a small subtitle encoder that translates those features into standard written subtitles, using no parallel verbatim-subtitle utterances; if that cross-task transfer is specific to Flemish broadcast speech, the gains will not generalise.

Editorial extensions

If this is right

  • If the approach holds, medium-resource languages with broadcast subtitle archives can train substantially better verbatim ASR without needing more manually transcribed speech.
  • The same model outputs a subtitle in standard written language alongside the verbatim transcript, so automatic subtitling and clean-text NLP inputs come from the same forward pass.
  • Because the method does not require forced alignment, filtering, pseudo-labelling, or iterative refinement of the weakly labelled data, it is relatively cheap to scale with broadcast archives.
  • Increasing the subtitle corpus from 400 to 14,000 hours keeps reducing WER and raising subtitle BLEU, suggesting the model will continue to benefit from even larger archives.
  • A 70M or 180M parameter model beats a 1.55B-parameter Whisper model and a Whisper model fine-tuned on verbatim data on Flemish verbatim WER, indicating that subtitle-trained models can compete with far larger systems.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the architecture is effectively learning an implicit spoken-to-standard-writing translation inside the subtitle encoder; in principle this could replace post-hoc inverse text normalisation and disfluency-removal pipelines for clean-text generation.
  • Editorial inference: the method's value likely depends on how strongly a language's subtitling conventions rewrite speech; with nearly verbatim subtitles the cascade may have less to learn, while with heavily rephrased subtitles the gains may be larger but the subtitle decoder's output will be farther from a transcript.
  • Editorial inference: a direct testable extension is to apply the identical recipe to another low-resource or medium-resource language with a large intralingual subtitle archive; if verbatim WER gains do not appear there, the Flemish result may be specific to the large spoken-written gap in Belgian Dutch.
  • Editorial inference: because the subtitle data is not released, reproducing the exact numbers requires access to similar broadcaster archives; open subtitle collections may offer a weaker but more portable test of the same idea.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 4 minor

Summary. The paper studies how weakly supervised TV subtitle transcripts can be combined with a small verbatim ASR corpus to improve both verbatim speech recognition and automatic subtitling for Flemish Dutch. The authors propose several encoder-decoder architectures, ranging from a naive mixed-training model and a shared task-decoder model to parallel and cascaded models with separate decoders and an additional subtitle encoder. The central empirical finding is that cascaded models, especially the cascaded encoder model with dual encoder features, outperform the CGN-only baseline and the alternative multitask architectures on verbatim WER while also producing competitive subtitle BLEU scores. The paper includes ablations on the Multi-Transformer decoder, a data-filtering study, long-form serialised output training, scaling experiments up to 14k hours of subtitles, and comparisons with Whisper and an ASR-plus-LLM subtitling pipeline.

Significance. If the results hold, this is a useful contribution to weakly supervised and low-resource ASR. The experimental methodology is generally careful: evaluation uses multiple held-out test sets (cgn-dev, subs-annot, subs-valid, subs-valid-14kh), statistical significance is assessed with MAPSSWE and bootstrap resampling, and the main trend is consistent across data sizes and model variants. The paper also ships open-source code and models, which is a concrete strength. The central claim, that explicitly separating verbatim and subtitle targets through cascaded encoders improves verbatim ASR, is well supported by the controlled comparisons in Tables 1 and 4. The main weakness is an overstated headline number in the scaling section, together with a few presentation issues and one unvalidated design choice in the cascaded-decoder variant.

major comments (1)
  1. [5.3.2 and Section 6] The claim of 'up to 50% relative WER reduction compared to the baseline ASR model' is not supported by the same-baseline comparisons reported in Figure 4 and Table 6. Against the long-form CGN-only baseline, the best XL results give a relative reduction of (10.61-6.49)/10.61 = 38.8% on cgn-dev-long and (14.17-8.63)/14.17 = 39.1% on subs-annot-long. The ~50% figure only appears when the comparison is made against the short-form CGN-only model evaluated on long-form test sets (15.12% and 16.85% in Table 4), which is not the appropriate baseline for the scaling experiment. The sentence in Section 5.3.2 and the corresponding sentence in Section 6 should be corrected to the actually supported values, or explicitly re-anchored to the short-form baseline with a clear caveat.
minor comments (4)
  1. [3.3.3 and footnote 5] The cascaded-decoder variant relies on forwarding an <unk> token through the ASR decoder to obtain a sentence embedding for subtitle data. The paper states this 'improves the optimisation' and the footnote acknowledges a training/testing mismatch, but no quantitative ablation is provided. Since this design choice affects only one of the proposed variants and the cascaded-encoder model performs similarly, this does not threaten the main conclusion, but it should be presented as a limitation or backed by an ablation.
  2. [Table 3] The first row of Table 3 is labelled 'Naive E2E ASR' with 0 hours of subtitles and shows the CGN-only baseline WERs; this row should be relabelled 'E2E ASR - CGN only' to avoid confusion with the naive model trained on subtitle data.
  3. [Figure 4 caption] The caption says 'Figure (a) shows WERs' and 'Figure (b) shows BLEU scores', but the three panels are (a) WER on cgn-dev, (b) WER on subs-annot, and (c) BLEU; the caption text should be corrected.
  4. [Figure 3] The x-axis labels 'Enc.' and 'Dec.' are not defined in the caption; add a sentence explaining that they refer to whether the subtitle encoder is conditioned on the ASR encoder outputs or on the ASR decoder features.

Circularity Check

0 steps flagged · score 1.0 of 10

No material circularity: the central results are empirical held-out evaluations, and the paper's self-citations are provenance, test-set, and hyperparameter references rather than load-bearing derivations.

full rationale

The derivation chain is empirical, not analytic. Models are trained on CGN verbatim data plus subtitle datasets (subs-720h, subs-2kh, subs-14kh) and evaluated on held-out test sets with reference transcripts and subtitles (cgn-dev, subs-annot, subs-valid, subs-valid-14kh). No fitted constant is renamed as a prediction, and no equation reduces a reported result to a training objective by construction. The cascaded subtitle encoder is trained through its own subtitle decoder loss, while the verbatim WER is produced by a separate verbatim decoder and compared against a CGN-only baseline on the same test sets (Tables 1, 4, and 6). The self-citations to [29] and [80] supply the parallel-decoder baseline, the cgn-dev/subs-annot splits, and the lambda=0.5 weighting; these are provenance, dataset, and hyperparameter references, not uniqueness constraints or unverified theorems, so they do not make the central claim circular. The subtitle evaluation sets do come from VRT material of the same broadcaster, and the paper itself notes in Appendix A.2 that the models 'are aware of the subtitling protocol as they were trained on subtitles from the same broadcaster'; this is a domain-match limitation rather than a reduction of the prediction to its input. The 'up to 50% relative WER reduction' claim in Section 5.3.2 is a support or anchoring issue, not a circularity issue: the same-baseline comparisons in Figure 4 and Table 6 support about 39-41% relative improvement, so the headline should be corrected or explicitly re-anchored to the short-form baseline. Overall, the paper is self-contained against held-out benchmarks and shows no significant circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claims rest on standard ML assumptions plus a strong domain assumption that subtitle text is a consistent, learnable normalization of spoken Flemish. No new physical entities are introduced. The free parameters are training hyperparameters that are held fixed across compared architectures; they affect absolute scores but not the relative ordering of models.

free parameters (4)
  • lambda_asr_equals_lambda_subs = 0.5/0.5
    Task-loss weighting between verbatim ASR and subtitling, set to 0.5 each; described as 'found optimal in previous work [29]' in Section 4.2.3.
  • CTC_weights_alpha_beta_gamma = 0.3, 0.3, 0.3
    CTC weight, intermediate CTC weight, and subtitle CTC weight, all set to 0.3 in Section 4.2.3 from standard ESPnet recipes.
  • BPE_vocab_size = 5000
    Unigram BPE vocabulary size trained jointly on verbatim and subtitle text; a design choice that affects tokenization for both decoders (Section 4.2.3).
  • checkpoint_averaging = 10 best checkpoints
    The 10 intermediate checkpoints with highest validation accuracy are averaged for evaluation; a model-selection choice that could affect reported numbers (Section 4.2.3).
assumptions (5)
  • domain assumption Subtitles are a learnable transformation of verbatim speech into standard written Flemish.
    The entire cascaded model (Section 3.3) assumes that a small subtitle encoder can learn to map ASR encoder features to subtitle decoder targets from unpaired data, without explicit parallel supervision.
  • domain assumption The shared ASR encoder representation is sufficient for both verbatim and subtitle decoding.
    Parallel and cascaded architectures backpropagate subtitle loss through the shared ASR encoder, assuming the representation is not harmed for verbatim CTC and decoding (Sections 3.2, 3.3).
  • standard math CTC assumes monotonic alignment; it is applied to verbatim targets only, except for subtitle CTC in some cascaded variants.
    The authors apply CTC on verbatim data only in the parallel model (Section 3.2) and add subtitle CTC only when a dedicated subtitle encoder exists (Section 3.3.5), relying on the standard monotonicity assumption of CTC.
  • ad hoc to paper Forwarding an <unk> token through the ASR decoder yields a useful sentence embedding for subtitle data in the cascaded decoder variant.
    Section 3.3.3 uses <unk> as the verbatim reference for subtitle data to train the cascaded decoder model, with an acknowledged train/test mismatch.
  • domain assumption Subtitle screen timings are accurate enough to define training segments without forced alignment.
    Section 4.1.2 builds the subtitle dataset by segmenting audio using on-screen subtitle timings, filtering only obvious non-speech annotations; no alignment is performed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Leveraging Broadcast Media Subtitle Transcripts for Automatic Speech Recognition and Subtitling." pith.science (2026). https://pith.science/paper/V4MJ4BNZ

@misc{pith2026250203212,
  author       = {Pith},
  title        = {Pith review of: Leveraging Broadcast Media Subtitle Transcripts for Automatic Speech Recognition and Subtitling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V4MJ4BNZ}},
  note         = {Machine review of arXiv:2502.03212}
}
read the original abstract

The recent advancement of speech recognition technology has been driven by large-scale datasets and attention-based architectures, but many challenges still remain, especially for low-resource languages and dialects. This paper explores the integration of weakly supervised transcripts from TV subtitles into automatic speech recognition (ASR) systems, aiming to improve both verbatim transcriptions and automatically generated subtitles. To this end, verbatim data and subtitles are regarded as different domains or languages, due to their distinct characteristics. We propose and compare several end-to-end architectures that are designed to jointly model both modalities with separate or shared encoders and decoders. The proposed methods are able to jointly generate a verbatim transcription and a subtitle. Evaluation on Flemish (Belgian Dutch) demonstrates that a model with cascaded encoders and separate decoders allows to represent the differences between the two data types most efficiently while improving on both domains. Despite differences in domain and linguistic variations, combining verbatim transcripts with subtitle data leads to notable ASR improvements without the need for extensive preprocessing. Additionally, experiments with a large-scale subtitle dataset show the scalability of the proposed approach. The methods not only improve ASR accuracy but also generate subtitles that closely match standard written text, offering several potential applications.

Figures

Figures reproduced from arXiv: 2502.03212 by the authors.

Figure 1
Figure 1. Overview of the proposed approach. Verbatim transcriptions from ASR datasets [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Comparison of all proposed models. (a) E2E ASR: Encoder-decoder ASR model with CTC regularisation. In naive E2E ASR, the subtitles are treated as verbatim transcriptions and both datasets are combined. (b) Shared task decoder model: similar to (a), but the decoder is conditioned on a task token to generate either a verbatim or a subtitle output. (c) Parallel model: the encoder is shared and there are two separate de… view at source ↗
Figure 3
Figure 3. Comparison between subtitle decoder blocks with only one cross-attention to the [PITH_FULL_IMAGE:figures/full_fig_p018_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Scaling experiments for the cascaded model with dual encoder features. On the [PITH_FULL_IMAGE:figures/full_fig_p023_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Leveraging Beam Search Information for Confidence Estimation in E2E ASR

    eess.AS 2026-07 conditional novelty 5.0 of 10

    A 0.6k-parameter module that scores ASR tokens and words using only beam-search scores, ranks, context sums, and top-k alternatives substantially reduces calibration error, especially worst-case MCE.

Reference graph

Works this paper leans on

92 extracted references · 62 canonical work pages · cited by 1 Pith paper

  1. [29]

    Poncelet, H

    J. Poncelet, H. Van hamme, Learning to jointly transcribe and subtitle for end-to- end spontaneous speech recognition, in: Proc. IEEE Spoken Language Technology Workshop (SLT), 2023, pp. 182–189. doi:10.1109/SLT54892.2023.10022420

    work page arXiv 2023

  2. [1]

    Vaswani, et al., Attention is all you need, in: Proc

    A. Vaswani, et al., Attention is all you need, in: Proc. Conf. on Neural Informa- tion Processing Systems (NeurIPS), 2017, pp. 5998–6008. URL https://proceedings.neurips.cc/paper_files/paper/2017/file/ 3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf

    2017

  3. [2]

    Gulati, et al., Conformer: Convolution-augmented transformer for speech recognition, in: Proc

    A. Gulati, et al., Conformer: Convolution-augmented transformer for speech recognition, in: Proc. Interspeech, 2020, pp. 5036–5040. doi:10.21437/ Interspeech.2020-3015. 25

    2020

  4. [3]

    Y. Zhang, et al., BigSSL: Exploring the frontier of large-scale semi-supervised learning for automatic speech recognition, IEEE Journal of Selected Topics in Signal Processing (JSTSP) 16 (6) (2022) 1519–1532. doi:10.1109/JSTSP.2022. 3182537

    work page doi:10.1109/jstsp.2022 2022

  5. [4]

    Zhang, et al., Pushing the limits of semi-supervised learning for automatic speech recognition, in: Proc

    Y. Zhang, et al., Pushing the limits of semi-supervised learning for automatic speech recognition, in: Proc. Conf. on Neural Information Processing Systems (NeurIPS): SAS Workshop, 2022

    2022

  6. [5]

    Babu, et al., XLS-R: Self-supervised cross-lingual speech representation learn- ing at scale, in: Proc

    A. Babu, et al., XLS-R: Self-supervised cross-lingual speech representation learn- ing at scale, in: Proc. Interspeech, 2022, pp. 2278–2282. doi:10.21437/ Interspeech.2022-143

    2022

  7. [6]

    S. Feng, B. M. Halpern, O. Kudina, O. Scharenborg, Towards inclusive auto- matic speech recognition, Computer, Speech and Language 84 (2024) 101567. doi:https://doi.org/10.1016/j.csl.2023.101567. URL https://www.sciencedirect.com/science/article/pii/ S0885230823000864

    work page arXiv 2024

  8. [7]

    W. Chen, X. Chang, Y. Peng, Z. Ni, S. Maiti, S. Watanabe, Reducing barriers to self-supervised learning: HuBERT pre-training with academic compute, in: Proc. Interspeech, 2023, pp. 4404–4408. doi:10.21437/Interspeech.2023-1176

    doi:10.21437/interspeech.2023-1176 2023

Show all 92 references
  1. [9]

    C.-C. Chiu, J. Qin, Y. Zhang, J. Yu, Y. Wu, Self-supervised learning with random- projection quantizer for speech recognition, in: Proc. Int. Conf. on Machine Learn- ing (ICML), 2022, pp. 3915–3924

    2022

  2. [10]

    Mohamed, et al., Self-supervised speech representation learning: A review, IEEE Journal of Selected Topics in Signal Processing (JSTSP) 16 (6) (2022) 1179–1210

    A. Mohamed, et al., Self-supervised speech representation learning: A review, IEEE Journal of Selected Topics in Signal Processing (JSTSP) 16 (6) (2022) 1179–1210. doi:10.1109/JSTSP.2022.3207050

    2022

  3. [11]

    Baevski, H

    A. Baevski, H. Zhou, A. Mohamed, M. Auli, Wav2vec 2.0: A framework for self-supervised learning of speech representations, in: Proc. Conf. on Neural In- formation Processing Systems (NeurIPS), 2020, pp. 12449–12460

    2020

  4. [12]

    W.-N. Hsu, B. Bolte, Y.-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, A. Mohamed, HuBERT: Self-supervised speech representation learning by masked prediction of hidden units, IEEE/ACM Trans. on Audio, Speech, and Language Processing 29 (2021) 3451–3460. doi:10.1109/TASLP.2021.3122291

    2021

  5. [13]

    Chen, et al., WavLM: Large-scale self-supervised pre-training for full stack speech processing, IEEE Journal of Selected Topics in Signal Processing (JSTSP) 16 (6) (2022) 1505–1518

    S. Chen, et al., WavLM: Large-scale self-supervised pre-training for full stack speech processing, IEEE Journal of Selected Topics in Signal Processing (JSTSP) 16 (6) (2022) 1505–1518. doi:10.1109/JSTSP.2022.3188113

    2022

  6. [14]

    Conneau, A

    A. Conneau, A. Baevski, R. Collobert, A. Mohamed, M. Auli, Unsupervised cross- lingual representation learning for speech recognition, in: Proc. Interspeech, 2021, pp. 2426–2430. doi:10.21437/Interspeech.2021-329. 26

    2021 doi

  7. [15]

    Lee, et al., Textless speech-to-speech translation on real data, in: Proc

    A. Lee, et al., Textless speech-to-speech translation on real data, in: Proc. Conf. North American Chapter of the Association for Computational Linguis- tics (NAACL): Human Language Technologies, 2022, pp. 860–872

    2022

  8. [16]

    Chen, et al., Joint prediction and denoising for large-scale multilingual self- supervised learning, in: Proc

    W. Chen, et al., Joint prediction and denoising for large-scale multilingual self- supervised learning, in: Proc. IEEE Automatic Speech Recognition and Under- standing Workshop (ASRU), 2023

    2023

  9. [17]

    Yang, et al., SUPERB: Speech processing universal performance benchmark, in: Proc

    S. Yang, et al., SUPERB: Speech processing universal performance benchmark, in: Proc. Interspeech, 2021, pp. 1194–1198

    2021

  10. [19]

    Hsu, et al., Robust wav2vec 2.0: Analyzing domain shift in self- supervised pre-training, in: Proc

    W.-N. Hsu, et al., Robust wav2vec 2.0: Analyzing domain shift in self- supervised pre-training, in: Proc. Interspeech, 2021, pp. 721–725. doi:10.21437/ Interspeech.2021-236

    2021

  11. [20]

    Radford, J

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. Mcleavey, I. Sutskever, Ro- bust speech recognition via large-scale weak supervision, in: Proc. Int. Conf. on Machine Learning (ICML), 2023, pp. 28492–28518

    2023

  12. [21]

    W. Chan, D. S. Park, C. A. Lee, Y. Zhang, Q. V. Le, M. Norouzi, SpeechStew: Simply mix all available speech recognition data to train one large neural network, in: Workshop on Machine Learning in Speech and Language Processing (MLSLP), 2021

    2021

  13. [22]

    Peng, et al., Reproducing Whisper-style training using an open-source toolkit and publicly available data, in: Proc

    Y. Peng, et al., Reproducing Whisper-style training using an open-source toolkit and publicly available data, in: Proc. IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2023

    2023

  14. [23]

    Likhomanenko, et al., Rethinking evaluation in ASR: Are our models robust enough?, in: Proc

    T. Likhomanenko, et al., Rethinking evaluation in ASR: Are our models robust enough?, in: Proc. Interspeech, 2021, pp. 311–315. doi:10.21437/Interspeech. 2021-1758

    2021 doi

  15. [24]

    J. D. Cintas, A. Remael, Audiovisual translation: Subtitling, Routledge, 2014

    2014

  16. [25]

    Karakanta, M

    A. Karakanta, M. Negri, M. Turchi, MuST-Cinema: a speech-to-subtitles corpus, in: Proc. Int. Conf. on Language Resources and Evaluation (LREC), 2020, pp. 3727–3734

    2020

  17. [26]

    Panayotov, G

    V. Panayotov, G. Chen, D. Povey, S. Khudanpur, LibriSpeech: An ASR corpus based on public domain audio books, in: Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2015, pp. 5206–5210

    2015

  18. [27]

    B. C. Roy, D. Roy, Fast transcription of unstructured audio recordings, in: Proc. Interspeech, 2009, pp. 1647–1650. doi:10.21437/Interspeech.2009-500

    2009 doi

  19. [28]

    Y. Yin, D. Mori, S. Fujimoto, ReazonSpeech: A free and massive corpus for Japanese ASR, in: Proc. 29th Annual Meeting of the Association for Natural Language Processing, 2023, pp. 1134–1139. 27

    2023

  20. [30]

    Ihori, H

    M. Ihori, H. Sato, T. Tanaka, R. Masumura, S. Mizuno, N. Hojo, Transcribing speech as spoken and written dual text using an autoregressive model, in: Proc. Interspeech, 2023, pp. 461–465. doi:10.21437/Interspeech.2023-1655

    2023 doi

  21. [31]

    Kanda, Y

    N. Kanda, Y. Gaur, X. Wang, Z. Meng, T. Yoshioka, Serialized output training for end-to-end overlapped speech recognition, in: Proc. Interspeech, 2020, pp. 2797–2801. doi:10.21437/Interspeech.2020-999

    2020 doi

  22. [32]

    J. Xu, F. Buet, J. Crego, E. Bertin-Lem´ ee, F. Yvon, Joint generation of captions and subtitles with dual decoding, in: Proc. Int. Conf. on Spoken Language Trans- lation (IWSLT), ACL, 2022, pp. 74–82. doi:10.18653/v1/2022.iwslt-1.7

    2022 doi

  23. [33]

    Lamel, J.-L

    L. Lamel, J.-L. Gauvain, G. Adda, Lightly supervised and unsupervised acoustic model training, Computer, Speech and Language 16 (1) (2002) 115–129. doi: https://doi.org/10.1006/csla.2001.0186

    2002

  24. [34]

    Lanchantin, et al., Selection of multi-genre broadcast data for the training of automatic speech recognition systems, in: Proc

    P. Lanchantin, et al., Selection of multi-genre broadcast data for the training of automatic speech recognition systems, in: Proc. Interspeech, 2016, pp. 3057–3061. doi:10.21437/Interspeech.2016-462

    2016 doi

  25. [35]

    Bang, M.-Y

    J.-U. Bang, M.-Y. Choi, S.-H. Kim, O.-W. Kwon, Automatic construction of a large-scale speech recognition database using multi-genre broadcast data with inaccurate subtitle timestamps, IEICE Trans. on Information and Systems E103.D (2) (2020) 406–415. doi:10.1587/transinf.2019EDP7234

    2020 doi

  26. [36]

    S. Ando, H. Fujihara, Construction of a large-scale Japanese ASR corpus on TV recordings, in: Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2021, pp. 6948–6952

    2021

  27. [38]

    Saz, et al., Lightly supervised alignment of subtitles on multi-genre broadcasts, in: Multimedia Tools and Applications, Vol

    O. Saz, et al., Lightly supervised alignment of subtitles on multi-genre broadcasts, in: Multimedia Tools and Applications, Vol. 77, 2018, pp. 30533–30550. doi: 10.1007/s11042-018-6050-1

    2018 doi

  28. [39]

    Manohar, D

    V. Manohar, D. Povey, S. Khudanpur, JHU Kaldi system for Arabic MGB-3 ASR challenge using diarization, audio-transcript alignment and transfer learning, in: Proc. IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2017, pp. 346–352. doi:10.1109/ASRU.2017.8268956

    2017

  29. [41]

    N. M. Guerreiro, R. Rei, F. Batista, Towards better subtitles: A multilingual approach for punctuation restoration of speech transcripts, Expert Systems with Applications 186 (2021) 115740. doi:https://doi.org/10.1016/j.eswa.2021. 115740

    2021 doi

  30. [42]

    Geislinger, B

    R. Geislinger, B. Milde, C. Biemann, Improved open source automatic subtitling for lecture videos, in: Proc. Conf. on Natural Language Processing (KONVENS), 2022, pp. 98–103

    2022

  31. [43]

    Milde, R

    B. Milde, R. Geislinger, I. Lindt, T. Baumann, Open source automatic lecture subtitling, in: Proc. Conf. on Electronical Speech Signal Processing (ESSV), 2021, pp. 128–135

    2021

  32. [44]

    D. Liu, J. Niehues, G. Spanakis, Adapting end-to-end speech recognition for readable subtitles, in: Proc. Int. Conf. on Spoken Language Translation (IWSLT), ACL, 2020, pp. 247–256. doi:10.18653/v1/2020.iwslt-1.30

    2020 doi

  33. [45]

    Bell, et al., The MGB challenge: Evaluating multi-genre broadcast media recognition, in: Proc

    P. Bell, et al., The MGB challenge: Evaluating multi-genre broadcast media recognition, in: Proc. IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2015, pp. 687–693. doi:10.1109/ASRU.2015.7404863

    2015

  34. [46]

    Ali, et al., The MGB-2 challenge: Arabic multi-dialect broadcast media recog- nition, in: Proc

    A. Ali, et al., The MGB-2 challenge: Arabic multi-dialect broadcast media recog- nition, in: Proc. IEEE Spoken Language Technology Workshop (SLT), 2016, pp. 279–284. doi:10.1109/SLT.2016.7846277

    2016

  35. [47]

    Lleida, et al., Albayzin 2018 evaluation: The IberSpeech-RTVE challenge on speech technologies for Spanish broadcast media, Applied Sciences 9 (24) (2019)

    E. Lleida, et al., Albayzin 2018 evaluation: The IberSpeech-RTVE challenge on speech technologies for Spanish broadcast media, Applied Sciences 9 (24) (2019). doi:10.3390/app9245412. URL https://www.mdpi.com/2076-3417/9/24/5412

    2019 doi

  36. [48]

    X. Che, S. Luo, H. Yang, C. Meinel, Automatic lecture subtitle generation and how it helps, in: Proc. IEEE Int. Conf. on Advanced Learning Technologies (ICALT), 2017, pp. 34–38. doi:10.1109/ICALT.2017.11

    2017 doi

  37. [49]

    S. Papi, M. Gaido, A. Karakanta, M. Cettolo, M. Negri, M. Turchi, Direct speech translation for automatic subtitling, Trans. of the Assoc. for Computational Lin- guistics 11 (2023) 1355–1376. doi:10.1162/tacl_a_00607. URL https://aclanthology.org/2023.tacl-1.77

    2023 doi

  38. [50]

    K¨ urzinger, D

    L. K¨ urzinger, D. Winkelbauer, L. Li, T. Watzel, G. Rigoll, CTC-segmentation of large corpora for German end-to-end speech recognition, in: Int. Conf. on Speech and Computer (SPECOM), 2020, pp. 267–278

    2020

  39. [51]

    Ihori, A

    M. Ihori, A. Takashima, R. Masumura, Parallel corpus for Japanese spoken-to- written style conversion, in: Proc. Int. Conf. on Language Resources and Evalu- ation (LREC), 2020, pp. 6346–6353

    2020

  40. [52]

    Liao, et al., Improving readability for automatic speech recognition transcrip- tion, ACM Trans

    J. Liao, et al., Improving readability for automatic speech recognition transcrip- tion, ACM Trans. on Asian and Low-Resource Language Information Processing 22 (5) (2023). doi:10.1145/3557894

    2023 doi

  41. [53]

    Nozaki, T

    J. Nozaki, T. Kawahara, K. Ishizuka, T. Hashimoto, End-to-end speech-to- punctuated-text recognition, in: Proc. Interspeech, 2022, pp. 1811–1815. doi: 10.21437/Interspeech.2022-5. 29

    2022 doi

  42. [54]

    Futami, et al., Streaming joint speech recognition and disfluency detection, in: Proc

    H. Futami, et al., Streaming joint speech recognition and disfluency detection, in: Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP),

  43. [56]

    Z. Wang, Y. Wang, S. Wang, W. Che, Adaptive unsupervised self-training for dis- fluency detection, in: Proc. Int. Conf. on Computational Linguistics (COLING), ICCL, 2022, pp. 7209–7218

    2022

  44. [57]

    J. Guo, T. N. Sainath, R. J. Weiss, A spelling correction model for end-to-end speech recognition, in: Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2019, pp. 5651–5655. doi:10.1109/ICASSP.2019.8683745

    2019

  45. [58]

    S. Li, X. Lu, S. Sakai, M. Mimura, T. Kawahara, Semi-supervised ensemble DNN acoustic model training, in: Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2017, pp. 5270–5274. doi:10.1109/ICASSP.2017. 7953162

    2017 doi

  46. [59]

    B. Li, T. N. Sainath, R. Pang, Z. Wu, Semi-supervised training for end-to-end models via weak distillation, in: Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2019, pp. 2837–2841. doi:10.1109/ICASSP.2019. 8682172

    2019 doi

  47. [60]

    Pratap, A

    V. Pratap, A. Hannun, G. Synnaeve, R. Collobert, Star Temporal Classification: Sequence modeling with partially labeled data, in: Proc. Conf. on Neural Information Processing Systems (NeurIPS), 2022, pp. 13392–13403. URL https://proceedings.neurips.cc/paper_files/paper/2022/fi...

    2022

  48. [62]

    Singh, et al., Training ASR models by generation of contextual information, in: Proc

    K. Singh, et al., Training ASR models by generation of contextual information, in: Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2020, pp. 7864–7868. doi:10.1109/ICASSP40776.2020.9053527

    2020

  49. [63]

    Chen, et al., GigaSpeech: An evolving, multi-domain ASR corpus with 10,000 hours of transcribed audio, in: Proc

    G. Chen, et al., GigaSpeech: An evolving, multi-domain ASR corpus with 10,000 hours of transcribed audio, in: Proc. Interspeech, 2021, pp. 3670–3674. doi: 10.21437/Interspeech.2021-1965

    2021 doi

  50. [64]

    Zhang, et al., WenetSpeech: A 10000+ hours multi-domain Mandarin corpus for speech recognition, in: Proc

    B. Zhang, et al., WenetSpeech: A 10000+ hours multi-domain Mandarin corpus for speech recognition, in: Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 6182–6186. doi:10.1109/ICASSP43922.2022. 9746682

    2022

  51. [65]

    Galvez, et al., The People’s Speech: A large-scale diverse English speech recog- nition dataset for commercial usage, in: Proc

    D. Galvez, et al., The People’s Speech: A large-scale diverse English speech recog- nition dataset for commercial usage, in: Proc. Conf. on Neural Information Pro- cessing Systems (NeurIPS): Track on Datasets and Benchmarks, 2021. 30 URL https://datasets-benchmarks-proceedings...

    2021

  52. [66]

    Wilken, P

    P. Wilken, P. Georgakopoulou, E. Matusov, SubER: A metric for automatic eval- uation of subtitle quality, in: Proc. Int. Conf. on Spoken Language Translation (IWSLT), ACL, 2022, pp. 1–10. doi:10.18653/v1/2022.iwslt-1.1. URL https://aclanthology.org/2022.iwslt-1.1

    2022 doi

  53. [67]

    Karakanta, F

    A. Karakanta, F. Buet, M. Cettolo, F. Yvon, Evaluating subtitle segmentation for end-to-end generation systems, in: Proc. Int. Conf. on Language Resources and Evaluation (LREC), ELRA, 2022, pp. 3069–3078. URL https://aclanthology.org/2022.lrec-1.328

    2022

  54. [68]

    Graves, S

    A. Graves, S. Fern´ andez, F. Gomez, J. Schmidhuber, Connectionist Temporal Classification: Labelling unsegmented sequence data with recurrent neural net- works, in: Proc. Int. Conf. on Machine Learning (ICML), ACM, 2006, p. 369–376. doi:10.1145/1143844.1143891. URL https://do...

    2006

  55. [69]

    J. Lee, S. Watanabe, Intermediate loss regularization for CTC-based speech recog- nition, in: Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2021, pp. 6224–6228. doi:10.1109/ICASSP39728.2021.9414594

    2021

  56. [70]

    Watanabe, T

    S. Watanabe, T. Hori, S. Kim, J. R. Hershey, T. Hayashi, Hybrid CTC/Attention architecture for end-to-end speech recognition, IEEE Journal of Selected Topics in Signal Processing (JSTSP) 11 (8) (2017) 1240–1253. doi:10.1109/JSTSP.2017. 2763455

    2017 doi

  57. [71]

    Szegedy, V

    C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, Z. Wojna, Rethinking the Inception architecture for computer vision, in: Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 2818–2826. doi:10.1109/CVPR.2016. 308

    2016 doi

  58. [72]

    Anastasopoulos, D

    A. Anastasopoulos, D. Chiang, Tied multitask learning for neural speech transla- tion, in: Proc. Conf. of the North American Chapter of the Association for Com- putational Linguistics (NAACL): Human Language Technologies, Vol. 1, 2018, pp. 82–91. doi:10.18653/v1/N18-1008

    2018 doi

  59. [73]

    Inaguma, S

    H. Inaguma, S. Dalmia, B. Yan, S. Watanabe, Fast-MD: Fast multi-decoder end- to-end speech translation with non-autoregressive hidden intermediates, in: Proc. IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2021, pp. 922–929. doi:10.1109/ASRU51503.2021.9687894

    2021

  60. [74]

    Dalmia, B

    S. Dalmia, B. Yan, V. Raunak, F. Metze, S. Watanabe, Searchable hidden in- termediates for end-to-end models of decomposable sequence tasks, in: Proc. Conf. of the North American Chapter of the Association for Computational Linguistics (NAACL): Human Language Technologies, 202...

    2021 doi

  61. [75]

    Helcl, J

    J. Helcl, J. Libovick´ y, D. Variˇ s, CUNI system for the WMT18 multimodal trans- lation task, in: Proc. Conf. on Machine Translation (WMT), ACL, 2018, pp. 616–623. doi:10.18653/v1/W18-6441. 31

    2018 doi

  62. [76]

    Chuang, Y.-S

    S.-P. Chuang, Y.-S. Chuang, C.-C. Chang, H.-y. Lee, Investigating the reordering capability in CTC-based non-autoregressive end-to-end speech translation, in: Findings of the Association for Computational Linguistics (ACL-IJCNLP), ACL, 2021, pp. 1068–1077. doi:10.18653/v1/2021...

    2021 doi

  63. [77]

    Yan, et al., CTC alignments improve autoregressive translation, in: Proc

    B. Yan, et al., CTC alignments improve autoregressive translation, in: Proc. Conf. of the European Chapter of the Association for Computational Linguistics, 2023, pp. 1623–1639. doi:10.18653/v1/2023.eacl-main.119. URL https://aclanthology.org/2023.eacl-main.119

    2023 doi

  64. [78]

    Oostdijk, The Spoken Dutch Corpus: Overview and first evaluation, in: Proc

    N. Oostdijk, The Spoken Dutch Corpus: Overview and first evaluation, in: Proc. Int. Conf. on Language Resources and Evaluation (LREC), Vol. 2, 2000

    2000

  65. [79]

    T. Ko, V. Peddinti, D. Povey, S. Khudanpur, Audio augmentation for speech recognition, in: Proc. Interspeech, 2015, pp. 3586–3589. doi:10.21437/ Interspeech.2015-711

    2015

  66. [80]

    Poncelet, H

    J. Poncelet, H. Van hamme, Comparison of self-supervised speech pre-training methods on Flemish Dutch, in: Proc. IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2021, pp. 169–176. doi:10.1109/ASRU51503. 2021.9688061

    2021

  67. [81]

    D. S. Park, et al., SpecAugment: A simple data augmentation method for au- tomatic speech recognition, in: Proc. Interspeech, 2019, pp. 2613–2617. doi: 10.21437/Interspeech.2019-2680

    2019 doi

  68. [82]

    Watanabe, et al., ESPnet: End-to-end speech processing toolkit, in: Proc

    S. Watanabe, et al., ESPnet: End-to-end speech processing toolkit, in: Proc. Interspeech, 2018, pp. 2207–2211. doi:10.21437/Interspeech.2018-1456

    2018 doi

  69. [83]

    D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, in: Proc. Int. Conf. on Learning Representations (ICLR), 2015. URL http://arxiv.org/abs/1412.6980

    2015 arXiv

  70. [84]

    Papineni, S

    K. Papineni, S. Roukos, T. Ward, W.-J. Zhu, BLEU: a method for automatic eval- uation of machine translation, in: Proc. 40th Annual Meeting of the Association for Computational Linguistics, 2002, p. 311–318. doi:10.3115/1073083.1073135. URL https://doi.org/10.3115/1073083.1073135

    2002

  71. [85]

    Post, A call for clarity in reporting BLEU scores, in: Proc

    M. Post, A call for clarity in reporting BLEU scores, in: Proc. Conf. on Machine Translation (WMT), ACL, 2018, pp. 186–191. URL https://www.aclweb.org/anthology/W18-6319

    2018

  72. [86]

    Pallet, W

    D. Pallet, W. Fisher, J. Fiscus, Tools for the analysis of benchmark speech recog- nition tests, in: Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 1990, pp. 97–100. doi:10.1109/ICASSP.1990.115546

    1990

  73. [87]

    Koehn, Statistical significance tests for machine translation evaluation, in: Proc

    P. Koehn, Statistical significance tests for machine translation evaluation, in: Proc. Conf. on Empirical Methods in Natural Language Processing, ACL, 2004, pp. 388–395. URL https://aclanthology.org/W04-3250 32

    2004

  74. [88]

    Vandekerckhove, A

    R. Vandekerckhove, A. De Houwer, A. Remael, Between language policy and linguistic reality: Intralingual subtitling on Flemish television, Pragmatics 19 (4) (2009) 609–628. doi:https://doi.org/10.1075/prag.19.4.05van. URL https://www.jbe-platform.com/content/journals/10.1075/p...

    2009 doi

  75. [89]

    Remael, A

    A. Remael, A. De Houwer, R. Vandekerckhove, Intralingual open subtitling in Flanders: audiovisual translation, linguistic variation and audience needs, Journal of Specialised Translation (JoSTrans) 10 (2008) 76–105. URL https://repository.uantwerpen.be/desktop/irua

    2008

  76. [90]

    Prieels, G

    L. Prieels, G. De Sutter, A mixed-method approach to the use of Colloquial Belgian Dutch in intralingual subtitling on Flemish television, Taal en Tongval 70 (2) (2018) 211–256. doi:https://doi.org/10.5117/TET2018.2.PRIE. URL https://www.aup-online.com/content/journals/10.5117...

    2018 doi

  77. [91]

    Vandekerckhove, Dialect loss and dialect vitality in Flanders, Int

    R. Vandekerckhove, Dialect loss and dialect vitality in Flanders, Int. Journal of the Sociology of Language 2009 (196-197) (2009) 73–97. doi:doi:10.1515/IJSL. 2009.017. URL https://doi.org/10.1515/IJSL.2009.017

    2009 doi

  78. [92]

    Vandekerckhove, ’Tussentaal’ as a source of change from below in Belgian Dutch

    R. Vandekerckhove, ’Tussentaal’ as a source of change from below in Belgian Dutch. a case study of substandardization processes in the chat language of Flem- ish teenagers, in: Germanic Language Histories ’from Below’ (1700-2000), De Gruyter, 2007, pp. 189–204. doi:doi:10.1515...

    doi

  79. [93]

    L. E. Shafey, H. Soltau, I. Shafran, Joint speech recognition and speaker di- arization via sequence transduction, in: Proc. Interspeech, 2019, pp. 396–400. doi:10.21437/Interspeech.2019-1943

    2019 doi

  80. [94]

    Kanda, et al., Streaming multi-talker ASR with token-level serialized out- put training, in: Proc

    N. Kanda, et al., Streaming multi-talker ASR with token-level serialized out- put training, in: Proc. Interspeech, 2022, pp. 3774–3778. doi:10.21437/ Interspeech.2022-7

    2022

  81. [95]

    J. Liu, C. Deng, Q. Zhang, Q. Chen, H. Yu, W. Wang, Recording for eyes, not echoing to ears: Contextualized spoken-to-written conversion of ASR transcripts (2024). arXiv:2408.09688. URL https://arxiv.org/abs/2408.09688

    2024 arXiv

  82. [96]

    Dubey, et al., The Llama 3 herd of models (2024)

    A. Dubey, et al., The Llama 3 herd of models (2024). arXiv:2407.21783. URL https://arxiv.org/abs/2407.21783

    2024 arXiv

  83. [97]

    ge”, common in spoken lan- guage, is translated to “je

    A. Q. Jiang, et al., Mistral 7b (2023). arXiv:2310.06825. URL https://arxiv.org/abs/2310.06825 33 Appendix A. Dual Outputs: V erbatim and Subtitle Annotations Appendix A.1. Examples of differences between verbatim and subtitle transcripts Section 5.2.2 has explained the genera...

    2023 arXiv

  84. [2023]

    doi:10.1109/ICASSP49357.2023.10094620

    2023

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.