{"id":"1047cfdb-c9c7-4020-9b2f-363c35635bda","arxiv_id":"2502.03212","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A cascaded dual-task ASR model trained jointly on exact transcripts and noisy TV subtitles cuts Flemish Dutch verbatim WER by up to half and produces standard-written subtitles without preprocessing or parallel data.","lead":"Using Flemish TV subtitles as weak training labels, this paper shows that an ASR model with cascaded encoders and separate decoders improves verbatim speech recognition by up to 50 percent relative WER and simultaneously generates clean subtitles, all without alignment or filtering of the subtitle data. The result matters for low-resource languages where exact transcripts are scarce but subtitled broadcasts are abundant.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline 'up to 50% relative WER reduction' is not supported by same-baseline, same-test-set comparisons in Figure 4 or Table 6; the largest supported reduction is about 41%.","rationale":"The paper's central empirical contribution is that weakly supervised subtitles improve verbatim WER. The reader's chosen weakest assumption (cross-task transfer generalizes poorly outside the broadcast domain) is a legitimate scope concern, but it does not threaten the reported Flemish results, which are domain-specific evidence. The more load-bearing issue is that the headline magnitude is overstated: no same-test-set comparison in the paper reaches a 50% relative WER reduction. The largest supported value is roughly 41% (subs-annot, XL model versus the long-form CGN-only baseline), and the long-form test sets give about 39%. This does not overturn the qualitative finding, which is supported by controlled architecture comparisons on fixed subtitle data, open-source code and released models, and significance testing. It does, however, change the strength of the central claim as advertised. The manuscript should either report the correct maximum or explicitly define the baseline as the short-form CGN-only model, in which case the 'unseen scores' phrasing is misleading because that baseline was not trained for long-form evaluation. The proposed recomputation from the published numbers settles the issue immediately, so the existing conditional verdict is retained without escalation.","tokens_in":26975,"tokens_out":13975,"duration_ms":117747,"concrete_test":"Recompute every relative WER reduction in Section 5.3.2 and Table 6 using (baseline - proposed)/baseline on the same test set, with the long-form CGN-only baseline (10.61 on cgn-dev-long, 14.17 on subs-annot-long). If no same-test-set comparison reaches 50%, revise the abstract, Discussion, and strongest claim to the actual maximum (~39-41%) and state the baseline unambiguously. Independently, run the released XL checkpoint on cgn-dev-long and subs-annot-long to confirm the 6.49 and 8.63 WERs and rule out an evaluation-pipeline discrepancy.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5.3.2 and the Discussion claim 'up to 50% relative WER reduction compared to the baseline ASR model'. Against the long-form CGN-only baseline reported in the same experiments (Figure 4 'No subs'; Table 6: 10.61% on cgn-dev-long, 14.17% on subs-annot-long), the best XL results are 6.49% and 8.63%, which are 38.8% and 39.1% relative reductions, not ~50%. The only route to ~50% is to compare against the short-form CGN-only model evaluated on long-form test sets (15.12/16.85 in Table 4, giving 57%/49%), a mismatched baseline that was not trained on long-form data, or to compare the cgn-dev baseline with the proposed cgn-dev-long result. The qualitative conclusion that subtitles improve verbatim ASR is well supported by controlled comparisons (e.g., Table 1, where cascaded models beat parallel and naive models on the same subtitle data), but the flagship quantitative claim is not supported as stated and should be corrected or explicitly re-anchored to the short-form baseline.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies how weakly supervised TV subtitle transcripts can be combined with a small verbatim ASR corpus to improve both verbatim speech recognition and automatic subtitling for Flemish Dutch. The authors propose several encoder-decoder architectures, ranging from a naive mixed-training model and a shared task-decoder model to parallel and cascaded models with separate decoders and an additional subtitle encoder. The central empirical finding is that cascaded models, especially the cascaded encoder model with dual encoder features, outperform the CGN-only baseline and the alternative multitask architectures on verbatim WER while also producing competitive subtitle BLEU scores. The paper includes ablations on the Multi-Transformer decoder, a data-filtering study, long-form serialised output training, scaling experiments up to 14k hours of subtitles, and comparisons with Whisper and an ASR-plus-LLM subtitling pipeline.","tokens_in":27264,"tokens_out":4842,"duration_ms":44661,"significance":"If the results hold, this is a useful contribution to weakly supervised and low-resource ASR. The experimental methodology is generally careful: evaluation uses multiple held-out test sets (cgn-dev, subs-annot, subs-valid, subs-valid-14kh), statistical significance is assessed with MAPSSWE and bootstrap resampling, and the main trend is consistent across data sizes and model variants. The paper also ships open-source code and models, which is a concrete strength. The central claim, that explicitly separating verbatim and subtitle targets through cascaded encoders improves verbatim ASR, is well supported by the controlled comparisons in Tables 1 and 4. The main weakness is an overstated headline number in the scaling section, together with a few presentation issues and one unvalidated design choice in the cascaded-decoder variant.","major_comments":[{"comment":"The claim of 'up to 50% relative WER reduction compared to the baseline ASR model' is not supported by the same-baseline comparisons reported in Figure 4 and Table 6. Against the long-form CGN-only baseline, the best XL results give a relative reduction of (10.61-6.49)/10.61 = 38.8% on cgn-dev-long and (14.17-8.63)/14.17 = 39.1% on subs-annot-long. The ~50% figure only appears when the comparison is made against the short-form CGN-only model evaluated on long-form test sets (15.12% and 16.85% in Table 4), which is not the appropriate baseline for the scaling experiment. The sentence in Section 5.3.2 and the corresponding sentence in Section 6 should be corrected to the actually supported values, or explicitly re-anchored to the short-form baseline with a clear caveat.","section":"5.3.2 and Section 6"}],"minor_comments":[{"comment":"The cascaded-decoder variant relies on forwarding an <unk> token through the ASR decoder to obtain a sentence embedding for subtitle data. The paper states this 'improves the optimisation' and the footnote acknowledges a training/testing mismatch, but no quantitative ablation is provided. Since this design choice affects only one of the proposed variants and the cascaded-encoder model performs similarly, this does not threaten the main conclusion, but it should be presented as a limitation or backed by an ablation.","section":"3.3.3 and footnote 5"},{"comment":"The first row of Table 3 is labelled 'Naive E2E ASR' with 0 hours of subtitles and shows the CGN-only baseline WERs; this row should be relabelled 'E2E ASR - CGN only' to avoid confusion with the naive model trained on subtitle data.","section":"Table 3"},{"comment":"The caption says 'Figure (a) shows WERs' and 'Figure (b) shows BLEU scores', but the three panels are (a) WER on cgn-dev, (b) WER on subs-annot, and (c) BLEU; the caption text should be corrected.","section":"Figure 4 caption"},{"comment":"The x-axis labels 'Enc.' and 'Dec.' are not defined in the caption; add a sentence explaining that they refer to whether the subtitle encoder is conditioned on the ASR encoder outputs or on the ASR decoder features.","section":"Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The paper's central empirical claim is sound and the evaluation is a cut above the typical standard for this area. The 'up to 50%' statement is an overstatement that should be corrected before acceptance; the correction is straightforward and does not undermine the main findings. The open-sourcing of code and models is a notable strength. The subtitle data itself cannot be released, so replication on the largest scale will be limited to the authors' resources, but the smaller-scale experiments are reproducible from the released artefacts."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read the Flemish subtitle-ASR paper. The architecture is the real contribution: a cascaded subtitle encoder attached to the ASR encoder, with Multi-Transformer decoders attending to both encoder outputs. This is a clean way to let weakly supervised subtitle data improve verbatim ASR without forced alignment or filtering, and the experiments show it works. On controlled comparisons (Tables 1 and 4), cascaded models consistently beat the parallel-decoder predecessor [29], the shared-task decoder, and the naive subtitle-as-verbatim baseline, across data scales and test sets, with significance tests. That qualitative claim is solid.\n\nThe paper also does several things right: multiple held-out test sets (cgn-dev, subs-annot, plus long-form variants), statistical significance testing, scaling experiments from 720h to 14kh, and released code and models. The linguistic analysis of tussentaal versus standard Dutch subtitles is a nice touch, and the appendix comparison of ASR+LLM subtitling pipelines adds perspective.\n\nSoft spots. The flagship \"up to 50% relative WER reduction\" does not survive contact with the paper's own tables. Against the same long-form baseline (10.61 on cgn-dev-long, 14.17 on subs-annot-long), the best XL numbers (6.49, 8.63) are about 39% relative reductions. The ~50% figure only appears when comparing against a short-form CGN-only model evaluated on long-form test sets, which is a mismatched baseline. The qualitative conclusion holds, but Section 5.3.2 and the Discussion need to be re-anchored to the correct baseline.\n\nSecond, subtitle quality is evaluated with BLEU only. BLEU is a weak proxy for usable subtitles; no segmentation, no human judgment, no SubER. The authors acknowledge this indirectly, but should add at least one more metric or a broader qualitative analysis beyond the appendix examples.\n\nThird, the training data (VRT subtitles) is proprietary, so exact reproduction is impossible. The released models and code help, but the 14kh scaling results are not independently checkable. That is a real limitation for a paper whose main empirical claim concerns data scaling.\n\nFourth, the generalization worry is real but not damning: all experiments are Flemish broadcast speech. The mechanism could plausibly transfer to other languages with similar subtitle norms, but we do not know that yet. I would flag it as a scope limitation rather than a flaw.\n\nBottom line: a solid, citable contribution to low-resource ASR and automatic subtitling. The headline number needs fixing, and BLEU-only subtitle evaluation is a weakness, but the core result is well supported. Send it to review; it needs revision, not rejection.","headline":"Solid architecture for using TV subtitles to improve verbatim ASR, but the flagship '50% WER reduction' overstates what the same-baseline numbers support.","tokens_in":27743,"tokens_out":3242,"would_cite":true,"duration_ms":26663,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TV subtitle transcripts, though inexact and misaligned, can be used as weakly supervised training data that improves both verbatim speech recognition and automatic subtitle generation in a single end-to-end model.","keywords":["Automatic Speech Recognition","Weak Supervision","End-to-End Modelling","Subtitles","Broadcast Media Data","Flemish Dutch","Cascaded Encoders","Multitask Learning"],"falsifier":"Randomly permute the audio–subtitle pairing in the 14,000-hour training set so each subtitle belongs to a different utterance, train the same cascaded dual-feature model, and check verbatim WER on the long-form test sets; if WER still drops by the same margin, the subtitle content is not the cause of the reported improvement.","tokens_in":26780,"feed_emoji":"📺","tokens_out":9886,"duration_ms":81328,"temperature":0.7,"pith_summary":"Many languages have abundant broadcast television with manually written intralingual subtitles, but subtitles are not exact transcripts: subtitlers shorten, rephrase, and normalise dialect into standard written language, and their timestamps are often misaligned. This paper tries to use such weakly supervised subtitle data anyway, in combination with a small manually transcribed verbatim corpus, to improve both exact speech-to-text and automatic subtitling for Flemish Dutch. The key move is to treat verbatim transcripts and subtitles as two different output languages and to train a single end-to-end model with a shared acoustic encoder, a small cascade encoder that translates acoustic features into subtitle features, and two separate decoders. On 14,000 hours of subtitles the model reports large verbatim error reductions, from 10.61% to 6.49% WER on one long-form test set and from 14.17% to 8.63% on another, while simultaneously producing subtitle text with a BLEU score of 61.76. If this transfers to other medium-resource languages, subtitle archives could become a practical substitute for expensive manual transcription.","feed_headline":"TV subtitles train a model that beats Whisper on Flemish ASR","feed_subtitle":"Cascaded dual decoder uses 14,000 hours of weakly labelled subtitles to cut verbatim word error and generate subtitles.","key_machinery":"The load-bearing mechanism is the cascaded dual-feature architecture. A 12-layer Conformer acoustic encoder produces verbatim-oriented features and is regularised with CTC on the verbatim data; a smaller Transformer subtitle encoder is stacked on its outputs and trained, through the subtitle decoder's loss, to rewrite those features into standard written subtitle text. Each decoder is a Multi-Transformer decoder with two cross-attention layers per block, one attending to the ASR encoder and one to the subtitle encoder, so the verbatim decoder can also exploit the subtitle representation and the subtitle decoder can retain fine-grained temporal information. The joint loss mixes verbatim ASR loss and subtitle cross-entropy, with optional subtitle CTC, and batches balance verbatim and subtitle utterances. This design is what lets gradients from subtitle-only data flow back into the shared acoustic encoder without corrupting the verbatim CTC path.","core_discovery":"The paper's central claim is that a joint model with cascaded encoders and separate decoders can learn from both domains without parallel data, and that the subtitle stream actively improves the verbatim branch. The authors argue that naively mixing subtitles into verbatim training is harmful, but that a dedicated subtitle encoder cascaded onto the ASR encoder absorbs the differences between spoken and written language, so the shared ASR encoder can be optimised for both objectives. The best variant, cascaded encoder with dual features, conditions both decoders on both encoder outputs through double cross-attention. The reported result is that, compared with a strong Conformer-based baseline trained only on the verbatim corpus, adding weakly supervised subtitles yields large relative WER improvements on long-form test sets in the large-scale experiment, with the paper describing up to 50% relative reduction and concrete long-form numbers going from 10.61% to 6.49% and from 14.17% to 8.63%. The same model produces subtitles that score 61.76 BLEU on a held-out 14,000-hour sample.","pith_inferences":["Editorial inference: the architecture is effectively learning an implicit spoken-to-standard-writing translation inside the subtitle encoder; in principle this could replace post-hoc inverse text normalisation and disfluency-removal pipelines for clean-text generation.","Editorial inference: the method's value likely depends on how strongly a language's subtitling conventions rewrite speech; with nearly verbatim subtitles the cascade may have less to learn, while with heavily rephrased subtitles the gains may be larger but the subtitle decoder's output will be farther from a transcript.","Editorial inference: a direct testable extension is to apply the identical recipe to another low-resource or medium-resource language with a large intralingual subtitle archive; if verbatim WER gains do not appear there, the Flemish result may be specific to the large spoken-written gap in Belgian Dutch.","Editorial inference: because the subtitle data is not released, reproducing the exact numbers requires access to similar broadcaster archives; open subtitle collections may offer a weaker but more portable test of the same idea."],"forward_implications":["If the approach holds, medium-resource languages with broadcast subtitle archives can train substantially better verbatim ASR without needing more manually transcribed speech.","The same model outputs a subtitle in standard written language alongside the verbatim transcript, so automatic subtitling and clean-text NLP inputs come from the same forward pass.","Because the method does not require forced alignment, filtering, pseudo-labelling, or iterative refinement of the weakly labelled data, it is relatively cheap to scale with broadcast archives.","Increasing the subtitle corpus from 400 to 14,000 hours keeps reducing WER and raising subtitle BLEU, suggesting the model will continue to benefit from even larger archives.","A 70M or 180M parameter model beats a 1.55B-parameter Whisper model and a Whisper model fine-tuned on verbatim data on Flemish verbatim WER, indicating that subtitle-trained models can compete with far larger systems."],"supporting_citations":[{"why":"Supplies the prior parallel-decoder baseline and the 720-hour subtitle setup that this work extends.","marker":"[29]"},{"why":"Provides the task-token conditioning idea used in the shared-task-decoder variant.","marker":"[30]"},{"why":"Defines the large-scale weakly supervised speech model used as the main external comparison.","marker":"[20]"},{"why":"Motivates serialised output training for long-form utterances and speaker-change tokens.","marker":"[31]"},{"why":"Provides the CTC objective used to regularise the ASR encoder on verbatim data.","marker":"[68]"},{"why":"Supplies the hybrid CTC/attention loss and joint decoding approach used throughout.","marker":"[70]"},{"why":"Provides the multi-sequence attention idea behind the Multi-Transformer decoder.","marker":"[74]"},{"why":"Supplies the manually transcribed Flemish Dutch verbatim corpus with rich tags used for training and evaluation.","marker":"[78]"},{"why":"Defines the BLEU metric used to score subtitle output quality.","marker":"[84]"}],"fun_headline_variants":["Subtitles slash Flemish ASR errors by 50% via cascaded encoders","14k subtitle hours halve Flemish WER, beat Whisper","TV captions train a dual decoder that cuts Flemish ASR errors","Weakly labelled subtitles boost Flemish ASR by half","Cascaded encoders turn subtitles into Flemish ASR gold"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The approach rests on the assumption that one acoustic encoder can learn a feature space that simultaneously supports exact verbatim decoding and a small subtitle encoder that translates those features into standard written subtitles, using no parallel verbatim-subtitle utterances; if that cross-task transfer is specific to Flemish broadcast speech, the gains will not generalise.","fun_headline_variants_meta":{"raw":{"variants":["Subtitles slash Flemish ASR errors by 50% via cascaded encoders","14k subtitle hours halve Flemish WER, beat Whisper","TV captions train a dual decoder that cuts Flemish ASR errors","Weakly labelled subtitles boost Flemish ASR by half","Cascaded encoders turn subtitles into Flemish ASR gold"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000231,"raw_usage":{"total_tokens":1514,"prompt_tokens":1000,"completion_tokens":514,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":616,"completion_tokens_details":{"reasoning_tokens":417}},"tokens_in":616,"tokens_out":514,"duration_ms":4816,"temperature":1.0,"reasoning_tokens":417,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T05:29:40.971434+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Randomly permute the audio–subtitle pairing in the 14,000-hour training set so each subtitle belongs to a different utterance, train the same cascaded dual-feature model, and check verbatim WER on the long-form test sets; if WER still drops by the same margin, the subtitle content is not the cause of the reported improvement.","supporting_citations":[{"cited_title":"Poncelet, H","cited_arxiv_id":null,"evidence_quote":"Supplies the prior parallel-decoder baseline and the 720-hour subtitle setup that this work extends."},{"cited_title":"Ihori, H","cited_arxiv_id":null,"evidence_quote":"Provides the task-token conditioning idea used in the shared-task-decoder variant."},{"cited_title":"Kanda, Y","cited_arxiv_id":null,"evidence_quote":"Motivates serialised output training for long-form utterances and speaker-change tokens."},{"cited_title":"Graves, S","cited_arxiv_id":null,"evidence_quote":"Provides the CTC objective used to regularise the ASR encoder on verbatim data."},{"cited_title":"Watanabe, T","cited_arxiv_id":null,"evidence_quote":"Supplies the hybrid CTC/attention loss and joint decoding approach used throughout."},{"cited_title":"Oostdijk, The Spoken Dutch Corpus: Overview and first evaluation, in: Proc","cited_arxiv_id":null,"evidence_quote":"Supplies the manually transcribed Flemish Dutch verbatim corpus with rich tags used for training and evaluation."},{"cited_title":"Papineni, S","cited_arxiv_id":null,"evidence_quote":"Defines the BLEU metric used to score subtitle output quality."}],"review_version":1}