{"id":"a4f972be-9c38-4e14-b213-0642ca53b369","arxiv_id":"2608.12549","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A new benchmark for full-video audio description generation and a streaming fine-tuned baseline that sets a new state of the art on the CMD-AD clip benchmark with 36.3 CIDEr.","lead":"The paper introduces StrAD, a benchmark of full-length videos with human-reviewed audio descriptions, and a streaming method that writes those descriptions on the fly without pre-set timestamps. It reports state-of-the-art results on the existing clip-level benchmark and shows the new full-video task is still far from solved.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1 line 8's cursor update contradicts Section 4.3 and, if implemented literally, would make the reported streaming results impossible.","rationale":"The reader's weakest assumption concerns the accuracy of StrAD's ground-truth annotations (single-annotator correction, no inter-annotator agreement). That is a legitimate benchmark-quality concern and warrants the reader's conditional verdict. However, I find a more immediately load-bearing issue in the method itself: Algorithm 1's cursor update contradicts the prose in Section 4.3. If the printed pseudocode were followed, the streaming loop would jump far ahead after each emitted AD, making the reported Table 2 metrics (Recall 59.5, SODA 2.4) unattainable. Since the prose and results are mutually consistent, the most likely explanation is a typographical error in the pseudocode, but this must be resolved before the streaming claims can be trusted. The concrete test (checking released code and rerunning streaming evaluation) settles the matter quickly. This does not change the verdict category: CONDITIONAL acceptance is still appropriate, with the added condition that the algorithm be corrected and verified. I therefore leave the reader's verdict unchanged while flagging a distinct, more technical concern.","tokens_in":26612,"tokens_out":7738,"duration_ms":71276,"concrete_test":"Inspect the released streaming-inference code (or obtain the exact update from the authors) and determine whether the cursor is updated to max(t, prediction.t_e) or to max(t, t−wv+prediction.t_e). If the former, rerun the streaming evaluation on StrAD and confirm that Table 2 values reproduce; if the latter, rerun the same evaluation and show that SODA, Recall, and CIDEr collapse because the cursor skips most timestamps. Update Algorithm 1 to match the verified implementation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The streaming method's core loop is defined in Algorithm 1. Line 8 updates the cursor as t←max(t, t−wv+prediction.t_e), but Section 4.3 states the intended rule: 'if the predicted end time t_e exceeds the current t, we set t=t_e.' These disagree. Under the pseudocode, after emitting an AD ending at t_e≈t+2s with wv=8s, the cursor jumps from t to t−8+t_e ≈ 2t+2 (e.g., from t=100 to ≈194), skipping roughly half the remaining video. Over StrAD's 22.3 hours, this would yield only a handful of ADs per video, so the reported Recall of 59.5 and SODA of 2.4 in Table 2 become numerically impossible. The prose rule is consistent with the results and likely reflects the actual implementation, but the printed algorithm is not. This is an internal inconsistency in the central technical contribution: a reader cannot tell from the paper whether the streaming loop advances correctly or skips most of the video. The pseudocode must be corrected, and the released code must be checked against the stated rule.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces StrAD, a benchmark for full-video audio description (AD) generation spanning diverse genres and long-form videos, along with three tasks: segment-level, streaming, and document-level. The authors propose two baselines: StrAD-Zero, a zero-shot pipeline extending AutoAD-Zero, and StrAD-FT, a fine-tuned multimodal model trained on CMD-AD. On segment-level benchmarks, StrAD-FT (Qwen-3.5) reports state-of-the-art CIDEr on CMD-AD (36.3) and strong results on StrAD (51.0), while remaining competitive on MAD-Eval (24.9). On the streaming task, StrAD-FT outperforms StrAD-Zero in SODA, CIDEr, localization, and overlap metrics, with real-time factor below 1 and fewer than 5B parameters. The paper also provides detailed reproducibility analyses, including validation against published predictions on an incomplete CMD-AD test set and a unified evaluation suite.","tokens_in":26804,"tokens_out":3703,"duration_ms":32334,"significance":"If the reported results hold, this is a valuable contribution: it is the first streaming, end-to-end AD generation method for full-length videos and the first benchmark of that kind with full-length, manually verified annotations. The paper strengthens credibility through careful metric validation: it checks its evaluation implementation against published predictions (Table 5), quantifies the effect of the incomplete CMD-AD test set, and reports a diagnostic breakdown (REP, OL, RTF) that honestly exposes remaining limitations. The segment-level SOTA on CMD-AD is supported by consistent CIDEr numbers and careful comparisons. However, the central streaming contribution is undermined by an internal inconsistency in Algorithm 1, and the benchmark's annotation reliability lacks inter-annotator agreement evidence.","major_comments":[{"comment":"Algorithm 1's cursor update, t←max(t, t−wv+prediction.t_e), directly contradicts the prose rule in Section 4.3: \"if the predicted end time t_e exceeds the current t, we set t=t_e.\" Under the pseudocode, after t is incremented by δ and an AD with t_e≈t+2s is emitted, the cursor jumps to roughly 2t (e.g., from t=100 to ≈194) because wv=8s, skipping most of the video. This would make the reported streaming Recall of 59.5 and SODA of 2.4 in Table 2 numerically impossible. The prose rule is consistent with the reported results and likely reflects the actual implementation, but the printed algorithm is not. The pseudocode must be corrected, and the released code must be verified against the stated rule. This is load-bearing because the streaming method is the paper's primary contribution.","section":"Algorithm 1, line 8 vs. Section 4.3"},{"comment":"The benchmark's ground truth is produced by automated Whisper/Silero/pyannote extraction followed by manual correction, but each video is reviewed by only one annotator with spot-checking by a second annotator, and no inter-annotator agreement is reported. Since all StrAD reference numbers and the streaming conclusions depend on this ground truth, systematic annotation errors cannot be ruled out. The authors should report IAA on a subset of videos (e.g., AD event boundary agreement and transcript accuracy) or otherwise validate the reliability of the manual correction step.","section":"Section A (StrAD annotation process)"}],"minor_comments":[{"comment":"The sentence \"Although this model achieves state-of-the-art results for CMD-AD and MAD-Eval\" is not accurate: the random-padding variant reaches 31.1 CIDEr on CMD-AD, which is below the reported StrAD-FT (Qwen-3.5) score of 36.3; it is state-of-the-art only on MAD-Eval (31.9 CIDEr).","section":"Section 5.2, random-padding variant"},{"comment":"There is a grammatical error: \"The main contributions of this paper is the first end-to-end streaming...\" should be \"are\" or the sentence should be restructured.","section":"Abstract/Introduction"},{"comment":"Several rows in Table 1, such as the AutoAD-II row showing \"2.08 | 19.5 51.3\", lack clear column separators for the LLM-AD-Eval and MAD-Eval columns, making the table hard to read; the table should use explicit column boundaries or be split.","section":"Table 1 formatting"},{"comment":"ActionScore is listed as a streaming quality metric, but it is not reported anywhere in the streaming evaluation (Table 2); the authors should either report it or clarify why it is omitted from the streaming results.","section":"Section D.2.1"}],"recommendation":"major_revision","confidential_remarks":"The Algorithm 1 inconsistency is severe and must be fixed before publication; it undermines the main technical contribution. Once corrected, the paper would be a strong candidate for acceptance, assuming the benchmark annotation reliability is also addressed. The authors should double-check the released code to ensure the actual cursor update matches Section 4.3."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read StrAD with some care. The short version: this is a genuinely useful benchmark paper, and the streaming-task framing is the right push for the field, but there is a printed pseudocode bug in the central algorithm that needs to be fixed before anyone relies on the method description.\n\nWhat is actually new: StrAD is the first AD benchmark made of full-length videos across genres (movies, docs, games, etc.) with three tasks: segment-level, streaming, document-level. The streaming formulation\\u2014sliding window, WAIT decisions, no ground-truth boundaries\\u2014is new for AD. The fine-tuned baselines are small (<5B) and faster than real-time; Qwen-3.5 StrAD-FT sets a strong CMD-AD CIDEr (36.3) and reasonable MAD-Eval results. The metric work is careful: they validate their implementations against published predictions, check the impact of the incomplete CMD-AD test set, and report the weak SODA numbers rather than hiding them. That is good practice.\n\nThe soft spots, in proportion. The biggest is an internal inconsistency: Algorithm 1 line 8 updates the cursor with t\\u2190max(t, t\\u2212w_v+prediction.t_e), while Section 4.3 and the reported results imply the intended rule t\\u2190max(t, prediction.t_e). If the pseudocode is read literally, the cursor jumps past most of the video and the Table 2 numbers become impossible. This is almost certainly a typo in the algorithm block, but it is a load-bearing one\\u2014readers cannot tell how the streaming loop actually advances. The released code needs to be checked against the prose rule and the pseudocode corrected.\n\nSecond, the StrAD ground truth is built with a single annotator per video plus spot-checking, with no inter-annotator agreement reported. For a benchmark meant to be a shared testbed, that is a real gap. The authors should either run a second full pass or report IAA on at least a subset.\n\nThird, the claim that StrAD-FT \\\"generates coherent full-video AD\\\" goes beyond what SODA 2.4 supports. The paper does acknowledge the limitations, but the contribution bullet is stronger than the evidence. A user study or tightened wording would fix it.\n\nMinor: benchmark depends on YouTube availability (they disclose this and provide alignment strategies, which is reasonable), the \\\"first\\\" claims are fine given the cited prior work, and the per-genre tables are useful.\n\nWho this is for: anyone working on audio description, dense video captioning, or long-video understanding. It deserves a real peer review; the benchmark and the careful evaluation are worth referee time. I would recommend acceptance if the pseudocode bug is fixed, the code/models/benchmark are released with a persistent identifier, and the annotation reliability is addressed.","headline":"Genuinely useful benchmark and a sensible streaming reformulation, but fix the Algorithm 1 pseudocode bug and tighten the annotation-reliability story before building on it.","tokens_in":27377,"tokens_out":2261,"would_cite":true,"duration_ms":18078,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes the first end-to-end streaming approach to audio description generation for full-length videos, together with a benchmark of 33 manually verified videos across five genres.","keywords":["audio description generation","streaming dense video captioning","long-form video understanding","vision-language models","multimodal LLM","temporal localization","accessibility","benchmark"],"falsifier":"Have two professional describers independently annotate a random subset of StrAD videos and measure agreement on AD event boundaries and wording (e.g., tIoU of intervals and BERTScore/CIDEr between their annotations). If agreement is low, or if the benchmark's pronoun heuristic is shown to misclassify third-person narration in documentaries as dialogue, then the reference numbers and the claimed streaming improvement would not be trustworthy.","tokens_in":26371,"feed_emoji":"🎬","tokens_out":11790,"duration_ms":88889,"temperature":0.7,"pith_summary":"The paper argues that audio description (AD) generation — the narration that makes video accessible to blind and low-vision viewers — should be reframed as a streaming task on full-length videos, not as clip captioning with pre-given timestamps. To make that reframing testable, it introduces StrAD, a benchmark of 33 full-length videos (22.3 hours) spanning movies, documentaries, short films, video games, and performances, with manually verified AD transcripts and timestamps. It also presents StrAD-FT, a fine-tuned multimodal model that sweeps a sliding window over the video and inserts timestamped AD sentences into speech gaps without any ground-truth boundaries. On segment-level benchmarks the paper reports 36.3 CIDEr on CMD-AD (a 10.0-point gain over the previous best training-free method), 51.0 CIDEr on StrAD, and 24.9 CIDEr on MAD-Eval; on the streaming task it reports 2.4 SODA versus 1.1 for a zero-shot baseline, at faster-than-real-time speed with under 5 billion parameters. The purpose of the benchmark is to make full-video AD generation measurable, so that accessibility tooling can scale beyond the small fraction of content that is currently described by hand.","feed_headline":"Streaming model writes audio descriptions for full videos","feed_subtitle":"StrAD-FT hits 36.3 CIDEr on CMD-AD and runs faster than real time.","key_machinery":"The load-bearing mechanism is the sliding-window streaming loop defined in Algorithm 1. A cursor advances through the video in four-second strides; whenever the transcript contains a speech gap of at least half a second, the model receives the preceding eight seconds of video plus the last 120 seconds of transcript and generated ADs, and either emits an AD with a start and end timestamp or returns a special WAIT token. Two design choices make this viable in practice: streaming training samples are created by moving a window with random stride over each video and labeling a sample WAIT when the window contains only dialogue, no AD, or a partial AD event; and 25 percent of the training budget is deliberately allocated to WAIT signals so the model learns when not to speak. The fine-tuned models are Phi-4-mm and Qwen-3.5, both under 5 billion parameters, adapted with LoRA on the language layers and full training of the vision and audio projection layers.","core_discovery":"The central claim is that AD generation can be executed as streaming dense video captioning: rather than receiving a clip boundary and producing one sentence, the model receives the video, its transcript, and previously generated ADs through a sliding window, and must decide both when to speak and what to say. The paper's fine-tuned model StrAD-FT is trained on this formulation using only CMD-AD data and, the authors report, matches or exceeds prior methods on the standard segment-level benchmarks while also producing coherent full-video AD without ground-truth timestamps. A zero-shot companion, StrAD-Zero, shows the same streaming interface works with prompted vision-language models. Ablations in the paper support a specific division of labor: video is the dominant signal for what to describe, while audio mainly tells the model when to narrate, reducing overlap with speech and cutting repetition. Together with the StrAD benchmark, these results are offered as the first end-to-end streaming solution to full-video AD generation.","pith_inferences":["Because the paper reports no inter-annotator agreement on StrAD, a natural next check is an independent re-annotation study on a random subset; if agreement is low, the benchmark reference scores and the streaming comparison would need to be re-interpreted.","The streaming formulation could transfer to live events and interactive media such as video games, where transcripts arrive incrementally; the reported real-time factor below 0.6 on a single A100 suggests the latency budget is already feasible.","The WAIT decision — learning when speaking would cover dialogue or intentional sound — is a reusable control task that could improve automated dubbing, commentary, or narration systems beyond accessibility.","Per-genre results show large variation (e.g., documentary CIDEr 65.0 vs game 40.3 for the Qwen-3.5 variant), hinting that genre-specific AD conventions such as describing game UI may need dedicated fine-tuning or evaluation."],"forward_implications":["AD generation no longer depends on ground-truth timestamps or character databases: a single streaming pass over a full video can place descriptions into natural speech gaps.","The benchmark gives the field a shared, automatically scored target on full-length videos with manually verified annotations, reducing the need for expensive per-video user studies.","The reported segment-level gains on CMD-AD (+10.0 CIDEr over Shot-by-shot) suggest that a small model trained on the streaming objective transfers well to the standard clip-based evaluation.","Audio input changes streaming behavior measurably: in the ablations, removing audio raises speech overlap from 3.9% to 5.3% and repetition from 11.5% to 16.3%, marking audio as a first-class input for the streaming task.","The large gap between segment-level performance (51.0 CIDEr) and streaming performance (2.4 SODA) defines temporal localization and narrative coherence as the main open problems the benchmark is designed to track."],"supporting_citations":[{"why":"Supplies the CMD-AD training and evaluation data plus the pronoun-based annotation pattern StrAD adapts, and provides the AutoAD-III baseline that StrAD-FT is compared against.","marker":"[14]"},{"why":"StrAD-Zero is built directly on AutoAD-Zero's three-stage prompting pipeline, and AutoAD-Zero serves as a zero-shot comparison baseline.","marker":"[51]"},{"why":"MAD and its MAD-Eval split form the standard segment-level benchmark where StrAD-FT must remain competitive.","marker":"[38]"},{"why":"Whisper is used to transcribe the AD audio track in StrAD's annotation pipeline.","marker":"[32]"},{"why":"Silero VAD detects the speech-free gaps that define candidate AD events for the benchmark.","marker":"[43]"},{"why":"pyannote speaker diarization separates AD narration from dialogue so the pronoun heuristic can label AD sentences.","marker":"[30]"},{"why":"CIDEr is the headline segment-level metric, and all state-of-the-art claims are expressed in its units.","marker":"[45]"},{"why":"SODA is the headline streaming metric used to score the full-video task.","marker":"[10]"},{"why":"Qwen-3.5-4B is the backbone of StrAD-Zero and of the best StrAD-FT variant.","marker":"[31]"},{"why":"Phi-4-mm is the other fine-tuned backbone and the one used in the modality ablations.","marker":"[1]"}],"fun_headline_variants":["Streaming audio descriptions for entire videos","On-the-fly video narration without timestamp cues","First streaming approach to full-video AD generation","StrAD model narrates videos as they play","Full-length video audio descriptions, no clip cuts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that StrAD's ground truth — AD transcripts and timestamps produced by Whisper transcription, Silero VAD, pyannote diarization, and a personal-pronoun heuristic, then corrected by one annotator per video with spot-checking by a second — is accurate enough to serve as an evaluation reference; systematic errors there would shift every StrAD score and the streaming conclusion.","fun_headline_variants_meta":{"raw":{"variants":["Streaming audio descriptions for entire videos","On-the-fly video narration without timestamp cues","First streaming approach to full-video AD generation","StrAD model narrates videos as they play","Full-length video audio descriptions, no clip cuts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000362,"raw_usage":{"total_tokens":2025,"prompt_tokens":1091,"completion_tokens":934,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":707,"completion_tokens_details":{"reasoning_tokens":865}},"tokens_in":707,"tokens_out":934,"duration_ms":8431,"temperature":1.0,"reasoning_tokens":865,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:06:12.732300+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have two professional describers independently annotate a random subset of StrAD videos and measure agreement on AD event boundaries and wording (e.g., tIoU of intervals and BERTScore/CIDEr between their annotations). If agreement is low, or if the benchmark's pronoun heuristic is shown to misclassify third-person narration in documentaries as dialogue, then the reference numbers and the claimed streaming improvement would not be trustworthy.","supporting_citations":[{"cited_title":"AutoAD III: The Prequel - Back to the Pixels","cited_arxiv_id":null,"evidence_quote":"Supplies the CMD-AD training and evaluation data plus the pronoun-based annotation pattern StrAD adapts, and provides the AutoAD-III baseline that StrAD-FT is compared against."},{"cited_title":"AutoAD-Zero: A Training-Free Framework for Zero-Shot Audio Description","cited_arxiv_id":null,"evidence_quote":"StrAD-Zero is built directly on AutoAD-Zero's three-stage prompting pipeline, and AutoAD-Zero serves as a zero-shot comparison baseline."},{"cited_title":"MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptions","cited_arxiv_id":null,"evidence_quote":"MAD and its MAD-Eval split form the standard segment-level benchmark where StrAD-FT must remain competitive."},{"cited_title":"Powerset multi-class cross entropy loss for neural speaker diarization","cited_arxiv_id":null,"evidence_quote":"pyannote speaker diarization separates AD narration from dialogue so the pronoun heuristic can label AD sentences."},{"cited_title":"CIDEr: Consensus-based image description evaluation","cited_arxiv_id":null,"evidence_quote":"CIDEr is the headline segment-level metric, and all state-of-the-art claims are expressed in its units."},{"cited_title":"Soda: Story oriented dense video captioning evaluation framework","cited_arxiv_id":null,"evidence_quote":"SODA is the headline streaming metric used to score the full-video task."}],"review_version":1}