{"id":"e8970cfe-014b-450c-b6c0-650ac6594f32","arxiv_id":"2504.12157","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"FocusedAD uses face tracking, soft prompts, and a video language model to generate character-focused movie audio descriptions, reporting state-of-the-art scores on MAD-eval-Named and the new Cinepile-AD benchmark.","lead":"The paper presents FocusedAD, a system that generates movie audio descriptions (narration for blind and visually impaired audiences) by focusing on main characters and using prior story context. It reports top scores on two benchmarks, including zero-shot results, and introduces a new test set, but the evaluation setup and missing baselines need scrutiny.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Zero-shot SOTA claim depends on an unverified disjointness between Storyboard-v2 training clips and the MAD-eval-Named/Cinepile-AD test movies; the paper reports no overlap check.","rationale":"The reader's weakest assumption identifies exactly the same load-bearing concern: the zero-shot SOTA numbers rest on MAD-eval-Named being independent of the Storyboard-v2 training data. The paper provides no direct evidence for this disjointness, and the data sources make overlap plausible: Storyboard20K mixes MovieNet and LSMDC, while MAD-eval-Named is an LSMDC-derived benchmark. The paper's only independence statement concerns LSMDC train/test splits, not Storyboard20K. This is the single most load-bearing issue because the entire empirical contribution depends on it; the architecture, ablations, and qualitative results are secondary and appear coherent. A missing AutoAD-II baseline and an unreported redundancy threshold are additional weaknesses, but they are less decisive because they affect the breadth of the comparison rather than the validity of the headline numbers. Since the reader already assigned CONDITIONAL, my stress-test does not move the verdict; it strengthens the rationale for conditioning acceptance on a concrete disjointness verification. The proposed check is feasible: source movie IDs can be listed from Storyboard20K and compared with the known MAD-eval-Named movie set, and Cinepile-AD movies can be enumerated from the released YouTube collection. Until such a check is performed, the zero-shot SOTA claim should be treated as unverified rather than accepted at face value.","tokens_in":15329,"tokens_out":4232,"duration_ms":43526,"concrete_test":"Have the authors release the list of source movie IDs for all Storyboard-v2 training clips, or a metadata file from Storyboard20K, and compute the intersection with the 10 MAD-eval-Named movie IDs and the Cinepile-AD movie IDs. If the intersection is empty, re-run Table 2 on the existing checkpoints and confirm the numbers. If the intersection is non-empty, remove all clips from overlapping movies from training and re-run the MAD-eval-Named and Cinepile-AD evaluations; if SPICE/METEOR/BertScore deltas versus the baselines shrink by more than a small margin, the zero-shot SOTA claim in the abstract should be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline result is that FocusedAD achieves zero-shot SOTA on MAD-eval-Named and Cinepile-AD. The load-bearing condition is that the Storyboard-v2 training data is disjoint from the evaluation movies. This condition is never established. Section 4.1 states that Storyboard-v2 is automatically constructed from Storyboard20K, which is sourced from MovieNet and LSMDC. Section 4.2 states that MAD-eval-Named is derived from LSMDC, with 10 movies retained after excluding films in LSMDC training/test splits. But no statement or check is reported that these 10 movies (or the Cinepile-AD movies) are absent from Storyboard20K or from the 11,250 Storyboard-v2 training triplets. Since the pipeline in Section 4.1 extracts complete movie clips from full-length films based on Storyboard20K timestamps, even a single overlapping movie would place training clips in the evaluation set. Because code and data are not yet released, an external reviewer cannot verify the split. If overlap exists, Table 2's SPICE 7.4 / METEOR 8.1 / BertScore 57.7 on MAD-eval-Named would not measure zero-shot generalization, and the central contribution would be substantially weakened. This concern is about internal verifiability, not about the plausibility of the architecture; the ablations in Table 1 are internally coherent, and the qualitative examples are encouraging, but the headline empirical claim cannot be accepted as stated without a demonstrated train/test disjointness.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FocusedAD, a framework for automatic movie audio description (AD) that combines a Character Perception Module (CPM), a Dynamic Prior Module (DPM) with learnable soft prompts, and a Focused Caption Module (FCM) built on a 7B MLLM. The authors also introduce an automated pipeline that constructs a character query bank via clustering and use it to build a training dataset, Storyboard-v2, from Storyboard20K. They evaluate the model zero-shot on MAD-eval-Named and a newly introduced Cinepile-AD test set, reporting state-of-the-art results on SPICE, METEOR, and BertScore, alongside a newly proposed redundancy metric R. The central claim is that a model trained on proxy movie data can outperform dedicated AD systems, including AutoAD-I, MM-Narrator, and general video MLLMs, without training on the target AD datasets.","tokens_in":15617,"tokens_out":2583,"duration_ms":25989,"significance":"If the empirical claims hold, FocusedAD would be a meaningful step toward practical automatic AD, particularly for character-centric narration and for the blind and visually impaired community. The proposed pipeline for constructing character query banks is sensible and addresses a real bottleneck in character identification. The ablations in Table 1 are internally coherent and provide evidence for the contribution of each module. However, the headline zero-shot SOTA claim rests on assumptions about data disjointness that are not verified, and the comparison table omits a key baseline, so the significance is conditional on resolving these issues.","major_comments":[{"comment":"The zero-shot claim requires that the Storyboard-v2 training clips, constructed from Storyboard20K (sourced from MovieNet and LSMDC), do not overlap with the evaluation movies in MAD-eval-Named (derived from LSMDC, with 10 retained movies) or in Cinepile-AD. No disjointness check is reported, and given that both the training and evaluation corpora draw from the same movie families, even a single overlapping movie would place training clips in the evaluation set and invalidate the zero-shot interpretation of Table 2. The authors should provide an explicit overlap analysis at the movie and clip level.","section":"§4.1, §4.2"},{"comment":"AutoAD-II is mentioned in the text as a fine-tuning-based state-of-the-art model, but its results are not included in Table 2. Since the paper claims SOTA over previous AD methods, omitting a strong and directly comparable baseline leaves the comparison incomplete and weakens the central empirical claim.","section":"§5.4, Table 2"},{"comment":"The reported BertScore of 24.2 for AutoAD-I is far below the range 50-60 reported for all other methods in the same table. This is likely an artifact of using a different evaluation configuration (e.g., tokenization, model variant, or rescaling). The authors should clarify the exact evaluation settings and, if necessary, re-run the AutoAD-I evaluation under the same protocol to ensure fair comparison.","section":"§5.4, Table 2"}],"minor_comments":[{"comment":"There is a missing space in 'understanding.To identify' in the abstract.","section":"Abstract"},{"comment":"The section header contains a typo: 'FocesedAD' should be 'FocusedAD'.","section":"§5.2"},{"comment":"Equation (6) uses y_k^a for the detected face but the text defines x_k^a as the detected face image; please unify the notation.","section":"§3.1, Eq. (6)"},{"comment":"In Figure 5, 'con’idence' is a typo for 'confidence'.","section":"Fig. 5"},{"comment":"The redundancy metric R relies on a threshold theta and a 'rigorously validated vocabulary', but the paper does not specify how theta is chosen, whether it is tuned on a validation set, or whether R correlates with human judgments of conciseness. Since R is used to support the model in Table 2, a sentence of justification or a reference to a validation study would be helpful.","section":"§5.1"}],"recommendation":"major_revision","confidential_remarks":"The central architecture and the proposed query-bank pipeline are interesting and the ablations are coherent, but the empirical SOTA claim currently rests on an unverified data-disjointness assumption. The authors should be asked to provide a concrete overlap analysis and to include AutoAD-II in the comparison, as well as to clarify the BertScore discrepancy. These issues are fixable within the manuscript's scope, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: FocusedAD is a well-engineered character-centric AD pipeline that does something genuinely useful: it clusters face embeddings into per-character \"best queries,\" tracks those characters through clips, and injects both character regions and prior AD/subtitle context as soft prompts into a 7B video LLM. The system-level integration is sensible, and the ablations are internally coherent. The best-query clustering is the most novel piece and it earns its keep in Table 1 (C2 vs B2). The application matters, too: automatic AD for BVI audiences is a real and underserved need.\n\nWhat I think is soft, in proportion:\n\nFirst, the zero-shot SOTA claim is load-bearing and not yet verifiable. Storyboard-v2 is built from MovieNet/LSMDC-derived Storyboard20K, and MAD-eval-Named is also LSMDC-derived. The paper says the test split excluded films from LSMDC train/test splits, but it never shows that the 10 retained movies are absent from Storyboard20K or from the 11,250 training triplets. Since the training pipeline extracts complete movie clips from full films using Storyboard20K timestamps, a single overlapping movie would put training clips into the evaluation set. Code and data are not released, and no overlap check is reported. That is a verifiability problem, not an architecture problem. The stress-test note is right.\n\nSecond, the comparison table is missing AutoAD-II even though the text describes it as a fine-tuning-based SOTA. If it is omitted because it uses MAD training data, that needs to be stated. Also, the AutoAD-I BertScore of 24.2 looks inconsistent with the other numbers in the table; I would want the source or a recalculation.\n\nThird, the new redundancy metric R is defined with a threshold theta, but theta is never reported. Without it, the R comparisons in Table 2 are not reproducible. This is minor and fixable, but it should be a one-line addition.\n\nSmaller items: the free parameters u, mu, and theta are all selected with some tuning, which is acceptable for a systems paper, but theta should be reported and the u tuning should be described as calibration rather than as part of the zero-shot claim. The qualitative examples are cherry-picked, as is usual.\n\nNone of these are fatal. The architecture is plausible, the ablation logic is coherent, and the method is a legitimate extension of known components. But the headline empirical claim cannot be accepted as stated until the train/test disjointness is demonstrated. This is a conditional for me: add an overlap check and release code/data, and I would take the SOTA numbers seriously. Without that, the contribution reduces to a solid engineering report with a promising method rather than a verified SOTA result.\n\nWho it is for: people working on movie understanding, accessible media, and multimodal captioning. Worth sending to a serious referee, with the split check and the missing baseline as required revisions.","headline":"A plausible systems paper whose headline zero-shot SOTA claim currently rests on an unverified train/test split; the architecture and ablations deserve a serious look, but the empirical claim needs a data-disjointness check before it should be cited.","tokens_in":16196,"tokens_out":2099,"would_cite":false,"duration_ms":20832,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FocusedAD claims that naming and tracking the active characters in a movie clip, then conditioning narration on story context, is enough to produce character-centric audio descriptions that outperform prior systems on zero-shot benchmarks.","keywords":["movie audio description","character-centric captioning","multi-modal large language models","character recognition","face clustering","soft prompting","zero-shot video captioning","accessibility for blind and visually impaired"],"falsifier":"Compare the list of film titles (and their clips) in the Storyboard20K-derived Storyboard-v2 training set against the ten movies in MAD-eval-Named and the clips in Cinepile-AD; any overlap would undermine the zero-shot interpretation. A second check: retrain or re-run the system with the ground-truth previous ADs replaced by the model's own previously generated ADs, and measure the score drop; if the drop is large, the benchmark's text-prior setting is doing much of the work.","tokens_in":15070,"feed_emoji":"🎬","tokens_out":5714,"duration_ms":53470,"temperature":0.7,"pith_summary":"The paper is trying to establish that movie audio description can be improved by making it character-centric: instead of describing entire scenes, a model should first find the active main characters, name them, and then narrate their actions and interactions. The authors propose FocusedAD, which combines a character perception module that tracks faces and links them to names, a dynamic prior module that feeds previous descriptions and subtitles into the generator through learnable soft prompts, and a focused caption module that pools visual features inside detected character regions. Training deliberately uses a proxy video dataset built from movie frames and character annotations rather than the target AD benchmark, so the reported gains on MAD-eval-Named and the new Cinepile-AD set are presented as zero-shot evidence. If the claim holds, automatic AD for blind and visually impaired audiences could be built from data that is easier to obtain, and generated narrations would carry the character names that AD guidelines require.","feed_headline":"Face tracking lifts movie narration quality past prior AD systems","feed_subtitle":"FocusedAD names the actors on screen, follows story context, and reports top zero-shot scores on two benchmarks.","key_machinery":"The load-bearing object is the character best query bank: for each main character, a set of face crops is clustered and a representative embedding is chosen by maximizing the ratio of inter-class distance to intra-class distance, so that characters remain distinguishable across lighting, makeup, and camera changes. Around that bank, the Character Perception Module performs face matching and bi-directional region propagation to mark which named characters are active in each clip; the Dynamic Prior Module converts previous ADs and subtitles into text priors and adds trainable soft prompts that adapt when zero, one, or many characters are present; and the Focused Caption Module pools image features inside each character mask to create character tokens, merges them temporally, and lets the LLM reason jointly over scene tokens, character tokens, and text tokens. The full pipeline is what the paper credits for shifting attention from generic objects to narrative-salient character actions.","core_discovery":"FocusedAD claims that explicit character grounding is the missing ingredient in current automated audio description. The model detects faces in key frames, associates each face with a character name using a clustered 'best query bank' built from portraits and movie frames, propagates the resulting regions forward and backward in time, and then feeds both scene-level and character-level visual tokens into an instruction-tuned language model together with prior ADs and subtitles. On the MAD-eval-Named benchmark the full system reports SPICE 7.4, METEOR 8.1, and BertScore 57.7, and on the proposed Cinepile-AD set SPICE 15.4, METEOR 14.7, and BertScore 64.5, exceeding dedicated AD systems such as AutoAD-I and MM-Narrator as well as general video models. The paper states that the model was fine-tuned only on its own Storyboard-v2 training set, not on the target AD datasets, making the comparison a zero-shot evaluation.","pith_inferences":["The same character query-bank construction could be applied to other identity-grounded video tasks, such as person-specific video question answering or long-form character tracking in TV series, though the paper does not test those settings.","A stricter test of the zero-shot claim would be to hold out every film whose clips or annotations appear in the Storyboard20K-derived training data, since the reported independence rests on dataset lineage.","One could test whether the soft prompt is actually carrying the narrative context by replacing the previous ADs with shuffled or wrong-context ADs; if scores drop sharply, the prompt is not just a style token."],"forward_implications":["Automated AD can be trained on proxy video data rather than on scarce AD-annotated film, lowering the data barrier for accessibility tools.","Character names, not just pronouns or generic labels, become a normal output of the captioning model, which matches professional AD style.","The dynamic soft prompt makes a single model handle scenes with any number of active characters without re-training.","The redundancy-aware metric introduced in the paper offers a way to quantify whether generated AD is concise enough for BVI listeners.","The Cinepile-AD test set provides a public-style benchmark derived from online clips, giving future AD systems a second zero-shot target."],"supporting_citations":[{"why":"Defines the MAD-eval-Named benchmark and the recurrent context approach that FocusedAD compares against.","marker":"[18]"},{"why":"AutoAD-II, the prior dedicated AD system that names characters and serves as a key baseline.","marker":"[17]"},{"why":"MM-Narrator, a long-form video narrator baseline that relies on in-context learning rather than fine-tuning.","marker":"[54]"},{"why":"Qwen2.5-VL, a general video MLLM baseline used for comparison.","marker":"[6]"},{"why":"VideoLLaMA3, a general video MLLM baseline used for comparison.","marker":"[9]"},{"why":"Storyboard20K, the video-generation dataset from which the Storyboard-v2 training set is built.","marker":"[48]"},{"why":"FaceNet, which supplies the face embeddings used for character recognition and query clustering.","marker":"[41]"},{"why":"SAM 2's memory encoder and memory bank, which provide the temporal character-region propagation mechanism.","marker":"[37]"},{"why":"CinePile, the source of the newly proposed Cinepile-AD test set.","marker":"[38]"},{"why":"The movie-description dataset from which MAD-eval-Named is derived, establishing the test-data lineage.","marker":"[39]"}],"fun_headline_variants":[],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that the results are zero-shot depends on the assumption that no clip from the evaluation movies appears in the Storyboard20K-derived training data, even though both sides draw from the same family of movie datasets; if training and evaluation films overlap, the reported numbers would measure memorization rather than generalization.","fun_headline_variants_meta":{"error":"Client error '402 Payment Required' for url 'https://api.deepseek.com/chat/completions'\nFor more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/402"},"cache_creation_input_tokens":0},"created_at":"2026-08-16T12:37:08.873090+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare the list of film titles (and their clips) in the Storyboard20K-derived Storyboard-v2 training set against the ten movies in MAD-eval-Named and the clips in Cinepile-AD; any overlap would undermine the zero-shot interpretation. A second check: retrain or re-run the system with the ground-truth previous ADs replaced by the model's own previously generated ADs, and measure the score drop; if the drop is large, the benchmark's text-prior setting is doing much of the work.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the MAD-eval-Named benchmark and the recurrent context approach that FocusedAD compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"AutoAD-II, the prior dedicated AD system that names characters and serves as a key baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"MM-Narrator, a long-form video narrator baseline that relies on in-context learning rather than fine-tuning."},{"cited_title":"From Benedict Cumberbatch to Sherlock Holmes: Character Identification in TV series without a Script","cited_arxiv_id":"1801.10442","evidence_quote":"CinePile, the source of the newly proposed Cinepile-AD test set."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The movie-description dataset from which MAD-eval-Named is derived, establishing the test-data lineage."}],"review_version":1}