{"id":"20eacf35-2a8b-4ad4-be90-f9bf1f33b10c","arxiv_id":"2412.08357","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A conditional DDPM that denoises importance scores, initialized by an unsupervised summarizer, reports top F-scores on three video summarization benchmarks.","lead":"This paper applies a denoising diffusion model to video summarization, generating frame importance scores by iterative denoising instead of direct prediction. The authors report state-of-the-art F-scores on TVSum, SumMe, and FPVSum, and argue that the generative approach tolerates subjective annotation noise.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's central mechanism—training DDPM on raw per-annotator scores makes summarization resistant to annotation noise—is never isolated: no ablation compares raw versus averaged annotation targets, so the reported gains could come from DDPM guidance or the unsupervised initialization rather…","rationale":"I read the paper as claiming a generative formulation whose key advantage is robustness to subjective annotation noise. For that claim to hold, it must be the raw per-annotator training distribution that produces the improvement. The paper provides no direct test of this: the training loop uses raw annotations, and all comparisons are against discriminative baselines or ablations that remove whole components. The reader's weakest assumption about the off-manifold x_T is a legitimate validity concern for DDPM theory, but the empirical central claim would survive even if x_T were only approximately on-manifold: the model could still be a learned conditional denoiser. The missing raw-versus-averaged ablation, by contrast, directly attacks the paper's stated mechanism. If the mechanism is not responsible, the SOTA results would be an unremarkable DDPM-plus-unsupervised refinement rather than a noise-robust generative summarization method. I therefore recommend retaining the CONDITIONAL verdict. I credit the paper for extensive comparisons, ablations, and a human study, and for reporting 5-split averages; the concern is not that the results are fabricated, but that their interpretation is underdetermined by the experiments.","tokens_in":11835,"tokens_out":3644,"duration_ms":39769,"concrete_test":"Retrain the full pipeline (DDPM with video-feature guidance, T=200, DR-DSN initialization, identical hyperparameters and 5-split protocol) using per-video averaged ground-truth importance scores, rescaled to [-1,1], as the training target instead of the raw per-annotator scores. If F-scores on TVSum, SumMe, and FPVSum stay within the reported split-to-split variation (ideally error bars over the 5 splits), the raw-annotation mechanism is not responsible for the headline gains; if averaged targets are much worse, the noise-robustness claim gains support. An additional check: report train versus test F-score gap for both variants to test the overfitting claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The Abstract and Introduction claim that learning the distribution of raw per-annotator importance scores, rather than averaged scores, makes the method 'more resistant to subjective annotation noise' and 'less prone to overfitting'. This is the paper's distinctive contribution and underpins the interpretation of Tables 1 and 2. However, Algorithm 1 simply loops over 'every importance scores annotation xg' when constructing training targets, and no experiment varies that choice. The ablations in Table 4 remove either DDPM ('w/o DDPM') or the unsupervised initialization ('w/o unsup'), but never replace raw annotations with averaged ground-truth scores. Consequently, the observed gains over discriminative baselines are confounded: they could be due to the DDPM architecture, the video-feature guidance, the DR-DSN starting point, or the 200-step schedule, with no evidence that the distribution over annotators is what helps. Similarly, the 'less prone to overfitting' claim is inferred from transfer-setting F-scores, but no train/test gap, variance across the 5 splits, or comparison against an averaged-target model is reported. A claim about noise robustness requires an experiment in which the only difference is the annotation-target distribution.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a diffusion-based generative method for video summarization. A Transformer-based noise predictor is trained to denoise importance-score trajectories, conditioned on video features, with raw per-annotator importance scores scaled to [-1,1] as training targets. At test time the method scales the output of an unsupervised summarizer (DR-DSN) to [-1,1], treats it as a T-step noised score x_T, and performs T=200 denoising steps to obtain final importance scores. Experiments on TVSum, SumMe, and FPVSum report state-of-the-art F-scores, correlation coefficients, ablations, a hyperparameter study, and a user study.","tokens_in":12111,"tokens_out":4800,"duration_ms":47170,"significance":"If the core claims hold, the paper would establish that generative modeling of importance-score distributions is more robust to annotation subjectivity than direct discriminative prediction, and that coupling DDPM with an unsupervised initializer is a practical answer to limited training data. The manuscript has strengths: the method is described in enough detail to be reimplemented, ablations show that both the DDPM component and the unsupervised initialization contribute, and the evaluation covers several datasets, three standard settings, and rank-correlation metrics. The paper also makes a falsifiable prediction about transfer performance. However, the distinctive claim regarding raw per-annotator targets is not experimentally isolated, the test-time assumption that the unsupervised output is a valid DDPM noisy state is unsupported, and key hyperparameters are selected on the test benchmarks. The current evidence supports a promising method but not the strong robustness and generalization interpretation stated in the Abstract.","major_comments":[{"comment":"The central novelty claim is that training on raw per-annotator importance scores, rather than averaged scores, makes the method 'more resistant to subjective annotation noise' and 'less prone to overfitting'. This claim is never directly tested. Algorithm 1 always uses raw annotations, and the ablations in Table 4 vary only the DDPM component and the unsupervised initialization; no variant replaces raw targets with averaged ground-truth scores. As a result, the gains over discriminative baselines are confounded: they could be due to the DDPM architecture, the video-feature guidance, the DR-DSN starting point, or the 200-step schedule. Please add an ablation in which the only change is raw versus averaged annotation targets, and report train/test gap or training-curve evidence for the overfitting claim; the Abstract should be calibrated to the outcome of that experiment.","section":"Abstract/Introduction; Algorithm 1; Table 4"},{"comment":"The testing process assumes that the scaled output of the unsupervised model, x_T = Scale(x_u), is a valid noisy state under q(x_T|x_0) with the chosen T=200. No distributional evidence is provided for this equivalence. If DR-DSN scores are not distributed like a 200-step noised ground-truth score, the reverse process starts off-manifold and the method is not a valid DDPM denoising. Please provide diagnostics, such as comparing the empirical distribution of scaled DR-DSN outputs with that of training-time x_T samples from Eq. (5), or an experiment that replaces the DR-DSN starting point with an actual 200-step noised ground-truth score during evaluation.","section":"Eq. (7); Algorithm 2; Figure 4"},{"comment":"Important configuration choices are made using the test benchmarks. Table 5 selects DR-DSN as the unsupervised model because it gives the 'best overall performance' on TVSum and SumMe, and Figure 4 selects T=200 on the same datasets. Since the final method uses these choices, the reported test F-scores are not fully independent and the state-of-the-art claim is weakened. Use validation splits or nested cross-validation to select T and the unsupervised model, then report test performance only after those choices are fixed.","section":"Analysis of Hyper-parameter; Table 5; Figure 4"},{"comment":"The experiments average over five random splits, but no standard deviations, per-split results, or significance tests are reported. Several headline differences are moderate in magnitude (for example, SumMe canonical: 58.7 vs. 56.0 for CFT-GIB; TVSum transfer: 60.9 vs. 59.7 for RR-STG) and may be within split-to-split variation. Reporting variance is necessary before concluding that the proposed method is state of the art under all settings.","section":"Quantitative Evaluation; Tables 1-3"}],"minor_comments":[{"comment":"The statement 'We make the first attempt to introduce DDPM into the field of video summarization' is contradicted by the cited baseline DMFF (Yu et al. 2024), which is a diffusion-based video summarization method; please qualify the novelty claim.","section":"Related Work"},{"comment":"The column header 'FVPSum' is a typo for 'FPVSum'.","section":"Ablation Study; Table 4"},{"comment":"The text refers to an appendix containing dataset details and additional visualization cases, but no appendix is present in the submitted version.","section":"Experiment"},{"comment":"Figure 4 labels the hyperparameter as 't', while the method defines the maximum noise addition steps as T; please make the notation consistent throughout.","section":"Analysis of Hyper-parameter; Figure 4"},{"comment":"The loss is written as L = ||epsilon - hat-epsilon||^2 with no normalization or expectation; please clarify whether this is the per-sample simplified DDPM objective and how the loop over annotations and steps is aggregated.","section":"Eq. (6)"}],"recommendation":"major_revision","confidential_remarks":"The central mechanism--raw-annotation distribution learning--is not isolated by the experiments, and the T/unsupervised-model selection on test sets weakens the SOTA claim. These are fixable with additional experiments and a recalibrated narrative. The 'first attempt' claim also needs correction given the cited DMFF method. If the raw-vs-averaged ablation and variance reporting are added, the paper could become a solid contribution to a video summarization venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid, well-tested engineering contribution that overclaims its conceptual novelty. The specific setup—conditional DDPM on video features, training on raw per-annotator importance scores, and starting the reverse process from an unsupervised model's output with a 200-step schedule—is new in the cited literature, and the reported F-scores are strong and internally consistent. The ablations show both the DDPM and the unsupervised init contribute, and the user study is a reasonable extra signal.\n\nWhat does not hold up is the load-bearing claim that training on raw annotations is what makes the model 'more resistant to subjective annotation noise.' There is no ablation that swaps raw per-annotator targets for averaged ground truth; Algorithm 1 just loops over annotations and no experiment varies that choice. So we cannot tell whether the gains come from the distribution over annotators, the DDPM formulation, the feature guidance, or the unsupervised initialization. The 'first attempt to introduce DDPM into video summarization' is also overstated—their own references include Yu et al. 2024 using a diffusion model for unsupervised video summarization. They should say 'first conditional DDPM for supervised video summarization' at most.\n\nTwo more soft spots. Minor: maximum noise step T and the choice of DR-DSN as the unsupervised initializer are selected on the test benchmarks, and no standard deviations or significance tests appear, so the SOTA margins are fragile. Moderate: the paper treats the unsupervised model output as if it were x_T, a 200-step noised version of ground truth, but gives no distributional justification; the argument is only that the noise weight at that step is moderate. If the unsupervised scores are off-manifold, the 'denoising' is really a learned refinement. This weakens the interpretation, not the empirical result.\n\nOn balance, I would send this to review. The missing raw-vs-averaged ablation is a straightforward experiment that would either support or kill the central claim, and the rest is competent enough for a referee to work with. The paper is useful to people applying diffusion to dense prediction and to the video summarization community, mainly as evidence that a controlled DDPM starting point works well.","headline":"Solid conditional-DDPM application with strong reported numbers, but the noise-robustness claim is unablated and the novelty claim is overstated.","tokens_in":12581,"tokens_out":3207,"would_cite":true,"duration_ms":30666,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes a denoising diffusion probabilistic model for video summarization that, trained on raw per-annotator importance scores and seeded by an unsupervised model's output, achieves state-of-the-art F-scores on TVSum, SumMe…","keywords":["video summarization","denoising diffusion probabilistic model","generative modeling","importance score prediction","subjective annotation noise","unsupervised video summarization","transfer learning","F-score evaluation"],"falsifier":"Compare the empirical distribution of the scaled unsupervised-model outputs against the theoretical $q(x_T|x_0)$ for $T=200$ with the paper's variance schedule; if the two distributions differ substantially, the 'denoising' is not a valid reverse diffusion process, and the reported gains would need a different explanation.","tokens_in":11638,"feed_emoji":"🎬","tokens_out":6942,"duration_ms":68203,"temperature":0.7,"pith_summary":"Video summarization is usually framed as a supervised regression task: a network predicts an importance score per frame from video features, and the summary is built from the top-scoring frames. This paper argues that the subjectivity of annotators corrupts that direct prediction, and instead frames summarization as a generative modeling problem. It trains a denoising diffusion probabilistic model (DDPM) to reconstruct importance scores from noise, conditioned on video frame features, using each annotator's raw scores rather than the averaged ground truth. Because data are scarce, the model starts its reverse denoising from the output of an unsupervised summarization model rather than from pure Gaussian noise. The reported result is improved F-scores and ranking correlations on TVSum, SumMe, and FPVSum, which the authors take as evidence that the generative formulation resists subjective annotation noise and generalizes better than discriminative predictors.","feed_headline":"Diffusion model sets new state of the art in video summarization","feed_subtitle":"Training a DDPM on raw per-annotator scores resists subjective noise and generalizes across domains.","key_machinery":"The key mechanism is the DDPM reverse process applied to frame importance scores, with video features as conditioning. The forward process defines $x_t = \\sqrt{\\bar\\alpha_t} x_0 + \\sqrt{1-\\bar\\alpha_t}\\epsilon$, and the noise predictor $\\epsilon_\\theta(x_t, f, t)$ uses Transformer attention, treating the noised importance scores as queries and video features as keys and values, to predict the noise component at step $t$. The design choice that makes the method work under data scarcity is to initialize the reverse process with the scaled output of an unsupervised video summarization model instead of Gaussian noise, so the model only needs to perform $T=200$ denoising steps. The training objective is $\\|\\epsilon - \\hat\\epsilon\\|^2$ over all annotators' raw scores, which lets the network learn the distribution of importance scores rather than a single averaged target.","core_discovery":"The paper's central claim is that modeling the distribution of per-annotator importance scores with a DDPM yields summaries that are more robust to subjective annotation noise and less prone to overfitting than discriminative score predictors. In training, ground-truth importance scores are linearly scaled to [-1,1] and noised for up to 200 steps; a Transformer-based noise predictor, with the noised scores as queries and video features as keys and values, is trained to estimate the added noise using a simple mean-squared error loss on each annotator's scores separately. At test time, the scaled output of an unsupervised video summarization model is treated as the starting state $x_T$ and is iteratively denoised for 200 steps to produce the final importance scores. The authors report the highest F-scores on all three datasets in all evaluated settings, along with the best Kendall's tau and Spearman's rho correlations, and interpret this as support for the generative formulation.","pith_inferences":["A testable extension is to apply the same recipe to other dense prediction tasks with noisy per-annotator labels, such as saliency or highlight detection, where learning the full label distribution may outperform learning the mean.","The optimal choice of $T=200$ is likely tied to the distribution of the chosen unsupervised model's output; the method may require retuning $T$ if a different unsupervised model is used.","The paper does not isolate how much of the improvement comes from the diffusion process itself versus from a learned refinement of the unsupervised scores; comparing against a lightweight residual correction head on top of the unsupervised output would separate those contributions.","If the distributional assumption is relaxed, the framework could be interpreted as a learned post-processing of the unsupervised model's scores, which would make the 'denoising' framing a useful but not strictly necessary description."],"forward_implications":["If the central claim is correct, generative score modeling replaces direct discriminative prediction as a robust formulation, achieving top F-scores on TVSum, SumMe, and FPVSum.","Training on raw per-annotator scores, rather than averaged scores, is what provides resistance to subjective annotation noise.","Starting the reverse process from an unsupervised model's output makes DDPM practical for small video summarization datasets, outperforming both pure DDPM and the unsupervised model alone in ablations.","The larger gains in the transfer settings indicate that the method overfits less to training data and generalizes across domains."],"supporting_citations":[{"why":"Supplies the DDPM formulation, forward/reverse process, noise-prediction loss, and variance schedule that the method adapts.","marker":"Ho, Jain, and Abbeel 2020"},{"why":"Provides the unsupervised video summarization model whose output seeds the denoising process and whose scores are used as the starting state $x_T$.","marker":"Zhou, Qiao, and Xiang 2018"},{"why":"Provides the TVSum dataset and its evaluation protocol, which the paper uses for canonical, augmented, and transfer settings.","marker":"Song et al. 2015"},{"why":"Provides the SumMe dataset used for F-score and correlation evaluations.","marker":"Gong et al. 2014"},{"why":"Provides the FPVSum dataset and the first-person video transfer evaluation protocol.","marker":"Ho, Chiu, and Wang 2018"},{"why":"Defines the canonical, augmented, and transfer evaluation settings and the dppLSTM baseline that the paper compares against.","marker":"Zhang et al. 2016"}],"fun_headline_variants":["Diffusion-based video summarization beats state of the art","Generative video summarization resists annotation noise","DDPM video summarization outperforms discriminative models","Video summarization via diffusion survives subjective scores"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the unsupervised model's scaled importance scores are distributed like ground-truth scores that have been corrupted by 200 steps of Gaussian noise, so that the reverse diffusion process starts on the manifold the model was trained on.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion-based video summarization beats state of the art","Generative video summarization resists annotation noise","DDPM video summarization outperforms discriminative models","Video summarization via diffusion survives subjective scores"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000166,"raw_usage":{"total_tokens":1249,"prompt_tokens":933,"completion_tokens":316,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":549,"completion_tokens_details":{"reasoning_tokens":255}},"tokens_in":549,"tokens_out":316,"duration_ms":4144,"temperature":1.0,"reasoning_tokens":255,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T17:54:27.715983+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare the empirical distribution of the scaled unsupervised-model outputs against the theoretical $q(x_T|x_0)$ for $T=200$ with the paper's variance schedule; if the two distributions differ substantially, the 'denoising' is not a valid reverse diffusion process, and the reported gains would need a different explanation.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the unsupervised video summarization model whose output seeds the denoising process and whose scores are used as the starting state $x_T$."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the TVSum dataset and its evaluation protocol, which the paper uses for canonical, augmented, and transfer settings."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the SumMe dataset used for F-score and correlation evaluations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the canonical, augmented, and transfer evaluation settings and the dppLSTM baseline that the paper compares against."}],"review_version":1}