{"id":"221f906d-0091-4b61-bdea-a3e29660168d","arxiv_id":"2508.04566","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"CLASP localizes dense audio-visual events under weak supervision by identifying cross-modal salient anchors and propagating their semantics along the timeline.","lead":"This paper introduces a weakly supervised version of dense audio-visual event localization, where the model sees only video-level labels. It proposes CLASP, which selects cross-modal salient anchors and propagates their semantics to produce temporal event boundaries, and reports state-of-the-art results on two benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"ActivityNet1.3 W-DAVEL label construction and mAP post-processing are unspecified, so the claimed 1.7% cross-dataset SOTA gain is not yet reproducible.","rationale":"The reader's weakest_assumption focuses on the agreement-score machinery: high JSD-based agreement between sigmoid outputs may not reliably indicate true events. That is a legitimate theoretical concern: Eq. (2) applies JSD to non-normalized multi-label sigmoid predictions, which is mathematically informal. However, the model is trained end-to-end and the same score is used for selection, so an imperfect score could still yield a useful ranking. The more decisive issue for a SOTA claim is benchmark validity. UnAV-100 has established audio-visual event annotations, so the primary result is credible pending minor details. ActivityNet1.3, by contrast, is not natively a DAVEL benchmark; using it requires undocumented choices about event classes, ground-truth construction, weak labels, and baseline adaptation. The paper also omits the inference post-processing needed to turn per-segment probabilities into the temporal detections that mAP/tIoU scoring requires. If the authors release the label-construction and post-processing artifacts, this concern can be settled. This supports the reader's CONDITIONAL verdict, but for a different reason than the reader's primary weakest assumption; hence 'partial' agreement.","tokens_in":14058,"tokens_out":8364,"duration_ms":109611,"concrete_test":"","verdict_should_be":"UNCHANGED","load_bearing_attack":"CLASP's headline claim is an empirical SOTA claim on two benchmarks. The UnAV-100 comparison is plausible, but the ActivityNet1.3 experiment (Table 2) is not verifiable as described. ActivityNet1.3 is an action-localization dataset; it does not natively contain audio-visual event labels or a distinction between audio and visual event modalities. The Datasets paragraph only says 'We also evaluate on this dataset' and gives statistics. It never specifies: (i) how the 203 activity classes were mapped to the C event classes used by Pa/Pv/p_av; (ii) how segment-level ground-truth audio-visual events were constructed; (iii) what video-level weak labels were used; or (iv) how the fully-supervised DAVEL baselines DAVE and CCNet were adapted to weak supervision. Moreover, 'Classification and Model Training' and 'Implementation Details' do not state how per-segment predictions p_av are converted into temporal event detections for mAP with tIoU 0.5-0.9. Thresholding at 0.5 alone does not define event intervals. Without these details, the reported 1.7% improvement over CCNet on ActivityNet1.3 cannot be reproduced or meaningfully evaluated. If the label construction is not a documented, deterministic transformation of original ActivityNet annotations, the cross-dataset pillar of the SOTA claim collapses.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a new weakly-supervised variant of Dense Audio-Visual Event Localization (W-DAVEL), in which only video-level event labels are available. The proposed method, CLASP, consists of three modules: Mutual Event Agreement Evaluation (MEAE) computes a cross-modal agreement score from independently predicted audio and visual event probabilities; Cross-modal Salient Anchor Identification (CSAI) selects global and local temporal anchors using that score; and Anchor-based Temporal Propagation (ATP) enhances per-timestamp features through cross-attention with the anchor features. The authors evaluate CLASP on UnAV-100 and ActivityNet1.3, comparing against prior AVEP and DAVEL methods under a weakly-supervised protocol, and report state-of-the-art results, including a 3.1% Avg mAP gain over CCNet on UnAV-100 and a 1.7% Avg mAP gain on ActivityNet1.3.","tokens_in":14493,"tokens_out":5258,"duration_ms":54411,"significance":"If the experimental claims are reliable, the paper makes a useful contribution by defining a practical weakly-supervised formulation of DAVEL and proposing a coherent bootstrapping mechanism that does not rely on external foundation models or manual pseudo-labels. The idea of using cross-modal agreement to identify reliable timestamps and then propagating semantic information from those anchors is intuitive and well aligned with the weak-supervision setting. However, the paper's central claim is an empirical SOTA claim, and several load-bearing experimental details are missing or inconsistent. The reported accuracy gains are meaningful only if the benchmark construction, baseline adaptation, metric computation, and hyperparameter selection protocol are fully documented.","major_comments":[{"comment":"The construction of the ActivityNet1.3 W-DAVEL benchmark is not specified. The paper does not state (i) how the 203 activity classes are mapped to the C event classes used by P_a/P_v/p_av, (ii) how segment-level audio-visual event ground truth is derived from action temporal annotations, (iii) what video-level weak labels are used, or (iv) how the fully-supervised baselines DAVE and CCNet are adapted to weak supervision. Without this, Table 2 and the claimed 1.7% improvement over CCNet are not reproducible.","section":"Experiments, 'Datasets' paragraph"},{"comment":"The reported 'Avg.' is not the arithmetic mean of the five tIoU mAP values as defined in the text. For example, in Table 1 the CLASP row (29.6, 26.3, 23.0, 19.7, 15.8) has mean 22.9, not the reported 30.0; the same discrepancy appears in every row of Tables 1-2 and all ablation tables. Since the paper's headline '3.1% mAP' gain over CCNet uses this number, the metric must be defined precisely or all Avg values corrected.","section":"Evaluation Metric and Tables 1-2"},{"comment":"Hyperparameters K, k, and M are selected based on experiments on UnAV-100, but the paper does not state whether these ablations are performed on the validation or test split. If the test set is used for model selection, the reported test performance is optimistically biased. Please clarify the split and, if necessary, re-run with validation-based selection.","section":"Implementation Details and Tables 3-5"},{"comment":"The paper says 'All methods are trained and evaluated under weakly supervised settings,' but DAVE and CCNet were designed for fully supervised DAVEL and the AVEP baselines for trimmed videos. The text never specifies how each baseline is adapted to W-DAVEL (loss change, MIL pooling, foreground suppression, etc.). Without a uniform protocol, the comparisons are not meaningful.","section":"Table 1, 'Evaluation on the UnAV-100 dataset'"},{"comment":"Jensen-Shannon divergence is defined for probability distributions that sum to one, but P_a and P_v are sigmoid multi-label scores whose rows do not normalize to 1. The agreement score s = 1 - d_JSD therefore lacks the theoretical interpretation claimed and may be driven by calibration differences. Please either normalize each row before computing JSD or justify the use of unnormalized scores.","section":"Methodology, MEAE, Eq. (2)"},{"comment":"The inference procedure after thresholding p_av at 0.5 is not described; the paper does not specify how per-segment event probabilities are grouped into event instances for tIoU-based mAP (e.g., contiguous segment merging, non-maximum suppression, or actionness smoothing). This step is necessary to reproduce Tables 1-2.","section":"Experiments, 'Implementation Details' and 'Evaluation Metric'"}],"minor_comments":[{"comment":"The abstract and several sections contain the typo 'DA VEL' instead of 'DAVEL'.","section":"Throughout"},{"comment":"The 'Avg.' column values are unexplained relative to the mean of the tIoU columns; beyond the major issue above, the captions should define exactly how 'Avg.' is computed.","section":"Tables 3-9"},{"comment":"The boxes labeled 'Linear' and 'Transformer' in Figure 2 are not all defined in the caption; a notation list or dimension annotations would improve clarity.","section":"Figure 2"},{"comment":"There are two 'Cheng et al.' entries with different years; the in-text citation for JoMoLD should be checked to ensure it refers to the correct year and publication.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The core idea is plausible and the paper is likely to be cited as a first W-DAVEL study, but the experimental reporting falls short of the standards needed for a SOTA claim. The missing ActivityNet1.3 label-construction details, the unexplained Avg metric, and the unspecified test/validation split for hyperparameter selection are all fixable in principle. I would encourage the editor to request a revision that documents the full experimental protocol and corrects the metric reporting before further consideration."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new thing here is the task itself: W-DAVEL, where you localize dense audio-visual events with only video-level labels. That is a real gap, and they build benchmarks on UnAV-100 and ActivityNet1.3 by adapting existing AVE/DAVEL methods to weak supervision. The anchor idea is also sound in spirit: pick timestamps where the audio and visual classifiers agree, fuse those features, and propagate them to the rest of the timeline. It is not a radical departure, but it is a sensible combination of known pieces applied to a setting nobody had defined before. Credit where due: the paper is clearly written, the ablations are systematic, and the UnAV-100 comparison includes the relevant prior methods.\n\nNow the soft spots, in order of severity. The ActivityNet1.3 result is not reproducible as described. That dataset has no native audio-visual event labels, yet the paper never specifies how the 203 action classes were mapped to the 100 event classes, how segment-level GT was built, what weak labels were used, or how the fully-supervised DAVEL baselines were adapted. The inference procedure for converting per-segment predictions into event intervals for tIoU mAP is also vague; thresholding at 0.5 does not define event boundaries. Until those details are documented as a deterministic transformation of the original annotations, the 1.7% cross-dataset SOTA claim is just an assertion.\n\nThe UnAV-100 experiments are more solid, but they tune K, k, M directly on the test set and report no variance. The ablation tables show the method is robust across hyperparameter choices, which helps, but three runs with seeds would be cheap and would put the 3.1% SOTA gap on firmer ground. The JSD on sigmoid outputs is mathematically informal, as the reader noted; sigmoid outputs are not normalized distributions. The fix is easy, either normalize before computing JSD or use a different agreement measure, and the qualitative figures suggest the anchor selection is doing something reasonable, so I would not call this load-bearing. Self-citations are heavy but the benchmark comparisons are independent of them.\n\nWho is this for? Anyone working on audio-visual event perception or weakly-supervised temporal localization in long videos. It is a niche subfield, and the paper is honest incremental progress, not a field re-organization. I would send it to a serious referee: the task definition is useful, the method is plausible, and the main issues are missing details and missing error bars rather than a broken central idea. If the authors supply the ActivityNet label construction and a seed-variance table, this becomes a straightforward accept-quality paper.","headline":"A legit new weakly-supervised DAVEL task with a plausible anchor-propagation method, but the ActivityNet experiment is currently under-specified and the UnAV-100 numbers lack error bars; worth refereeing after those are fixed.","tokens_in":14846,"tokens_out":665,"would_cite":true,"duration_ms":10238,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that timestamps where audio and visual classifiers agree can serve as anchors, and propagating those anchors along the timeline yields 30.0% mAP on UnAV-100 and 21.3% on ActivityNet1.3, the best reported results.","keywords":["weakly-supervised learning","audio-visual event localization","dense event localization","cross-modal salient anchors","temporal semantic propagation","untrimmed video understanding","multiple instance learning","Jensen-Shannon divergence"],"falsifier":"Shuffle the pairing in a trained CLASP model: compute the agreement score between each audio timestamp's prediction and the visual prediction of a different timestamp (or a different video), leaving the rest of the pipeline untouched. If average mAP on UnAV-100 stays near 30.0%, genuine cross-modal agreement is not the active ingredient; if it collapses, agreement is load-bearing. A cheaper complementary check: on a labeled subset, count how many of the chosen top-K anchors fall inside ground-truth event segments - the method's premise requires nearly all of them to.","tokens_in":14045,"feed_emoji":"🎬","tokens_out":13761,"duration_ms":121913,"temperature":0.7,"pith_summary":"This paper introduces W-DAVEL, a setting in which a model must localize every audio-visual event in a long, untrimmed video - predicting both which event categories occur and when - using only video-level labels, with no temporal boundaries given. The central claim is that such localization is achievable by exploiting cross-modal salient anchors: a small set of timestamps where independently trained audio and visual classifiers produce the most consistent event predictions. The CLASP method computes a mutual agreement score from the Jensen-Shannon divergence between the two modality predictions, selects the top-scoring timestamps both globally and inside local windows, fuses the chosen audio-visual features, and propagates those anchor semantics to every timestamp through cross-attention. On the two benchmarks the paper establishes, CLASP reports 30.0% average mAP on UnAV-100 and 21.3% on ActivityNet1.3, surpassing the previous best method by 3.1 and 1.7 points. If the claim holds, reliable timestamps can substitute for expensive frame-level annotation in dense event localization.","feed_headline":"Weak labels alone yield 30.0 mAP on dense audio-visual localization","feed_subtitle":"Timestamps where audio and visual classifiers agree become anchors that spread event semantics across the untrimmed video.","key_machinery":"The load-bearing object is the mutual agreement score $s = 1 - d_{\\mathrm{JSD}}$, where $d_{\\mathrm{JSD}}$ is the Jensen-Shannon divergence between per-timestamp sigmoid predictions $P_a$ and $P_v$ from two modality-specific classifiers (Eqs. 1-3). One scalar per timestamp ranks reliability: global anchor identification takes the top-$K$ over the whole video; local anchor identification takes the top-$k$ inside each of $M$ temporal windows. The chosen audio and visual features are summed within each modality and fused into $Z_{av} \\in \\mathbb{R}^{K \\times d}$ (Eq. 4). Anchor-based Temporal Propagation then applies multi-head cross-attention, $\\hat{F}_A = F_A + \\mathrm{MHA}(F_A, Z_{av}, Z_{av","core_discovery":"The paper's claim, stated on its own terms: under video-level supervision alone, some timestamps can already be predicted with confidence - the cross-modal salient anchors - and these are exactly the timestamps where independently trained audio and visual event classifiers agree most. CLASP turns that intuition into a pipeline: two modality-specific classifiers emit per-timestamp probabilities; an agreement score, 1 minus the Jensen-Shannon divergence between the two outputs, ranks every timestamp; the top-K globally and the top-k within each of M local windows identify anchor features, which are summed per modality and fused into a compact multimodal anchor representation; and a cross-atten","pith_inferences":["A test the paper leaves implicit: replace the agreement score with a properly normalized probability distance (JSD on softmax-normalized predictions, or cosine similarity between class-conditional embeddings) to check whether the informal use of JSD on sigmoid outputs costs accuracy; the ablations suggest the score function itself is a tunable axis.","The anchor-propagation idea should transfer to other weakly supervised dense tasks - temporal action proposals or weakly supervised audio-visual segmentation - wherever a few trustworthy timestamps can seed semantic spreading. This is an editorial inference, not a paper claim.","Because anchors are chosen per video at inference time by the model's own agreement, CLASP could be bootstrapped on unlabeled videos, using high-agreement timestamps as pseudo-labels for a second training round; the paper does not explore this.","The method should struggle exactly where agreement is spuriously high - long homogeneous scenes such as an empty road with steady traffic noise, where background timestamps may outrank short genuine events; stress-testing on such videos would separate true anchor semantics from mere classifier consensus."],"forward_implications":["Video-level labels alone reach 30.0% average mAP on UnAV-100 and 21.3% on ActivityNet1.3, establishing the first W-DAVEL benchmarks with margins of 3.1 and 1.7 points over the previous best method.","Global and local anchor identification are complementary: ablations show that removing either one lowers performance (29.5% or 29.8% vs. 30.0% average mAP), so both scales of selection contribute.","Anchors fused from both modalities beat single-modality anchors (30.0% vs. 29.6% or 29.4%), confirming that cross-modal agreement, not one modality's confidence, drives the gain.","The anchor counts are stable hyperparameters: K=10 global anchors, k=4 per window, and M=14 windows give the best results, with graceful degradation when varied.","The gap between this weak-supervision result (30.0%) and the roughly 50% average mAP of fully supervised DAVEL methods quantifies the headroom that remains in the task."],"supporting_citations":[{"why":"Supplies the UnAV-100 benchmark, the DAVE baseline, and the feature-extraction protocol that every comparison in the paper inherits.","marker":"Geng et al. 2023"},{"why":"CCNet, the prior state-of-the-art method whose UnAV-100 and ActivityNet1.3 scores define the margins CLASP claims to beat.","marker":"Zhou et al. 2025c"},{"why":"Defines the Jensen-Shannon divergence that the agreement score is built from (1 minus divergence between the two modality predictions).","marker":"Fuglede and Topsoe 2004"},{"why":"Provides the audio-visual event localization formulation and the MIL-pooling objective that turns video-level labels into trainable per-segment predictions.","marker":"Tian et al. 2018"},{"why":"Supplies the multi-head cross-attention mechanism used in the Anchor-based Temporal Propagation module.","marker":"Vaswani et al. 2017"},{"why":"VGGish, the pretrained model that produces the audio features fed into the pipeline.","marker":"Hershey et al. 2017"},{"why":"Two-stream I3D, the pretrained model that produces the visual features fed into the pipeline.","marker":"Carreira and Zisserman 2017"},{"why":"ActivityNet1.3, the second benchmark used to show the method generalizes beyond UnAV-100.","marker":"Heilbron et al. 2015"}],"fun_headline_variants":["Weak labels plus audio-visual agreement spot dense events","CLASP: cross-modal anchors boost weakly-supervised audio-visual localization","Agreeing audio-visual timestamps become anchors for weak DAVEL","Audio-visual agreement pinpoints anchors for weakly-supervised event localization","Weak supervision: use audio-visual agreement to anchor event localization"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The load-bearing premise is that a timestamp where the audio and visual classifiers agree is genuinely an event moment; if both classifiers are confidently wrong in the same way, or background segments routinely score high agreement, the anchors and the propagation amplify that shared error instead of correcting it.","fun_headline_variants_meta":{"raw":{"variants":["Weak labels plus audio-visual agreement spot dense events","CLASP: cross-modal anchors boost weakly-supervised audio-visual localization","Agreeing audio-visual timestamps become anchors for weak DAVEL","Audio-visual agreement pinpoints anchors for weakly-supervised event localization","Weak supervision: use audio-visual agreement to anchor event localization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000314,"raw_usage":{"total_tokens":1635,"prompt_tokens":773,"completion_tokens":862,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":517,"completion_tokens_details":{"reasoning_tokens":772}},"tokens_in":517,"tokens_out":862,"duration_ms":9209,"temperature":1.0,"reasoning_tokens":772,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T23:52:46.413996+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Shuffle the pairing in a trained CLASP model: compute the agreement score between each audio timestamp's prediction and the visual prediction of a different timestamp (or a different video), leaving the rest of the pipeline untouched. If average mAP on UnAV-100 stays near 30.0%, genuine cross-modal agreement is not the active ingredient; if it collapses, agreement is load-bearing. A cheaper complementary check: on a labeled subset, count how many of the chosen top-K anchors fall inside ground-truth event segments - the method's premise requires nearly all of them to.","supporting_citations":[],"review_version":1}