{"id":"44e108e4-d375-4763-874b-acb506e58e19","arxiv_id":"2412.19563","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A reinforcement learning agent removes modality-specific noisy labels during audio-visual video parsing, guided by validation F-scores and a soft inter-reward, improving parsing accuracy on the LLP dataset.","lead":"This paper trains a reinforcement learning agent to clean noisy event labels during weakly-supervised audio-visual video parsing, using the parsing model's validation accuracy as a reward. It reports improved F-scores on the Look, Listen, and Parse dataset and shows the cleaner can be added to other parsing models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Validation-set reward during training confounds the comparison: reported test gains may stem from validation leakage rather than the RL denoiser.","rationale":"The reader's weakest assumption identified validation leakage as the key risk, and this stress-test agrees. The central claim is that RLLD outperforms existing label denoising methods and improves SOTA parsers. For that claim to hold, the denoising policy must learn to remove modality-specific noise in a way that transfers to unseen test data. Because the policy is rewarded with validation-set F-scores during training, the observed test improvement could in principle be caused by overfitting to the 649-video validation split, whose labels are otherwise unavailable to the baselines. This is not an internal inconsistency; it is a correctness risk in the experimental comparison. The proposed test, training with the reward on a held-out training split, would settle whether the gain survives without access to validation labels. Since the reader already conditioned the verdict on addressing validation leakage, the verdict remains CONDITIONAL, and no adjustment is needed.","tokens_in":14366,"tokens_out":6002,"duration_ms":55428,"concrete_test":"Retrain RLLD from scratch with R_terminal computed on a randomly held-out 10% split of the training set, using the official validation split only for early stopping and hyperparameter selection. Then compare the test F-scores (Table 1) against JoMoLD and MA. Run at least 5 seeds and report mean and standard deviation. If the Event@AV margin over JoMoLD (53.4 vs 51.4) shrinks to below 1 point or reverses, the claimed advantage is attributable to validation leakage rather than the RL label-denoising mechanism.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing concern is that the denoiser is trained directly on the validation set. In Algorithm 1, every episode computes R_terminal = A(s_a, s_v, a_a, a_v) (Eq. 10), the F-score of the current task network on the official validation split, and uses it in the REINFORCE update (Eq. 15) to adjust the denoising policy. Thus validation labels act as training labels for the denoiser, not merely as a model-selection signal. This breaks the comparison with JoMoLD and MA in Table 1, since those baselines never see validation labels during training. The reported test improvements (e.g., Event@AV 53.4 vs 51.4 for JoMoLD) may reflect the extra supervision from 649 validation videos rather than the RL/joint-training design. The small margins and absence of error bars make this concern concrete. The paper is transparent about using validation rewards, but the inference that RLLD provides better label denoising is not yet supported without ruling out this confound.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RLLD, a reinforcement-learning-based label denoising module for weakly-supervised audio-visual video parsing (AVVP). A denoising policy is trained with REINFORCE to remove modality-specific noisy labels before these labels are used to train a parsing network; the reward combines an immediate soft inter-reward, measuring consistency between smoothed original labels and revised labels, with a terminal reward equal to the parsing model's F-score on the official validation split. Experiments on the LLP dataset compare RLLD with existing denoisers (JoMoLD, MA) and report small improvements, and the paper also reports additional gains when RLLD is integrated into CPSP and VALOR.","tokens_in":14615,"tokens_out":6594,"duration_ms":56779,"significance":"If the validation-reward protocol can be made sound, the paper would be a useful contribution: it is, to my knowledge, the first RL formulation of label denoising for AVVP, and the joint training of denoiser and parser is a natural way to address the disconnect between denoising and parsing. The integration experiments with CPSP and VALOR are a useful feature, and the paper is transparent about the validation-based reward. However, the central quantitative claim is currently weakened by the training-on-validation protocol and by the absence of uncertainty estimates; these issues are fixable, but they are load-bearing for the claim of superior denoising.","major_comments":[{"comment":"The terminal reward in Eq. (10) is the F-score of the task network on the official 649-video validation split, and Algorithm 1 (steps 5-7) uses this reward in the REINFORCE update of Eq. (15). Validation labels therefore act as training labels for the denoising policy, not merely as a model-selection signal. The baselines in Table 1 (e.g., JoMoLD, MA) never receive validation labels during training, so the comparison is not controlled: the reported test improvements (e.g., Event@AV 53.4 vs 51.4 for JoMoLD) could reflect overfitting to the validation split rather than a genuinely better denoising strategy. Please address this by, at minimum, ablating the reward source (e.g., using a held-out portion of the training set for the reward), reporting the validation-versus-test gap for the learned policy, and either giving baselines the same validation-based feedback or explicitly arguing why that feedback would not change their results.","section":"Algorithm 1, Eq. (10), Sec. 4.1"},{"comment":"Table 1 reports single F-score values with no standard deviations, number of seeds, or significance tests. The margins over JoMoLD are about 1.0 point at segment-level audio (61.6 vs 60.6), 2.4 points at segment-level visual (64.6 vs 62.2), and 2.0 points at event-level Event@AV (53.4 vs 51.4); these are small relative to typical run-to-run variation in this setting. Since Section 4.5 already reports five random initializations for the convergence curves, the authors have the infrastructure to report mean +/- std for the final test metrics; they should do so and add a suitable significance test before claiming 'superior performance'.","section":"Table 1, Secs. 4.2 and 4.5"},{"comment":"The ablation in Table 2 does not isolate the claimed components: the 'Full Setting' row is 'RLLD+CL', while the ablated rows are 'RLLD w/o initialized labels' and 'RLLD w/o soft inter-reward'. As presented, the ablated variants do not include the CL loss, so the difference attributed to the soft inter-reward (e.g., segment-level audio 61.8 vs 63.4) conflates the reward term with the contrastive loss. Please rerun the ablations on the same base configuration (all variants with or all variants without CL) and report both configurations.","section":"Table 2, Sec. 4.4"}],"minor_comments":[{"comment":"The formula for R1 is difficult to parse: the variables l_soft^t and l_t^in(j) are not defined precisely, the indices j and m are unexplained, and the expression as typeset appears to mix sums and division incorrectly. Please rewrite Eq. (9) with clear definitions of every symbol.","section":"Eq. (9)"},{"comment":"The text says 'R(S) is computed by Eq.(9)', but Eq. (9) defines R_inter only; the full reward R(S) is defined in Eq. (11). Please correct the reference.","section":"Eq. (13)"},{"comment":"The video-level loss L_video in Eq. (7) is defined but does not appear in Algorithm 1 or in the training description. Please clarify whether L_video is used and, if so, where.","section":"Eqs. (6)-(8), Algorithm 1"},{"comment":"The state definition mixes indices: s_i_audio uses f^t_audio and y^i_a, while i and t are both used for the video index. Please use one consistent subscript/superscript convention.","section":"Eq. (1)"},{"comment":"The text says 'Table 3 presents the F-score results', but the main comparison table is Table 1. Please correct the cross-reference.","section":"Sec. 4.2"}],"recommendation":"major_revision","confidential_remarks":"The main issue is methodological rather than novelty-related. If the authors can convincingly rule out validation-set overfitting (e.g., with a reward-source ablation and multi-seed significance testing), the paper is likely acceptable. I would not reject on the basis of the validation-reward design alone, because it is explicitly disclosed and can be addressed experimentally. The related-work coverage seems adequate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe real contribution here is the joint RL denoising loop: a binary action policy removes modality-specific label noise, and the reward comes partly from the parser's F-score on the validation split. That integration is new for AVVP, and the authors are upfront about it. The experiments also show the denoiser can be bolted onto existing parsers like VALOR and CPSP, with small but consistent gains.\n\nThe soft spot is the one you'd expect. The denoiser is trained with a terminal reward that is exactly the task network's F-score on the 649-video validation set. That means validation labels leak into training for the denoiser. The baselines (JoMoLD, MA, and the rest) never see those labels during training. So the reported test improvements — a few F-score points — could be partly explained by the extra supervision from validation, not by the RL/joint-training design. The paper doesn't report error bars or significance tests, and the margins are small. The ablation 'w/o soft inter-reward' still uses the validation reward, so it doesn't isolate this effect. What's missing is a version of RLLD that trains without validation rewards, or a comparison against baselines that also get validation labels, or a cross-validation scheme.\n\nI don't think the idea is wrong. The validation reward is a reasonable choice if the goal is a direct signal for denoising, and the paper is transparent about it. But the evidence as presented doesn't yet show that RLLD is better denoising rather than better at exploiting validation labels. That's fixable: run the method on a split where validation labels are never used, or at least report test results with standard deviations over random seeds and a baseline with the same validation supervision.\n\nThe paper is worth a serious referee. The framing is clear, the related work is covered, and the method can be repaired with a cleaner evaluation. For a reading group, the validation-leakage discussion is the most instructive piece — it's a nice case study in how an intuitive training signal can compromise a comparison.\n\nI'd recommend sending it to review, with a request for a revised evaluation that isolates the denoiser's contribution.","headline":"Novel RL-based label denoising for AVVP, but the validation-set reward confounds the experimental comparison; needs a revised evaluation to be convincing.","tokens_in":15095,"tokens_out":2641,"would_cite":false,"duration_ms":35158,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Label denoising for audio-visual video parsing should be trained jointly with the parser, using validation F-scores as the reward, and this jointly trained denoiser outperforms prior denoisers and improves existing parsers.","keywords":["audio-visual video parsing","weakly-supervised learning","label denoising","reinforcement learning","REINFORCE","modality-specific noise","LLP dataset","video event localization"],"falsifier":"Train RLLD with the same validation-reward rule but on a split of the validation set held out from the original validation data, or with permuted validation labels, and compare test F-scores; if deleting labels that raise the validation reward does not also raise test scores, the reported gains come from validation-set memorization rather than better denoising.","tokens_in":14217,"feed_emoji":"🎧","tokens_out":6033,"duration_ms":47034,"temperature":0.7,"pith_summary":"This paper argues that label denoising for weakly-supervised audio-visual video parsing should not be a separate preprocessing step. It proposes RLLD, a reinforcement-learning agent that decides which event labels to remove from each modality, trained jointly with the parser so that denoising is guided by the parser's own validation performance. The agent's state combines input features with the original noisy labels, and its reward combines a validation F-score with a soft inter-reward comparing revised labels to smoothed originals. If the central claim holds, denoisers that optimize parsing outcomes directly outperform denoisers that estimate noise in a disconnected stage, and the same denoiser can be dropped into existing parsers to improve them.","feed_headline":"RL label denoiser lifts audio-visual parsing F-scores","feed_subtitle":"Trained on validation feedback, it beats prior denoisers and boosts CPSP and VALOR parsers.","key_machinery":"The load-bearing object is the reinforcement-learning label-denoising policy. At each step the state is the concatenation of pretrained audio and visual features with the original video-level labels; self-attention and cross-attention produce hidden states, and a Bernoulli sample over class probabilities decides which labels to delete from each modality. The revised labels supervise both video-level and modality-level losses of a HAN-style task network. Since the validation F-score reward is non-differentiable, REINFORCE with a Monte Carlo estimate updates the policy; the soft inter-reward, built from a smoothed-label divergence and cosine similarity between revised and smoothed labels, supplies immediate feedback, while a terminal reward equal to the parser's validation F-score supplies the task-aligned signal.","core_discovery":"On the LLP benchmark, RLLD reports segment-level audio F-score 61.6 and event-level Event@Audio-Visual 53.4, above JoMoLD's 60.6 and 51.4; adding a contrastive loss gives 63.4 and 54.0. Integrated into CPSP and VALOR, it raises their reported F-scores on most metrics, for example VALOR segment-level Type from 62.0 to 62.7 and event-level Event from 54.2 to 54.9. The authors attribute the gains to using validation results as a reward, which aligns the denoising policy with the actual parsing objective, and to the soft inter-reward, which provides step-by-step feedback during learning.","pith_inferences":["A clean test of the method would hold out a third split of the validation set, since using validation F-scores as a training reward makes the validation set part of the training signal and could inflate test results through selection.","The same reward scheme could transfer to other weakly-supervised sequence labeling problems with noisy per-modality labels, such as action localization in video, because the machinery does not depend on audio-visual specifics.","The ablation results suggest the soft inter-reward acts mainly as a regularizer that keeps the policy close to the original labels; without it performance drops, implying the agent's freedom to delete labels is useful only when tempered.","The paper itself notes (Section 4.6) that reinforcement learning's trial-and-error nature makes training slow, so practical adoption depends on whether the reported F-score gains justify roughly triple the training time of prior denoisers."],"forward_implications":["If RLLD works as claimed, label denoising in audio-visual video parsing can be optimized end-to-end toward parsing performance instead of being a separate noise-estimation stage.","Because the reward is just the parser's validation F-score, the denoiser can in principle be attached to any AVVP parser that has a validation set, not only the HAN backbone used in the paper.","Removing labels rather than estimating noise ratios gives downstream components such as contrastive learning cleaner modality-specific supervision, which may explain the gains when RLLD is added to CPSP and VALOR.","The paper's training-time comparison (about 8 hours versus 2.5-3 hours for MA and JoMoLD) means the F-score improvements come with a substantial compute cost."],"supporting_citations":[{"why":"Introduces the LLP dataset and the HAN baseline used as the task network and comparison point.","marker":"[31]"},{"why":"JoMoLD is the main label-denoising baseline RLLD compares against and whose configuration is followed.","marker":"[6]"},{"why":"MA is the other modality-specific denoising method RLLD is compared with.","marker":"[38]"},{"why":"VALOR is one of the state-of-the-art parsers RLLD is integrated into for enhancement experiments.","marker":"[43]"},{"why":"CPSP is the other state-of-the-art parser used to test whether RLLD transfers to existing models.","marker":"[44]"},{"why":"REINFORCE supplies the policy-gradient update used to train the denoising network.","marker":"[37]"}],"fun_headline_variants":["Joint RL denoising boosts audio-visual parsing","Reinforced label denoiser lifts AV parsing scores","RL denoiser with validation feedback improves parsing","Joint training of denoiser and parser outperforms prior"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The denoiser is trained with a reward equal to the parser's F-score on the validation set, so the method assumes that validation F-score is a trustworthy signal that transfers to unseen test videos rather than being overfit by the policy.","fun_headline_variants_meta":{"raw":{"variants":["Joint RL denoising boosts audio-visual parsing","Reinforced label denoiser lifts AV parsing scores","RL denoiser with validation feedback improves parsing","Joint training of denoiser and parser outperforms prior"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000147,"raw_usage":{"total_tokens":1149,"prompt_tokens":871,"completion_tokens":278,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":487,"completion_tokens_details":{"reasoning_tokens":214}},"tokens_in":487,"tokens_out":278,"duration_ms":3560,"temperature":1.0,"reasoning_tokens":214,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T00:11:32.538962+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train RLLD with the same validation-reward rule but on a split of the validation set held out from the original validation data, or with permuted validation labels, and compare test F-scores; if deleting labels that raise the validation reward does not also raise test scores, the reported gains come from validation-set memorization rather than better denoising.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the LLP dataset and the HAN baseline used as the task network and comparison point."},{"cited_title":"Cheng, Z","cited_arxiv_id":null,"evidence_quote":"JoMoLD is the main label-denoising baseline RLLD compares against and whose configuration is followed."},{"cited_title":"Wu and Y","cited_arxiv_id":null,"evidence_quote":"MA is the other modality-specific denoising method RLLD is compared with."},{"cited_title":"Yung-Hsuan, C","cited_arxiv_id":null,"evidence_quote":"VALOR is one of the state-of-the-art parsers RLLD is integrated into for enhancement experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CPSP is the other state-of-the-art parser used to test whether RLLD transfers to existing models."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"REINFORCE supplies the policy-gradient update used to train the denoising network."}],"review_version":1}