{"id":"346e1cb3-c124-4c9c-abd5-c6834b2d3ea4","arxiv_id":"2608.08622","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A training-free decoding framework that adaptively reweights attention toward video tokens and erases key visual evidence per frame to suppress hallucinated predictions, achieving 72.60% accuracy on EventHallusion with LLaVA-Video-7B.","lead":"This paper presents VADER, a training-free method that cuts hallucinated answers in video question-answering by adaptively steering model attention toward video evidence and suppressing language-prior-driven predictions. It reports strong benchmark gains, including 72.60% accuracy on EventHallusion with LLaVA-Video-7B, and could make video AI systems more reliable without retraining.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Gradient-attention saliency (Eq. 3) is not causally validated; if uninformative, VFR's adaptive policy reduces to a fixed intervention, undermining the claimed mechanism.","rationale":"The reader's weakest assumption identifies the gradient-attention saliency as unvalidated, and I agree this is the most load-bearing concern for the paper's central claim. The paper's headline contribution is a video-conditioned intervention policy (VFR), and the entire claim of adaptivity rests on the diagnostic quality of I^{(l)}_{ij} (Eq. 3) and the resulting layer selection and amplitude (Eqs. 6-7). The manuscript offers only qualitative saliency visualizations and a comparison against a single fixed layer set, which is insufficient to establish that the saliency is informative. Moreover, the diagnostic target is the model's own initial prediction, which may be hallucinated; in that case the gradient points along the evidence path that produced the error, so VFR could systematically reinforce the wrong answer in exactly the cases the method aims to fix. The existing ablation (Table 3 vs Table 4) shows adaptive selection outperforms one arbitrary fixed configuration, but this does not rule out the possibility that random or simple heuristics perform equally well or better. If so, the 'adaptive' part of the method is not the source of the gains, and the paper's novelty would be substantially weakened, even if the empirical numbers survive. This concern is therefore directly load-bearing on the stated contribution and on the interpretation of the experimental results. The proposed concrete test—comparison against random and uniform layer selection with matched seeds and downstream accuracy, plus stratification by initial prediction correctness—would settle whether the attribution is causal or incidental. Because the paper's empirical gains are plausible and the method is clearly specified, the appropriate verdict remains CONDITIONAL: the claim is promising but needs this additional mechanistic validation before full acceptance.","tokens_in":13432,"tokens_out":10393,"duration_ms":112277,"concrete_test":"Run VADER on EventHallusion and VideoHallucer with LLaVA-Video-7B, replacing VFR's layer selection (Eq. 6) with (a) per-sample random selection of K_l=5 layers (10 seeds) and (b) a fixed uniform spread of 5 layers across the decoder, keeping all other components and hyperparameters fixed. If adaptive selection does not exceed both baselines by more than the seed-level standard deviation, the saliency is not demonstrably informative. Also stratify results by correctness of the initial prediction used in L_diag to check whether the policy helps only when the model was already correct.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends on VFR's gradient-attention saliency (Eq. 3) faithfully identifying where visual evidence influences the current decision. This saliency is computed with respect to the model's own initial prediction (L_diag = -log p(y_hat|V,q)); when that prediction is a hallucination, the gradients highlight the very attention pathways that support the wrong answer, and VFR's selected layers and amplitude a (Eqs. 6-7) could amplify rather than correct the error. No causal validation is provided: the only evidence is qualitative saliency maps (Fig. 3) and one fixed-layer baseline (Table 3). The fixed-layer baseline (layers 7,14,15,16,17) is a single arbitrary choice; adaptive selection improves over it by 0.5-2.1 points across benchmarks, but a random or uniformly spaced layer set might perform equally well. If the attribution is not informative, the 'video-conditioned policy' is not actually diagnosing evidence flow, and the reported gains could stem solely from the broad video-enhancement/system-suppression intervention applied at any layers. This would not invalidate the empirical improvements, but it would undercut the claimed mechanism and the novelty of VFR.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes VADER, a training-free framework for reducing hallucinations in video large language models (VideoLLMs). VADER combines two modules: Visual Focus Reallocation (VFR), which diagnoses layer-wise visual-to-text attention flow via a gradient-attention saliency measure and adaptively selects decoder layers and a reweighting amplitude to shift pre-softmax attention from system tokens to video tokens; and Selective Evidence Erasure (SEE), which masks per-frame high-importance visual tokens inside the decoder to construct a prior-biased contrastive branch. The two branches are fused through contrastive decoding. Experiments on VidHalluc, VideoHallucer, EventHallusion, and general video understanding benchmarks across three VideoLLMs report substantial gains; the headline result is 72.60% accuracy on EventHallusion with LLaVA-Video-7B, surpassing the training-based methods TPO and RRPO.","tokens_in":13611,"tokens_out":5936,"duration_ms":62765,"significance":"If the results hold, VADER shows that training-free, input-adaptive decoding interventions can improve video grounding without retraining, and the module ablations support the complementarity of VFR and SEE. Strengths include a clearly specified algorithm, a consistent training-free protocol across backbones and benchmarks, and evaluations on both hallucination-specific and general video-understanding tasks. However, the evidence for the adaptive mechanism itself is weaker than the evidence for the overall gains, and the hyperparameter selection procedure calls the headline number into question; both issues need to be addressed before the central claims are fully convincing.","major_comments":[{"comment":"The global configuration (β=0.05, λ=1.5, K_l=5, l*=8, ρ=0.3) and the SEE insertion depth are explicitly selected using EventHallusion with LLaVA-Video-7B, and the same setting is then reported as the headline result (72.60% on EventHallusion in Table 1). This constitutes test-set tuning: the hyperparameters are fit to the benchmark on which the main improvement over TPO and RRPO is claimed. Please either report results with hyperparameters chosen on a disjoint validation split of EventHallusion, or provide a held-out benchmark that was not used for any configuration choice; without this, the comparison on EventHallusion is not conclusive.","section":"§4.1, §4.4, Table 1"},{"comment":"The gradient-attention saliency I^{(l)}_{ij} is computed with respect to the model's own initial prediction ŷ1, which may itself be a hallucination. There is no validation that this saliency identifies the visual-to-text evidence flow that supports the correct answer; the only supporting evidence is the qualitative visualization in Figure 3 and a comparison to a single fixed-layer set in Table 3. If the attribution is noisy or systematically miscalibrated for hallucinated inputs, the adaptive policy could select the wrong layers and the mechanism claim would not hold. Please add a quantitative sanity check (e.g., correlation of saliency with some ground-truth measure of evidence importance, or a control comparing adaptive selection with randomly chosen layer sets) and justify using the first-token diagnostic for all subsequent decoding steps.","section":"§3.2, Eq. (3), Table 3"},{"comment":"The claim that SEE yields a prior-biased branch that is 'difficult to compensate' through cross-frame redundancy is not directly measured. Masking at a single insertion layer l* may be partially undone in higher layers, and the paper does not show that the branch is systematically biased toward language priors beyond the one case study in Figure 5. Please provide an analysis of the negative branch (e.g., agreement between z_prior and a text-only model, or a measure of how much evidence survives the mask across layers), or at least explicitly qualify this claim.","section":"§3.3, Figure 5"}],"minor_comments":[{"comment":"The set T_dec ('decoder-side positions actually intervened by VFR') is never precisely defined; please clarify whether it includes all textual positions (system plus context) and whether it affects rows only for the current decoding token or all rows.","section":"§3.2, Eq. (10)"},{"comment":"No error bars or multiple runs are reported, so the gains of 0.5–2.1 points over the fixed-layer baseline could be within run-to-run variance; deterministic decoding is mentioned for the prior branch, but the main branch sampling strategy is not specified.","section":"§4.2, Tables 3–4"},{"comment":"The term 'source-setting' is used without definition; please clarify what 'source' refers to in this context.","section":"§4.4"},{"comment":"The heatmaps in Figure 3 are difficult to read in print; consider adding color bars and axis labels to make the saliency visualizations interpretable.","section":"Figure 3"},{"comment":"The acronym VADER is used immediately after the expansion 'Video-Adaptive Debiasing via Evidence Reweighting,' but the acronym definition appears in the full text; consider defining it at first use in the abstract for clarity.","section":"Abstract"},{"comment":"The comparison with TPO and RRPO is presented without noting that those methods involve additional training or reward-model supervision; the caption should state this to avoid misleading readers about the training-free claim.","section":"Table 1 caption"}],"recommendation":"major_revision","confidential_remarks":"The paper is potentially interesting, but the test-set tuning issue is significant: the headline EventHallusion number is obtained under hyperparameters selected on that very benchmark, so it should not be presented as a fair comparison with TPO/RRPO until the configuration is chosen without peeking at EventHallusion. The adaptive mechanism also needs stronger empirical support than a single fixed-layer baseline and qualitative saliency maps. If these issues are addressed convincingly, the paper could be a useful contribution to training-free hallucination mitigation for VideoLLMs."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"VADER is a training-free, plug-in method for cutting hallucination in video LLMs by reweighting attention and erasing high-importance tokens. Its two pieces—VFR's per-input layer selection and SEE's per-frame decoder-internal erasure—are genuinely new relative to fixed-layer or input-perturbation baselines like TCD and SEASON. The reported gains are large: +9.03 on EventHallusion for LLaVA-Video-7B, with improvements on VideoHallucer and minor gains on general benchmarks. The ablations show both modules contribute, and the fixed-layer comparison (Table 3) does give some support to adaptive selection. The paper is clearly written and the method is specified precisely enough to reimplement.\n\nWhere I'd push back: the stress-test concern about gradient-attention saliency is real. Eq. 3 uses the model's own prediction as the diagnostic target, and when that prediction is wrong, the gradients may highlight the pathways supporting the hallucination. VFR then amplifies video attention in those layers. The paper shows only qualitative saliency maps and one fixed-layer baseline; there is no causal validation that the adaptive policy is doing what the mechanism claims. This is not fatal to the empirical story—the intervention might still help—but it undercuts the claimed novelty of 'diagnosing evidence flow,' and the adaptive-vs-fixed comparison is a single, arbitrary set.\n\nSecond, more load-bearing: the global configuration (beta, lambda, K_l, l_star, rho) is tuned once on EventHallusion with LLaVA-Video-7B, and the headline 72.60% is on that same benchmark. There are no error bars, no code, no multiple runs. For a training-free decoding method the variance may be low, but benchmark-level overfitting is still possible. The authors say they apply the config unchanged elsewhere, which is honest, but it does not remove the risk. Independent reimplementation is needed before we can treat the gains as solid.\n\nThird, minor: the SEE insertion layer l* is selected on the source setting using the same benchmark, and the text does not report sensitivity for beta or lambda, only for the SEE keep ratio.\n\nOverall: worth taking seriously as a CVPR-style contribution, but I would not cite the headline numbers until code and/or error bars appear. My recommendation: send to peer review—there is enough substance and novelty that a referee can test the mechanism claims—but ask for code and a deeper ablation of the adaptive policy, ideally with a null test where layers are chosen randomly or by a fixed uniform schedule.","headline":"Training-free hallucination mitigation with real gains, but the adaptive mechanism is under-validated and the headline result is tuned on the same benchmark it leads with.","tokens_in":14247,"tokens_out":1934,"would_cite":false,"duration_ms":22309,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A training-free decoder intervention that diagnoses each video–question pair and reweights attention toward video evidence can reduce event-level hallucination in video language models, reaching 72.60% on EventHallusion with LLaVA-Video-7B.","keywords":["video hallucination","contrastive decoding","attention reweighting","training-free debiasing","event-level grounding","temporal consistency","gradient saliency","large vision-language models"],"falsifier":"Replace VADER's adaptive layer selection with attention-mass-only selection and with a random selection of five layers on EventHallusion while keeping SEE fixed; if either replacement matches the adaptive result within noise, the attribution signal is not doing the work the paper assigns it.","tokens_in":13166,"feed_emoji":"🎬","tokens_out":9369,"duration_ms":77555,"temperature":0.7,"pith_summary":"VADER is a training-free decoding method that tries to stop video language models from producing fluent answers unsupported by the video. Its central claim is that hallucination can be reduced by making the intervention per-input rather than fixed: for each video–question pair, the model's own attention and gradients identify which decoder layers carry visual-to-text evidence, and those layers get a calibrated reallocation of pre-softmax attention from system tokens toward video tokens. A second branch erases the most-attended visual tokens within each frame, so predictions that stay confident without that evidence are exposed as prior-driven and suppressed by contrastive decoding. The paper argues this works because video frames are redundant enough that input-level perturbations can be recovered, while decoder-internal, per-frame erasure cannot be. If correct, this gives a plug-and-play way to improve grounding and temporal consistency across existing video LLMs without retraining.","feed_headline":"Training-free VADER hits 72.6% on event-level video hallucination","feed_subtitle":"A per-video diagnostic reweights attention to visual evidence, then subtracts a prior-biased branch to suppress unsupported answers.","key_machinery":"The load-bearing object is the gradient-attention saliency tensor $I^{(l)}_{ij}=\\frac{1}{H}\\sum_h |A^{(l,h)}_{ij} \\odot \\partial \\mathcal{L}_{\\mathrm{diag}}/\\partial A^{(l,h)}_{ij}|$, computed from the model's own first prediction $\\hat y_1$; it claims to measure which token interactions are both attended to and influential for the current output. VFR reads the cross-modal part of this tensor (text positions attending to video positions) to choose the intervention layers and the logarithmic amplitude $a=\\log(1+\\beta x)$. SEE's mechanism is an additive $-\\infty$ mask on the attention-score columns of high-importance video tokens at a fixed insertion layer $l^\\star$, with a per-frame retention budget that keeps the lowest-attended tokens in every frame. The contrastive fusion $\\bar z=(1+\\lambda)z_{\\mathrm{gnd}}-\\lambda z_{\\mathrm{prior}}$ is the final mechanism that converts the two branches into a suppression rule.","core_discovery":"On its own terms, the paper establishes that a dual-branch contrastive decoding scheme, with both branches constructed inside the frozen decoder, meaningfully reduces event-level hallucination in video LLMs. The grounded branch (VFR) uses gradient–attention saliency to select the top-K layers with the strongest cross-modal flow and reweights the system-token and video-token blocks of the pre-softmax score matrix by $(1-a)$ and $(1+a)$. The prior branch (SEE) masks, at a fixed insertion layer, the highest-attended visual tokens per frame and keeps low-attended context, producing logits that reflect language priors; the contrast $\\bar z=(1+\\lambda)z_{\\mathrm{gnd}}-\\lambda z_{\\mathrm{prior}}$ suppresses candidates that remain confident after erasure. On LLaVA-Video-7B, the method reaches 72.60% accuracy on EventHallusion, above the training-based TPO and RRPO, and it improves average VideoHallucer scores by +7.29 points while leaving MVBench, VideoMME, and TVBench averages positive or neutral.","pith_inferences":["The gradient-attention saliency is never validated against a ground-truth attribution; a direct test would be to replace the saliency-based layer selection with attention-mass-only selection and with random layers on the same benchmarks, which would show how much of the gain depends on the attribution being correct.","Because the method is training-free and architecture-agnostic, the same two-branch recipe may transfer to other multimodal decoding settings, such as image-based LVLMs with long contexts or audio-visual models, where per-frame redundancy has a distinct analogue.","The contrastive formulation suggests a testable prediction: VADER should help most on questions where the wrong answer is supported by coarse scene context and where the correct answer depends on localized, sparse evidence — a comparison the paper only illustrates qualitatively in Figure 5."],"forward_implications":["On LLaVA-Video-7B, VADER improves EventHallusion accuracy from 63.57% to 72.60%, surpassing the training-based TPO (63.33%) and RRPO (67.97%) without any parameter updates.","VADER improves average VideoHallucer accuracy by +3.90, +10.30, and +7.29 points on Qwen3-VL-8B, LLaVA-OneVision-7B, and LLaVA-Video-7B, with the largest gains on event-ordering and temporal-consistency subsets.","Ablations show that the adaptive policy matters: fixed-layer VFR yields smaller gains than per-video layer selection, and VFR-only or SEE-only settings each underperform the combined method.","General video understanding is preserved: on MVBench, VideoMME, and TVBench, VADER raises the average by +0.6 and +1.5 points on LLaVA-OV-7B and LLaVA-Video-7B, while several competing interventions are neutral or slightly negative."],"supporting_citations":[{"why":"It supplies the EventHallusion benchmark, which measures event-level hallucination, and the TCD contrastive-decoding baseline that VADER must beat.","marker":"Zhang et al. 2024a"},{"why":"It provides the VidHalluc benchmark and the DINO-HEAL training-free baseline used in the main comparisons.","marker":"Li, Im, and Fazli 2025"},{"why":"It defines VideoHallucer, the benchmark whose temporal and event-level subsets show VADER's largest gains.","marker":"Wang et al. 2024c"},{"why":"It supplies LLaVA-Video-7B, the primary backbone on which VADER reaches 72.60% on EventHallusion and on which the global configuration is calibrated.","marker":"Zhang et al. 2024c"},{"why":"It supplies Qwen3-VL-8B, one of the three video LLM backbones used to show the method transfers across architectures.","marker":"Bai et al. 2025a"},{"why":"It supplies LLaVA-OneVision-7B, another backbone in the evaluation, used to check cross-model generality.","marker":"Li et al. 2024a"},{"why":"It is TPO, the training-based preference-optimization method whose EventHallusion accuracy VADER surpasses without retraining.","marker":"Li et al. 2025"},{"why":"It is RRPO, the training-based baseline that VADER outperforms on EventHallusion with LLaVA-Video-7B.","marker":"Sarkar and Etemad 2025"},{"why":"It is SEASON, a training-free self-diagnostic contrastive method that provides the strongest reference point on LLaVA-OV-7B.","marker":"Wu et al. 2025a"}],"fun_headline_variants":["Adaptive debiasing cuts video hallucination, hits 72.6% accuracy","VADER's adaptive debiasing hits 72.6% on EventHallusion","No-training VADER uses dual-branch contrast to hit 72.6% event accuracy","Training-free VADER debiases evidence flow, hits 72.6% on video hallucination"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the product of attention mass and gradient magnitude, read from the model's own first prediction, reliably shows which decoder layers and which video tokens actually drive the current answer.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive debiasing cuts video hallucination, hits 72.6% accuracy","VADER's adaptive debiasing hits 72.6% on EventHallusion","No-training VADER uses dual-branch contrast to hit 72.6% event accuracy","Training-free VADER debiases evidence flow, hits 72.6% on video hallucination"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.002547,"raw_usage":{"total_tokens":9790,"prompt_tokens":1010,"completion_tokens":8780,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":626,"completion_tokens_details":{"reasoning_tokens":8682}},"tokens_in":626,"tokens_out":8780,"duration_ms":65721,"temperature":1.0,"reasoning_tokens":8682,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:28:40.644781+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replace VADER's adaptive layer selection with attention-mass-only selection and with a random selection of five layers on EventHallusion while keeping SEE fixed; if either replacement matches the adaptive result within noise, the attribution signal is not doing the work the paper assigns it.","supporting_citations":[],"review_version":1}