{"id":"27ab131e-6c25-450a-b123-5ff6e04f123c","arxiv_id":"2505.03420","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"On LLaVA and InstructBLIP, updating 0.003 percent of parameters at test time with rewards from a trained CLIP evaluator reduces object hallucination rates by about 15 to 17 percent in single-run AMBER results.","lead":"This paper introduces a test-time method that updates only tiny layer-normalization weights of a vision-language model, guided by reinforcement rewards from a CLIP-based evaluator, to cut object hallucinations in image captions by 15 to 17 percent. The appeal is that deployed models could be improved per image without retraining or extra models.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The policy-gradient derivation in §III-A is not instantiated by Algorithm 1: beam search is not sampling from πθ, so the implemented update is not ∇θJ(θ) and the reported reductions cannot be attributed to the stated RL objective.","rationale":"The reader's weakest assumption identifies the same load-bearing concern I would raise: the RL derivation in Section III-A is the paper's only formal support for the claim that updating LayerNorm gamma with a policy-gradient step reduces hallucinations. The mismatch between the expectation under πθ and Algorithm 1's beam-search candidate generation is concrete and directly attacks the stated mechanism. If the gradient estimator is invalid, the paper's central contribution is not a policy-gradient RL method as claimed, even if the empirical numbers are reproducible as a heuristic. The implausible VCD baseline in Table III and the absence of code and error bars are additional serious issues, but the estimator flaw is the most fundamental because it undermines the interpretation of every self-comparison in Table I. My read does not change the reader's rejection; if anything, the beam-search/RL mismatch makes the request for corrected derivations and code essential before the work can be assessed.","tokens_in":9188,"tokens_out":10006,"duration_ms":113337,"concrete_test":"Re-run Algorithm 1 exactly as written, but replace beam search with B ancestral samples drawn from the current policy πθ at each of the 5 adaptation steps, keeping the same LayerNorm-gamma update, reward model, learning rates, and AMBER evaluation. If the CHAIR/Hal reductions reported in Table I vanish or reverse, the beam-search candidate distribution is the active ingredient rather than the stated policy-gradient objective; if the gains persist, the method still requires an off-policy/importance-weighting derivation before the RL claim can stand.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III-A defines the objective as J(θ)=E_{πθ}[r(y,v)] and the gradient as ∇θJ(θ)=E_{πθ}[∇θ log π(y|s;θ)·r(y,v)], which requires captions sampled from the policy. Algorithm 1 line 3 instead generates B candidate captions using deterministic beam search. Beam-search outputs are not samples from the softmax policy πθ, and no importance-weighting or off-policy correction is supplied, so the loss Lpolicy=−log π(y|s;θ)·r(y,v) is not an unbiased estimator of ∇θJ(θ). This is not a cosmetic mismatch: beam search preferentially selects high-likelihood sequences, so the update reinforces the beam-selected set even before reward information enters, and the gradient is confounded with the candidate-generation distribution. Consequently, the reported CHAIR and Hal improvements in Table I cannot be attributed to optimizing the stated reinforcement-learning objective without a corrected derivation or a sampling-based implementation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a test-time adaptation (TTA) method to mitigate object hallucinations in vision-language models (VLMs). The method freezes all model parameters except layer-normalization gamma parameters (approximately 0.003% of the total), generates B candidate captions via beam search, evaluates them with a custom CLIP-based hallucination evaluation model that produces Semantic Alignment Scores (SAS) and Non-Hallucination Probability (NHP) rewards, and updates the gamma parameters with a policy-gradient-style loss. Experiments on the AMBER benchmark report reductions in CHAIR, Hal, and Cog for LLaVA-7B and InstructBLIP-7B, and claim substantial improvements over the Visual Contrastive Decoding (VCD) baseline. Evaluation is performed on external AMBER ground truth, so the headline results are not circular with respect to the reward model's own outputs.","tokens_in":9408,"tokens_out":5063,"duration_ms":48281,"significance":"The idea of adapting only layer-normalization parameters at test time is interesting and potentially efficient, and the within-model comparisons show consistent directional improvements on CHAIR, Hal, and Cog for both models. The use of an external benchmark (AMBER) for evaluation is a strength and avoids immediate circularity. However, the paper's central mechanism is undermined by the mismatch between the stated policy-gradient derivation and the beam-search-based implementation, and the SOTA comparison contains implausible baseline numbers. If these issues are resolved, the method could be a useful lightweight hallucination-mitigation technique; as it stands, the theoretical and empirical support are insufficient.","major_comments":[{"comment":"The policy-gradient update in §III-A is not an unbiased estimator of ∇θJ(θ) because Algorithm 1 generates candidate captions with deterministic beam search rather than sampling from πθ. The objective J(θ)=Eπθ[r(y,v)] and gradient ∇θJ(θ)=Eπθ[∇θ log π(y|s;θ) r(y,v)] require samples from the policy. Beam search is a deterministic argmax-like procedure, and without importance weights or an off-policy correction, the loss Lpolicy=−logπ(y|s;θ) r(y,v) is not an unbiased estimate of the policy gradient. The beam-search selection may reinforce high-likelihood sequences independently of reward, confounding the reported gains. The authors should either sample from the policy, apply an appropriate off-policy correction, or re-derive the method as a deterministic risk/reranking objective and explain the observed improvements in those terms.","section":"III-A, Algorithm 1"},{"comment":"The VCD baseline numbers in Table III are inconsistent with the unadapted model in Table I. For LLaVA-7B, Table I reports w/o TTA as CHAIR=7.8, Cover=51.0, Hal=36.4, Cog=4.2, while Table III reports w/ VCD as CHAIR=80.5, Cover=10.3, Hal=97.5, Cog=9.2. VCD is designed to mitigate hallucinations, so such a large degradation is implausible unless the VCD implementation is broken, hyperparameters are inappropriate, or the evaluation protocol differs. This invalidates the claimed 91.7% CHAIR reduction and 68.4% Hal improvement over SOTA. The authors must report a correctly configured VCD baseline with the same evaluation code, include implementation details, and verify the numbers.","section":"Table III"},{"comment":"No error bars, confidence intervals, or significance tests are reported for any of the main results. The improvements are small in absolute terms (for example, CHAIR 7.8→6.7 for LLaVA-7B), so without repeated runs or significance testing the robustness of the claim is unclear. In addition, several hyperparameters (β, learning rates, beam size B=5, number of adaptation steps=5, and the reward normalization procedure) are set without sensitivity analysis; the method's sensitivity to these choices should be examined.","section":"Tables I, IV; §IV hyperparameters"}],"minor_comments":[{"comment":"Reference [8] does not point to InstructBLIP; it cites a survey. The correct InstructBLIP paper (Dai et al.) should be cited.","section":"References"},{"comment":"The normalization of SAS and NHP scores is only described in prose as subtracting the mean across candidates; please define the normalization operator explicitly in an equation.","section":"III-B"},{"comment":"The description of using 'LLaMA3 to generate negative image-caption pairs' lacks details; please specify the prompts, filtering criteria, and how you ensure the generated negatives contain object hallucinations rather than merely being dissimilar.","section":"IV"},{"comment":"Step 8 says 'Generate the refined caption y* using the updated VLM' but does not specify the decoding method; please state whether beam search with the same B is used or a different decoding strategy, and ensure the evaluation protocol matches.","section":"Algorithm 1"},{"comment":"The comparison between CLIP-based models and LLaVA-based models for hallucination detection is potentially misleading because LLaVA is evaluated on a binary classification task it was not trained for; please clarify the task setup or remove this comparison if the settings are not comparable.","section":"Table V"},{"comment":"The abstract claims 'outperforms state-of-the-art baselines with a 68.3% improvement', but this number depends on the implausible VCD baseline in Table III; please qualify or revise the claim.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The central technical claim relies on a policy-gradient derivation that is not instantiated by the beam-search algorithm, and the VCD baseline numbers in Table III are implausible. Both are load-bearing and must be addressed. I recommend major revision with a request to either correct the derivation or revise the algorithm, and to rerun the VCD comparison with a verified implementation. If the corrected numbers no longer support the stated improvements, the contribution may need to be reframed or further validated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear Colleague,\n\nQuick take: the headline claim—that updating only LayerNorm gamma with a CLIP-based reward reduces object hallucination rates by 15–17% on LLaVA and InstructBLIP—may be real, but this paper doesn't yet support it. The RL derivation in III-A doesn't match Algorithm 1, and the VCD baseline in Table III is so implausible that it casts doubt on the whole comparison.\n\nWhat's new: adapting a VLM at test time by updating only LayerNorm gamma (about 0.003% of parameters) to reduce hallucination is a neat, parameter-efficient idea. I haven't seen that exact combination. The custom evaluator—frozen CLIP plus one learnable query token, multi-stage cross-attention, triplet loss, and dual SAS/NHP rewards—is also a legitimate new component. Credit where due: the main results are evaluated on AMBER ground truth, not on the reward model's own scores, so the central empirical claim isn't circular. The paper is also honest about scope, limiting itself to object hallucinations.\n\nWhere it falls apart: Algorithm 1 generates candidate captions with beam search, but Section III-A defines the policy gradient as an expectation under the policy's softmax. Beam search is deterministic and does not sample from πθ, so the implemented update is not ∇θJ(θ) as stated. The authors need to either sample from πθ or reframe the loss as reward-weighted regression on beam-selected outputs. This isn't cosmetic—it changes what the method actually is. Second, Table III shows VCD on LLaVA with CHAIR 80.5 and Hal 97.5, while the unadapted LLaVA in Table I has CHAIR 7.8 and Hal 36.4. That's backwards. VCD is designed to reduce hallucination, not make it an order of magnitude worse. Either the numbers are misreported or the baseline is broken. Third, there are no error bars or significance tests on any of the improvements, and no code is released. Given the effect sizes (a few points on CHAIR), we can't tell if the gains are stable.\n\nThe paper deserves a serious referee because the idea is interesting and the flaws are identifiable and fixable. But I wouldn't accept it as-is. The authors need to correct the derivation (or reframe), rerun the VCD baseline properly, add variance estimates, and release code. Then the empirical claim can actually be checked.\n\nRecommendation: send to peer review with expectation of major revision. If the derivation and baseline issues are addressed, this could be a useful contribution to the hallucination-mitigation subfield.","headline":"Plausible idea and honest AMBER evaluation, but the RL derivation doesn't match the algorithm and the VCD baseline is implausible—needs major revision before it can be believed.","tokens_in":9914,"tokens_out":3872,"would_cite":false,"duration_ms":34432,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that updating only LayerNorm gamma parameters with a policy-gradient step, guided by a CLIP-based dual reward, reduces object hallucination in vision-language captioning at test time, reporting CHAIR reductions of 14.1%…","keywords":["vision-language models","hallucination mitigation","test-time adaptation","reinforcement learning","layer normalization","CLIP reward","image captioning","object hallucination"],"falsifier":"On a held-out set of AMBER images, compare three variants: the full TTA pipeline, a variant that scores the same beam candidates but simply picks the highest-reward caption without any LayerNorm update, and a variant that updates a random 0.003% of parameters instead of LayerNorm gamma. If the re-ranking or random-parameter variant matches the full pipeline's CHAIR and Hal reductions, the claimed policy-gradient-to-LayerNorm mechanism is not what is causing the gains.","tokens_in":8982,"feed_emoji":"🖼️","tokens_out":6208,"duration_ms":58114,"temperature":0.7,"pith_summary":"The paper proposes a test-time adaptation framework that reduces object hallucinations in vision-language captioning without retraining or auxiliary models. The captioning model is treated as a reinforcement-learning policy, and for each test image only the layer-normalization gamma parameters, about 0.003% of the total, are updated. The reward comes from a lightweight CLIP-based evaluator that combines a semantic-alignment score with a non-hallucination probability. On LLaVA 7B and InstructBLIP 7B, the authors report CHAIR reductions of 14.1% and 26.1%, Hal reductions of 15.4% and 17.3%, and a roughly 68% lower hallucination rate than a contrastive-decoding baseline. The paper's point is that tiny, sample-specific normalization updates can steer a frozen pretrained model away from hallucinated objects.","feed_headline":"Only 0.003% of weights need updating to cut VLM caption hallucinations by 26%","feed_subtitle":"Test-time reinforcement learning adjusts layer-norm gamma per image, with no retraining or extra models.","key_machinery":"The load-bearing machinery is the pairing of two components: first, the LayerNorm gamma parameters as the only learnable parameters of the frozen VLM, providing a low-dimensional steering mechanism during inference; second, a hallucination evaluation model built from a frozen CLIP with one learnable query token refined through N-stage cross-attention and trained with a triplet loss, which outputs the dual rewards SAS and NHP. The policy-gradient objective $L_{\\mathrm{policy}} = -\\mathbb{E}_{\\pi_\\theta}[\\log \\pi(y|s;\\theta)\\, r(y,v)]$ is what formally converts those rewards into gamma updates, with beam search supplying the candidate captions.","core_discovery":"The central discovery the paper aims to establish is that object hallucinations in deployed VLMs can be mitigated at inference time by treating the VLM as a policy and applying policy-gradient updates to only the LayerNorm gamma parameters. For each test image, the policy generates B beam-search captions; a CLIP-based hallucination evaluator scores each candidate with dual rewards, the Semantic Alignment Score from cosine similarity and the Non-Hallucination Probability from a classifier logit; the aggregated reward then drives a policy-gradient loss that updates gamma. After each sample the parameters reset to their pretrained values. The paper reports consistent reductions on CHAIR, Hal, and Cog for both LLaVA 7B and InstructBLIP 7B, and claims the approach outperforms visual contrastive decoding on these metrics.","pith_inferences":["Because the policy-gradient estimate is computed from deterministic beam-search outputs rather than sampled from $\\pi_\\theta$, the update may not truly maximize expected reward; a simpler best-of-B re-ranking guided by the same dual rewards might reproduce the reported gains.","The LayerNorm-gamma update may act as a test-time feature re-scaling that shifts activation distributions toward the current image, which would connect this work to test-time normalization methods even outside the RL framing.","The dual-reward evaluator could be combined with the same loop to address attribute and relation hallucinations, which the paper leaves to future work.","Varying the number of adaptation steps or the beam size could reveal whether hallucination reductions saturate or reverse, separating the contribution of the per-sample update from the evaluator's ranking quality."],"forward_implications":["Deployed captioning models can be adapted per test image without retraining, making hallucination mitigation available on demand in high-stakes applications.","Because only LayerNorm gamma changes and is reset after each sample, the pretrained model's general knowledge is preserved while sample-specific biases are not accumulated.","The dual-reward design shows that semantic alignment alone is insufficient: adding the non-hallucination classifier logit yields measurable further reductions in CHAIR, Hal, and Cog.","The framework transfers across two different 7B-scale VLMs, suggesting the parameter-efficient update is not tied to a single architecture.","The CLIP-based evaluator, trained on object, attribute, and relation hallucinations, can be reused for broader hallucination detection beyond the current scope."],"supporting_citations":[{"why":"Supplies the visual-contrastive-decoding baseline the paper must outperform and whose limitations in auto-regressive generation motivate the TTA design.","marker":"[4]"},{"why":"The LLaVA 7B captioning model used as one of the two policy models for test-time adaptation.","marker":"[7]"},{"why":"The InstructBLIP 7B testbed as cited by the paper, the second model on which the method is evaluated.","marker":"[8]"},{"why":"Provides the policy-gradient theorem that gives the RL update rule for the gamma parameters.","marker":"[20]"},{"why":"CLIP, whose frozen vision and text encoders supply the embeddings and cosine-similarity used to compute the SAS reward and to build the evaluator.","marker":"[21]"},{"why":"The AMBER benchmark whose CHAIR, Hal, Cover, and Cog metrics are used to measure hallucination reduction.","marker":"[24]"},{"why":"The PixelProse subset used to train and test the CLIP-based hallucination evaluation model.","marker":"[25]"},{"why":"LLaMA3 is used to generate negative image-caption pairs containing hallucinations for the triplet training of the evaluator.","marker":"[26]"}],"fun_headline_variants":["Tweak 0.003% of weights to cut VLM hallucinations by up to 17%","Test-time RL on layer norms reduces VLM captions' hallucinations","Adjust only layer-norm gamma to slash VLM hallucinations","Zero retraining: 0.003% parameter update cuts VLM hallucination rates","Policy-gradient layer-norm tweak beats SOTA on VLM hallucination"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The derivation treats a policy-gradient update computed from beam-search candidate captions as a valid estimate of the expected-reward gradient, but beam search is deterministic and is not sampling from the model's own distribution.","fun_headline_variants_meta":{"raw":{"variants":["Tweak 0.003% of weights to cut VLM hallucinations by up to 17%","Test-time RL on layer norms reduces VLM captions' hallucinations","Adjust only layer-norm gamma to slash VLM hallucinations","Zero retraining: 0.003% parameter update cuts VLM hallucination rates","Policy-gradient layer-norm tweak beats SOTA on VLM hallucination"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000384,"raw_usage":{"total_tokens":2006,"prompt_tokens":892,"completion_tokens":1114,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":508,"completion_tokens_details":{"reasoning_tokens":1026}},"tokens_in":508,"tokens_out":1114,"duration_ms":8474,"temperature":1.0,"reasoning_tokens":1026,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:52:02.797287+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a held-out set of AMBER images, compare three variants: the full TTA pipeline, a variant that scores the same beam candidates but simply picks the highest-reward caption without any LayerNorm update, and a variant that updates a random 0.003% of parameters instead of LayerNorm gamma. If the re-ranking or random-parameter variant matches the full pipeline's CHAIR and Hal reductions, the claimed policy-gradient-to-LayerNorm mechanism is not what is causing the gains.","supporting_citations":[{"cited_title":"Miti- gating object hallucinations in large vision-language models through vi- sual contrastive decoding,","cited_arxiv_id":null,"evidence_quote":"Supplies the visual-contrastive-decoding baseline the paper must outperform and whose limitations in auto-regressive generation motivate the TTA design."},{"cited_title":"Llama 3.2 multimodal (version 2023),","cited_arxiv_id":null,"evidence_quote":"LLaMA3 is used to generate negative image-caption pairs containing hallucinations for the triplet training of the evaluator."}],"review_version":1}