{"id":"945d6643-44a0-4974-a420-e7401745bdf4","arxiv_id":"2411.17760","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A judge-free self-improvement pipeline for multimodal LLMs that generates hallucinated caption pairs with a controlled decoding ratio, filters and swaps them with CLIP scores, and trains with DPO, reporting reduced hallucination on Object HalBench and a new IC dataset.","lead":"Researchers propose a way for multimodal AI models to improve their image descriptions without using another large AI model as a judge. They generate deliberately flawed captions with a controllable 'hallucination knob', use a lightweight image-text similarity scorer to pick the better caption of each pair, and fine-tune the model to prefer accurate captions.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported gains likely reflect selection of the best of 10 splits and 3 distributions; no error bars or validation-based model choice are given, so the central empirical claim is not established.","rationale":"The reader's weakest_assumption focuses on whether average sentence-level CLIPScore reliably identifies hallucinated captions, which is an important mechanistic risk. However, the single most load-bearing concern for the central empirical claim is the evaluation protocol: the reported numbers appear to be the best F1 over 10 splits and 3 sampling distributions, selected on the same test set, with no error bars or validation-based decision rule. This directly undermines the claim that the method 'outperforms conventional techniques' and 'achieves superior precision and recall.' The reader's rationale does mention test-set selection and missing error bars, but the reader's formal weakest_assumption is about CLIPScore. Therefore I partially agree. I did not find a machine-checked proof or reproducible code that would independently support the headline numbers. The concern is concrete and falsifiable by re-reporting the full distribution of results or by using a held-out validation set for model selection. Since the reader already recommended REJECT and this stress-test confirms that the reported evidence is not trustworthy as presented, the verdict remains REJECT (UNCHANGED from the reader's verdict).","tokens_in":14633,"tokens_out":7677,"duration_ms":70238,"concrete_test":"Recompute Table 2 by reporting the mean and standard deviation of F1 over all 10 splits for each hratio distribution, and explicitly state which split and distribution produced the Object HalBench numbers in Table 1. Then apply a pre-registered model-selection rule: hold out a random half of the IC dataset as validation, choose the split and distribution with the highest validation F1, and evaluate the chosen model on the held-out test half only. If the mean F1 is within noise of the 6.58 baseline, or the held-out test gain is not clearly positive, the claimed improvement is a selection artifact rather than a property of the method.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of significantly better performance rests on numbers that appear to be the maximum over many evaluated configurations. Section 4.2 states that the 100k generated pairs are sorted by CLIP-score difference and organized into 10 splits, and a separate LLaVA model is trained for each split. Figures 4-6 report the 'Best F1' of 7.70, 7.67, and 7.76 for fixed-hratio, uniform, and Gaussian sampling, respectively. Table 2 lists '+Ours' as achieving 7.76 F1, matching the best Gaussian split. Thus the headline result is a maximum over 10 splits and 3 distributions, all evaluated on the same IC test set, without reporting the other folds or any variance estimate. The Object HalBench result (Table 1) does not state which split produced it, but if it is from the same model selected on IC, the reported 51.6->9.4 drop is also from a single cherry-picked run. This makes the empirical support for the framework unverifiable: even a method with no true improvement would be expected to yield a best-of-10 F1 well above its mean, and the paper never shows the mean or spread. This concern is independent of whether CLIPScore is a valid preference signal; even if the mechanism is sound, the demonstrated superiority is not established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a model-level judge-free self-improvement framework for multimodal large language models. It generates preference pairs by mixing conditional and unconditional decoding paths with a controllable hallucination ratio, then filters and swaps pairs using average sentence-level CLIPScore, and finally trains the seed LLaVA-1.5-13B model with DPO. The authors introduce a 150-image IC dataset evaluated by GPT-4o for precision and recall, and report large reductions on Object HalBench (51.6/24.6 to 9.4/5.1) and an F1 improvement on IC (6.58 to 7.76). The core claim is that this framework outperforms conventional techniques without requiring an MLLM judge, at significantly lower computational cost.","tokens_in":14930,"tokens_out":7071,"duration_ms":69338,"significance":"If the empirical claims were properly supported, the framework would be a valuable contribution: it removes the MLLM judge from the verification loop, proposes a simple controllable procedure for negative generation, and introduces a new hallucination-focused benchmark with both precision and recall. The pipeline is clearly described, and the qualitative examples and the explicit limitation section are useful. However, the evidence as presented is not sufficient to establish the central claim. The headline numbers appear to be selected over many test-set configurations, no error bars or significance tests are provided, the closest prior work (CLIP-DPO) is not evaluated as a baseline, and the validity of the CLIPScore-based preference signal is not quantitatively checked. These are experimental-protocol issues rather than flaws in the formulation, so the approach remains promising but unproven.","major_comments":[{"comment":"The reported F1 of 7.76 in Table 2 is the maximum over the ten CLIP-difference splits for the Gaussian hratio distribution, as shown in Fig. 6. The paper does not report the mean or variance over the other splits, nor does it specify a held-out validation set for selecting among the ten splits and three distribution families. Consequently, the headline improvement from 6.58 to 7.76 is a best-of-30 selection on the IC test set, and the claim that the framework outperforms conventional techniques is not established by the presented evidence. Please report all per-split results with error bars, select the final model on a validation set, and state the selection rule explicitly.","section":"§4.2, Table 2, Figs. 4–6"},{"comment":"The paper cites CLIP-DPO [16] as closely related work and states that the proposed approach extends it to long captions, but CLIP-DPO is never evaluated as a baseline. Table 1 mixes different model families, sizes, training data, and feedback types, so the comparison does not support the claim that the proposed method outperforms conventional techniques. A matched comparison using the same seed model, data budget, and evaluation protocol is necessary; otherwise the 9.4/5.1 Object HalBench result has no controlled reference point.","section":"§4.3, Table 1"},{"comment":"The controllability of the hallucination ratio is not quantitatively validated. The paper claims that higher hratio injects more hallucination, but the only evidence is the qualitative reconstruction in Fig. 3. Similarly, the swap rule in §3.4 rests on the assumption that average sentence-level CLIPScore orders captions by faithfulness; the paper reports no swap rate, no agreement with object-level annotations or human judgements, and no oracle ablation. Without such validation, the preference labels may be noisy, and the observed gains cannot be reliably attributed to the proposed inversion and filtering mechanism.","section":"§3.3, §3.4"},{"comment":"The IC dataset contains only 150 images, GPT-4o scoring is stochastic, and the paper provides no confidence intervals, repeated evaluations, or significance tests. The reported difference between 6.58 and 7.76 F1 could be within rating noise. The Object HalBench result is likewise a single run with no seeds or variance reported. Statistical reliability is essential for the central empirical claim; please provide multiple evaluation runs, inter-annotator agreement for the GPT-4o judge, and appropriate uncertainty measures.","section":"§4.1, §4.2, §4.3"},{"comment":"The experimental setup omits several details needed for reproducibility: the DPO hyperparameters (β, learning rate, number of epochs), the exact boundaries of the ten CLIP-score-difference splits, and the random seeds used for data generation and training. Without these, another group cannot reproduce the results. Please report these details and, if possible, release the IC dataset and code.","section":"§4.2"}],"minor_comments":[{"comment":"The category name \"veichle\" should be corrected to \"vehicle\".","section":"Table 3"},{"comment":"The y-axis label \"Score (%)\" conflicts with the 1-to-10 scale described in §4.3 and Table 2; please clarify the units or remove the percentage sign.","section":"Figs. 4–6"},{"comment":"The text \"llavav1 5 mix665k\" should be formatted as \"LLaVA-v1.5-mix665k\" for readability.","section":"§4.2"},{"comment":"The distributions pc and pu in Eq. (3) are not formally defined; please state that they are per-token vocabulary distributions and clarify how the weighted combination is computed when the two paths have different tokenizers.","section":"§3.3"}],"recommendation":"major_revision","confidential_remarks":"For the editor: I am concerned that the manuscript reports a best-of-30 test-set result without disclosing the selection rule. After a revision that reports all splits, uses a validation-based selection protocol, and adds matched baselines such as CLIP-DPO, the headline numbers may change materially. The authors should be encouraged to provide the complete per-configuration results and statistical measures."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the core idea is reasonable and the price is right, but the paper does not show what it claims. The headline IC F1 of 7.76 is the best among 10 splits and three hratio distributions, all evaluated on the same test set. Table 2 reports that maximum without saying so. Object HalBench's dramatic drop (51.6 to 9.4) is presumably from the same selected run, so the central result is unverifiable.\n\nCredit: the pipeline is coherent and genuinely judge-free: mixing conditional and unconditional decoding with a controllable hratio, then swapping preference labels when average sentence-level CLIPScore disagrees, then DPO. The swap-based inversion for long captions is a reasonable extension of CLIP-DPO's idea. The new IC dataset with precision/recall via GPT-4o is a useful testbed, though it needs human validation. The qualitative reconstructions are suggestive.\n\nSoft spots: (1) selection on the test set, no disclosure in Table 2; (2) no error bars or significance tests on 150 images; (3) no direct comparison to CLIP-DPO or other self-improvement baselines; (4) CLIPScore as a reliable preference signal for long captions is not validated, and the swap rate is not reported; (5) the controllable hallucination ratio is not quantitatively validated.\n\nWho is this for? People working on MLLM alignment/hallucination might find the efficiency angle attractive, and the dataset could be reused. But as it stands, the claimed gains are not established. It deserves a serious referee only if the authors can supply a proper protocol: report all splits, means and variances, and include a CLIP-DPO baseline. My recommendation: ask for a major revision with those changes; if the authors won't release the per-split numbers, I'd skip it.","headline":"A cheap judge-free self-improvement recipe that is likely oversold: the headline F1 is the best of 30 evaluated configurations, with no error bars or controlled baselines.","tokens_in":15454,"tokens_out":2867,"would_cite":false,"duration_ms":26826,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A model can improve its own image captioning without any AI judge, using only a lightweight CLIP score.","keywords":["multimodal large language models","self-improvement","hallucination mitigation","preference optimization","direct preference optimization","CLIPScore","controllable decoding","judge-free"],"falsifier":"Rate a random sample of the preference pairs with human annotators who mark every hallucinated object or attribute. If the CLIPScore ordering agrees with the human ordering no better than chance, or if DPO trained on pairs sorted by a random or human-inverted score retains the same gains, then the reported improvements cannot be attributed to the CLIP-based label inversion.","tokens_in":14460,"feed_emoji":"🖼️","tokens_out":4839,"duration_ms":42221,"temperature":0.7,"pith_summary":"The paper claims that an MLLM can improve its own captioning by generating preference pairs through a controllable hallucination mechanism, cleaning those pairs with a lightweight CLIP-based verifier, and training with DPO—no large-model judge required. The method mixes conditional and unconditional token distributions with a hallucination ratio to create negative captions, then uses average sentence-level CLIPScore to swap any pair whose negative scores higher than its positive. The authors report that LLaVA-1.5-13B drops from 51.6 to 9.4 on Object HalBench response-level metric, and its precision-recall F1 on their new IC dataset rises from 6.58 to 7.76. If valid, this is a cheaper route to self-improvement that avoids the cost and bias of MLLM judges.","feed_headline":"Judge-free self-training cuts LLaVA hallucinations fivefold","feed_subtitle":"A lightweight CLIP score verifies preference pairs, replacing expensive MLLM judges in the loop.","key_machinery":"The load-bearing mechanism is a hallucination-controlled decoding path combined with a lightweight verifier. A hallucination ratio hratio governs the interpolation $t = (1 - h_{\\text{ratio}}) \\cdot t_c + h_{\\text{ratio}} \\cdot t_u$ between the conditional token distribution (image + instruction) and the unconditional distribution (instruction only), producing negative captions that are more hallucinated as hratio grows. The verifier is average sentence-level CLIPScore, computed per caption via a frozen contrastive image-text encoder, which decides whether the initial positive is actually better than the initial negative; if not, the pair is swapped. DPO then trains the seed model on the filtered pairs. The identity linking them is that a cheap, sentence-averaged CLIP score can stand in for an MLLM judge in deciding preference direction.","core_discovery":"The central claim is that a model-level judge is not necessary for self-improvement of multimodal LLMs: a controllable decoding intervention plus a frozen contrastive encoder can generate and verify preference pairs good enough for DPO to reduce hallucination substantially. Concretely, the paper asserts that mixing the conditional next-token distribution with the unconditional (image-free) distribution at a sampled ratio hratio produces captions that are controllably more hallucinated, and that when these candidate pairs are labeled by average sentence-level CLIPScore—swapping whenever the initial negative outscores the initial positive and keeping pairs with moderate score differences—the resulting DPO training improves both precision and recall over the seed model while using far less compute than MLLM-judge pipelines.","pith_inferences":["The sentence-level CLIPScore inversion is a generic data-cleaning heuristic: any preference dataset where a cheap proxy score is available but noisy could be cleaned by the same swap-on-negative-difference rule, independent of the hratio mechanism.","If CLIPScore misses attribute-level errors, the method might improve object hallucination while leaving attribute hallucinations intact; a targeted benchmark on attributes would test this boundary.","The hratio interpolation could also serve as an inference-time hallucination dial, letting a user trade detail against fidelity without retraining.","The new IC dataset, with its GPT-4o precision and recall ratings, offers a reusable template for evaluating hallucination control beyond object-level checks."],"forward_implications":["Hallucination reduction on Object HalBench reaches state-of-the-art levels without human or MLLM feedback, suggesting the pipeline can be applied to other MLLMs that expose token distributions.","The same seed model's own generations are the only data needed, so the framework requires no external annotations and scales with unlabeled image collections.","Because the verifier is a frozen CLIP encoder, the approach avoids the cost of running a large judge at data-generation time, making the loop feasible for iterative rounds.","Moderate CLIP-score differences between pairs yield the best DPO gains, implying a practical selection criterion for preference datasets."],"supporting_citations":[{"why":"Supplies the reference-free CLIPScore metric used to compute average sentence-level image-caption correspondence and decide preference-pair swaps.","marker":"[8]"},{"why":"Provides the DPO objective that the paper uses to train the seed model on the generated and filtered preference pairs.","marker":"[18]"},{"why":"Demonstrates CLIP-based ranking for short descriptions, which the paper extends to long captions within its self-improvement pipeline.","marker":"[16]"},{"why":"Presents a prior negative-sampling method (POVID) that the paper contrasts with and builds upon in its controllable decoding approach.","marker":"[30]"},{"why":"Introduces M3ID, a related hallucination-control method whose sampling strategy the paper compares to its own simpler approach.","marker":"[7]"},{"why":"RLAIF-V represents the MLLM-as-judge paradigm that the paper aims to replace with a lightweight verifier.","marker":"[26]"},{"why":"Object HalBench is the public benchmark used to evaluate the improved model's object-level hallucination rates.","marker":"[19]"},{"why":"Defines LLaVA-1.5-13B, the seed model that the framework improves upon.","marker":"[13]"}],"fun_headline_variants":["Judge-free self-improvement cuts hallucinations 5x","No MLLM judge, no collapse: self-improve with CLIP","Efficient MLLM self-improvement via CLIP-scored pairs","Control hallucinations with a frozen CLIP, not a big judge","Self-improving MLLMs: cheap CLIP replaces costly judge"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the average sentence-level CLIPScore reliably orders two long captions by how hallucinated they are, so swapping a pair based on that score and keeping pairs with moderate differences actually cleans the DPO training signal.","fun_headline_variants_meta":{"raw":{"variants":["Judge-free self-improvement cuts hallucinations 5x","No MLLM judge, no collapse: self-improve with CLIP","Efficient MLLM self-improvement via CLIP-scored pairs","Control hallucinations with a frozen CLIP, not a big judge","Self-improving MLLMs: cheap CLIP replaces costly judge"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000756,"raw_usage":{"total_tokens":3310,"prompt_tokens":847,"completion_tokens":2463,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":463,"completion_tokens_details":{"reasoning_tokens":2373}},"tokens_in":463,"tokens_out":2463,"duration_ms":18217,"temperature":1.0,"reasoning_tokens":2373,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:37:31.538761+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rate a random sample of the preference pairs with human annotators who mark every hallucinated object or attribute. If the CLIPScore ordering agrees with the human ordering no better than chance, or if DPO trained on pairs sorted by a random or human-inverted score retains the same gains, then the reported improvements cannot be attributed to the CLIP-based label inversion.","supporting_citations":[{"cited_title":"Direct preference optimization: Your language model is secretly a reward model","cited_arxiv_id":null,"evidence_quote":"Provides the DPO objective that the paper uses to train the seed model on the generated and filtered preference pairs."},{"cited_title":"Multi-modal hal- lucination control by visual information grounding","cited_arxiv_id":null,"evidence_quote":"Introduces M3ID, a related hallucination-control method whose sampling strategy the paper compares to its own simpler approach."},{"cited_title":"Improved baselines with visual instruction tuning","cited_arxiv_id":null,"evidence_quote":"Defines LLaVA-1.5-13B, the seed model that the framework improves upon."}],"review_version":1}