Pith. sign in

REVIEW 5 major objections 4 minor 2 cited by

Efficient Self-Improvement in Multimodal Large Language Models: A Model-Level Judge-Free Approach

T0 review · 5 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A model can improve its own image captioning without any AI judge, using only a lightweight CLIP score.

desk verdict A cheap judge-free self-improvement recipe that is likely oversold: the headline F1 is the best of 30 evaluated configurations, with no error bars or controlled baselines. read the letter →

arxiv 2411.17760 v1 pith:OGKJOJWK submitted 2024-11-26 cs.CL cs.AIcs.CVcs.LG

Shijian Deng , Wentian Zhao , Yu-Jhe Li , Kun Wan , Daniel Miranda , Ajinkya Kale , Yapeng Tian This is my paper · ORCID
classification cs.CLcs.AIcs.CVcs.LG
keywords multimodallargelanguagemodelsself-improvementhallucinationmitigationpreferenceoptimizationdirectCLIPScorecontrollabledecodingjudge-free
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that an MLLM can improve its own captioning by generating preference pairs through a controllable hallucination mechanism, cleaning those pairs with a lightweight CLIP-based verifier, and training with DPO—no large-model judge required. The method mixes conditional and unconditional token distributions with a hallucination ratio to create negative captions, then uses average sentence-level CLIPScore to swap any pair whose negative scores higher than its positive. The authors report that LLaVA-1.5-13B drops from 51.6 to 9.4 on Object HalBench response-level metric, and its precision-recall F1 on their new IC dataset rises from 6.58 to 7.76. If valid, this is a cheaper route to self-improvement that avoids the cost and bias of MLLM judges.

What carries the argument

The load-bearing mechanism is a hallucination-controlled decoding path combined with a lightweight verifier. A hallucination ratio hratio governs the interpolation $t = (1 - h_{\text{ratio}}) \cdot t_c + h_{\text{ratio}} \cdot t_u$ between the conditional token distribution (image + instruction) and the unconditional distribution (instruction only), producing negative captions that are more hallucinated as hratio grows. The verifier is average sentence-level CLIPScore, computed per caption via a frozen contrastive image-text encoder, which decides whether the initial positive is actually better than the initial negative; if not, the pair is swapped. DPO then trains the seed model on the filtered pairs. The identity linking them is that a cheap, sentence-averaged CLIP score can stand in for an MLLM judge in deciding preference direction.

What would settle it

Rate a random sample of the preference pairs with human annotators who mark every hallucinated object or attribute. If the CLIPScore ordering agrees with the human ordering no better than chance, or if DPO trained on pairs sorted by a random or human-inverted score retains the same gains, then the reported improvements cannot be attributed to the CLIP-based label inversion.

Watch

Extended reading notes

Core claim

The central claim is that a model-level judge is not necessary for self-improvement of multimodal LLMs: a controllable decoding intervention plus a frozen contrastive encoder can generate and verify preference pairs good enough for DPO to reduce hallucination substantially. Concretely, the paper asserts that mixing the conditional next-token distribution with the unconditional (image-free) distribution at a sampled ratio hratio produces captions that are controllably more hallucinated, and that when these candidate pairs are labeled by average sentence-level CLIPScore—swapping whenever the initial negative outscores the initial positive and keeping pairs with moderate score differences—the resulting DPO training improves both precision and recall over the seed model while using far less compute than MLLM-judge pipelines.

Load-bearing premise

The method assumes that the average sentence-level CLIPScore reliably orders two long captions by how hallucinated they are, so swapping a pair based on that score and keeping pairs with moderate differences actually cleans the DPO training signal.

Editorial extensions

If this is right

  • Hallucination reduction on Object HalBench reaches state-of-the-art levels without human or MLLM feedback, suggesting the pipeline can be applied to other MLLMs that expose token distributions.
  • The same seed model's own generations are the only data needed, so the framework requires no external annotations and scales with unlabeled image collections.
  • Because the verifier is a frozen CLIP encoder, the approach avoids the cost of running a large judge at data-generation time, making the loop feasible for iterative rounds.
  • Moderate CLIP-score differences between pairs yield the best DPO gains, implying a practical selection criterion for preference datasets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The sentence-level CLIPScore inversion is a generic data-cleaning heuristic: any preference dataset where a cheap proxy score is available but noisy could be cleaned by the same swap-on-negative-difference rule, independent of the hratio mechanism.
  • If CLIPScore misses attribute-level errors, the method might improve object hallucination while leaving attribute hallucinations intact; a targeted benchmark on attributes would test this boundary.
  • The hratio interpolation could also serve as an inference-time hallucination dial, letting a user trade detail against fidelity without retraining.
  • The new IC dataset, with its GPT-4o precision and recall ratings, offers a reusable template for evaluating hallucination control beyond object-level checks.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes a model-level judge-free self-improvement framework for multimodal large language models. It generates preference pairs by mixing conditional and unconditional decoding paths with a controllable hallucination ratio, then filters and swaps pairs using average sentence-level CLIPScore, and finally trains the seed LLaVA-1.5-13B model with DPO. The authors introduce a 150-image IC dataset evaluated by GPT-4o for precision and recall, and report large reductions on Object HalBench (51.6/24.6 to 9.4/5.1) and an F1 improvement on IC (6.58 to 7.76). The core claim is that this framework outperforms conventional techniques without requiring an MLLM judge, at significantly lower computational cost.

Significance. If the empirical claims were properly supported, the framework would be a valuable contribution: it removes the MLLM judge from the verification loop, proposes a simple controllable procedure for negative generation, and introduces a new hallucination-focused benchmark with both precision and recall. The pipeline is clearly described, and the qualitative examples and the explicit limitation section are useful. However, the evidence as presented is not sufficient to establish the central claim. The headline numbers appear to be selected over many test-set configurations, no error bars or significance tests are provided, the closest prior work (CLIP-DPO) is not evaluated as a baseline, and the validity of the CLIPScore-based preference signal is not quantitatively checked. These are experimental-protocol issues rather than flaws in the formulation, so the approach remains promising but unproven.

major comments (5)
  1. [§4.2, Table 2, Figs. 4–6] The reported F1 of 7.76 in Table 2 is the maximum over the ten CLIP-difference splits for the Gaussian hratio distribution, as shown in Fig. 6. The paper does not report the mean or variance over the other splits, nor does it specify a held-out validation set for selecting among the ten splits and three distribution families. Consequently, the headline improvement from 6.58 to 7.76 is a best-of-30 selection on the IC test set, and the claim that the framework outperforms conventional techniques is not established by the presented evidence. Please report all per-split results with error bars, select the final model on a validation set, and state the selection rule explicitly.
  2. [§4.3, Table 1] The paper cites CLIP-DPO [16] as closely related work and states that the proposed approach extends it to long captions, but CLIP-DPO is never evaluated as a baseline. Table 1 mixes different model families, sizes, training data, and feedback types, so the comparison does not support the claim that the proposed method outperforms conventional techniques. A matched comparison using the same seed model, data budget, and evaluation protocol is necessary; otherwise the 9.4/5.1 Object HalBench result has no controlled reference point.
  3. [§3.3, §3.4] The controllability of the hallucination ratio is not quantitatively validated. The paper claims that higher hratio injects more hallucination, but the only evidence is the qualitative reconstruction in Fig. 3. Similarly, the swap rule in §3.4 rests on the assumption that average sentence-level CLIPScore orders captions by faithfulness; the paper reports no swap rate, no agreement with object-level annotations or human judgements, and no oracle ablation. Without such validation, the preference labels may be noisy, and the observed gains cannot be reliably attributed to the proposed inversion and filtering mechanism.
  4. [§4.1, §4.2, §4.3] The IC dataset contains only 150 images, GPT-4o scoring is stochastic, and the paper provides no confidence intervals, repeated evaluations, or significance tests. The reported difference between 6.58 and 7.76 F1 could be within rating noise. The Object HalBench result is likewise a single run with no seeds or variance reported. Statistical reliability is essential for the central empirical claim; please provide multiple evaluation runs, inter-annotator agreement for the GPT-4o judge, and appropriate uncertainty measures.
  5. [§4.2] The experimental setup omits several details needed for reproducibility: the DPO hyperparameters (β, learning rate, number of epochs), the exact boundaries of the ten CLIP-score-difference splits, and the random seeds used for data generation and training. Without these, another group cannot reproduce the results. Please report these details and, if possible, release the IC dataset and code.
minor comments (4)
  1. [Table 3] The category name "veichle" should be corrected to "vehicle".
  2. [Figs. 4–6] The y-axis label "Score (%)" conflicts with the 1-to-10 scale described in §4.3 and Table 2; please clarify the units or remove the percentage sign.
  3. [§4.2] The text "llavav1 5 mix665k" should be formatted as "LLaVA-v1.5-mix665k" for readability.
  4. [§3.3] The distributions pc and pu in Eq. (3) are not formally defined; please state that they are per-token vocabulary distributions and clarify how the weighted combination is computed when the two paths have different tokenizers.

Circularity Check

1 steps flagged · score 4.0 of 10

No definitional circularity: the CLIP-based preference pipeline is self-contained and Object HalBench is an external target. The main circularity-like flaw is that the headline IC F1 (7.76) is a best-of-10-splits/3-distributions maximum selected on the test set and then reported as a single 'Ours' result.

  1. fitted input called prediction [Sec 4.2 (Experiment Setup), Figs 4-6, Table 2]
    ""The pairs were then sorted by CLIP score difference, from low to high, and organized into 10 splits, each containing 10k pairs. For each split, we trained a LLaVA model and conducted inference on the IC dataset and other benchmarks to gather results." (Sec 4.2); "Best F1: 7.76" (Fig. 6); Table 2: "+ Ours 7.74 7.78 7.76 1.18"."

    The paper trains one LLaVA model per CLIP-difference split (10 splits) and repeats this for fixed-hratio, uniform, and Gaussian hratio distributions. Figures 4-6 report the best F1 among the ten splits for each distribution (7.70, 7.67, 7.76), and Table 2 presents the best of these (7.76) as the single '+Ours' result. The split index and hratio distribution are therefore free parameters chosen after observing IC test-set F1, making the reported number a maximum over 10x3 evaluated configurations rather than an out-of-sample prediction of one fixed model. Best-of-N selection is statistically forced to exceed the average, so the claimed improvement from 6.58 to 7.76 is partly a selection artifact and not an independent derivation from the framework.

full rationale

The method itself is not circular by construction: positive/negative pairs are generated by mixing conditional and unconditional decoding with hratio (Eq. 3), CLIPScore is used as an external lightweight verifier to swap labels when the initial negative scores higher (Sec 3.4), and DPO trains the seed model on the filtered pairs (Sec 3.5). None of these steps defines the target benchmarks in terms of the method's own outputs. Object HalBench is an external public benchmark, and the IC dataset is scored by GPT-4o, not by the CLIP model used for preference filtering, so the central derivation is not self-referential. No load-bearing self-citations or imported uniqueness theorems appear; reference [16] (CLIP-DPO) is external prior work. The only significant circularity-like issue is empirical: the headline IC F1 is selected as the best of 10 splits and 3 sampling distributions on the same test set, then reported as a single model's result. This is a fitted-input-called-prediction problem rather than a definitional equivalence. Because the framework's components are independent and there is an external benchmark, the circularity burden is moderate, not maximal. The paper's own limitations section (Sec 5) honestly notes missing recursive self-improvement and scaling studies, but those are scope limitations, not circular steps.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on hand-chosen h_ratio distributions, a CLIP-score-based label inversion whose validity is assumed, and a final model selected by best IC F1 across 30 trained variants. These choices are not independently justified.

free parameters (4)
  • Hallucination ratio distribution (Gaussian mu, sigma; uniform bounds) = mu=0.5, sigma=0.15 for Gaussian; uniform [0,1]
    Chosen by hand; controls negative sample difficulty. The paper reports the Gaussian variant achieves best F1, but no sensitivity analysis or held-out selection is given. Sec 4.2.
  • CLIP score difference split index = The split with best IC F1 (7.76), i.e., split 10 in Gaussian experiments
    Pairs sorted by CLIP score difference are split into 10 groups; each split is used to train a model, and the best on the IC evaluation set is reported. This is a free parameter fitted to the test benchmark. Sec 4.2, Figs 4-6, Table 2.
  • Preference dataset size and split size = 100k pairs, 10k per split
    Arbitrary scale; no justification. Sec 4.2.
  • Swap threshold = 0 (any negative CLIP score higher triggers swap)
    No margin; sensitivity not analyzed. Sec 3.4.
assumptions (4)
  • standard math DPO objective and reference model setup as defined by Rafailov et al. [18]
    Used in Sec 3.1; standard, no issue.
  • domain assumption The unconditional decoding path pu (image removed) produces tokens that are increasingly hallucinated as h_ratio increases
    Sec 3.3, Eq. 3; central to negative sample generation but not independently validated beyond qualitative examples.
  • domain assumption Average sentence-level CLIPScore is a valid measure of caption faithfulness for long captions and can identify when the negative sample is better than the positive
    Sec 3.4; CLIPScore was designed for short captions; the paper extends it by averaging sentences without validation against human labels.
  • domain assumption GPT-4o-based precision/recall scoring on the new IC dataset is a reliable ground truth for hallucination evaluation
    Sec 4.1, Fig 13; no human agreement or calibration reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Efficient Self-Improvement in Multimodal Large Language Models: A Model-Level Judge-Free Approach." pith.science (2026). https://pith.science/paper/OGKJOJWK

@misc{pith2026241117760,
  author       = {Pith},
  title        = {Pith review of: Efficient Self-Improvement in Multimodal Large Language Models: A Model-Level Judge-Free Approach},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OGKJOJWK}},
  note         = {Machine review of arXiv:2411.17760}
}
read the original abstract

Self-improvement in multimodal large language models (MLLMs) is crucial for enhancing their reliability and robustness. However, current methods often rely heavily on MLLMs themselves as judges, leading to high computational costs and potential pitfalls like reward hacking and model collapse. This paper introduces a novel, model-level judge-free self-improvement framework. Our approach employs a controlled feedback mechanism while eliminating the need for MLLMs in the verification loop. We generate preference learning pairs using a controllable hallucination mechanism and optimize data quality by leveraging lightweight, contrastive language-image encoders to evaluate and reverse pairs when necessary. Evaluations across public benchmarks and our newly introduced IC dataset designed to challenge hallucination control demonstrate that our model outperforms conventional techniques. We achieve superior precision and recall with significantly lower computational demands. This method offers an efficient pathway to scalable self-improvement in MLLMs, balancing performance gains with reduced resource requirements.

Figures

Figures reproduced from arXiv: 2411.17760 by the authors.

Figure 1
Figure 1. Comparison of three different improvement paradigms. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of our framework. Our efficient self-improvement framework combines two main strategies: (a) We use a simple yet [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Image reconstruction examples. To further demonstrate the effectiveness of our training framework, we use a text-to-image [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Performance comparison of DPO training using various [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 7
Figure 7. Figure 7: Comparison of model performance when adding cer [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 6
Figure 6. Figure 6: Performance comparison of DPO training using vari [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 8
Figure 8. Figure 8: Examples of qualitative results. With the same input image and instruction prompt ”Describe image in detail,” the caption [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Examples from the IC dataset. The dataset covers a wide range of diverse categories important for image captioning tasks, [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Qualitative comparison of animation image captions. The left panel shows the description generated by LLaVA 1.5 13b, while [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: Qualitative comparison of document image captions. The left panel contains the description generated by LLaVA 1.5 13b, while [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]
Figure 12
Figure 12. Figure 12: Qualitative comparison of common sense violation image captions. The left panel shows the description generated by LLaVA [PITH_FULL_IMAGE:figures/full_fig_p015_12.png]
Figure 13
Figure 13. Figure 13: Prompts used for GPT-4o evaluation. We first use (a) and (b) to obtain precision and recall analysis separately, and then, [PITH_FULL_IMAGE:figures/full_fig_p016_13.png]
Figure 14
Figure 14. Figure 14: GPT-4o analyzing the precision of a caption given a food image and rating the score. [PITH_FULL_IMAGE:figures/full_fig_p016_14.png]
Figure 15
Figure 15. Figure 15: GPT-4o analyzing the recall of a caption given a handwritten image and rating the score. [PITH_FULL_IMAGE:figures/full_fig_p017_15.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. CARL: Constraint-Aware Reinforcement Learning for Planning with LLMs

    cs.AI 2026-07 conditional novelty 6.5 of 10

    CARL improves LLM planning by rewarding the KL divergence between output distributions on constrained versus unconstrained queries, yielding large gains on BlocksWorld, TravelPlanner, and T-Eval.

  2. InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Model

    cs.CV 2025-01 conditional novelty 6.0 of 10

    IXC-2.5-Reward is an open-source multimodal reward model that achieves 70.0% macro accuracy on VL-RewardBench and improves LVLM chat via PPO.

Reference graph

Works this paper leans on

41 extracted references · 18 canonical work pages · cited by 2 Pith papers

  1. [16]

    Clip-dpo: Vision-language models as a source of preference for fixing hallucinations in lvlms

    Yassine Ouali, Adrian Bulat, Brais Martinez, and Georgios Tzimiropoulos. Clip-dpo: Vision-language models as a source of preference for fixing hallucinations in lvlms. arXiv preprint arXiv:2408.10433, 2024. 2

    arXiv 2024

  2. [1]

    Pixtral 12b

    Pravesh Agrawal, Szymon Antoniak, Emma Bou Hanna, Devendra Chaplot, Jessica Chudnovsky, Saurabh Garg, Theophile Gervet, Soham Ghosh, Am ´elie H ´eliou, Paul Ja- cob, et al. Pixtral 12b. arXiv preprint arXiv:2410.07073 ,

    arXiv

  3. [2]

    Understand- ing alignment in multimodal llms: A comprehensive study

    Elmira Amirloo, Jean-Philippe Fauconnier, Christoph Roes- mann, Christian Kerl, Rinu Boney, Yusu Qian, Zirui Wang, Afshin Dehghan, Yinfei Yang, Zhe Gan, et al. Understand- ing alignment in multimodal llms: A comprehensive study. arXiv preprint arXiv:2407.02477, 2024. 2

    arXiv 2024

  4. [3]

    Qwen-vl: A frontier large vision-language model with versatile abilities

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966, 2023. 2, 6

    arXiv 2023

  5. [4]

    Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24185–24198, 2024. 2

    2024

  6. [5]

    Enhancing large vision language models with self-training on image comprehension

    Yihe Deng, Pan Lu, Fan Yin, Ziniu Hu, Sheng Shen, James Zou, Kai-Wei Chang, and Wei Wang. Enhancing large vision language models with self-training on image comprehension. arXiv preprint arXiv:2405.19716, 2024. 1, 2

    arXiv 2024

  7. [6]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Ab- hishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 ,

    arXiv

  8. [7]

    Multi-modal hal- lucination control by visual information grounding

    Alessandro Favero, Luca Zancato, Matthew Trager, Sid- dharth Choudhary, Pramuditha Perera, Alessandro Achille, Ashwin Swaminathan, and Stefano Soatto. Multi-modal hal- lucination control by visual information grounding. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14303–14312, 2024. 1, 2

    work page 2024

Show all 41 references
  1. [8]

    Clipscore: A reference-free evaluation met- ric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation met- ric for image captioning. arXiv preprint arXiv:2104.08718,

    arXiv

  2. [9]

    Opera: Alleviating hallucination in multi- modal large language models via over-trust penalty and retrospection-allocation

    Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Con- ghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. Opera: Alleviating hallucination in multi- modal large language models via over-trust penalty and retrospection-allocation. In Proceedings of the IEEE/CVF Confer...

    2024

  3. [10]

    Mitigating object hal- lucinations in large vision-language models through visual contrastive decoding

    Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. Mitigating object hal- lucinations in large vision-language models through visual contrastive decoding. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recogn...

    2024

  4. [11]

    Silkie: Preference distillation for large visual lan- guage models

    Lei Li, Zhihui Xie, Mukai Li, Shunian Chen, Peiyi Wang, Liang Chen, Yazheng Yang, Benyou Wang, and Lingpeng Kong. Silkie: Preference distillation for large visual lan- guage models. arXiv preprint arXiv:2312.10665, 2023. 6

    2023 arXiv

  5. [12]

    Mini-gemini: Mining the potential of multi-modality vision language models

    Yanwei Li, Yuechen Zhang, Chengyao Wang, Zhisheng Zhong, Yixin Chen, Ruihang Chu, Shaoteng Liu, and Jiaya Jia. Mini-gemini: Mining the potential of multi-modality vision language models. arXiv preprint arXiv:2403.18814,

    arXiv

  6. [13]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306, 2024. 5, 6

    2024

  7. [14]

    Llava-next: Im- proved reasoning, ocr, and world knowledge, 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Im- proved reasoning, ocr, and world knowledge, 2024. 6

    2024

  8. [15]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024. 2

    2024

  9. [17]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Car- roll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Ad- vances in neural information processing systems, 35:...

    2022

  10. [18]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christo- pher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024. 2

    2024

  11. [19]

    Object hallucination in image cap- tioning

    Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object hallucination in image cap- tioning. arXiv preprint arXiv:1809.02156, 2018. 5

    2018 arXiv

  12. [20]

    Aligning large multi- modal models with factually augmented rlhf

    Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang-Yan Gui, Yu-Xiong Wang, Yiming Yang, et al. Aligning large multi- modal models with factually augmented rlhf. arXiv preprint arXiv:2309.14525, 2023. 6

    2023 arXiv

  13. [21]

    A survey on self-evolution of large lan- guage models

    Zhengwei Tao, Ting-En Lin, Xiancai Chen, Hangyu Li, Yuchuan Wu, Yongbin Li, Zhi Jin, Fei Huang, Dacheng Tao, and Jingren Zhou. A survey on self-evolution of large lan- guage models. arXiv preprint arXiv:2404.14387, 2024. 2

    2024 arXiv

  14. [22]

    Planbench: An extensible benchmark for evaluating large language mod- els on planning and reasoning about change

    Karthik Valmeekam, Matthew Marquez, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati. Planbench: An extensible benchmark for evaluating large language mod- els on planning and reasoning about change. Advances in Neural Information Processing Systems, 36, 2024. 1

    2024

  15. [23]

    Llms still can’t plan; can lrms? a preliminary evaluation of openai’s o1 on planbench

    Karthik Valmeekam, Kaya Stechly, and Subbarao Kamb- hampati. Llms still can’t plan; can lrms? a preliminary evaluation of openai’s o1 on planbench. arXiv preprint arXiv:2409.13373, 2024. 1

    2024 arXiv

  16. [24]

    Cogvlm: Visual expert for pretrained language models

    Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, et al. Cogvlm: Visual expert for pretrained language models. arXiv preprint arXiv:2311.03079, 2023. 2

    2023 arXiv

  17. [25]

    Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional hu- man feedback

    Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun, et al. Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional hu- man feedback. In Proceedings of the IEEE/CVF Conference on ...

    2024

  18. [26]

    Rlaif-v: Aligning mllms through open-source ai feedback for super gpt-4v trustworthiness

    Tianyu Yu, Haoye Zhang, Yuan Yao, Yunkai Dang, Da Chen, Xiaoman Lu, Ganqu Cui, Taiwen He, Zhiyuan Liu, Tat-Seng Chua, et al. Rlaif-v: Aligning mllms through open-source ai feedback for super gpt-4v trustworthiness. arXiv preprint arXiv:2405.17220, 2024. 1, 2

    2024

  19. [27]

    Less is more: Mitigat- ing multimodal hallucination from an eos decision perspec- tive

    Zihao Yue, Liang Zhang, and Qin Jin. Less is more: Mitigat- ing multimodal hallucination from an eos decision perspec- tive. arXiv preprint arXiv:2402.14545, 2024. 6

    2024 arXiv

  20. [28]

    Beyond hallucinations: Enhanc- ing lvlms through hallucination-aware direct preference op- timization

    Zhiyuan Zhao, Bin Wang, Linke Ouyang, Xiaoyi Dong, Ji- aqi Wang, and Conghui He. Beyond hallucinations: Enhanc- ing lvlms through hallucination-aware direct preference op- timization. arXiv preprint arXiv:2311.16839, 2023. 6

    2023 arXiv

  21. [29]

    Analyzing and mitigating object hallucination in large vision-language models

    Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. Analyzing and mitigating object hallucination in large vision-language models. arXiv preprint arXiv:2310.00754,

    arXiv

  22. [30]

    Aligning modalities in vision large lan- guage models via preference fine-tuning

    Yiyang Zhou, Chenhang Cui, Rafael Rafailov, Chelsea Finn, and Huaxiu Yao. Aligning modalities in vision large lan- guage models via preference fine-tuning. arXiv preprint arXiv:2402.11411, 2024. 1, 2, 6 Efficient Self-Improvement in Multimodal Large Language Models: A Model-Le...

    2024 arXiv

  23. [31]

    Setting: The caption accurately describes the picnic setting with a blanket on the grass, which is indeed depicted in the image

  24. [32]

    - The salad is correctly noted as being part of the spread, but the caption could be more specific about the contents of the salad

    Food Items: - The mention of pizza aligns with the visible pizza on the wooden board. - The salad is correctly noted as being part of the spread, but the caption could be more specific about the contents of the salad. - The mention of a bowl of fruit is somewhat misleading; wh...

  25. [33]

    However, it inaccurately refers to a wine glass; the image shows glasses of what appears to be a juice or iced tea rather than wine glasses

    Beverages: - The caption correctly states the presence of multiple cups. However, it inaccurately refers to a wine glass; the image shows glasses of what appears to be a juice or iced tea rather than wine glasses

  26. [34]

    - The mention of a potted plant is inaccurate; while there is foliage in the background, it cannot be clearly identified as a potted plant

    Decorative Elements: - The vase with flowers is accurately referenced, adding a decorative touch. - The mention of a potted plant is inaccurate; while there is foliage in the background, it cannot be clearly identified as a potted plant

  27. [35]

    The scene includes forks but not knives

    Utensils: - The caption mentions utensils such as forks, knives, and spoons, which aligns with what is laid out on the blanket, but it specifies fewer types of utensils than actually present. The scene includes forks but not knives

  28. [36]

    Seating Description: - The caption states that a chair is positioned close to the blanket, which is not visible in the image; instead, there are cushions, not a chair. Overall, the caption captures the general essence of the picnic scene, but it contains inaccuracies regarding...

  29. [37]

    Notebook Paper and Handwritten Essay: The caption correctly notes that the image depicts a piece of notebook paper with a handwritten essay, which matches the visual content

  30. [38]

    Topic of the Essay: It accurately identifies that the essay concerns the ancient Greeks and their mathematical contributions, which is a central theme in the text shown in the image

  31. [39]

    It connects this diagram to the essay's content, although it does not specify what the diagram illustrates

    Diagram in Blue Ink: The caption mentions a diagram drawn in blue ink, which aligns with what is visible in the image. It connects this diagram to the essay's content, although it does not specify what the diagram illustrates

  32. [40]

    Cursive Writing: The caption highlights that the essay is written in cursive, reflecting the personal touch and historical context, which corresponds to the style seen in the image

  33. [41]

    Overall, the caption effectively captures the primary elements of the image, including the handwriting, topic, and visual characteristics

    Detailed Exploration: The assessment of the essay as a detailed exploration of the ancient Greeks’ mathematical achievements aligns with the informative nature of the text present in the image. Overall, the caption effectively captures the primary elements of the image, includ...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.