Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

AdaViP: Aligning Multi-modal LLMs via Adaptive Vision-enhanced Preference Optimization

T0 review · 3 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read AdaViP claims that training multimodal LLMs to prefer intact images over images with key objects removed sharply cuts hallucination, beating GPT-4V on Object HalBench.

desk verdict A sensible next step over V-DPO/MDPO with a clean adaptive DPO loss, but the central removal-selection mechanism is unvalidated and the reporting needs fixes; deserves a serious referee. read the letter →

arxiv 2504.15619 v1 pith:YI267IIE submitted 2025-04-22 cs.CV

Jinda Lu , Jinghan Li , Yuan Gao , Junkang Wu , Jiancan Wu , Xiang Wang , Xiangnan He This is my paper · ORCID
classification cs.CV
keywords multimodallargelanguagemodelshallucinationmitigationpreferenceoptimizationdirectvision-basedpairsadaptiveweightingobjectremovaltrustworthiness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that preference alignment for multimodal LLMs has been too language-centric: contrasting a good answer against a bad answer for the same image does not force the model to look carefully at the image. AdaViP therefore creates a second kind of rejected sample by removing a key visual object from the image while keeping the preferred answer, so the model must learn which visual details the answer depends on. A cascade of visual foundation models proposes candidate removals, and a vision-language similarity model selects the removal that most lowers similarity to the preferred response. The training loss then treats the preferred sample as beating both the language-based and vision-based rejected samples, with weights on the two rejection types set adaptively from their implicit rewards. The authors report that this lowers object hallucination to the point where open 7B and 13B models outperform GPT-4V on the Object HalBench non-hallucination rate.

What carries the argument

The load-bearing mechanism is the constructed vision-based preference pair plus an adaptive three-way DPO objective. In the construction stage, an image tagging model lists object categories, an open-set detector localizes them, a segmentation model produces masks, and an inpainting model removes each object to create candidate images; a vision-language similarity model scores each candidate against the preferred response decomposed into sentences, and the lowest-scoring candidate becomes the rejected image (Eq. 9). In the optimization stage, the loss (Eq. 11) is a Plackett-Luce-style softmax over the preferred sample versus both rejected samples, so the weight given to the language-based and vision-based rejected samples is $\omega_{+-} = \mathrm{Sigmoid}(\hat{r}(D^{+-})-\hat{r}(D^{-+}))$ and $\omega_{-+} = \mathrm{Sigmoid}(\hat{r}(D^{-+})-\hat{r}(D^{+-}))$, respectively, where $\hat{r}$ is the implicit DPO reward. This weighting is what prevents the easily distinguishable vision-rejected samples from dominating training.

What would settle it

A human study on a random sample of constructed pairs would settle it: if annotators frequently find that the selected removed object is not mentioned or implied by the preferred response, or if replacing similarity-based selection with random removal selection leaves most of the hallucination gains intact, then the selection mechanism is not carrying the result.

Watch

Extended reading notes

Core claim

On the paper's own terms, the claim is that hallucination in multimodal LLMs can be reduced by making visual context a first-class citizen of preference optimization. Given a preferred triple of question, image, and preferred response, AdaViP constructs a rejected triple with the same question and response but an image from which a key object has been removed; the removed object is chosen as the one that most lowers the similarity between the perturbed image and the sentence-decomposed preferred response. The model is trained with a DPO-style loss in which the preferred sample must beat both the language-rejected sample and the vision-rejected sample, and the relative weights of the two rejection types are set by a sigmoid over their implicit reward differences. In the paper's experiments this produces Object HalBench non-hallucination rates of 93.7% response-level and 96.4% mention-level for a 7B model, and 95.4% and 97.7% for a 13B model, above GPT-4V's 86.4% and 92.7%; the same recipe also improves AMBER and MMHal-Bench scores.

Load-bearing premise

The pipeline assumes that the candidate image whose similarity to the preferred answer is lowest is the one whose removed object the answer actually depends on; if that ranking is wrong, the rejected image teaches the model to distrust an irrelevant visual change.

Editorial extensions

If this is right

  • If the paper is right, object hallucination in multimodal LLMs can be attacked by changing the preference data rather than the architecture: no new model components are needed, only constructed image perturbations and a modified loss.
  • The adaptive weighting is necessary: the paper reports that a fixed equal-weight combination of vision and language preferences drops AMBER discriminative accuracy from 64.8% to 51.8%, while the adaptive loss raises it to 79.9%.
  • During training the model relies more on vision-based preferences early and shifts toward language preferences later, implying that the visual signal is learned first and then integrated with language preferences.
  • The gains transfer from a 7B to a 13B model of the same family, so the mechanism is not tied to one model size.
  • Because AdaViP's 7B and 13B variants exceed GPT-4V on Object HalBench non-hallucination rates, the paper's approach is a candidate route for open-source models to close the trustworthiness gap with proprietary systems.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test whether the similarity-based selector is the bottleneck; a stronger scorer or human-verified removals could plausibly harden the negatives further or, conversely, expose that the training loss is doing most of the work.
  • The constructed pairs are essentially hard negatives for visual grounding, so the same data could serve as a probing task: asking a trained model what changed between the original and rejected image would directly measure whether the model has learned to track the removed object.
  • The adaptive weighting scheme should transfer to other alignment settings where one rejection type is much easier than another, such as safety versus helpfulness, although the paper demonstrates it only for vision versus language.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes AdaViP, a preference-alignment method for multimodal LLMs that augments conventional language-based DPO with vision-based preference pairs. The vision-based rejected sample is constructed by cascading RAM, GroundingDINO, SAM, and LaMa to detect, segment, and remove objects from the preferred image, then selecting via CLIP the candidate removal least similar to the preferred response. The training loss unifies the preferred, vision-rejected, and language-rejected triples under a Plackett-Luce-style objective, yielding adaptive weights for the two rejected samples. Experiments on LLaVA-1.5 7B/13B report lower object hallucination on Object HalBench, AMBER, and MMHal-Bench than several published baselines, including values above GPT-4V on Object HalBench.

Significance. The loss derivation is internally consistent: Equation (11) follows from the Plackett-Luce model in Equation (10), and the gradient in Equation (12) and the weights in Equation (13) are correct algebraic consequences. The adaptive weighting requires no additional tuned hyperparameter, and the approach builds on off-the-shelf visual foundation models, making it a practical recipe if the empirical claims hold. If the reported numbers are reproduced, AdaViP would be a strong method for reducing object hallucination in open-source MLLMs. However, the evidence does not yet validate the CLIP-based removal mechanism, and the state-of-the-art comparisons are not re-run under identical conditions, so the significance is conditional on additional experiments.

major comments (3)
  1. [§3.2, Eq. (9)] The construction of the rejected image Il rests on the assumption that CLIP similarity to the sentence-decomposed preferred response yw ranks removals by semantic load-bearingness. The paper provides no human validation, no failure analysis of the RAM/GroundingDINO/SAM/LaMa cascade, and no comparison against random removal; Figure 3 shows only selected successful cases. If the cascade removes an object that yw does not depend on, the resulting triple (Q, Il, yw) is a noisy preference, and the central claim that strategic key-element removal drives the gains is unsupported. Please add (i) a quantitative precision/recall study of the removal cascade and CLIP ranking, and (ii) an ablation comparing CLIP-selected removals against random removals and a no-removal control.
  2. [§4.2, Table 2] The ablation does not isolate the two claimed components. It compares DPO, equal-weight vision+language DPO, and adaptive weighting, but there is no row with adaptive weighting while omitting vision pairs, nor a row with vision pairs built from random removals. Consequently, the improved F1 and accuracy under '+Adaptive' could be due entirely to the weighting scheme, and the table cannot establish that CLIP-selected object removal contributes. Please add the two missing rows.
  3. [§4.1 and Table 1] The state-of-the-art claim rests on baseline numbers 'sourced from [8,30,37]' rather than re-run under the same evaluation protocol, and no standard deviations or multiple seeds are reported. Given the large reported margins, please re-evaluate at least the closest baselines (RLAIF-V, V-DPO, MDPO) on Object HalBench and AMBER, and report variance. Also reconcile Table 2's base LLaVA-1.5-7B overall F1 of 74.7 with Table 1's 77.7 for the same model and AMBER Discriminative metric.
minor comments (6)
  1. [Abstract] The abstract says AdaViP achieves '93.7% and 96.4% reductions in response-level and mentioned-level hallucination,' but Table 1 reports non-hallucination rates; please rephrase to avoid equating a rate with a reduction.
  2. [Eq. (1)] Equation (1) has an unbalanced parenthesis in sigma(r*(yw|Q,I) - (r*(yl|Q,I)); insert the missing closing parenthesis.
  3. [Table 2] The base LLaVA-1.5-7B row reports an overall F1 of 74.7, while Table 1 reports 77.7 for the same model and AMBER Discriminative F1; please clarify which subset or metric is used in each table.
  4. [§3.2, Eq. (9)] The variable n is reused for the number of RAM categories in Equation (5) and the number of decomposed sentences in Equation (9); use a separate symbol such as m for the sentence count.
  5. [§4.3] The bullet claiming 'over 70%' improvement on AMBER Generative is ambiguous: the reported CHAIR drop from 3.5 to 1.8 is about 49%, while the Cog. drop from 1.0 to 0.3 is 70%; please specify which generative metric is meant.
  6. [Figure 5] Figure 5 needs axis labels, a legend, and a description of what each curve averages over; as printed, the weight and reward dynamics are hard to reproduce. Also fix the typo 'essboard' in Figure 3.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the preference-pair construction, the Plackett-Luce-style loss, and the benchmark evaluations are all self-contained and not equivalent to the paper's inputs.

full rationale

The paper's derivation chain does not reduce to its inputs. Vision-based rejected images are built by an external cascade (RAM, GroundingDINO, SAM, LaMa) and selected by CLIP similarity to the preferred response (Eq. 9); CLIP is a frozen, externally trained model, not the trained MLLM's own likelihood, so the construction is not self-definitional. The adaptive optimization loss (Eqs. 10-13) is derived from the Plackett-Luce model using the standard implicit reward of DPO (Eq. 4); beta is taken from prior DPO work, and the weights in Eq. 13 are softmax consequences of the loss rather than fitted coefficients. The reported gains are measured on external benchmarks (Object HalBench human annotations, AMBER labels, MMHal-Bench with GPT-4), and none of these benchmarks are used as fitting targets. The self-citations in the paper (e.g., Refs. [4] and [18]) are not load-bearing: the Plackett-Luce formula is cited as inspiration alongside the original Plackett-Luce reference, and the derivation is written out explicitly in the text. No fitted input is renamed as a prediction, no uniqueness theorem is imported from the authors' prior work, and no ansatz is smuggled in solely via self-citation. Thus, despite possible concerns about the validity of the CLIP-based removal selection, no circularity is present.

Assumptions & free parameters 1 free parameters · 6 assumptions · 0 invented entities

The method rests on standard DPO assumptions plus several domain assumptions about the reliability of the vision foundation-model cascade and CLIP-based candidate selection. The most paper-specific axiom is the three-outcome Plackett-Luce ordering, proposed without independent validation. No invented entities are introduced.

free parameters (1)
  • beta (DPO temperature) = 0.1
    Standard DPO coefficient taken from RLAIF-V/DPO; not fitted to benchmarks, but the scale of the log-ratio reward and the adaptive weights depends on it.
assumptions (6)
  • domain assumption The language-based preference annotations from RLAIF-V (LLaVA-Next 34B) are a valid proxy for human preferences.
    All training preferences come from [37]; if the automatic labels are noisy, both DPO and AdaViP inherit the errors. Section 4.1.
  • domain assumption CLIP score between a candidate edited image and the decomposed preferred response selects the key visual element worth removing.
    Eq. 9 assumes CLIP similarity ranks semantic importance; no human validation of the selected removals is reported. Section 3.2.
  • domain assumption RAM, GroundingDINO, SAM, and Big LaMa reliably detect, segment, and inpaint each candidate object without distorting the rest of the image.
    The pipeline in Eqs. 5-8 depends on the correctness of each external model; failures would create noisy vision-based preferences. Section 3.2.
  • ad hoc to paper The Plackett-Luce model with the three outcomes D++, D+-, and D-+ captures the intended multi-modal preference ordering.
    Eq. 10 unifies vision and language preferences into one ranking; this modeling choice is proposed without independent validation. Section 3.3.
  • standard math DPO's implicit reward identity and the initialization of pi_ref as pi_SFT hold.
    Taken from [23]; standard DPO theory is assumed throughout Sections 2 and 3.
  • domain assumption Baseline numbers collected from other papers are directly comparable to the authors' runs.
    Table 1 relies on external baseline numbers rather than re-running all methods, and evaluation versions or protocols may differ. Section 4.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AdaViP: Aligning Multi-modal LLMs via Adaptive Vision-enhanced Preference Optimization." pith.science (2026). https://pith.science/paper/YI267IIE

@misc{pith2026250415619,
  author       = {Pith},
  title        = {Pith review of: AdaViP: Aligning Multi-modal LLMs via Adaptive Vision-enhanced Preference Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YI267IIE}},
  note         = {Machine review of arXiv:2504.15619}
}
read the original abstract

Preference alignment through Direct Preference Optimization (DPO) has demonstrated significant effectiveness in aligning multimodal large language models (MLLMs) with human preferences. However, existing methods focus primarily on language preferences while neglecting the critical visual context. In this paper, we propose an Adaptive Vision-enhanced Preference optimization (AdaViP) that addresses these limitations through two key innovations: (1) vision-based preference pair construction, which integrates multiple visual foundation models to strategically remove key visual elements from the image, enhancing MLLMs' sensitivity to visual details; and (2) adaptive preference optimization that dynamically balances vision- and language-based preferences for more accurate alignment. Extensive evaluations across different benchmarks demonstrate our effectiveness. Notably, our AdaViP-7B achieves 93.7% and 96.4% reductions in response-level and mentioned-level hallucination respectively on the Object HalBench, significantly outperforming current state-of-the-art methods.

Figures

Figures reproduced from arXiv: 2504.15619 by the authors.

Figure 1
Figure 1. Comparison of preference construction methods and op [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of our Adaptive Vision-aware Preference optimization (AdaViP). Given a preferred sample, the vision-based rejected one is constructed by locating and removing key elements of the image (Left). Subsequently, to effectively balance the vision- and language￾based preference pair, we propose an adaptive loss that modulates the optimization procedure based on the relative rewards (right). where πθ and πref repre… view at source ↗
Figure 3
Figure 3. Visualization results of our vision-based rejected image [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: The dynamic weight between the vision- and language [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Experience Augmented Policy Optimization for LLM Reasoning

    cs.LG 2026-06 unverdicted novelty 6.0 of 10

    EAPO reuses prior RL policy experience adaptively at decision points in LLM rollouts with adapted importance sampling and reports gains over prior RLVR methods on math benchmarks.

Reference graph

Works this paper leans on

44 extracted references · 20 canonical work pages · cited by 1 Pith paper

  1. [1]

    Qwen-vl: A versatile vision-language model for un- derstanding, localization, text reading, and beyond

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for un- derstanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966, 1(2):3, 2023. 1

    arXiv 2023

  2. [2]

    Rank analysis of incomplete block designs: I

    Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired compar- isons. Biometrika, 39(3/4):324–345, 1952. 2

    1952

  3. [3]

    End-to- end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to- end object detection with transformers. In European confer- ence on computer vision, pages 213–229. Springer, 2020. 3

    2020

  4. [4]

    On softmax direct preference optimization for recommendation

    Yuxin Chen, Junfei Tan, An Zhang, Zhengyi Yang, Leheng Sheng, Enzhi Zhang, Xiang Wang, and Tat-Seng Chua. On softmax direct preference optimization for recommendation. arXiv preprint arXiv:2406.09215, 2024. 4

    arXiv 2024

  5. [5]

    Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24185–24198, 2024. 1

    2024

  6. [6]

    Fine- grained verifiers: Preference modeling as next-token pre- diction in vision-language alignment

    Chenhang Cui, An Zhang, Yiyang Zhou, Zhaorun Chen, Gelei Deng, Huaxiu Yao, and Tat-Seng Chua. Fine- grained verifiers: Preference modeling as next-token pre- diction in vision-language alignment. arXiv preprint arXiv:2410.14148, 2024. 8

    arXiv

  7. [7]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Ab- hishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 ,

    arXiv

  8. [8]

    Token pref- erence optimization with self-calibrated visual-anchored rewards for hallucination mitigation

    Jihao Gu, Yingyao Wang, Meng Cao, Pi Bu, Jun Song, Yancheng He, Shilong Li, and Bo Zheng. Token pref- erence optimization with self-calibrated visual-anchored rewards for hallucination mitigation. arXiv preprint arXiv:2412.14487, 2024. 6, 8

    arXiv 2024

Show all 44 references
  1. [9]

    Opera: Alleviating hallucination in multi- modal large language models via over-trust penalty and retrospection-allocation

    Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Con- ghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. Opera: Alleviating hallucination in multi- modal large language models via over-trust penalty and retrospection-allocation. In Proceedings of the IEEE/CVF Confer...

    2024

  2. [10]

    Vcoder: Ver- satile vision encoders for multimodal large language models

    Jitesh Jain, Jianwei Yang, and Humphrey Shi. Vcoder: Ver- satile vision encoders for multimodal large language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 27992–28002, 2024. 8

    2024

  3. [11]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF international confer- ence on computer vision, pages 4015–4026, 2023. 2, 3

    2023

  4. [12]

    Mitigating object hal- lucinations in large vision-language models through visual contrastive decoding

    Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. Mitigating object hal- lucinations in large vision-language models through visual contrastive decoding. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recogn...

    2024

  5. [13]

    Mitigating hallucination in large multi-modal models via robust instruction tuning

    Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Ya- coob, and Lijuan Wang. Mitigating hallucination in large multi-modal models via robust instruction tuning. In The Twelfth International Conference on Learning Representa- tions, 2023. 8

    2023

  6. [14]

    Llava-next: Im- proved reasoning, ocr, and world knowledge, 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Im- proved reasoning, ocr, and world knowledge, 2024. 5, 8

    2024

  7. [15]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024. 1

    2024

  8. [16]

    A survey on hallucination in large vision-language models

    Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiu- tian Zhao, Ke Wang, Liping Hou, Rongjun Li, and Wei Peng. A survey on hallucination in large vision-language models. arXiv preprint arXiv:2402.00253, 2024. 8

    2024 arXiv

  9. [17]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European Conference on Computer Vision , pages 38–55. Springe...

    2024

  10. [18]

    Dama: Data- and model-aware alignment of multi-modal llms

    Jinda Lu, Junkang Wu, Jinghan Li, Xiaojun Jia, Shuo Wang, YiFan Zhang, Junfeng Fang, Xiang Wang, and Xiangnan He. Dama: Data- and model-aware alignment of multi-modal llms. arXiv preprint arXiv:2502.01943, 2025. 1

    2025 arXiv

  11. [19]

    Simpo: Simple preference optimization with a reference-free reward

    Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with a reference-free reward. arXiv preprint arXiv:2405.14734, 2024. 8

    2024 arXiv

  12. [20]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Car- roll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Ad- vances in neural information processing systems, 35:...

    2022

  13. [21]

    The analysis of permutations

    Robin L Plackett. The analysis of permutations. Journal of the Royal Statistical Society Series C: Applied Statistics , 24 (2):193–202, 1975. 4

    1975

  14. [22]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  15. [23]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christo- pher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024. 1, 3, 4, 6, 8

    2024

  16. [24]

    Object hallucination in image cap- tioning

    Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object hallucination in image cap- tioning. arXiv preprint arXiv:1809.02156, 2018. 5, 6

    2018 arXiv

  17. [25]

    Proximal policy optimization algo- rithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Rad- ford, and Oleg Klimov. Proximal policy optimization algo- rithms. arXiv preprint arXiv:1707.06347, 2017. 2, 8 9

    2017 arXiv

  18. [26]

    Objects365: A large-scale, high-quality dataset for object detection

    Shuai Shao, Zeming Li, Tianyuan Zhang, Chao Peng, Gang Yu, Xiangyu Zhang, Jing Li, and Jian Sun. Objects365: A large-scale, high-quality dataset for object detection. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 8430–8439, 2019. 3

    2019

  19. [27]

    Aligning large multi- modal models with factually augmented rlhf

    Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang-Yan Gui, Yu-Xiong Wang, Yiming Yang, et al. Aligning large multi- modal models with factually augmented rlhf. In Findings of the Association for Computational Linguistics, 2024. 5, 6, 8

    2024

  20. [28]

    Resolution-robust large mask inpainting with fourier convolutions

    Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, and Victor Lempitsky. Resolution-robust large mask inpainting with fourier convolutions. InProceedings of the IEEE/CVF winter...

    2022

  21. [29]

    Eyes wide shut? exploring the visual shortcomings of multimodal llms

    Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9568–9578, 2024. 8

    2024

  22. [30]

    mdpo: Conditional preference optimization for multimodal large language mod- els

    Fei Wang, Wenxuan Zhou, James Y Huang, Nan Xu, Sheng Zhang, Hoifung Poon, and Muhao Chen. mdpo: Conditional preference optimization for multimodal large language mod- els. arXiv preprint arXiv:2406.11839, 2024. 1, 4, 5, 6, 8

    2024 arXiv

  23. [31]

    An llm-free multi-dimensional benchmark for mllms hallu- cination evaluation

    Junyang Wang, Yuhang Wang, Guohai Xu, Jing Zhang, Yukai Gu, Haitao Jia, Ming Yan, Ji Zhang, and Jitao Sang. An llm-free multi-dimensional benchmark for mllms hallu- cination evaluation. arXiv preprint arXiv:2311.07397, 2023. 5, 6

    2023 arXiv

  24. [32]

    V-dpo: Mitigating hallucination in large vision language models via vision-guided direct preference optimization

    Yuxi Xie, Guanzhen Li, Xiao Xu, and Min-Yen Kan. V-dpo: Mitigating hallucination in large vision language models via vision-guided direct preference optimization. In Findings of the Association for Computational Linguistics: EMNLP , pages 13258–13273, 2024. 1, 4, 5, 7, 8

    2024

  25. [33]

    Miti- gating object hallucination via concentric causal attention

    Yun Xing, Yiheng Li, Ivan Laptev, and Shijian Lu. Miti- gating object hallucination via concentric causal attention. Advances in neural information processing systems, 2024. 5

    2024

  26. [34]

    Hallucidoctor: Mitigating hallucinatory toxicity in visual instruction data

    Qifan Yu, Juncheng Li, Longhui Wei, Liang Pang, Wen- tao Ye, Bosheng Qin, Siliang Tang, Qi Tian, and Yueting Zhuang. Hallucidoctor: Mitigating hallucinatory toxicity in visual instruction data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio...

    2024

  27. [35]

    Gradient surgery for multi-task learning

    Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. Advances in neural information process- ing systems, 33:5824–5836, 2020. 4

    2020

  28. [36]

    Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional hu- man feedback

    Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun, et al. Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional hu- man feedback. In Proceedings of the IEEE/CVF Conference on ...

    2024

  29. [37]

    Rlaif-v: Aligning mllms through open-source ai feedback for super gpt-4v trustworthiness

    Tianyu Yu, Haoye Zhang, Yuan Yao, Yunkai Dang, Da Chen, Xiaoman Lu, Ganqu Cui, Taiwen He, Zhiyuan Liu, Tat-Seng Chua, et al. Rlaif-v: Aligning mllms through open-source ai feedback for super gpt-4v trustworthiness. arXiv preprint arXiv:2405.17220, 2024. 1, 5, 6, 8

    2024

  30. [38]

    Less is more: Mitigat- ing multimodal hallucination from an eos decision perspec- tive

    Zihao Yue, Liang Zhang, and Qin Jin. Less is more: Mitigat- ing multimodal hallucination from an eos decision perspec- tive. In Proceedings of the 62nd Annual Meeting of the Asso- ciation for Computational Linguistics , pages 11766–11781,

  31. [39]

    Dino: Detr with improved denoising anchor boxes for end-to-end object de- tection

    Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel Ni, and Heung-Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object de- tection. In The Eleventh International Conference on Learn- ing Representations, 2023. 3

    2023

  32. [40]

    Automated multi-level preference for mllms

    Mengxi Zhang, Wenhao Wu, Yu Lu, Yuxin Song, Kang Rong, Huanjin Yao, Jianbo Zhao, Fanglong Liu, Yifan Sun, Haocheng Feng, et al. Automated multi-level preference for mllms. Advances in Neural Information Processing Systems,

  33. [41]

    Recognize anything: A strong image tagging model

    Youcai Zhang, Xinyu Huang, Jinyu Ma, Zhaoyang Li, Zhaochuan Luo, Yanchun Xie, Yuzhuo Qin, Tong Luo, Yaqian Li, Shilong Liu, et al. Recognize anything: A strong image tagging model. arXiv preprint arXiv:2306.03514 ,

    arXiv

  34. [42]

    Mitigating object hallucination in large vision- language models via classifier-free guidance

    Linxi Zhao, Yihe Deng, Weitong Zhang, and Quanquan Gu. Mitigating object hallucination in large vision- language models via classifier-free guidance. arXiv preprint arXiv:2402.08680, 2024. 8

    2024 arXiv

  35. [43]

    Beyond hallucinations: Enhanc- ing lvlms through hallucination-aware direct preference op- timization

    Zhiyuan Zhao, Bin Wang, Linke Ouyang, Xiaoyi Dong, Ji- aqi Wang, and Conghui He. Beyond hallucinations: Enhanc- ing lvlms through hallucination-aware direct preference op- timization. arXiv preprint arXiv:2311.16839, 2023. 5

    2023 arXiv

  36. [44]

    Aligning modalities in vision large lan- guage models via preference fine-tuning

    Yiyang Zhou, Chenhang Cui, Rafael Rafailov, Chelsea Finn, and Huaxiu Yao. Aligning modalities in vision large lan- guage models via preference fine-tuning. arXiv preprint arXiv:2402.11411, 2024. 4, 5, 8 10

    2024 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.