Pith. sign in

REVIEW 5 major objections 6 minor 2 cited by

Unsupervised Visual Chain-of-Thought Reasoning via Preference Optimization

T0 review · 5 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read UV-CoT replaces human-annotated bounding boxes with AI-ranked preferences, letting a multimodal model learn which image region to zoom into, and reports gains over supervised visual chain-of-thought on six benchmarks and four unseen…

desk verdict A useful label-free pipeline for visual CoT, but the headline 'unsupervised superiority' only holds after adding 10% of the supervised data; the evaluator-reliability concern is real but not fatal. read the letter →

arxiv 2504.18397 v2 pith:A5JWW6FM submitted 2025-04-25 cs.CV

Kesen Zhao , Beier Zhu , Qianru Sun , Hanwang Zhang This is my paper · ORCID
classification cs.CV
keywords visualchain-of-thoughtpreferenceoptimizationmultimodallargelanguagemodelsbounding-boxlocalizationquestionansweringzero-shotgeneralizationScore-DPOregionselection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

UV-CoT is an attempt to train image-level chain-of-thought reasoning without any human bounding-box annotations. The target multimodal large language model (MLLM) proposes several candidate crops for a question, answers using each crop, and an evaluator model scores the answers; these scores become preference pairs that train the target model through a modified direct-preference-optimization loss called Score-DPO, which uses the score gap as an extra decision margin. The paper reports that this unsupervised recipe outperforms the supervised Visual-CoT-7B on TextVQA and VSR with 0% labels, and with only 10% labeled data beats it by 2.1 percentage points on average across six benchmarks. On four unseen datasets, zero-shot UV-CoT is reported to outperform all baselines, with the largest gains on high-resolution reasoning, suggesting the ranking formulation generalizes better than supervised imitation.

What carries the argument

The load-bearing object is the automatic preference-data pipeline plus the Score-DPO loss. The pipeline turns a single image-question pair into pairwise training data: stochastic decoding of the target model gives $n$ seed boxes, each box is used to answer the question, and an evaluator model scores each candidate with a current score plus a discounted next-step score, $s_i = s_i^{\mathrm{cur}} + \gamma s_i^{\mathrm{nxt}}$. This scoring design is what lets the model judge a box by whether it helps reasoning continue, not merely by whether it contains the answer text. Score-DPO then carries the signal into training by inserting the mapped score gap $g(s_w)-g(s_l)$ as an offset inside the sigmoid of the standard DPO objective; the paper derives this offset from Gumbel-distributed rewards, so the loss explicitly optimizes both the ordering and the magnitude of preference. Iterative learning regenerates preference data with the updated model over four rounds, keeping the data distribution aligned with the evolving policy.

What would settle it

Take a random sample of the preference pairs generated by Algorithm 1 and have humans judge which crop is more useful for answering each question. If the evaluator's preferred crop agrees with the human choice at no better than chance on pairs where the model answered correctly, the preference signal is not learning useful visual grounding; alternatively, train UV-CoT on the same pairs but with the evaluator scores replaced by random scores and check whether the reported gains vanish.

Watch

Extended reading notes

Core claim

The paper's central claim is that the key-region selection step in visual chain-of-thought reasoning can be learned from relative comparisons instead of absolute labels. Starting from a question, the target multimodal model samples several bounding boxes, crops each one, and answers; an evaluator scores each response and a look-ahead estimate, $s_i = s_i^{\mathrm{cur}} + \gamma s_i^{\mathrm{nxt}}$, so the score reflects not just whether the answer came out right but whether the crop helps the next step. The preferred and dispreferred chains are then used to train the target model with Score-DPO, $\sigma(\beta \log \frac{\pi_\theta(y_w|x)}{\pi_{\mathrm{ref}}(y_w|x)} - \beta \log \frac{\pi_\theta(y_l|x)}{\pi_{\mathrm{ref}}(y_l|x)} - (g(s_w)-g(s_l)))$, which is DPO with a Gumbel-motivated offset equal to the mapped score difference. The paper claims this produces bounding boxes that are more useful for downstream answers than those from supervised Visual-CoT, makes the model beat a larger evaluator model on average, and generalizes to unseen datasets without retraining.

Load-bearing premise

The load-bearing premise is that the evaluator model's score for a candidate crop, computed as the quality of the answer that crop enables plus a look-ahead term, is a faithful measure of whether that crop is the right region to focus on; if the evaluator instead prefers crops that merely contain the answer text or other superficial cues, the preference data used for training is corrupted.

Editorial extensions

If this is right

  • Bounding-box annotation is no longer a prerequisite for visual CoT: preference pairs can be generated automatically, so the same recipe can be applied to new datasets or domains without human labeling.
  • Zero-shot transfer improves because the model learns a relative ranking of regions rather than memorizing annotation patterns; the paper reports it outperforms all baselines on four unseen datasets.
  • A small amount of supervised data can still be used: UV-CoT trained with 10% of Visual-CoT's labels outperforms the fully supervised Visual-CoT-7B by 2.1 percentage points on average across six benchmarks.
  • The learned region-selection behavior transfers at inference time: adding UV-CoT-generated bounding boxes to models that were not fine-tuned for CoT improves their scores, so the box-producing step itself is a reusable capability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the evaluator's preferences are the active ingredient, then improving the evaluator's look-ahead quality should directly improve region selection; a cheap test is to vary $\gamma$ or the number of sampled next responses and watch whether downstream accuracy tracks the resulting score quality.
  • The ranking formulation is essentially weakly supervised visual grounding: the model is never told what a useful region looks like, only which of two crops led to a better answer. That suggests UV-CoT could be repurposed as a general-purpose weak localizer for tasks beyond VQA, such as referring expression comprehension or visual grounding.
  • Because UV-CoT beats its own evaluator on average, the gap between teacher and student may be a general property of preference optimization rather than a quirk of this setup; if so, iterative self-evaluation might push performance past the initial evaluator's ceiling without ever introducing human labels.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. UV-CoT proposes to train MLLMs for visual chain-of-thought reasoning without human bounding-box annotations. The target model samples candidate bounding boxes, uses each cropped region to answer the question, and an evaluator MLLM scores the resulting responses (Eq. 1); these scores define preference pairs. The target model is trained with Score-DPO (Eq. 2), which augments the DPO objective with a score-dependent margin, over four iterative rounds. Experiments compare against Visual-CoT-7B, LLaVA-1.5-{7B,13B}, MiniCPM-o-8B, and OmniLMM-12B on six VQA benchmarks, with additional zero-shot results on four unseen benchmarks. The reported results are that UV-CoT(10%) exceeds Visual-CoT-7B by 2.1% on average, while the fully unsupervised UV-CoT (0% label) averages 0.473 versus 0.482 for Visual-CoT-7B on the same six benchmarks.

Significance. If fully validated, the core idea of replacing human bounding-box annotations with model-generated preference data would be practically significant: it removes the main annotation bottleneck of Visual-CoT and could improve generalization to unseen tasks. The paper also has concrete strengths: the Gumbel-based derivation in Appendix B is mathematically correct and reduces to a shifted sigmoid objective, an ablation isolates the contribution of sDPO and iterative learning, and the authors release code. Nevertheless, the evidence for the headline claim is currently mixed: the fully unsupervised model does not beat the supervised baseline on average, and key design choices that the method depends on are underspecified. The central claim therefore needs either stronger evidence or more careful qualification.

major comments (5)
  1. [§4.2, Table 1] The paper's central quantitative claim that UV-CoT outperforms the supervised Visual-CoT-7B is not supported by the fully unsupervised model: UV-CoT (0% label) averages 0.473 versus 0.482 for Visual-CoT-7B, a decrease of 0.9 points. The claimed 2.1-point average gain comes only from UV-CoT (10% label), which uses 10% of the labeled Visual-CoT data and is therefore not fully unsupervised. The abstract and introduction should be revised to state explicitly which variant achieves state-of-the-art performance and to avoid implying that pure unsupervised learning surpasses supervised visual CoT on average.
  2. [§3.1, Eq. (1)] The validity of the entire preference dataset rests on the untested assumption that the evaluator MLLM's score s_i is a faithful proxy for the usefulness of the cropped region. No experiment in the paper compares the evaluator's region-quality judgments against human annotations or ground-truth region relevance. The bounding-box evaluation in Fig. 3 uses GPT-4o as the scorer, which is another MLLM and does not break the circularity; Table 5 and Fig. 4 provide only indirect evidence about final answers. Since the central claim is that automatically generated preferences substitute for human bounding-box annotations, this missing validation is load-bearing and should be addressed with either a human study of region relevance or a comparison of the preference ordering against ground-truth box utility on a subset.
  3. [§3.2, Eq. (2)] The function g(·) is central to sDPO but is never specified. The offset g(sw) − g(sl) determines the margin in the loss; without knowing g (for example, identity, log, or a scaled score), the reader cannot reproduce the method or interpret why sDPO outperforms naive DPO in Table 4. Please specify g and report any associated hyperparameters, along with the values of n, k, T, and γ used in Algorithm 1 and Eq. (1).
  4. [Tables 1–3] All comparisons are reported as point estimates without variance, multiple seeds, or significance testing. Many differences are small (for example, 0.632 versus 0.652 on Flickr30k and 0.536 versus 0.546 on GQA), and the headline average gain of 2.1% could be within run-to-run noise. Please report standard deviations or bootstrap confidence intervals for the main comparisons, at least for the leading baseline Visual-CoT-7B and the two UV-CoT variants.
  5. [§4.3 and Appendix C.1] The zero-shot section is internally confusing about training exposure. Section 4.3 says the default UV-CoT is tested on SROIE, DUDE, Visual7W, and V∗ Bench 'without any training exposure to these datasets,' but Appendix C.1 lists Visual7W among the source datasets used for preference data, and Table 6 includes all ten datasets in one table. The paper should specify exactly which model variant was trained on which splits, and clarify that UV-CoT∗ is trained on unlabeled data from parts of the zero-shot datasets and is therefore not itself a zero-shot model.
minor comments (6)
  1. [Appendix C.1] The text says 'nine source datasets' while Table 6 lists ten datasets including V∗ Bench; please reconcile the count.
  2. [Figure 4] The figure contains the typos 'Prefered' and 'Dis-prefered'; these should read 'Preferred' and 'Dis-preferred.'
  3. [§4.2] The bullet 'UV-CoT outperforms the supervised Visual-CoT' should explicitly refer to UV-CoT(10%) rather than the 0%-label model, since Table 1 shows the latter is worse on average.
  4. [Appendix B, Eq. (10)] Equation (10) is written with π∗ while the loss in Eq. (6) uses πθ; a sentence explaining the standard DPO parameterization step would improve readability.
  5. [§3.1, Eq. (1)] Please report the number of random samples used to estimate the expectation in the definition of s_i^nxt, and the value of γ used in the main experiments.
  6. [Abstract and §1] The phrase 'state-of-the-art' is stronger than Table 1 supports for the fully unsupervised 0%-label model; consider qualifying the claim with the 10%-label variant or reporting the 0%-label comparison more prominently.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the training signal comes from an external evaluator MLLM and all benchmark results are measured on independent datasets with no fitted constants.

full rationale

The paper's central claim is empirical: UV-CoT, trained on preference data auto-generated by an external evaluator MLLM (OmniLMM-12B) and the target model's own bounding-box outputs, improves visual question answering relative to supervised Visual-CoT-7B and other baselines. The preference scores in Eq. (1) are produced by feval, an independently pretrained model, not by benchmark labels or by the target model's final test predictions. The sDPO loss in Eq. (2) is a training objective derived from standard DPO with an additional offset g(sw)-g(sl); this offset is a design choice, not a parameter fitted to the six in-domain or four zero-shot benchmark results. No load-bearing premise is justified by a self-citation chain: the authors' own prior references are not used to establish the existence or uniqueness of the method, and the comparison baseline Visual-CoT-7B is external work. The Gumbel-based derivation in Appendix B is a standard mathematical reformulation of the Bradley-Terry model with an offset, explicitly credited to ODPO [1], and it does not smuggle in the benchmark outcomes. The main caveat, that the evaluator MLLM's scores may not perfectly reflect true region relevance, is a validity risk about the quality of the training signal, not a circular reduction: the evaluator is never trained on the test benchmarks, so the reported gains remain an independent empirical outcome rather than an artifact of construction.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The paper's central contribution is a training pipeline, not a derived physical/mathematical law. The key unproven inputs are the quality of the evaluator's preferences and the diversity of seed boxes; the hyperparameters beta, gamma, g, n, k are free design choices with material effect (ablation w/o gamma drops 8.8%). The 10% labeled SFT component is a free ingredient in the best-performing variant.

free parameters (6)
  • beta = 0.1
    DPO temperature in Eq. (2), chosen by hand.
  • gamma = not reported in text
    Weight for next-step score in Eq. (1); ablation 'w/o gamma' shows 8.8% average drop, so the value matters.
  • g = unspecified
    Monotonically increasing map from scores to logit space in Eq. (2); its form is not given.
  • n = not reported
    Number of seed bounding boxes generated at each step in Algorithm 1; not specified.
  • k = not reported
    Number of preference pairs constructed in Algorithm 1; not specified.
  • 10% SFT data = 25k samples
    UV-CoT(10%) uses 10% of the labeled Visual-CoT data; the best numbers come from this variant, so the labeled-data assumption is a free ingredient for the headline result.
assumptions (5)
  • standard math Bradley-Terry model of preferences
    Used in Eq. (4) and App. B, standard in DPO literature.
  • standard math Gumbel-max trick / Gumbel difference is logistic
    Used in App. B.4 to derive the sDPO loss; cited to ODPO [1].
  • domain assumption Evaluator MLLM scores correlate with region usefulness
    Eq. (1) treats feval scores as a proxy for bounding-box quality; this is the load-bearing assumption discussed in weakest_assumption.
  • domain assumption Target model's stochastic decoding produces diverse, useful seed boxes
    Algorithm 1 relies on n random seeds yielding candidate regions that include the key region; if seeds are too similar or poor, preference data is degenerate.
  • domain assumption GPT-4o score is a valid measure of answer quality
    All reported numbers are GPT-4o scores (App. C.2), so the entire comparison rests on this proxy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unsupervised Visual Chain-of-Thought Reasoning via Preference Optimization." pith.science (2026). https://pith.science/paper/A5JWW6FM

@misc{pith2026250418397,
  author       = {Pith},
  title        = {Pith review of: Unsupervised Visual Chain-of-Thought Reasoning via Preference Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A5JWW6FM}},
  note         = {Machine review of arXiv:2504.18397}
}
read the original abstract

Chain-of-thought (CoT) reasoning greatly improves the interpretability and problem-solving abilities of multimodal large language models (MLLMs). However, existing approaches are focused on text CoT, limiting their ability to leverage visual cues. Visual CoT remains underexplored, and the only work is based on supervised fine-tuning (SFT) that relies on extensive labeled bounding-box data and is hard to generalize to unseen cases. In this paper, we introduce Unsupervised Visual CoT (UV-CoT), a novel framework for image-level CoT reasoning via preference optimization. UV-CoT performs preference comparisons between model-generated bounding boxes (one is preferred and the other is dis-preferred), eliminating the need for bounding-box annotations. We get such preference data by introducing an automatic data generation pipeline. Given an image, our target MLLM (e.g., LLaVA-1.5-7B) generates seed bounding boxes using a template prompt and then answers the question using each bounded region as input. An evaluator MLLM (e.g., OmniLLM-12B) ranks the responses, and these rankings serve as supervision to train the target MLLM with UV-CoT by minimizing negative log-likelihood losses. By emulating human perception--identifying key regions and reasoning based on them--UV-CoT can improve visual comprehension, particularly in spatial reasoning tasks where textual descriptions alone fall short. Our experiments on six datasets demonstrate the superiority of UV-CoT, compared to the state-of-the-art textual and visual CoT methods. Our zero-shot testing on four unseen datasets shows the strong generalization of UV-CoT. The code is available in https://github.com/kesenzhao/UV-CoT.

Figures

Figures reproduced from arXiv: 2504.18397 by the authors.

Figure 1
Figure 1. Comparison of Visual-CoT [35] and our UV-CoT. Left: Visual-CoT relies on human-annotated bounding boxes to identify key regions. The model is trained via supervised fine-tuning to maximize the likelihood of the labeled data. Right: UV-CoT eliminates the need for human annotation. Given an image, the target model generates seed bounding boxes and answers questions based on these regions, respectively. An evaluator ML… view at source ↗
Figure 2
Figure 2. Illustration of UV-CoT reasoning. dresses this by directly linking reward functions to opti￾mal policies, eliminating reward model fine-tuning. Further improvements include IPO [2], which mitigates overfitting with a bounded preference function, and KTO [7], which removes the need for paired preference data, relying instead on single examples labeled as either ‘good’ or ‘bad’. These preference learning techniques ar… view at source ↗
Figure 3
Figure 3. (a&b) Bounding box evaluation on (a) training datasets and (b) zero-shot datasets. Our UV-CoT performs better than Visual￾CoT. (c) Model performance under varying visual token sizes. Our UV-CoT demonstrates better token efficiency. MLLM DUDE SROIE Visual7w Average LLaVA-1.5-7B 0.165 0.147 0.340 0.217 LLaVA-1.5-13B 0.174 0.159 0.352 0.228 MiniCPM-o-8B 0.182 0.165 0.341 0.229 OmniLMM-12B 0.194 0.166 0.357 0.239 Visual… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Visualization of preference data generated by Algorithm 1. Preferred BBoxes are in [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Visualization of our UV-CoT inference. Model-generated bounding boxes are shown in red. enabling MLLMs to focus on evaluating final answers. 4.7. Other Detailed Analyses Visual token efficiency. Compared to standard MLLM generation, image-level CoT doubles the number o…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. HiDe: Rethinking The Zoom-IN method in High Resolution MLLMs via Hierarchical Decoupling

    cs.CV 2025-09 unverdicted novelty 7.0 of 10

    HiDe uses token-wise attention decoupling and layout-preserving decoupling to build compact crops that push Qwen2.5-VL and InternVL3 to state-of-the-art scores on high-resolution VQA benchmarks.

  2. HART: High-Resolution Annotation-Free Reasoning Technique through a Closed-loop Framework

    cs.CV 2026-02 conditional novelty 6.0 of 10

    HART uses a closed-loop 'crop-and-answer' training scheme plus a dynamic-weight GRPO variant to improve LMM grounding and high-resolution reasoning without bounding-box annotations.

Reference graph

Works this paper leans on

58 extracted references · 33 canonical work pages · cited by 2 Pith papers

  1. [1]

    Direct preference optimization with an offset

    Afra Amini, Tim Vieira, and Ryan Cotterell. Direct preference optimization with an offset. arXiv preprint arXiv:2402.10571, 2024. 4, 11

    arXiv 2024

  2. [2]

    A general theoretical paradigm to un- derstand learning from human preferences

    Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bi- lal Piot, Remi Munos, Mark Rowland, Michal Valko, and Daniele Calandriello. A general theoretical paradigm to un- derstand learning from human preferences. In International Conference on Artificial Intelligence and Statistics, 2024. 3

    work page 2024

  3. [3]

    Qwen-vl: A frontier large vision-language model with versatile abilities

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966, 2023. 1, 2

    arXiv 2023

  4. [4]

    Rank analysis of incomplete block designs: I

    Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired compar- isons. Biometrika, 39(3/4):324–345, 1952. 4

    1952

  5. [5]

    Towards reasoning era: A survey of long chain-of-thought for reasoning large language models

    Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jian- nan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. Towards reasoning era: A survey of long chain-of-thought for reasoning large language models. arXiv preprint arXiv:2503.09567, 2025. 2

    arXiv 2025

  6. [6]

    The method of paired comparisons

    Herbert Aron David. The method of paired comparisons . London, 1963. 11

    work page 1963

  7. [7]

    Kto: Model alignment as prospect theoretic optimization

    Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Ju- rafsky, and Douwe Kiela. Kto: Model alignment as prospect theoretic optimization. arXiv preprint arXiv:2402.01306 ,

    arXiv

  8. [8]

    Towards revealing the mystery behind chain of thought: a theoretical perspective

    Guhao Feng, Bohang Zhang, Yuntian Gu, Haotian Ye, Di He, and Liwei Wang. Towards revealing the mystery behind chain of thought: a theoretical perspective. NeurIPS, 2023. 2

    work page 2023

Show all 58 references
  1. [9]

    Towards revealing the mystery be- hind chain of thought: a theoretical perspective

    Guhao Feng, Bohang Zhang, Yuntian Gu, Haotian Ye, Di He, and Liwei Wang. Towards revealing the mystery be- hind chain of thought: a theoretical perspective. In NeurIPS,

  2. [10]

    Psycollm: Enhancing llm for psychological understanding and evalua- tion

    Jinpeng Hu, Tengteng Dong, Luo Gang, Hui Ma, Peng Zou, Xiao Sun, Dan Guo, Xun Yang, and Meng Wang. Psycollm: Enhancing llm for psychological understanding and evalua- tion. IEEE Transactions on Computational Social Systems ,

  3. [11]

    Mini-monkey: Alleviating the semantic saw- tooth effect for lightweight mllms via complementary image pyramid

    Mingxin Huang, Yuliang Liu, Dingkang Liang, Lianwen Jin, and Xiang Bai. Mini-monkey: Alleviating the semantic saw- tooth effect for lightweight mllms via complementary image pyramid. arXiv preprint arXiv:2408.02034, 2024. 1

    2024 arXiv

  4. [12]

    Icdar2019 com- petition on scanned receipt ocr and information extraction

    Zheng Huang, Kai Chen, Jianhua He, Xiang Bai, Dimosthe- nis Karatzas, Shijian Lu, and CV Jawahar. Icdar2019 com- petition on scanned receipt ocr and information extraction. In ICDAR, 2019. 5, 12

    2019

  5. [13]

    Gqa: A new dataset for real-world visual reasoning and compositional question answering

    Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In CVPR, 2019. 5, 12

    2019

  6. [14]

    G3: an effective and adaptive framework for worldwide geolocalization using large multi- modality models

    Pengyue Jia, Yiding Liu, Xiaopeng Li, Xiangyu Zhao, Yuhao Wang, Yantong Du, Xiao Han, Xuetao Wei, Shuaiqiang Wang, and Dawei Yin. G3: an effective and adaptive framework for worldwide geolocalization using large multi- modality models. In NeurIPS, 2024. 2

    2024

  7. [15]

    Devils in middle layers of large vision- language models: Interpreting, detecting and mitigating ob- ject hallucinations via attention lens

    Zhangqi Jiang, Junkai Chen, Beier Zhu, Tingjin Luo, Yankun Shen, and Xu Yang. Devils in middle layers of large vision- language models: Interpreting, detecting and mitigating ob- ject hallucinations via attention lens. In CVPR, 2025. 2

    2025

  8. [16]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In ICCV, 2023. 12

    2023

  9. [17]

    Large language models are zero-shot reasoners

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. In NeurIPS, 2022. 1, 2

    2022

  10. [18]

    Step-dpo: Step-wise preference op- timization for long-chain reasoning of llms

    Xin Lai, Zhuotao Tian, Yukang Chen, Senqiao Yang, Xian- gru Peng, and Jiaya Jia. Step-dpo: Step-wise preference op- timization for long-chain reasoning of llms. arXiv preprint arXiv:2406.18629, 2024. 3

    2024 arXiv

  11. [19]

    Rlaif: Scaling reinforcement learning from human feedback with ai feed- back

    Harrison Lee, Samrat Phatale, Hassan Mansoor, Kellie Ren Lu, Thomas Mesnard, Johan Ferret, Colton Bishop, Ethan Hall, Victor Carbune, and Abhinav Rastogi. Rlaif: Scaling reinforcement learning from human feedback with ai feed- back. 2023. 2

    2023

  12. [20]

    Visual spa- tial reasoning

    Fangyu Liu, Guy Emerson, and Nigel Collier. Visual spa- tial reasoning. Transactions of the Association for Computa- tional Linguistics, 11:635–651, 2023. 5, 12

    2023

  13. [21]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In CVPR,

  14. [22]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. NeurIPS, 2024. 1, 5

    2024

  15. [23]

    Multimodal recommender systems: A survey

    Qidong Liu, Jiaxi Hu, Yutian Xiao, Xiangyu Zhao, Jingtong Gao, Wanyu Wang, Qing Li, and Jiliang Tang. Multimodal recommender systems: A survey. ACM Computing Surveys, 57(2):1–17, 2024. 2

    2024

  16. [24]

    Deepseek-vl: towards real-world vision- language understanding

    Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang, et al. Deepseek-vl: towards real-world vision- language understanding. arXiv preprint arXiv:2403.05525,

    arXiv

  17. [25]

    Maddison and Danny Tarlow

    Chris J. Maddison and Danny Tarlow. Gumbel machinery,

  18. [26]

    Docvqa: A dataset for vqa on document images

    Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. Docvqa: A dataset for vqa on document images. In WACV,

  19. [27]

    Infographicvqa

    Minesh Mathew, Viraj Bagal, Rub `en Tito, Dimosthenis Karatzas, Ernest Valveny, and CV Jawahar. Infographicvqa. In WACV, 2022. 5, 12, 13

    2022

  20. [28]

    Chatgpt, 2023

    OpenAI. Chatgpt, 2023. Accessed: Mar. 4, 2025. 5

    2023

  21. [29]

    Precise localization of memories: A fine-grained neuron-level knowledge editing technique for llms

    Haowen Pan, Xiaozhi Wang, Yixin Cao, Zenglin Shi, Xun Yang, Juanzi Li, and Meng Wang. Precise localization of memories: A fine-grained neuron-level knowledge editing technique for llms. In ICLR, 2025. 2

    2025

  22. [30]

    Iterative rea- soning preference optimization

    Richard Yuanzhe Pang, Weizhe Yuan, He He, Kyunghyun Cho, Sainbayar Sukhbaatar, and Jason Weston. Iterative rea- soning preference optimization. In NeurIPS, 2025. 3

    2025

  23. [31]

    Flickr30k entities: Collecting region-to-phrase corre- spondences for richer image-to-sentence models

    Bryan A Plummer, Liwei Wang, Chris M Cervantes, Juan C Caicedo, Julia Hockenmaier, and Svetlana Lazeb- nik. Flickr30k entities: Collecting region-to-phrase corre- spondences for richer image-to-sentence models. In ICCV,

  24. [32]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In ICML, 2021. 2

    2021

  25. [33]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christo- pher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. NeurIPS, 2023. 2

    2023

  26. [34]

    Proximal policy optimization algo- rithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Rad- ford, and Oleg Klimov. Proximal policy optimization algo- rithms. arXiv preprint arXiv:1707.06347, 2017. 4, 11

    2017 arXiv

  27. [35]

    Visual cot: Advancing multi-modal language models with a comprehen- sive dataset and benchmark for chain-of-thought reasoning

    Hao Shao, Shengju Qian, Han Xiao, Guanglu Song, Zhuofan Zong, Letian Wang, Yu Liu, and Hongsheng Li. Visual cot: Advancing multi-modal language models with a comprehen- sive dataset and benchmark for chain-of-thought reasoning. NeurIPS, 2025. 1, 2, 5, 12

    2025

  28. [36]

    Towards vqa models that can read

    Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read. In CVPR,

  29. [37]

    Aligning large multi- modal models with factually augmented rlhf

    Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang-Yan Gui, Yu-Xiong Wang, Yiming Yang, et al. Aligning large multi- modal models with factually augmented rlhf. arXiv preprint arXiv:2309.14525, 2023. 3

    2023 arXiv

  30. [38]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 2

    2023 arXiv

  31. [39]

    Document understanding dataset and evaluation (dude)

    Jordy Van Landeghem, Rub `en Tito, Łukasz Borchmann, Michał Pietruszka, Pawel Joziak, Rafal Powalski, Dawid Ju- rkiewicz, Micka¨el Coustaty, Bertrand Anckaert, Ernest Val- veny, et al. Document understanding dataset and evaluation (dude). In ICCV, 2023. 5, 12

    2023

  32. [40]

    Chain-of-thought prompting elicits reasoning in large lan- guage models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large lan- guage models. In NeurIPS, 2022. 1, 2

    2022

  33. [41]

    V?: Guided visual search as a core mechanism in multimodal llms

    Penghao Wu and Saining Xie. V?: Guided visual search as a core mechanism in multimodal llms. In CVPR, 2024. 5, 12

    2024

  34. [42]

    Llava-o1: Let vision language models reason step- by-step

    Guowei Xu, Peng Jin, Li Hao, Yibing Song, Lichao Sun, and Li Yuan. Llava-o1: Let vision language models reason step- by-step. arXiv preprint arXiv:2411.10440, 2024. 1, 2

    2024 arXiv

  35. [43]

    Mm-react: Prompting chatgpt for multimodal reasoning and action

    Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Ehsan Azarnasab, Faisal Ahmed, Zicheng Liu, Ce Liu, Michael Zeng, and Lijuan Wang. Mm-react: Prompting chatgpt for multimodal reasoning and action. arXiv preprint arXiv:2303.11381, 2023. 1, 2

    2023 arXiv

  36. [44]

    Minicpm-v: A gpt-4v level mllm on your phone

    Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level mllm on your phone. arXiv preprint arXiv:2408.01800, 2024. 5

    2024 arXiv

  37. [45]

    Reformulating vision-language foundation models and datasets towards universal multimodal assistants

    Tianyu Yu, Jinyi Hu, Yuan Yao, Haoye Zhang, Yue Zhao, Chongyi Wang, Shan Wang, Yinxv Pan, Jiao Xue, Dahai Li, et al. Reformulating vision-language foundation models and datasets towards universal multimodal assistants. arXiv preprint arXiv:2310.00653, 2023. 1

    arXiv

  38. [46]

    Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback

    Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun, et al. Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback. In CVPR, 2024. 3

    2024

  39. [47]

    Rlaif-v: Aligning mllms through open-source ai feedback for super gpt-4v trustworthiness

    Tianyu Yu, Haoye Zhang, Yuan Yao, Yunkai Dang, Da Chen, Xiaoman Lu, Ganqu Cui, Taiwen He, Zhiyuan Liu, Tat-Seng Chua, et al. Rlaif-v: Aligning mllms through open-source ai feedback for super gpt-4v trustworthiness. arXiv preprint arXiv:2405.17220, 2024. 3, 4

    2024

  40. [48]

    Notellm-2: Multimodal large representation models for rec- ommendation

    Chao Zhang, Haoxin Zhang, Shiwei Wu, Di Wu, Tong Xu, Xiangyu Zhao, Yan Gao, Yao Hu, and Enhong Chen. Notellm-2: Multimodal large representation models for rec- ommendation. arXiv preprint arXiv:2405.16789, 2024. 2

    2024 arXiv

  41. [49]

    Automatic chain of thought prompting in large language models

    Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. Automatic chain of thought prompting in large language models. arXiv preprint arXiv:2210.03493, 2022. 1, 2

    2022 arXiv

  42. [50]

    Multimodal chain-of- thought reasoning in language models

    Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. Multimodal chain-of- thought reasoning in language models. arXiv preprint arXiv:2302.00923, 2023. 1, 2

    2023 arXiv

  43. [51]

    Egotextvqa: Towards egocentric scene-text aware video question answering

    Sheng Zhou, Junbin Xiao, Qingyun Li, Yicong Li, Xun Yang, Dan Guo, Meng Wang, Tat-Seng Chua, and Angela Yao. Egotextvqa: Towards egocentric scene-text aware video question answering. In CVPR, 2025. 2

    2025

  44. [52]

    Boosting few-shot learning via atten- tive feature regularization

    Xingyu Zhu, Shuo Wang, Jinda Lu, Yanbin Hao, Haifeng Liu, and Xiangnan He. Boosting few-shot learning via atten- tive feature regularization. In AAAI, 2024. 2

    2024

  45. [53]

    Enhancing zero-shot vision models by label-free prompt distribution learning and bias correcting

    Xingyu Zhu, Beier Zhu, Yi Tan, Shuo Wang, Yanbin Hao, and Hanwang Zhang. Enhancing zero-shot vision models by label-free prompt distribution learning and bias correcting. NeurIPS, 2024

    2024

  46. [54]

    Selective vision-language subspace projection for few-shot clip

    Xingyu Zhu, Beier Zhu, Yi Tan, Shuo Wang, Yanbin Hao, and Hanwang Zhang. Selective vision-language subspace projection for few-shot clip. In ACMMM, 2024. 2

    2024

  47. [55]

    Visual7w: Grounded question answering in images

    Yuke Zhu, Oliver Groth, Michael Bernstein, and Li Fei- Fei. Visual7w: Grounded question answering in images. In CVPR, 2016. 5, 12

    2016

  48. [56]

    Fine-tuning language models from human preferences

    Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593, 2019. 2 A. Outline We begin by presenting an overview of our Append...

    1909 arXiv

  49. [58]

    Proof: Define the random variable I = arg maxl,w{Rl,Rw}

    =σ(∆ˆrθ), where ∆ˆrθ =rθ(x,yw)−rθ(x,yl). Proof: Define the random variable I = arg maxl,w{Rl,Rw}. The goal is to show: P (I =w) = exp(ˆrθ(x,yw)) exp(ˆrθ(x,yw)) + exp(ˆrθ(x,yl)). (11) For notation simplicity, let ˆrw = ˆrθ(x,yw), ˆrl = ˆrθ(x,yl), andgˆrw∼ Gumbel(ˆrw, 1). Then: ...

  50. [2017]

    Available online. 4, 11

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.