Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

Reinforcing VLMs to Use Tools for Detailed Visual Reasoning Under Resource Constraints

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read GRPO and one zoom call lift a 3B VLM by 5.7 points on V*Bench

desk verdict A useful engineering recipe for teaching small VLMs to zoom via GRPO, but the headline causal claim about the tool is not proven without an RL-without-tool control. read the letter →

arxiv 2506.14821 v3 pith:IQGIMFN2 submitted 2025-06-10 cs.LG cs.AIcs.CV

Sunil Kumar , Bowen Zhao , Leo Dirac , Paulina Varshavskaya This is my paper
classification cs.LGcs.AIcs.CV
keywords reinforcementlearningvision-languagemodelstoolusezoomGRPOvisualquestionansweringfine-grainedreasoningresourceefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper shows that a small vision-language model can be taught to call a zoom tool through reinforcement learning, closing part of the gap with much larger models on fine-grained visual question answering. Training a 3-billion-parameter model with GRPO, a structured reward, and a deliberately simple tool interface raises its accuracy on V*Bench from 74.4% to 80.1%, matching a 7-billion-parameter baseline and beating a hierarchical visual search system, all with a single targeted zoom per image. The recipe works without any ground-truth tool-use trajectories and under a modest four-GPU budget, but it does not transfer to 4K/8K images, where the model was trained only on downscaled inputs. The paper identifies the training data mix, the reward shape, and the tool-interface simplicity as the decisive factors.

What carries the argument

The load-bearing mechanism is a GRPO training loop with a one-call zoom tool accepting a single keypoint coordinate, a structured reward $R = \alpha R_c + \beta R_f + \gamma R_t$ mixing a hard VQA score, a soft edit-distance partial credit, a format reward, and a small tool-use reward, plus masking of tool-generated tokens during the policy loss. A YAML-like tool invocation format replaces JSON because the small model can emit it reliably, and the 400x400 crop is upscaled to the input image's longer dimension so the model sees detail at a comparable scale. The training mix filters TextVQA to examples where the base model scores below 0.5 in an 8-shot evaluation, roughly 7,800 of 34,600 examples, so the model is pushed to use the tool where it matters.

What would settle it

A decisive test is to ablate the learned keypoint selection: run the same GRPO recipe but feed the zoom tool a fixed center or random keypoint. If V*Bench accuracy stays near 80%, the RL policy is not what selects the region and the claim that the model 'learns to zoom where it matters' collapses.

Watch

Extended reading notes

Core claim

The central claim is that group-relative policy optimization can teach a small VLM to invoke a zoom tool productively for fine-grained visual reasoning, provided the tool interface is easy to emit, the reward separates correctness from formatting from tool success, and the training set over-represents examples the base model fails. The trained 3B model learns to emit a single keypoint coordinate, receives a 400x400 crop upscaled to the input size as extra image tokens, and answers with access to both the original and the crop. On V*Bench this yields 80.1% overall accuracy, a 5.7-point gain over the untrained 3B baseline, roughly matching a 7B model, and beating the SEAL visual search system while using far fewer crop evaluations. The same model does not improve on HR-Bench 4K/8K or on in-domain TextVQA, and the authors trace the failure to the resolution gap between low-resolution training and high-resolution inference.

Load-bearing premise

The method assumes that a keypoint-selection policy learned on images downsized to at most 1024 pixels on the long side transfers to original images at much higher resolutions at inference time.

Editorial extensions

If this is right

  • A small VLM can learn to call an external tool from a reward signal alone, without any supervised tool-use trajectories, under a four-GPU budget.
  • A single keypoint-based zoom call is enough to match or exceed systems that evaluate multiple crops per query, cutting inference cost.
  • The paper identifies the simple tool interface, token masking, crop upscaling, and a difficult-example-heavy data mix as the decisive components of the recipe.
  • The failure to improve on HR-Bench 4K/8K is attributed to the training/inference resolution gap, so gains on V*Bench should not be expected to transfer to higher native resolutions without bridging that gap.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The gain on V*Bench may be partly attributable to the zoom tool providing additional image tokens rather than to the RL policy's keypoint selection; ablating keypoint choice with random or center crops would separate these effects.
  • A straightforward extension is to train with mixed-resolution inputs or multi-scale augmentation to close the resolution gap; if that recovers HR-Bench performance, the recipe would generalize beyond V*Bench's resolution range.
  • The one-call zoom limit means the method is suited to single-ROI questions; tasks requiring multiple distinct regions would need iterative calls or a multi-crop tool, which the current interface does not support.
  • The same GRPO recipe should transfer to other visual tools (e.g., OCR or depth estimators) by swapping the tool description and keeping the keypoint interface, provided the reward still gives dense enough feedback.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper presents an RL recipe for teaching a 3B vision-language model (Qwen2.5-VL-3B-Instruct) to call a zoom tool before answering visual questions. Training uses GRPO with a reward combining answer correctness, format compliance, and tool-use success, a simplified YAML-like tool interface, a single tool call per query, crop upscaling, and a TextVQA training subset filtered to examples where the base model scores below 0.5. On V*Bench the method reports 80.1% overall accuracy versus 74.4% for the base model (+5.7%), matching Qwen2.5-VL-7B (79.1%) and beating SEAL (75.4%) at the 3B scale. On TextVQA validation the method degrades by 5.9 points, and on HR-Bench 4K/8K it shows no improvement or a 5.7-point drop, which the authors attribute to a training/inference resolution gap.

Significance. If the main result is reproducible, the paper offers a practical and relatively cheap recipe: a 3B model, four A100s, one zoom call, and no tool-use trajectory supervision. The design contains several credible components—masking tool-generated image tokens in the GRPO loss, upscaling the crop, a compact tool interface, and a hard-data mix—and the efficiency comparison against SEAL is favorable. However, the central causal claim that the gain comes from the tool is not currently established, because the recipe changes several things at once and the reported numbers come from a single best run. The paper is a useful systems contribution in its current form, but it needs a control and variance reporting before the attribution can be accepted.

major comments (3)
  1. [Section 3, Table 1, and Abstract] The causal attribution in the abstract ('thanks to the detailed visual information gathered from the external tool') is not supported by the experimental design. The recipe contains at least four interventions that could improve V*Bench without tool access: GRPO on a filtered hard subset of TextVQA, the correctness/format reward structure, the simplified tool-calling interface, and crop upscaling. Figure 4 compares the RL-with-zoom model to the base model, not to an RL-trained model without the zoom tool, and the crop-upscale ablation still calls the tool in both arms. I request a control trained with the same rewards, data mix, and rollouts but with the zoom tool disabled (or replaced by a no-op tool); without it the paper cannot distinguish 'GRPO on this data helps' from 'the zoom tool helps.'
  2. [Section 3, Training details] The paper states, 'We present the results from the training run that performed the best,' meaning Table 1 reports a single run with no variance information. GRPO training with no KL regularization and clip-higher can be noisy, and selecting the best run can overstate the +5.7% V*Bench gain. Report mean and standard deviation across at least three independent seeds, or otherwise justify why the selected run is representative.
  3. [Section 2, 'Image resolution,' and Table 1] The statement that downscaled training 'helps the model generalize well on high-resolution visual tasks at inference time' is contradicted by the HR-Bench results in Table 1: HR-Bench 4K shows no improvement over the base model and HR-Bench 8K shows a 5.7-point overall drop. The authors' resolution-gap conjecture in Section 3 is reasonable, but the claim in Section 2 should be scoped to the resolutions actually tested (V*Bench, roughly 2K), or the paper should include high-resolution training to support the broader generalization claim.
minor comments (6)
  1. [Figure 2] The JSON example contains the typo 'boudning box'; it should read 'bounding box'.
  2. [Section 2, Training] The regular expression in the paragraph beginning 'Once the tool is used' is garbled in the manuscript ('[ˆ<]*)</think>...'); please present it in monospace and verify that it matches the intended grammar.
  3. [Table 1] The table uses 'HR-Bench' and 'HR-bench' inconsistently; choose one capitalization throughout.
  4. [General] The text repeatedly renders 'YAML-like' as 'Y AML-like' (e.g., Figure 2 and Section 2); fix the spacing.
  5. [Section 3, Results] The phrase 'RL substantially improves' should be qualified as 'on V*Bench,' because Table 1 shows decreases on TextVQA and HR-Bench 8K.
  6. [Figure 4] Label the x-axis with units (pixels of the long side) and add a marker for the training resolution of 1024 pixels to make the resolution-gap discussion easier to follow.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: evaluation is on external benchmarks and the training objective is not definitionally tied to the reported metric.

full rationale

The reported gains are measured on external benchmarks (V*Bench, HR-Bench, and TextVQA validation) that are not used to define the training objective. The GRPO update uses a hand-specified reward R = αRc + βRf + γRt combining answer correctness, formatting, and tool-use success; the tool-use reward is a training input, not a synonym for the evaluated accuracy. The data-mix filter selects TextVQA training examples where the base model scores below 0.5 in an 8-shot run; this is a training-data selection heuristic, not a fitted parameter whose value is then reported as the prediction. No load-bearing self-citation appears: the GRPO and DeepSeek-R1 references are standard prior-method citations, and no uniqueness or ansatz claim is imported from the authors' own prior work. The absence of an RL-without-zoom control weakens the causal attribution of the V*Bench gain to the zoom tool, but that is an experimental-design and attribution limitation, not circularity: no equation or definition makes the evaluated accuracy equal to the reward or to the training filter by construction. The HR-Bench degradation is explicitly acknowledged as a resolution-gap limitation, further indicating that the paper does not redefine failure away. Therefore no pattern from the circularity taxonomy is exhibited, and the derivation is self-contained with respect to the benchmarks it claims to predict.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The method's success depends on several hand-tuned reward weights, a data-filter threshold, and a fixed crop size; these are chosen empirically rather than derived, but none of them reduce the evaluation to a fitted quantity.

free parameters (6)
  • alpha (answer correctness reward weight) = 1
    Hand-tuned; balances correctness against format and tool rewards (Section 2, Reward shaping).
  • beta (format reward weight) = 1
    Hand-tuned; encourages proper tool/answer formatting.
  • gamma (tool-use reward weight) = 0.1
    Small positive reward for any successful zoom call; tuned to encourage tool use without overwhelming correctness.
  • lambda (hard/soft reward blend) = 0.5
    Blends exact VQA score with edit-distance-based partial credit.
  • data filter threshold = 0.5 VQA score
    Training examples with base-model 8-shot VQA score < 0.5 are kept; chosen to over-represent difficult examples (Section 2, Data mix).
  • zoom crop size = 400x400 pixels
    Fixed crop size centered on keypoint; chosen as a compromise between detail and memory.
assumptions (4)
  • domain assumption GRPO with the described reward shaping will optimize tool-use behavior in a small VLM without ground-truth tool trajectories.
    The paper relies on RL to discover tool use; no proof that the reward signal is sufficient, and Figure 3 shows it is learned in their runs.
  • ad hoc to paper A fixed 400x400 crop centered on a keypoint contains enough information to answer the query when the keypoint is well chosen.
    The tool design assumes the ROI is captured by this crop; no analysis of keypoint error tolerance.
  • domain assumption The VQA score on TextVQA training examples is a reliable measure of an example's difficulty for tool-use learning.
    Used to filter training data; no validation that this proxy holds for other benchmarks.
  • domain assumption The model can learn to emit a single well-formed tool call in the YAML-like format.
    The paper finds JSON format leads to collapse; the simplified format is assumed to be learnable by a 3B model.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reinforcing VLMs to Use Tools for Detailed Visual Reasoning Under Resource Constraints." pith.science (2026). https://pith.science/paper/IQGIMFN2

@misc{pith2026250614821,
  author       = {Pith},
  title        = {Pith review of: Reinforcing VLMs to Use Tools for Detailed Visual Reasoning Under Resource Constraints},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IQGIMFN2}},
  note         = {Machine review of arXiv:2506.14821}
}
read the original abstract

Despite tremendous recent advances in large model reasoning ability, vision-language models (VLMs) still struggle with detailed visual reasoning, especially when compute resources are limited. To address this challenge, we draw inspiration from methods like Deepseek-r1 for VLMs and train smaller-scale models with Group Relative Policy Optimization (GRPO) to use external tools such as zoom. The greatest benefit is obtained with a combination of GRPO learning, a simple reward structure, a simplified tool-calling interface, allocating additional tokens to the result of the tool call, and a training data mix that over-represents visually difficult examples. Compared to similarly-sized baseline models, our method achieves better performance on some visual question-answering (VQA) tasks, thanks to the detailed visual information gathered from the external tool.

Figures

Figures reproduced from arXiv: 2506.14821 by the authors.

Figure 1
Figure 1. In contrast to direct prompting where the model fails to capture the visual details in high-resolution images, RL [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Our YAML-like interface for tool-use calls uses [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Accumulated GRPO advantage for policy trajecto [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Overall accuracy of VLMs on V ∗Bench with var￾ied input image resolutions at inference time. 2025; Yang et al. 2025; Deng et al. 2025b; Wang et al. 2025b). However, existing work often requires expensive human annotation or distillation from larger models for gen￾erali…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Beyond Zooming: Learning Multi-Tool Visual Reasoning for Ultra-High-Resolution Remote Sensing

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Training on multi-tool visual reasoning trajectories (zoom, grounding, lines) with an attention-focused RL objective improves UHR remote-sensing VQA accuracy over single-tool zoom-in and larger base models.

Reference graph

Works this paper leans on

36 extracted references · 7 canonical work pages · cited by 1 Pith paper

  1. [1]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023

    arXiv 2023

  2. [2]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025

    arXiv 2025

  3. [3]

    Words or vision: Do vision-language models have blind faith in text? arXiv preprint arXiv:2503.02199, 2025 a

    Ailin Deng, Tri Cao, Zhirui Chen, and Bryan Hooi. Words or vision: Do vision-language models have blind faith in text? arXiv preprint arXiv:2503.02199, 2025 a

    arXiv 2025

  4. [4]

    Openvlthinker: An early exploration to complex vision-language reasoning via iterative self-improvement

    Yihe Deng, Hritik Bansal, Fan Yin, Nanyun Peng, Wei Wang, and Kai-Wei Chang. Openvlthinker: An early exploration to complex vision-language reasoning via iterative self-improvement. arXiv preprint arXiv:2503.17352, 2025 b

    arXiv 2025

  5. [5]

    How well can vision language models see image details? arXiv preprint arXiv:2408.03940, 2024

    Chenhui Gou, Abdulwahab Felemban, Faizan Farooq Khan, Deyao Zhu, Jianfei Cai, Hamid Rezatofighi, and Mohamed Elhoseiny. How well can vision language models see image details? arXiv preprint arXiv:2408.03940, 2024

    arXiv 2024

  6. [6]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    arXiv 2025

  7. [7]

    Visual programming: Compositional visual reasoning without training

    Tanmay Gupta and Aniruddha Kembhavi. Visual programming: Compositional visual reasoning without training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14953--14962, June 2023

    work page 2023

  8. [8]

    Vision-r1: Incentivizing reasoning capability in multimodal large language models

    Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-r1: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749, 2025

    arXiv 2025

Show all 36 references
  1. [9]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  2. [10]

    Search-r1: Training llms to reason and leverage search engines with reinforcement learning

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516, 2025

    2025 arXiv

  3. [11]

    Dyfo: A training-free dynamic focus visual search for enhancing lmms in fine-grained visual understanding

    Geng Li, Jinglin Xu, Yunzhen Zhao, and Yuxin Peng. Dyfo: A training-free dynamic focus visual search for enhancing lmms in fine-grained visual understanding. arXiv preprint arXiv:2504.14920, 2025

    2025 arXiv

  4. [12]

    API -bank: A comprehensive benchmark for tool-augmented LLM s

    Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. API -bank: A comprehensive benchmark for tool-augmented LLM s. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical ...

    2023 doi

  5. [13]

    Infimm-hd: A leap forward in high-resolution multimodal understanding

    Haogeng Liu, Quanzeng You, Xiaotian Han, Yiqi Wang, Bohan Zhai, Yongfei Liu, Yunzhe Tao, Huaibo Huang, Ran He, and Hongxia Yang. Infimm-hd: A leap forward in high-resolution multimodal understanding. arXiv preprint arXiv:2403.01487, 2024

    2024 arXiv

  6. [14]

    Textcot: Zoom in for enhanced multimodal text-rich image understanding

    Bozhi Luan, Hao Feng, Hong Chen, Yonghui Wang, Wengang Zhou, and Houqiang Li. Textcot: Zoom in for enhanced multimodal text-rich image understanding. arXiv preprint arXiv:2404.09797, 2024

    2024 arXiv

  7. [15]

    Feast your eyes: Mixture-of-resolution adaptation for multimodal large language models

    Gen Luo, Yiyi Zhou, Yuxin Zhang, Xiawu Zheng, Xiaoshuai Sun, and Rongrong Ji. Feast your eyes: Mixture-of-resolution adaptation for multimodal large language models. arXiv preprint arXiv:2403.03003, 2024

    2024 arXiv

  8. [16]

    Yun Peng, Shuqing Li, Wenwei Gu, Yichen Li, Wenxuan Wang, Cuiyun Gao, and Michael R. Lyu. Revisiting, benchmarking and exploring api recommendation: How far are we? IEEE Transactions on Software Engineering, 49 0 (4): 0 1876--1897, 2023. doi:10.1109/TSE.2022.3197063

    2023

  9. [17]

    Toolrl: Reward is all tool learning needs

    Cheng Qian, Emre Can Acikgoz, Qi He, Hongru Wang, Xiusi Chen, Dilek Hakkani-T \"u r, Gokhan Tur, and Heng Ji. Toolrl: Reward is all tool learning needs. arXiv preprint arXiv:2504.13958, 2025

    2025 arXiv

  10. [18]

    Vision language models are blind

    Pooyan Rahmanzadehgervi, Logan Bolton, Mohammad Reza Taesiri, and Anh Totti Nguyen. Vision language models are blind. In Proceedings of the Asian Conference on Computer Vision (ACCV), pages 18--34, December 2024

    2024

  11. [19]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  12. [20]

    Zoomeye: Enhancing multimodal llms with human-like zooming capabilities through tree-based image exploration

    Haozhan Shen, Kangjia Zhao, Tiancheng Zhao, Ruochen Xu, Zilun Zhang, Mingwei Zhu, and Jianwei Yin. Zoomeye: Enhancing multimodal llms with human-like zooming capabilities through tree-based image exploration. arXiv preprint arXiv:2411.16044, 2024

    2024 arXiv

  13. [21]

    Vlm-r1: A stable and generalizable r1-style large vision-language model

    Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, et al. Vlm-r1: A stable and generalizable r1-style large vision-language model. arXiv preprint arXiv:2504.07615, 2025

    2025 arXiv

  14. [22]

    Towards vqa models that can read

    Amanpreet Singh, Vivek Natarjan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8317--8326, 2019

    2019

  15. [23]

    Dragonfly: Multi-resolution zoom-in encoding enhances vision-language models

    Rahul Thapa, Kezhen Chen, Ian Covert, Rahul Chalamala, Ben Athiwaratkun, Shuaiwen Leon Song, and James Zou. Dragonfly: Multi-resolution zoom-in encoding enhances vision-language models. arXiv preprint arXiv:2406.00977, 2024

    2024 arXiv

  16. [24]

    Eyes Wide Shut ? Exploring the Visual Shortcomings of Multimodal LLMs

    Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes Wide Shut ? Exploring the Visual Shortcomings of Multimodal LLMs . pages 9568--9578, 2024. URL https://openaccess.thecvf.com/content/CVPR2024/html/Tong_Eyes_Wide_Shut_Exploring_the_Visual_Short...

    2024

  17. [25]

    Mllm-tool: A multimodal large language model for tool agent learning

    Chenyu Wang, Weixin Luo, Sixun Dong, Xiaohua Xuan, Zhengxin Li, Lin Ma, and Shenghua Gao. Mllm-tool: A multimodal large language model for tool agent learning. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 6678--6687, 2025 a . doi:10.1109/...

    2025

  18. [26]

    Vl-rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning

    Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. Vl-rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning. arXiv preprint arXiv:2504.08837, 2025 b

    2025 arXiv

  19. [27]

    Divide, conquer and combine: A training-free framework for high-resolution image perception in multimodal large language models

    Wenbin Wang, Liang Ding, Minyan Zeng, Xiabin Zhou, Li Shen, Yong Luo, Wei Yu, and Dacheng Tao. Divide, conquer and combine: A training-free framework for high-resolution image perception in multimodal large language models. Proceedings of the AAAI Conference on Artificial Inte...

    2025 doi

  20. [28]

    Visual chatgpt: Talking, drawing and editing with visual foundation models

    Chenfei Wu, Shengming Yin, Weizhen Qi, Xiaodong Wang, Zecheng Tang, and Nan Duan. Visual chatgpt: Talking, drawing and editing with visual foundation models. arXiv preprint arXiv:2303.04671, 2023

    2023 arXiv

  21. [29]

    V?: Guided visual search as a core mechanism in multimodal llms

    Penghao Wu and Saining Xie. V?: Guided visual search as a core mechanism in multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13084--13094, June 2024

    2024

  22. [30]

    R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization

    Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, et al. R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization. arXiv preprint arXiv:2503.10615, 2025

    2025 arXiv

  23. [31]

    Mm-react: Prompting chatgpt for multimodal reasoning and action

    Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Ehsan Azarnasab, Faisal Ahmed, Zicheng Liu, Ce Liu, Michael Zeng, and Lijuan Wang. Mm-react: Prompting chatgpt for multimodal reasoning and action. arXiv preprint arXiv:2303.11381, 2023

    2023 arXiv

  24. [32]

    Predicting goal-directed human attention using inverse reinforcement learning

    Zhibo Yang, Lihan Huang, Yupei Chen, Zijun Wei, Seoyoung Ahn, Gregory Zelinsky, Dimitris Samaras, and Minh Hoai. Predicting goal-directed human attention using inverse reinforcement learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition ...

    2020

  25. [33]

    Dapo: An open-source llm reinforcement learning system at scale, 2025

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen...

    2025 arXiv

  26. [34]

    Vision-r1: Evolving human-free alignment in large vision-language models via vision-guided reinforcement learning

    Yufei Zhan, Yousong Zhu, Shurong Zheng, Hongyin Zhao, Fan Yang, Ming Tang, and Jinqiao Wang. Vision-r1: Evolving human-free alignment in large vision-language models via vision-guided reinforcement learning. arXiv preprint arXiv:2503.18013, 2025

    2025 arXiv

  27. [35]

    Llava-uhd v2: an mllm integrating high-resolution feature pyramid via hierarchical window transformer

    Yipeng Zhang, Yifan Liu, Zonghao Guo, Yidan Zhang, Xuesong Yang, Chi Chen, Jun Song, Bo Zheng, Yuan Yao, Zhiyuan Liu, et al. Llava-uhd v2: an mllm integrating high-resolution feature pyramid via hierarchical window transformer. arXiv preprint arXiv:2412.13871, 2024

    2024 arXiv

  28. [36]

    thinking with images

    Ziwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang, Chao Shen, and Xing Yu. Deepeyes: Incentivizing "thinking with images" via reinforcement learning, 2025. URL https://arxiv.org/abs/2505.14362

    2025 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.