Pith. sign in

REVIEW 4 major objections 4 minor 4 cited by

WeThink: Toward General-purpose Vision-Language Reasoning via Reinforcement Learning

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 120K diverse QA dataset plus hybrid-reward RL improves a 7B vision-language model across 14 benchmarks.

desk verdict Solid dataset-and-recipe paper for R1-style multimodal RL, worth peer review, but the training/eval contamination question needs an answer before the generalization claim is fully credible. read the letter →

arxiv 2506.07905 v1 pith:VLDJTS2K submitted 2025-06-09 cs.CV

Jie Yang , Feipeng Ma , Zitian Wang , Dacheng Yin , Kang Rong , Fengyun Rao , Ruimao Zhang This is my paper
classification cs.CV
keywords multimodalreasoningreinforcementlearningvisualquestionansweringdatasetsynthesischain-of-thoughthybridrewardvision-languagemodelsscalabledatapipeline
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that general-purpose vision-language reasoning can be trained by reinforcement learning, not just math or perception skills, provided the RL runs on a large, diverse set of reasoning-centric questions with flexible rewards. It builds an automated pipeline that turns images into context-aware question-answer pairs with explicit reasoning paths, releases a dataset of over 120K such pairs from 18 image sources, and trains a 7B vision-language model with a hybrid reward combining exact rule checks on choice and fill-in-the-blank answers with a judge-model assessment of descriptive answers. Across 14 benchmarks the trained model beats its base by an average of +3.9 points on six math benchmarks and +1.2 points on eight general multimodal benchmarks, and adding more diverse web images raises the gains further. The central claim matters because it suggests data diversity plus hybrid-reward RL, rather than domain-specific recipes, is the lever for general multimodal reasoning.

What carries the argument

The load-bearing mechanism is the Scalable Multimodal QA Synthesis pipeline together with a hybrid reward system. The pipeline pairs two large models in multi-turn dialogue: one gives a coarse image caption and answers follow-up clarification requests, while the other spots missing visual details and formulates questions that must combine reasoning with at least one of recognition, knowledge, OCR, spatial awareness, or math. Answers are kept only when two independent verifiers agree or a third judge confirms them, and chains of thought are shortened by a refinement step. During RL, the reward is a weighted sum of a format reward and an accuracy reward that switches between exact string matching for multiple-choice and fill-in-the-blank questions and a judge-model score for descriptive answers, all optimized with group-relative policy optimization, which normalizes rewards across a group of sampled responses.

What would settle it

Compute exact and near-duplicate overlap between every evaluation benchmark image and question and the 120K training set; if a substantial fraction of test items are duplicates or near-duplicates of training items, the reported average gains cannot be attributed to general multimodal reasoning.

Watch

Extended reading notes

Core claim

The paper's claim, stated on its own terms, is that direct reinforcement learning on a diverse, reasoning-path-annotated question set is sufficient to improve a strong open 7B multimodal model broadly. It reports that training on all question types in the dataset outperforms training on math questions only, with the math benchmark average rising from 41.4 to 44.1 and the general benchmark average from 70.6 to 71.8. It also reports that direct RL without cold-start supervised fine-tuning beats the cold-start route in this setting, and that adding roughly 20K internet images through the same pipeline raises the math average to 44.7 and the general average to 72.3. The key empirical finding is that question-domain diversity, not just answer verifiability, is what transfers to general multimodal tasks.

Load-bearing premise

The reported gains are computed on evaluation benchmarks whose images and question types may overlap with the 18 training sources; if that overlap is substantial, the improvements could reflect memorization rather than general reasoning.

Editorial extensions

If this is right

  • Direct RL on a diverse reasoning dataset can serve as a general post-training recipe for strong multimodal models, reducing the need for cold-start supervised fine-tuning.
  • Expanding data diversity through an automated pipeline yields further gains, suggesting continuous data collection can keep improving model performance.
  • Rule-based rewards alone are insufficient for general multimodal reasoning; hybrid rewards with model-based assessment extend RL to subjective and descriptive answers.
  • Chain-of-thought quality matters: refined, concise reasoning paths support both SFT and RL, while long raw chains can degrade a well-optimized base model.
  • A 7B open model can reach stronger math and general multimodal benchmark results using RL plus synthetic data, without architectural changes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the gains survive a clean separation between training and evaluation images, the same pipeline should transfer to other base models and language settings, making data synthesis the main bottleneck for general multimodal reasoning.
  • The pipeline could be turned into a closed loop: evaluate on held-out diverse benchmarks, mine failures, synthesize questions for those failure images, and retrain; the paper's scalability result implies this would keep improving.
  • A natural stress test is to train on questions from only one ability family, such as OCR or recognition, and measure whether cross-domain gains come from diversity itself or from a shared reasoning format; the paper does not isolate this.
  • The hybrid-reward design suggests descriptive-answer RL could extend to open-ended multimodal generation tasks beyond question answering, such as captioning or embodied visual QA.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces WeThink, a 120K multimodal QA dataset with reasoning paths generated by a pipeline in which Qwen2.5-VL-72B describes images and DeepSeek-R1 formulates questions through multi-turn refinement; answers are constructed and filtered by DeepSeek-R1, Qwen2.5-VL-72B, and Gemini 2.5 Pro. The authors then apply supervised fine-tuning and GRPO-based reinforcement learning with a hybrid rule/model-based reward to Qwen2-VL-7B and Qwen2.5-VL-7B, reporting improvements across six mathematical and eight general multimodal benchmarks. They also report that direct RL outperforms cold-start SFT followed by RL, that hybrid rewards outperform either reward alone, and that adding more diverse data from the Internet yields further gains.

Significance. If the empirical claims hold, the paper's main contributions are valuable: an open-source dataset with reasoning paths, an automated and potentially scalable synthesis pipeline, and a systematic comparison of SFT versus RL and reward types on a 7B MLLM. The use of VLMEvalKit and the release of the dataset help reproducibility, and the finding that better-conditioned CoT data matters for SFT is clearly demonstrated in Table 2. However, the paper's headline generalization claim is currently under-supported by the uncontrolled overlap between training sources and evaluation benchmarks, by the absence of statistical uncertainty estimates, and by the confounded diversity comparison.

major comments (4)
  1. [Section 3.1 / Table 1 vs Section C.2] The central claim that RL on WeThink yields generalizable gains (Tables 5 and 6, +3.9 math and +1.2 general) is not distinguishable from benchmark memorization because training and evaluation image sources overlap. Table 1 lists AI2D (12,024 images) and ChartQA (22,865 images) among the 18 WeThink sources, while C.2 evaluates on AI2D (Test) and MathVista_MINI, whose chart questions substantially derive from ChartQA. Since Section 3.2's Optional Contextual Constraints allow prior questions from the source datasets to guide question generation, contamination can occur at the question-text level as well as the image level. The paper reports no deduplication, no exclusion of overlapping images or questions, and no per-benchmark overlap analysis. The largest gains are concentrated in chart/diagram-heavy benchmarks (WeMath +9.9, LogicVista +3.4), which is exactly where ChartQA/AI2D training data is concentrated. Please quantify the overlap (e.g., exact or near-duplicate image hashing and question string matching) and re-report all benchmark numbers after removing overlapping samples, or supplement with evaluations on benchmarks disjoint from the training sources.
  2. [Tables 3-6] Every reported number is a single run with no error bars, no multiple seeds, and no significance testing. RL training with GRPO at temperature 1.0 is stochastic, and the claimed effects are small relative to benchmark noise: the general-task average gain is +1.2, and individual general benchmarks fluctuate from -0.4 (AI2D) to +4.6 (MMVet); several math gains are under 2 points (MathVision +0.8). Without variance estimates or significance tests, the central quantitative claims (e.g., Table 4's All vs Math comparison and Table 3's Rule+Model vs Rule comparison) are not statistically supported. Please report mean and standard deviation over at least three RL runs, or bootstrap over evaluation samples, for the main comparisons.
  3. [Table 4 / Section 4.3] The paper uses the comparison 'Math' vs 'All' in Table 4 to conclude that increasing question-domain diversity improves performance (Section 4.3). This comparison is confounded by training-data quantity: the 'All' condition uses the full WeThink dataset while 'Math' uses only math-type questions, so the gains could come from more data rather than from diversity. Since the stated finding is specifically about diversity, please add a matched control that trains on the same number of samples drawn from the math-type subset (or otherwise equalizes sample count and training compute) before attributing the improvement to domain diversity.
  4. [Appendix B.3 / Table 3] The hybrid reward claim rests on the model-based reward for descriptive questions, but the paper provides no validation of the DeepSeek-V3 judge. Table 3 shows that mixing model-based reward with rule-based reward improves the average by only 1.2 points over rule alone, while model-only reward degrades performance (-1.1), and the descriptive answers used in RL are generated and verified by the same model family (DeepSeek-R1 answers, DeepSeek-V3 judge, Qwen2.5-VL cross-check). Without a judge-agreement study or a per-question-type breakdown of reward accuracy, it is unclear whether the hybrid reward's benefit comes from the model-based signal or from the extra diversity and number of training questions. Please report judge agreement against a human-annotated sample and ablate the model-based reward while holding question type and sample size fixed.
minor comments (4)
  1. [Introduction and Figure 5] There are several typos: 'an novel' in the Introduction, 'resoning' in Section 4.3, and 'difficutity' in the Figure 5 caption; these should be corrected.
  2. [Related Work and Limitations] The Related Work section does not discuss benchmark contamination or data leakage in MLLM training, and the Limitations section does not mention the evaluation-overlap risk identified in this report; adding such a discussion would improve the paper's completeness.
  3. [Section 3.3] The text says 'we perform two rounds of verification' but then describes three stages (Preliminary Question Filtering, Answer Construction and Quality Control, CoT Refinement); please clarify whether the two rounds refer only to the filtering stage.
  4. [References [33] and Section 3.3] The pipeline description mentions 'Gemini 2.5 Pro' but reference [33] is the Gemini 1.5 technical report; please update the citation to the correct Gemini 2.5 documentation.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the RL pipeline is evaluated on external benchmarks and no fitted parameter is renamed as a prediction.

full rationale

The paper's central claim is empirical: RL fine-tuning on the synthesized WeThink dataset improves accuracy on 14 external benchmarks (Tables 5 and 6). The training rewards (rule-based, model-based, and format rewards in Appendix B.3) are computed against WeThink's own generated answers, while the reported evaluation numbers come from independent benchmark suites through VLMEvalKit (Section C.2). There is no equation or definition that makes a predicted quantity equal to a fitted input by construction. The citations to the authors' prior R1-OneVision work ([10]) appear only in a survey-style list of existing R1-style MLLM RL methods and are not load-bearing for the dataset construction, reward design, or evaluation. The claimed 'scalable multimodal QA synthesis' pipeline is a data-generation procedure, not a derivation of benchmark performance, and the benchmark scores are not defined in terms of the training data. The training/evaluation overlap concern (Table 1 lists ChartQA with 22,865 images and AI2D with 12,024 images as training sources, while Section C.2 evaluates on AI2D Test and MathVista_MINI, whose chart questions partly come from chart sources) is a possible benchmark-contamination threat to the generalization claim, but it is not a circularity under the specified criteria: the reported numbers are not logically forced by the paper's own definitions or self-cited premises. In line with the hard rule to flag circularity only when a specific reduction can be quoted, no circular step is identified.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on trusting the teacher models that generated the dataset: Qwen2.5-VL-72B descriptions, DeepSeek-R1 questions/answers/CoTs, Gemini 2.5 Pro verification, and DeepSeek-V3 rewards. It also assumes evaluation benchmarks are not contaminated by training images from the same source datasets, and that the EasyR1/veRL implementation of GRPO is correct. These are domain assumptions, not standard mathematical axioms.

free parameters (3)
  • Reward coefficient alpha_accuracy = 0.7
    Hand-set in Appendix B.3 to weight the accuracy component of the final reward; affects RL training dynamics and the reported improvements.
  • Reward coefficient alpha_format = 0.3
    Hand-set in Appendix B.3 to weight the format component; complementary to alpha_accuracy.
  • Max clarification rounds = 3
    In the Information Request Protocol (Appendix A), DeepSeek-R1 may request up to three rounds of extra visual details; this design choice controls how much visual information the generated questions can use.
assumptions (4)
  • domain assumption Qwen2.5-VL-72B's image descriptions and DeepSeek-R1's questions, answers, and CoTs are accurate enough to serve as training supervision.
    The whole data pipeline relies on these teacher models; Gemini 2.5 Pro re-verifies only mismatched cases, and no human audit is described (Sections 3.2-3.3).
  • domain assumption The training images do not overlap with evaluation benchmark images in a way that inflates reported scores.
    AI2D and ChartQA appear in both the training image sources (Table 1) and the evaluation set (Section C.2); the paper does not perform any leakage analysis.
  • domain assumption DeepSeek-V3 provides reliable correctness judgments for descriptive answers in the RL reward.
    The model-based reward for DES questions is assigned by DeepSeek-V3, which belongs to the same model family as DeepSeek-R1 that generated the ground truths; no calibration or human agreement is reported (Appendix B.3).
  • domain assumption The EasyR1/veRL implementation of GRPO and the evaluation harness (VLMEvalKit) are correct.
    The experimental results depend on these external software packages, and the paper does not provide a verification or reproducibility harness (Section 4.3).

how reviews work

0 comments
Cite this review

Pith. "Pith review of WeThink: Toward General-purpose Vision-Language Reasoning via Reinforcement Learning." pith.science (2026). https://pith.science/paper/VLDJTS2K

@misc{pith2026250607905,
  author       = {Pith},
  title        = {Pith review of: WeThink: Toward General-purpose Vision-Language Reasoning via Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VLDJTS2K}},
  note         = {Machine review of arXiv:2506.07905}
}
read the original abstract

Building on the success of text-based reasoning models like DeepSeek-R1, extending these capabilities to multimodal reasoning holds great promise. While recent works have attempted to adapt DeepSeek-R1-style reinforcement learning (RL) training paradigms to multimodal large language models (MLLM), focusing on domain-specific tasks like math and visual perception, a critical question remains: How can we achieve the general-purpose visual-language reasoning through RL? To address this challenge, we make three key efforts: (1) A novel Scalable Multimodal QA Synthesis pipeline that autonomously generates context-aware, reasoning-centric question-answer (QA) pairs directly from the given images. (2) The open-source WeThink dataset containing over 120K multimodal QA pairs with annotated reasoning paths, curated from 18 diverse dataset sources and covering various question domains. (3) A comprehensive exploration of RL on our dataset, incorporating a hybrid reward mechanism that combines rule-based verification with model-based assessment to optimize RL training efficiency across various task domains. Across 14 diverse MLLM benchmarks, we demonstrate that our WeThink dataset significantly enhances performance, from mathematical reasoning to diverse general multimodal tasks. Moreover, we show that our automated data pipeline can continuously increase data diversity to further improve model performance.

Figures

Figures reproduced from arXiv: 2506.07905 by the authors.

Figure 1
Figure 1. WeThink-VL-7B, fine-tuned on Qwen2.5-VL-7B [ [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The automatic process of question formulation for a given image. As illustrated by the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The automatic process of answer construction and quality control. First, DeepSeek-R1 [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The distribution analysis of question domains and [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: We show the difficutity distribution of question sample in the WeThink dataset. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: The impact of cold start data quality on performance. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: The impact of cold start SFT for RL on all math-type questions in WeThink. Previous experiments relying solely on CoT supervision reveal its insufficiency in exploring visual-language reson￾ing, particularly for powerful base models. Reinforcement Learning (RL), as a m…

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Trace: A Taxonomy-Guided Environment for Multidomain Visual Reasoning

    cs.CV 2026-07 conditional novelty 6.0 of 10

    RLVR training on 64,000 procedurally generated Trace instances improves Qwen2.5-VL macro-average on 24 external visual reasoning benchmarks by 3.51 points at 3B and 4.06 points at 7B.

  2. $M^3-Verse$: A "Spot the Difference" Challenge for Large Multimodal Models

    cs.CV 2025-12 conditional novelty 6.0 of 10

    A new benchmark tests whether large multimodal models can compare paired 'before and after' videos to detect scene changes, and finds current models perform near random.

  3. MathReal: We Keep It Real! A Real Scene Benchmark for Evaluating Math Reasoning in Multimodal Large Language Models

    cs.CV 2025-08 unverdicted novelty 5.0 of 10

    A new benchmark of 2,000 K-12 math questions photographed in real scenes finds that multimodal LLMs perform significantly worse on authentic handheld images than on clean benchmarks.

  4. Interpretable Open-Vocabulary Referring Object Detection with Reverse Contrast Attention

    cs.CV 2025-07 reject novelty 5.0 of 10

    A training-free attention reweighting method, Reverse Contrast Attention, is claimed to improve referring object detection in 11 of 15 VLMs, but the custom FitAP metric ranks boxes by IoU with ground truth, which make...

Reference graph

Works this paper leans on

125 extracted references · 19 canonical work pages · cited by 4 Pith papers

  1. [1]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025

    arXiv 2025

  2. [2]

    Openai o3 and o4-mini system card, 2025

    OpenAI. Openai o3 and o4-mini system card, 2025. Accessed: 2025-05-14

    2025

  3. [3]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms.arXiv preprint arXiv:2501.12599, 2025

    arXiv 2025

  4. [4]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    arXiv 2025

  5. [5]

    Sft memorizes, rl generalizes: A comparative study of foundation model post-training.arXiv preprint arXiv:2501.17161, 2025

    Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. Sft memorizes, rl generalizes: A comparative study of foundation model post-training.arXiv preprint arXiv:2501.17161, 2025

    arXiv 2025

  6. [6]

    s1: Simple test-time scaling.arXiv preprint arXiv:2501.19393, 2025

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling.arXiv preprint arXiv:2501.19393, 2025

    arXiv 2025

  7. [7]

    Sft or rl? an early investigation into training r1-like reasoning large vision-language models.arXiv preprint arXiv:2504.11468, 2025

    Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. Sft or rl? an early investigation into training r1-like reasoning large vision-language models.arXiv preprint arXiv:2504.11468, 2025

    arXiv 2025

  8. [8]

    Open- vlthinker: An early exploration to complex vision-language reasoning via iterative self- improvement.arXiv preprint arXiv:2503.17352, 2025

    Yihe Deng, Hritik Bansal, Fan Yin, Nanyun Peng, Wei Wang, and Kai-Wei Chang. Open- vlthinker: An early exploration to complex vision-language reasoning via iterative self- improvement.arXiv preprint arXiv:2503.17352, 2025

    arXiv 2025

Show all 125 references
  1. [9]

    R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization.arXiv preprint arXiv:2503.12937, 2025

    Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization.arXiv preprint arXiv:2503.12937, 2025. 10

    2025 arXiv

  2. [10]

    R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization.arXiv preprint arXiv:2503.10615, 2025

    Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, et al. R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization.arXiv preprint arXiv:2503.10615, 2025

    2025 arXiv

  3. [11]

    Visualprm: An effective process reward model for multimodal reasoning.arXiv preprint arXiv:2503.10291, 2025

    Weiyun Wang, Zhangwei Gao, Lianjie Chen, Zhe Chen, Jinguo Zhu, Xiangyu Zhao, Yangzhou Liu, Yue Cao, Shenglong Ye, Xizhou Zhu, et al. Visualprm: An effective process reward model for multimodal reasoning.arXiv preprint arXiv:2503.10291, 2025

    2025 arXiv

  4. [12]

    Vision-r1: Incentivizing reasoning capability in multimodal large language models.arXiv preprint arXiv:2503.06749, 2025

    Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-r1: Incentivizing reasoning capability in multimodal large language models.arXiv preprint arXiv:2503.06749, 2025

    2025 arXiv

  5. [13]

    Vl- rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning

    Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. Vl- rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning. arXiv preprint arXiv:2504.08837, 2025

    2025 arXiv

  6. [14]

    Skywork r1v2: Multimodal hybrid reinforcement learning for reasoning.arXiv preprint arXiv:2504.16656, 2025

    Yichen Wei, Yi Peng, Xiaokun Wang, Weijie Qiu, Wei Shen, Tianyidan Xie, Jiangbo Pei, Jianhao Zhang, Yunzhuo Hao, Xuchen Song, et al. Skywork r1v2: Multimodal hybrid reinforcement learning for reasoning.arXiv preprint arXiv:2504.16656, 2025

    2025 arXiv

  7. [15]

    Noisyrollout: Reinforcing visual reasoning with data augmentation

    Xiangyan Liu, Jinjie Ni, Zijian Wu, Chao Du, Longxu Dou, Haonan Wang, Tianyu Pang, and Michael Qizhe Shieh. Noisyrollout: Reinforcing visual reasoning with data augmentation. arXiv preprint arXiv:2504.13055, 2025

    2025

  8. [16]

    Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    2023

  9. [17]

    Seamlessm4t: Massively multilingual & multimodal machine translation.arXiv preprint arXiv:2308.11596, 2023

    Loïc Barrault, Yu-An Chung, Mariano Cora Meglioli, David Dale, Ning Dong, Paul-Ambroise Duquenne, Hady Elsahar, Hongyu Gong, Kevin Heffernan, John Hoffman, et al. Seamlessm4t: Massively multilingual & multimodal machine translation.arXiv preprint arXiv:2308.11596, 2023

    2023 arXiv

  10. [18]

    Wavlm: Large-scale self-supervised pre- training for full stack speech processing.IEEE Journal of Selected Topics in Signal Processing, 16(6):1505–1518, 2022

    Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, et al. Wavlm: Large-scale self-supervised pre- training for full stack speech processing.IEEE Journal of Selected Topics in Signal Processing, 16(6):...

    2022

  11. [19]

    Videochat: Chat-centric video understanding.arXiv preprint arXiv:2305.06355, 2023

    KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. Videochat: Chat-centric video understanding.arXiv preprint arXiv:2305.06355, 2023

    2023 arXiv

  12. [20]

    Video-llama: An instruction-tuned audio-visual language model for video understanding.arXiv preprint arXiv:2306.02858, 2023

    Hang Zhang, Xin Li, and Lidong Bing. Video-llama: An instruction-tuned audio-visual language model for video understanding.arXiv preprint arXiv:2306.02858, 2023

    2023 arXiv

  13. [21]

    Minigpt-4: En- hancing vision-language understanding with advanced large language models.arXiv preprint arXiv:2304.10592, 2023

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: En- hancing vision-language understanding with advanced large language models.arXiv preprint arXiv:2304.10592, 2023

    2023 arXiv

  14. [22]

    Minicpm-v: A gpt-4v level mllm on your phone.arXiv preprint arXiv:2408.01800, 2024

    Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level mllm on your phone.arXiv preprint arXiv:2408.01800, 2024

    2024 arXiv

  15. [23]

    Cogvlm: Visual expert for pretrained language models

    Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Song XiXuan, et al. Cogvlm: Visual expert for pretrained language models. Advances in Neural Information Processing Systems, 37:121475–121499, 2024

    2024

  16. [24]

    Sharegpt4v: Improving large multi-modal models with better captions

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi-modal models with better captions. InEuropean Conference on Computer Vision, pages 370–387. Springer, 2024

    2024

  17. [25]

    Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond.arXiv preprint arXiv:2308.12966, 2023

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond.arXiv preprint arXiv:2308.12966, 2023. 11

    2023 arXiv

  18. [26]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024

    2024 arXiv

  19. [27]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306, 2024

    2024

  20. [28]

    Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024

    2024 arXiv

  21. [29]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. InProceedings of the IEEE/CVF Conference on Computer Visi...

    2024

  22. [30]

    How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites.arXiv preprint arXiv:2404.16821, 2024

    Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites.arXiv preprint arXiv:2404.16821, 2024

    2024 arXiv

  23. [31]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271, 2024

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271, 2024

    2024 arXiv

  24. [32]

    Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  25. [33]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context.arXiv preprint arXiv:2403.05530, 2024

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context.arXiv preprint arXiv:2403.05530, 2024

    2024 arXiv

  26. [34]

    Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  27. [35]

    Claude.https://www.anthropic.com/

    Anthropic. Claude.https://www.anthropic.com/

  28. [36]

    Grok.https://x.ai/

    xAI. Grok.https://x.ai/

  29. [37]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  30. [38]

    Large language models are zero-shot reasoners.Advances in neural information processing systems, 35:22199–22213, 2022

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners.Advances in neural information processing systems, 35:22199–22213, 2022

    2022

  31. [39]

    Mul- timodal chain-of-thought reasoning in language models.arXiv preprint arXiv:2302.00923, 2023

    Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. Mul- timodal chain-of-thought reasoning in language models.arXiv preprint arXiv:2302.00923, 2023

    2023 arXiv

  32. [40]

    Chameleon: Plug-and-play compositional reasoning with large language models.Advances in Neural Information Processing Systems, 36:43447–43478, 2023

    Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, and Jianfeng Gao. Chameleon: Plug-and-play compositional reasoning with large language models.Advances in Neural Information Processing Systems, 36:43447–43478, 2023

    2023

  33. [41]

    Layoutllm: Layout instruction tuning with large language models for document understanding

    Chuwei Luo, Yufan Shen, Zhaoqing Zhu, Qi Zheng, Zhi Yu, and Cong Yao. Layoutllm: Layout instruction tuning with large language models for document understanding. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15630–15640, 2024. 12

    2024

  34. [42]

    Mm-react: Prompting chatgpt for multimodal reasoning and action.arXiv preprint arXiv:2303.11381, 2023

    Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Ehsan Azarnasab, Faisal Ahmed, Zicheng Liu, Ce Liu, Michael Zeng, and Lijuan Wang. Mm-react: Prompting chatgpt for multimodal reasoning and action.arXiv preprint arXiv:2303.11381, 2023

    2023 arXiv

  35. [43]

    Cocot: Contrastive chain-of-thought prompting for large multimodal models with multiple image inputs.arXiv preprint arXiv:2401.02582, 2024

    Daoan Zhang, Junming Yang, Hanjia Lyu, Zijian Jin, Yuan Yao, Mingkai Chen, and Jiebo Luo. Cocot: Contrastive chain-of-thought prompting for large multimodal models with multiple image inputs.arXiv preprint arXiv:2401.02582, 2024

    2024 arXiv

  36. [44]

    Compositional chain-of- thought prompting for large multimodal models

    Chancharik Mitra, Brandon Huang, Trevor Darrell, and Roei Herzig. Compositional chain-of- thought prompting for large multimodal models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14420–14431, 2024

    2024

  37. [45]

    Ddcot: Duty-distinct chain-of-thought prompting for multimodal reasoning in language models.Advances in Neural Information Processing Systems, 36:5168–5191, 2023

    Ge Zheng, Bin Yang, Jiajin Tang, Hong-Yu Zhou, and Sibei Yang. Ddcot: Duty-distinct chain-of-thought prompting for multimodal reasoning in language models.Advances in Neural Information Processing Systems, 36:5168–5191, 2023

    2023

  38. [46]

    Visual-o1: Understanding am- biguous instructions via multi-modal multi-turn chain-of-thoughts reasoning.arXiv preprint arXiv:2410.03321, 2024

    Minheng Ni, Yutao Fan, Lei Zhang, and Wangmeng Zuo. Visual-o1: Understanding am- biguous instructions via multi-modal multi-turn chain-of-thoughts reasoning.arXiv preprint arXiv:2410.03321, 2024

    2024 arXiv

  39. [47]

    Visual chain-of-thought prompting for knowledge-based visual reasoning

    Zhenfang Chen, Qinhong Zhou, Yikang Shen, Yining Hong, Zhiqing Sun, Dan Gutfreund, and Chuang Gan. Visual chain-of-thought prompting for knowledge-based visual reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 1254–1262, 2024

    2024

  40. [48]

    Llava-o1: Let vision language models reason step-by-step.arXiv preprint arXiv:2411.10440, 2024

    Guowei Xu, Peng Jin, Li Hao, Yibing Song, Lichao Sun, and Li Yuan. Llava-o1: Let vision language models reason step-by-step.arXiv preprint arXiv:2411.10440, 2024

    2024 arXiv

  41. [49]

    Insight-v: Exploring long-chain visual reasoning with multimodal large language models.arXiv preprint arXiv:2411.14432, 2024

    Yuhao Dong, Zuyan Liu, Hai-Long Sun, Jingkang Yang, Winston Hu, Yongming Rao, and Ziwei Liu. Insight-v: Exploring long-chain visual reasoning with multimodal large language models.arXiv preprint arXiv:2411.14432, 2024

    2024 arXiv

  42. [50]

    Llamav-o1: Rethinking step-by-step visual reasoning in llms.arXiv preprint arXiv:2501.06186, 2025

    Omkar Thawakar, Dinura Dissanayake, Ketan More, Ritesh Thawkar, Ahmed Heakl, Noor Ahsan, Yuhao Li, Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, et al. Llamav-o1: Rethinking step-by-step visual reasoning in llms.arXiv preprint arXiv:2501.06186, 2025

    2025 arXiv

  43. [51]

    Reinforced mllm: A survey on rl-based reasoning in multimodal large language models.arXiv preprint arXiv:2504.21277, 2025

    Guanghao Zhou, Panjia Qiu, Cen Chen, Jie Wang, Zheming Yang, Jian Xu, and Minghui Qiu. Reinforced mllm: A survey on rl-based reasoning in multimodal large language models.arXiv preprint arXiv:2504.21277, 2025

    2025 arXiv

  44. [52]

    Training language models to follow instructions with human feedback.Advances in neural information processing systems, 35:27730–27744, 2022

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback.Advances in neural information processing systems, 35:27730...

    2022

  45. [53]

    Relation-r1: Cognitive chain-of-thought guided reinforcement learning for unified relational comprehension.arXiv preprint arXiv:2504.14642, 2025

    Lin Li, Wei Chen, Jiahui Li, and Long Chen. Relation-r1: Cognitive chain-of-thought guided reinforcement learning for unified relational comprehension.arXiv preprint arXiv:2504.14642, 2025

    2025

  46. [54]

    Compile scene graphs with reinforcement learning.arXiv preprint arXiv:2504.13617, 2025

    Zuyao Chen, Jinlin Wu, Zhen Lei, Marc Pollefeys, and Chang Wen Chen. Compile scene graphs with reinforcement learning.arXiv preprint arXiv:2504.13617, 2025

    2025 arXiv

  47. [55]

    Embodied-r: Collaborative framework for activating embodied spatial reasoning in foundation models via reinforcement learning.arXiv preprint arXiv:2504.12680, 2025

    Baining Zhao, Ziyou Wang, Jianjie Fang, Chen Gao, Fanhang Man, Jinqiang Cui, Xin Wang, Xinlei Chen, Yong Li, and Wenwu Zhu. Embodied-r: Collaborative framework for activating embodied spatial reasoning in foundation models via reinforcement learning.arXiv preprint arXiv:2504.1...

    2025 arXiv

  48. [56]

    Improved visual-spatial reasoning via r1-zero-like training.arXiv preprint arXiv:2504.00883, 2025

    Zhenyi Liao, Qingsong Xie, Yanhao Zhang, Zijian Kong, Haonan Lu, Zhenyu Yang, and Zhijie Deng. Improved visual-spatial reasoning via r1-zero-like training.arXiv preprint arXiv:2504.00883, 2025

    2025 arXiv

  49. [57]

    Embodied-reasoner: Synergizing visual search, reasoning, and action for embodied interactive tasks.arXiv preprint arXiv:2503.21696, 2025

    Wenqi Zhang, Mengna Wang, Gangao Liu, Xu Huixin, Yiwei Jiang, Yongliang Shen, Guiyang Hou, Zhe Zheng, Hang Zhang, Xin Li, et al. Embodied-reasoner: Synergizing visual search, reasoning, and action for embodied interactive tasks.arXiv preprint arXiv:2503.21696, 2025. 13

    2025 arXiv

  50. [58]

    aha moment

    Hengguang Zhou, Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsieh. R1-zero’s" aha moment" in visual reasoning on a 2b non-sft model.arXiv preprint arXiv:2503.05132, 2025

    2025 arXiv

  51. [59]

    Perception-r1: Pioneering perception policy with reinforcement learning.arXiv preprint arXiv:2504.07954, 2025

    En Yu, Kangheng Lin, Liang Zhao, Jisheng Yin, Yana Wei, Yuang Peng, Haoran Wei, Jian- jian Sun, Chunrui Han, Zheng Ge, et al. Perception-r1: Pioneering perception policy with reinforcement learning.arXiv preprint arXiv:2504.07954, 2025

    2025 arXiv

  52. [60]

    Vlm-r1: A stable and generalizable r1-style large vision-language model.arXiv preprint arXiv:2504.07615, 2025

    Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, et al. Vlm-r1: A stable and generalizable r1-style large vision-language model.arXiv preprint arXiv:2504.07615, 2025

    2025 arXiv

  53. [61]

    Visual-rft: Visual reinforcement fine-tuning.arXiv preprint arXiv:2503.01785, 2025

    Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement fine-tuning.arXiv preprint arXiv:2503.01785, 2025

    2025 arXiv

  54. [62]

    Boosting the generalization and reasoning of vision language models with curriculum reinforcement learning

    Huilin Deng, Ding Zou, Rui Ma, Hongchen Luo, Yang Cao, and Yu Kang. Boosting the generalization and reasoning of vision language models with curriculum reinforcement learning. arXiv preprint arXiv:2503.07065, 2025

    2025 arXiv

  55. [63]

    Seg- zero: Reasoning-chain guided segmentation via cognitive reinforcement.arXiv preprint arXiv:2503.06520, 2025

    Yuqi Liu, Bohao Peng, Zhisheng Zhong, Zihao Yue, Fanbin Lu, Bei Yu, and Jiaya Jia. Seg- zero: Reasoning-chain guided segmentation via cognitive reinforcement.arXiv preprint arXiv:2503.06520, 2025

    2025 arXiv

  56. [64]

    Crowdvlm-r1: Expanding r1 ability to vision language model for crowd counting using fuzzy group relative policy reward.arXiv preprint arXiv:2504.03724, 2025

    Zhiqiang Wang, Pengbin Feng, Yanbin Lin, Shuzhang Cai, Zongao Bian, Jinghua Yan, and Xingquan Zhu. Crowdvlm-r1: Expanding r1 ability to vision language model for crowd counting using fuzzy group relative policy reward.arXiv preprint arXiv:2504.03724, 2025

    2025

  57. [65]

    Reason-rft: Reinforcement fine-tuning for visual reasoning.arXiv preprint arXiv:2503.20752, 2025

    Huajie Tan, Yuheng Ji, Xiaoshuai Hao, Minglan Lin, Pengwei Wang, Zhongyuan Wang, and Shanghang Zhang. Reason-rft: Reinforcement fine-tuning for visual reasoning.arXiv preprint arXiv:2503.20752, 2025

    2025

  58. [66]

    Othink- mr1: Stimulating multimodal generalized reasoning capabilities via dynamic reinforcement learning.arXiv preprint arXiv:2503.16081, 2025

    Zhiyuan Liu, Yuting Zhang, Feng Liu, Changwang Zhang, Ying Sun, and Jun Wang. Othink- mr1: Stimulating multimodal generalized reasoning capabilities via dynamic reinforcement learning.arXiv preprint arXiv:2503.16081, 2025

    2025 arXiv

  59. [67]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. InComputer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings,...

    2014

  60. [68]

    Segment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4015–4026, 2023

    2023

  61. [69]

    Visual genome: Connecting language and vision using crowdsourced dense image annotations.International journal of computer vision, 123:32–73, 2017

    Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations.International journal of computer v...

    2017

  62. [70]

    Gqa: A new dataset for real-world visual reasoning and compositional question answering

    Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6700–6709, 2019

    2019

  63. [71]

    Dual-glance model for deciphering social relationships

    Junnan Li, Yongkang Wong, Qi Zhao, and Mohan S Kankanhalli. Dual-glance model for deciphering social relationships. InProceedings of the IEEE international conference on computer vision, pages 2650–2659, 2017

    2017

  64. [72]

    Towards vqa models that can read

    Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8317–8326, 2019. 14

    2019

  65. [73]

    Docvqa: A dataset for vqa on document images

    Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. Docvqa: A dataset for vqa on document images. InProceedings of the IEEE/CVF winter conference on applications of computer vision, pages 2200–2209, 2021

    2021

  66. [74]

    Ocr-vqa: Visual question answering by reading text in images

    Anand Mishra, Shashank Shekhar, Ajeet Kumar Singh, and Anirban Chakraborty. Ocr-vqa: Visual question answering by reading text in images. In2019 international conference on document analysis and recognition (ICDAR), pages 947–952. IEEE, 2019

    2019

  67. [75]

    Chartqa: A benchmark for question answering about charts with visual and logical reasoning.arXiv preprint arXiv:2203.10244, 2022

    Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: A benchmark for question answering about charts with visual and logical reasoning.arXiv preprint arXiv:2203.10244, 2022

    2022 arXiv

  68. [76]

    An augmented benchmark dataset for geometric question answering through dual parallel text encoding

    Jie Cao and Jing Xiao. An augmented benchmark dataset for geometric question answering through dual parallel text encoding. InProceedings of the 29th international conference on computational linguistics, pages 1511–1520, 2022

    2022

  69. [77]

    Learn to explain: Multimodal reasoning via thought chains for science question answering.Advances in Neural Information Processing Systems, 35:2507–2521, 2022

    Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering.Advances in Neural Information Processing Systems, 35:2507–2521, 2022

    2022

  70. [78]

    A diagram is worth a dozen images

    Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. A diagram is worth a dozen images. InComputer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14, pages 235–251. S...

    2016

  71. [79]

    Clevr-math: A dataset for compositional language, visual and mathematical reasoning.arXiv preprint arXiv:2208.05358, 2022

    Adam Dahlgren Lindström and Savitha Sam Abraham. Clevr-math: A dataset for compositional language, visual and mathematical reasoning.arXiv preprint arXiv:2208.05358, 2022

    2022 arXiv

  72. [80]

    Large-scale classification of fine-art paintings: Learning the right metric on the right feature.arXiv preprint arXiv:1505.00855, 2015

    Babak Saleh and Ahmed Elgammal. Large-scale classification of fine-art paintings: Learning the right metric on the right feature.arXiv preprint arXiv:1505.00855, 2015

    2015 arXiv

  73. [81]

    Quality at a glance: An audit of web-crawled multilingual datasets.Transactions of the Association for Computational Linguistics, 10:50–72, 2022

    Julia Kreutzer, Isaac Caswell, Lisa Wang, Ahsan Wahab, Daan van Esch, Nasanbayar Ulzii- Orshikh, Allahsera Tapo, Nishant Subramani, Artem Sokolov, Claytone Sikasote, et al. Quality at a glance: An audit of web-crawled multilingual datasets.Transactions of the Association for C...

    2022

  74. [82]

    Mm-vet: Evaluating large multimodal models for integrated capabilities

    Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprint arXiv:2308.02490, 2023

    2023 arXiv

  75. [83]

    Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts.arXiv preprint arXiv:2310.02255, 2023

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts.arXiv preprint arXiv:2310.02255, 2023

    arXiv

  76. [84]

    Measuring multimodal mathematical reasoning with math-vision dataset

    Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with math-vision dataset. Advances in Neural Information Processing Systems, 37:95095–95169, 2024

    2024

  77. [85]

    Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? InEuropean Conference on Computer Vision, pages 169–186

    Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Yu Qiao, et al. Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? InEuropean Conference on Computer Vision, pages 169–186. Springer, 2024

    2024

  78. [86]

    Dynamath: A dynamic visual benchmark for evaluating mathematical reasoning robustness of vision language models.arXiv preprint arXiv:2411.00836, 2024

    Chengke Zou, Xingang Guo, Rui Yang, Junyu Zhang, Bin Hu, and Huan Zhang. Dynamath: A dynamic visual benchmark for evaluating mathematical reasoning robustness of vision language models.arXiv preprint arXiv:2411.00836, 2024

    2024 arXiv

  79. [87]

    We-math: Does your large multimodal model achieve human-like mathematical reasoning?arXiv preprint arXiv:2407.01284, 2024

    Runqi Qiao, Qiuna Tan, Guanting Dong, Minhui Wu, Chong Sun, Xiaoshuai Song, Zhuoma GongQue, Shanglin Lei, Zhe Wei, Miaoxuan Zhang, et al. We-math: Does your large multimodal model achieve human-like mathematical reasoning?arXiv preprint arXiv:2407.01284, 2024

    2024 arXiv

  80. [88]

    Logicvista: Multimodal llm logical reasoning benchmark in visual contexts.arXiv preprint arXiv:2407.04973, 2024

    Yijia Xiao, Edward Sun, Tianyu Liu, and Wei Wang. Logicvista: Multimodal llm logical reasoning benchmark in visual contexts.arXiv preprint arXiv:2407.04973, 2024. 15

    2024 arXiv

  81. [89]

    Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. InProceedings of the IEEE/CVF Conference on Co...

    2024

  82. [90]

    Mmbench: Is your multi-modal model an all-around player? InEuropean conference on computer vision, pages 216–233

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? InEuropean conference on computer vision, pages 216–233. Springer, 2024

    2024

  83. [91]

    Are we on the right way for evaluating large vision-language models?arXiv preprint arXiv:2403.20330, 2024

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. Are we on the right way for evaluating large vision-language models?arXiv preprint arXiv:2403.20330, 2024

    2024 arXiv

  84. [92]

    Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models

    Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, et al. Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models. InProcee...

    2024

  85. [93]

    Ocrbench: on the hidden mystery of ocr in large multimodal models.Science China Information Sciences, 67(12):220102, 2024

    Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, Xu-Cheng Yin, Cheng-Lin Liu, Lianwen Jin, and Xiang Bai. Ocrbench: on the hidden mystery of ocr in large multimodal models.Science China Information Sciences, 67(12):220102, 2024

    2024

  86. [94]

    Grok-1.5 vision preview: Connecting the digital and physical worlds with our first multimodal model

    X.AI Corp. Grok-1.5 vision preview: Connecting the digital and physical worlds with our first multimodal model. 2024

    2024

  87. [95]

    Vlmevalkit: An open-source toolkit for evaluating large multi-modality models

    Haodong Duan, Junming Yang, Yuxuan Qiao, Xinyu Fang, Lin Chen, Yuan Liu, Xiaoyi Dong, Yuhang Zang, Pan Zhang, Jiaqi Wang, et al. Vlmevalkit: An open-source toolkit for evaluating large multi-modality models. InProceedings of the 32nd ACM international conference on multimedia,...

    2024

  88. [96]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  89. [97]

    Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    2024 arXiv

  90. [98]

    Easyr1: An efficient, scalable, multi-modality rl training framework

    Yaowei Zheng, Junting Lu, Shenzhi Wang, Zhangchi Feng, Dongdong Kuang, and Yuwen Xiong. Easyr1: An efficient, scalable, multi-modality rl training framework. https://github. com/hiyouga/EasyR1, 2025

    2025

  91. [99]

    Vision Only

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv:2409.19256, 2024. 16 A The Details of Scalable Multimodal QA Synthesis We formulate the q...

    2024 arXiv

  92. [100]

    Total number of possible outcomes: There are 38 numbers on the roulette wheel

  93. [101]

    The Four Stages of a Man

    Favorable outcomes: There is only 1 way to win by betting on the exact three numbers 1-2-3. The probability P of winning is given by the ratio of the number of favorable outcomes to the total number of possible outcomes: P= Number of favorable outcomes Total number of possible...

    1900

  94. [102]

    Visit V1 and then follow the edges to its neighbors

  95. [103]

    Let’s follow the adjacency list step-by-step:

    Continue this process, visiting each vertex and its neighbors, ensuring that each vertex is visited only once. Let’s follow the adjacency list step-by-step:

  96. [104]

    V1 points to V5, so visit V5

  97. [105]

    V5 points to V4, so visit V4

  98. [106]

    V4 points to V7, so visit V7

  99. [107]

    V7 points to V6, so visit V6

  100. [108]

    V6 has no further neighbors, so backtrack to V5

  101. [109]

    Since V5 has already been visited, backtrack to V4

  102. [110]

    Since V4 has already been visited, backtrack to V1

  103. [111]

    Since V1 has already been visited, backtrack to V2

  104. [112]

    V2 has no further neighbors, so backtrack to V3

  105. [113]

    The sequence of vertices visited is: V1, V5, V4, V7, V6, V2, V3

    V3 has no further neighbors, so backtrack to V1. The sequence of vertices visited is: V1, V5, V4, V7, V6, V2, V3. Thus, the correct answer is: A. V1,V5,V4,V7,V6,V2,V3 WeThink-VL-7B: 24 <think> To determine the correct vertex sequence obtained by traversing the graph from verte...

  106. [114]

    V1 -> V5 (first child)

  107. [115]

    V5 -> V4 (first child)

  108. [116]

    V4 -> V7 (first child)

  109. [117]

    V7 -> V6 (first child)

  110. [118]

    V6 -> (no further children, backtrack)

  111. [119]

    V4 -> (already visited, backtrack)

  112. [120]

    V5 -> V3 (next child)

  113. [121]

    V3 -> V2 (first child)

  114. [122]

    V2 -> (no further children, backtrack)

  115. [123]

    V3 -> (already visited, backtrack)

  116. [124]

    V5 -> (already visited, backtrack)

  117. [125]

    Let’s match this with the given options: A

    V1 -> (already visited, backtrack) The sequence obtained is: V1, V5, V4, V7, V6, V3, V2. Let’s match this with the given options: A. V1,V5,V4,V7,V6,V2,V3 B. V1,V2,V3,V4,V7,V6,V5 C. V1,V5,V4,V7,V6,V3,V2 D. V1,V5,V6,V4,V7,V2,V3 The correct sequence is: V1, V5, V4, V7, V6, V3, V2...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.