Pith. sign in

REVIEW 3 major objections 5 minor 227 references

Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Models

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that reinforcement fine-tuning (RFT), not supervised fine-tuning alone, is what gives multimodal large language models their reasoning power across vision, audio, video, GUI, and agent settings.

desk verdict A useful map of RFT-for-MLLM work whose central causal claim outruns the evidence it surveys. read the letter →

arxiv 2505.18536 v1 pith:XJZ2HTY4 submitted 2025-05-24 cs.CL cs.AIcs.CV

Haoyuan Sun , Jiaqi Wu , Bo Xia , Yifu Luo , Yifei Zhao , Kai Qin , Xufei Lv , Tiantian Zhang
show 2 more authors
Yongzhe Chang Xueqian Wang
This is my paper
classification cs.CLcs.AIcs.CV
keywords reinforcementfine-tuningmultimodallargelanguagemodelsreasoningGRPOPPOchain-of-thoughtRLpost-trainingpositionpaper
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This position paper argues that reinforcement fine-tuning (RFT), the stage of training a large model with reinforcement learning after supervised instruction tuning, is the mechanism that turns multimodal large language models into reasoners. The authors contend that the recipe behind text-only reasoning systems such as OpenAI-o1 and DeepSeek-R1 is now being applied successfully across vision, video, audio, omni-multimodal, GUI, metaverse, and embodied-agent settings. They organize roughly a hundred recent works into five claims: RFT works across diverse modalities, across diverse tasks and domains, better training algorithms are emerging, benchmarks are maturing, and engineering frameworks are lowering the barrier to entry. If the paper is right, the path to multimodal reasoning runs through post-training with reinforcement signals, making reward design and RL infrastructure central research targets.

What carries the argument

The key machinery is the RFT training loop itself: a policy model initialized from an SFT model, a reward signal that is either an outcome reward (rule-based correctness) or a process reward model, and an on-policy RL optimizer, chiefly PPO (proximal policy optimization with a critic value model) or GRPO (group relative policy optimization, which replaces the critic with group-normalized rewards). The paper renames the common taxonomy as critic-model-driven versus critic-model-free algorithms, arguing this fits classical RL better than the earlier value-based/value-free distinction. This loop is what transforms a model that can describe images or audio into one that can reason step by step. The paper treats this loop as the portable ingredient that carries reasoning gains across modalities.

What would settle it

A controlled study that trains the same base model on the same curated multimodal reasoning data with the same compute budget under two conditions, one with the RL objective and one with continued supervised fine-tuning, would settle the matter: if the SFT-only model matches the RL model's benchmark gains, the position is falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is stated explicitly as its position: Reinforcement Fine-Tuning (RFT) powers reasoning capability of Multimodal Large Language Models (MLLMs). By RFT it means the post-training phase in which a model, typically initialized from a supervised fine-tuned base, is optimized by an RL objective such as PPO or GRPO with rule-based outcome rewards or learned process rewards. The paper reads the surge of results after DeepSeek-R1 as evidence that this paradigm transfers from text to multimodal domains: RFT reportedly improves visual math and multi-discipline reasoning, audio question answering, GUI action prediction, 3D metaverse spatial reasoning, video reasoning, medical and embodied vision, and even text-to-image generation. It classifies the community's progress into five successes and proposes five future directions: generalization across modalities, tasks, and domains; combining outcome and process rewards; safety of reasoning MLLMs; data augmentation; and better algorithms and reward paradigms.

Load-bearing premise

The claim rests on the assumption that the reasoning gains reported in the surveyed works are caused by RFT itself, and not by earlier supervised fine-tuning, the choice of reward, curated data, extra compute, or test-time filtering.

Editorial extensions

If this is right

  • If RFT is the powering ingredient, training pipelines for multimodal reasoners should allocate marginal compute to RL post-training rather than only to further supervised data collection.
  • Rule-based outcome rewards should keep working across tasks where correctness is verifiable, lowering the cost of multimodal reasoning training.
  • The critic-model-free line (GRPO and variants) is likely to dominate in multimodal settings because it avoids the memory and stability cost of a critic.
  • Benchmarks should be designed to test reasoning under RL training, not just static accuracy, since the paper reports trends toward harder, interactive, and process-oriented benchmarks.
  • A generalizable reasoning model spanning modalities is a plausible near-term goal if RFT transfers as the cited works suggest.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's position would be stronger with controlled ablations: the surveyed works mostly compare RFT against an SFT baseline, so gains could partly come from more training compute, better data curation, or reward design rather than from the RL algorithm per se.
  • One untested extension suggested by the position is unified reward models shared across modalities; the paper mentions multimodal reward training as a direction but does not propose a single cross-modal reward standard.
  • If RFT is modality-agnostic, abrupt reasoning gains akin to vision's 'aha moments' should appear in audio and GUI agents too; the paper documents such cases anecdotally but does not measure them systematically.
  • Safety may become the binding constraint: the paper flags reward hacking, jailbreaks, and overthinking as future work, which follows directly from giving verifiable rewards more weight.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This position paper argues that reinforcement fine-tuning (RFT) powers the reasoning capability of multimodal large language models (MLLMs). After an introductory background on reinforcement learning and multimodal reasoning, the paper organizes recent work into five claimed successes: diverse modalities, diverse tasks and domains, better training algorithms, abundant benchmarks, and thriving engineering frameworks. It then proposes five future research directions. A companion GitHub repository collects the surveyed work. The paper contains no new experiments or quantitative meta-analysis; its central claim rests on a qualitative synthesis of recent preprints and open-source projects.

Significance. If the central claim were established, the paper would make a useful case for RFT as a general post-training mechanism for multimodal reasoning, organizing a fast-moving literature and giving newcomers a structured map of methods, benchmarks, and frameworks. The paper's strengths are its breadth of coverage (vision, video, audio, omni, GUI, medical, embodied, metaverse, agents, and generation), its taxonomy of critic-model-driven versus critic-model-free RFT algorithms, and the accompanying curated repository with links to code and models. However, the paper makes no original measurements, and the central causal claim is substantially stronger than the evidence it marshals: the cited works typically couple RFT with SFT cold starts, curated data selection, reward shaping, or additional compute, and the survey neither isolates the RFT component nor engages with null or mixed results. The paper is therefore more an annotated bibliography than a supported position statement in its current form.

major comments (3)
  1. [Section 3, opening paragraph and Successes 1-3] The central claim that 'RFT has significantly enhanced the reasoning abilities of MLLMs' is not supported by the cited evidence because no cited work isolates the RFT component from confounds. For example, Virgo [69] and LMM-R1 [72] use SFT cold starts, ThinkLite-VL [97] uses MCTS-filtered data, R1-VL [76] adds step-wise rewards, and OpenVLThinker [77] iterates SFT and GRPO with evolving data. The survey never reports an ablation in which only the RL component is toggled while base model, data, and inference budget are held fixed, nor does it discuss null or negative results. In particular, VLAA-Thinking [98] is cited as a success, yet its title and content directly investigate 'SFT or RL?' and report nuanced trade-offs; ignoring that result makes the causal attribution to RFT underdetermined. As a position paper, the argument must at least acknowledge these confounds and justify why RFT, rather than SFT data quality, reward design, or extra compute, is the operative factor.
  2. [Section 3, Successes 4-5] The sections on 'Abundant Benchmarks' and 'Thriving Engineering Frameworks' do not provide evidence for the causal claim that RFT powers reasoning. The proliferation of evaluation benchmarks and open-source RL frameworks shows community activity and lowered engineering barriers, but it says nothing about whether RFT is responsible for observed capability gains. Including these as 'successes' of RFT conflates the existence of tools with the efficacy of the method, weakening the argument's focus.
  3. [Appendix A and Section 3 text] The survey gives the same evidential weight to unreviewed code repositories as to full papers. Works such as VAGEN [56], MMR1 [73], R1-Track [91], and Open-R1-Video [102] are listed under 'Open-Source Projects (Repository without Paper)' and yet are cited in the text as demonstrating RFT effectiveness (e.g., 'VAGEN has advanced the training of VLM-based visual agents'). Without published results, benchmarks, or ablations, these items cannot support the strong causal claim. The authors should either reclassify such entries as community resources only or find peer-reviewed evidence for the specific claims made.
minor comments (5)
  1. [Section 2.1] The text says 'A donates the action space' and 's0 is the inital state'; both should be corrected to 'denotes' and 'initial state', respectively.
  2. [Section 2.3, Eq. (7)] The definition of the group relative advantage contains a typo: the paper writes 'ˆAi,t = eri = (r(oi,a) − mean(...))/std(...)', which mixes an exponential with the normalization formula. The GRPO advantage is the normalized reward, not an exponentiated one; please correct the equation or the surrounding explanation.
  3. [Section 3, bulleted questions in the introduction to Section 3] The sentence 'the community has made significant process' should read 'significant progress'.
  4. [Section 4, TO DO 3] The phrase 'According to [28], this point can generally be divided into three components' is unclear, since the three listed components (reward hacking, jailbreak attacks, overthinking) do not appear to be the organization of reference [28]; please clarify the attribution or rephrase.
  5. [Appendix A] Inconsistent naming appears for 'MAYE [144]' versus 'MAYA [144]' in the main text, and the appendix contains spacing artifacts such as 'OpenLLaV A-Video-R1' and 'GUI-R1 [52]' versus 'GUI-R1 [53]' in Section 3; these should be harmonized.

Circularity Check

0 steps flagged · score 0.0 of 10

No construction-circularity: the survey's position is an external synthesis of cited empirical results, not a derivation from its own assumptions.

full rationale

The paper is a position paper/survey. Its POSITION is stated in Section 1 and repeated in Section 3, but the supporting evidence is a list of externally reported RFT-for-MLLM papers and benchmarks (Sections 3 and Appendix A), each with its own independent training recipes and evaluations. There is no fitted parameter later renamed as a prediction, no target quantity defined in terms of itself, no unique-solution theorem imported from the authors' own prior work, and no equation in which the output is identical to the input by construction. The closest concern is evidentiary rather than circular: each cited work includes SFT cold starts, data filtering, reward shaping, or extra compute (e.g., Virgo [69], LMM-R1 [72], ThinkLite-VL [97], OpenVLThinker [77]), so the survey does not isolate the RFT component; and VLAA-Thinking [98], titled 'SFT or RL?', is cited as evidence of RFT success without its trade-off conclusion being discussed. That is a threat to the strength of the causal generalization, but it is a confound/rigor critique, not a case where the paper's derivation reduces to its inputs. The paper's background sections restate standard RL definitions from external textbooks and papers. No self-citation is load-bearing. Accordingly the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper contributes no fitted parameters and no new postulated entities. Its claim rests on the trustworthiness and attribution of the surveyed empirical literature, plus standard RL background. No independent verification of the cited numbers is included.

assumptions (3)
  • domain assumption Reported improvements in the cited works are attributable to RFT rather than to auxiliary factors such as SFT stages, data curation, reward design, or added compute.
    Section 3 and Appendix A present each surveyed paper as evidence for RFT's effectiveness, but no controlled comparison isolating the RFT component is provided.
  • domain assumption The surveyed works, including unreviewed preprints and open-source repositories, are treated as valid demonstrations of the claim.
    Section 3 and Appendix A mix peer-reviewed papers, preprints, and repositories without quality weighting or negative-result accounting.
  • standard math Standard reinforcement learning background, including MDPs, PPO, and GRPO, as presented in Section 2, is correct.
    The survey relies on standard equations for PPO and GRPO, but the position claim does not depend on the details of these derivations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Models." pith.science (2026). https://pith.science/paper/XJZ2HTY4

@misc{pith2026250518536,
  author       = {Pith},
  title        = {Pith review of: Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XJZ2HTY4}},
  note         = {Machine review of arXiv:2505.18536}
}
read the original abstract

Standing in 2025, at a critical juncture in the pursuit of Artificial General Intelligence (AGI), reinforcement fine-tuning (RFT) has demonstrated significant potential in enhancing the reasoning capability of large language models (LLMs) and has led to the development of cutting-edge AI models such as OpenAI-o1 and DeepSeek-R1. Moreover, the efficient application of RFT to enhance the reasoning capability of multimodal large language models (MLLMs) has attracted widespread attention from the community. In this position paper, we argue that reinforcement fine-tuning powers the reasoning capability of multimodal large language models. To begin with, we provide a detailed introduction to the fundamental background knowledge that researchers interested in this field should be familiar with. Furthermore, we meticulously summarize the improvements of RFT in powering reasoning capability of MLLMs into five key points: diverse modalities, diverse tasks and domains, better training algorithms, abundant benchmarks and thriving engineering frameworks. Finally, we propose five promising directions for future research that the community might consider. We hope that this position paper will provide valuable insights to the community at this pivotal stage in the advancement toward AGI. Summary of works done on RFT for MLLMs is available at https://github.com/Sun-Haoyuan23/Awesome-RL-based-Reasoning-MLLMs.

Figures

Figures reproduced from arXiv: 2505.18536 by the authors.

Figure 1
Figure 1. An overview of works done on reinforcement fine-tuning (RFT) for multimodal large [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

227 extracted references · 1 canonical work pages

  1. [98]

    Sft or rl? an early investigation into training r1-like reasoning large vision- language models

    Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. Sft or rl? an early investigation into training r1-like reasoning large vision- language models. arXiv preprint arXiv:2504.11468, 2025

    arXiv 2025

  2. [69]

    Virgo: A preliminary exploration on reproducing o1-like mllm

    Yifan Du, Zikang Liu, Yifan Li, Wayne Xin Zhao, Yuqi Huo, Bingning Wang, Weipeng Chen, Zheng Liu, Zhongyuan Wang, and Ji-Rong Wen. Virgo: A preliminary exploration on reproducing o1-like mllm. arXiv preprint arXiv:2501.01904, 2025

    arXiv 2025

  3. [72]

    Lmm-r1: Empowering 3b lmms with strong reasoning abilities through two-stage rule-based rl

    Yingzhe Peng, Gongrui Zhang, Miaosen Zhang, Zhiyuan You, Jie Liu, Qipeng Zhu, Kai Yang, Xingzhong Xu, Xin Geng, and Xu Yang. Lmm-r1: Empowering 3b lmms with strong reasoning abilities through two-stage rule-based rl. arXiv preprint arXiv:2503.07536, 2025

    arXiv 2025

  4. [97]

    Sota with less: Mcts-guided sample selection for data-efficient visual reasoning self-improvement

    Xiyao Wang, Zhengyuan Yang, Chao Feng, Hongjin Lu, Linjie Li, Chung-Ching Lin, Kevin Lin, Furong Huang, and Lijuan Wang. Sota with less: Mcts-guided sample selection for data-efficient visual reasoning self-improvement. arXiv preprint arXiv:2504.07934, 2025

    arXiv 2025

  5. [76]

    R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization

    Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization. arXiv preprint arXiv:2503.12937, 2025

    arXiv 2025

  6. [77]

    Open- vlthinker: An early exploration to complex vision-language reasoning via iterative self- improvement

    Yihe Deng, Hritik Bansal, Fan Yin, Nanyun Peng, Wei Wang, and Kai-Wei Chang. Open- vlthinker: An early exploration to complex vision-language reasoning via iterative self- improvement. arXiv preprint arXiv:2503.17352, 2025

    arXiv 2025

  7. [56]

    Vagen: Training vlm agents with multi-turn reinforcement learning, 2025

    Kangrui Wang, Pingyue Zhang, Zihan Wang, Qineng Wang, Yaning Gao, Linjie Li, Zhengyuan Yang, Chi Wan, Hanyang Chen, Yiping Lu, and Manling Li. Vagen: Training vlm agents with multi-turn reinforcement learning, 2025. URL https://github.com/RAGEN-AI/ VAGEN

    2025

  8. [73]

    Mmr1: Ad- vancing the frontiers of multimodal reasoning

    Sicong Leng, Jing Wang, Jiaxi Li, Hao Zhang, Zhiqiang Hu, Boqiang Zhang, Hang Zhang, Yuming Jiang, Xin Li, Deli Zhao, Fan Wang, Yu Rong, Aixin Sun, and Shijian Lu. Mmr1: Ad- vancing the frontiers of multimodal reasoning. https://github.com/LengSicong/ MMR1, 2025

    2025

  9. [91]

    R1-track: Direct application of mllms to visual object tracking via reinforcement learning

    Biao Wang. R1-track: Direct application of mllms to visual object tracking via reinforcement learning. https://github.com/Wangbiao2/R1-Track, 2025

    2025

  10. [102]

    Open-r1-video

    Xiaodong Wang and Peixi Peng. Open-r1-video. https://github.com/ Wang-Xiaodong1899/Open-R1-Video , 2025

    2025

Show all 227 references
  1. [1]

    Reinforcement learning: An introduction, volume 1

    Richard S Sutton, Andrew G Barto, et al. Reinforcement learning: An introduction, volume 1. MIT press Cambridge, 1998

    1998

  2. [2]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  3. [3]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:2773...

    2022

  4. [4]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  5. [5]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  6. [6]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  7. [7]

    Introducing openai o1

    OpenAI. Introducing openai o1. https://openai.com/o1/, 2024

    2024

  8. [8]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  9. [9]

    Mind with eyes: from language reasoning to multimodal reasoning

    Zhiyu Lin, Yifei Gao, Xian Zhao, Yunfan Yang, and Jitao Sang. Mind with eyes: from language reasoning to multimodal reasoning. arXiv preprint arXiv:2503.18071, 2025

    2025 arXiv

  10. [10]

    Reinforced mllm: A survey on rl-based reasoning in multimodal large language models

    Guanghao Zhou, Panjia Qiu, Cen Chen, Jie Wang, Zheming Yang, Jian Xu, and Minghui Qiu. Reinforced mllm: A survey on rl-based reasoning in multimodal large language models. arXiv preprint arXiv:2504.21277, 2025

    2025 arXiv

  11. [11]

    A markovian decision process

    Richard Bellman. A markovian decision process. Journal of mathematics and mechanics, pages 679–684, 1957

    1957

  12. [12]

    Convergence of q-learning: A simple proof

    Francisco S Melo. Convergence of q-learning: A simple proof. Institute Of Systems and Robotics, Tech. Rep, pages 1–4, 2001

    2001

  13. [13]

    Playing atari with deep reinforcement learning.arXiv preprint arXiv:1312.5602, 2013

    V olodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning.arXiv preprint arXiv:1312.5602, 2013

    2013 arXiv

  14. [14]

    Human-level control through deep reinforcement learning

    V olodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. nature, 518(7540):529–533, 2015

    2015

  15. [15]

    Deep reinforcement learning with double q-learning

    Hado Van Hasselt, Arthur Guez, and David Silver. Deep reinforcement learning with double q-learning. In Proceedings of the AAAI conference on artificial intelligence, volume 30, 2016. 10

    2016

  16. [16]

    Dueling network architectures for deep reinforcement learning

    Ziyu Wang, Tom Schaul, Matteo Hessel, Hado Hasselt, Marc Lanctot, and Nando Freitas. Dueling network architectures for deep reinforcement learning. In International conference on machine learning, pages 1995–2003. PMLR, 2016

    1995

  17. [17]

    Rainbow: Combining improvements in deep reinforcement learning

    Matteo Hessel, Joseph Modayil, Hado Van Hasselt, Tom Schaul, Georg Ostrovski, Will Dabney, Dan Horgan, Bilal Piot, Mohammad Azar, and David Silver. Rainbow: Combining improvements in deep reinforcement learning. In Proceedings of the AAAI conference on artificial intelligence,...

    2018

  18. [18]

    Policy gradi- ent methods for reinforcement learning with function approximation

    Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. Policy gradi- ent methods for reinforcement learning with function approximation. Advances in neural information processing systems, 12, 1999

    1999

  19. [19]

    Actor-critic algorithms

    Vijay Konda and John Tsitsiklis. Actor-critic algorithms. Advances in neural information processing systems, 12, 1999

    1999

  20. [20]

    Asynchronous methods for deep reinforce- ment learning

    V olodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforce- ment learning. In International conference on machine learning, pages 1928–1937. PmLR, 2016

    1928

  21. [21]

    Trust region policy optimization

    John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. Trust region policy optimization. In International conference on machine learning, pages 1889–1897. PMLR, 2015

    2015

  22. [22]

    Llm as a mastermind: A survey of strategic reasoning with large language models

    Yadong Zhang, Shaoguang Mao, Tao Ge, Xun Wang, Adrian de Wynter, Yan Xia, Wenshan Wu, Ting Song, Man Lan, and Furu Wei. Llm as a mastermind: A survey of strategic reasoning with large language models. arXiv preprint arXiv:2404.01230, 2024

    2024 arXiv

  23. [23]

    Reasoning with large language models, a survey

    Aske Plaat, Annie Wong, Suzan Verberne, Joost Broekens, Niki van Stein, and Thomas Back. Reasoning with large language models, a survey. arXiv preprint arXiv:2407.11511, 2024

    2024

  24. [24]

    Towards large reasoning models: A survey of reinforced reasoning with large language models

    Fengli Xu, Qianyue Hao, Zefang Zong, Jingwei Wang, Yunke Zhang, Jingyi Wang, Xiaochong Lan, Jiahui Gong, Tianjian Ouyang, Fanjin Meng, et al. Towards large reasoning models: A survey of reinforced reasoning with large language models. arXiv preprint arXiv:2501.09686, 2025

    2025 arXiv

  25. [25]

    A tutorial on llm reasoning: Relevant methods behind chatgpt o1

    Jun Wang. A tutorial on llm reasoning: Relevant methods behind chatgpt o1. arXiv preprint arXiv:2502.10867, 2025

    2025 arXiv

  26. [26]

    Thinking machines: A survey of llm based reasoning strategies

    Dibyanayan Bandyopadhyay, Soham Bhattacharjee, and Asif Ekbal. Thinking machines: A survey of llm based reasoning strategies. arXiv preprint arXiv:2503.10814, 2025

    2025 arXiv

  27. [27]

    Towards reasoning era: A survey of long chain-of-thought for reasoning large language models

    Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. Towards reasoning era: A survey of long chain-of-thought for reasoning large language models. arXiv preprint arXiv:2503.09567, 2025

    2025 arXiv

  28. [28]

    100 days after deepseek-r1: A survey on replication studies and more directions for reasoning language models

    Chong Zhang, Yue Deng, Xiang Lin, Bin Wang, Dianwen Ng, Hai Ye, Xingxuan Li, Yao Xiao, Zhanfeng Mo, Qi Zhang, et al. 100 days after deepseek-r1: A survey on replication studies and more directions for reasoning language models. arXiv preprint arXiv:2505.00551, 2025

    2025 arXiv

  29. [29]

    Investigating inference-time scaling for chain of multi-modal thought: A preliminary study

    Yujie Lin, Ante Wang, Moye Chen, Jingyao Liu, Hao Liu, Jinsong Su, and Xinyan Xiao. Investigating inference-time scaling for chain of multi-modal thought: A preliminary study. arXiv preprint arXiv:2502.11514, 2025

    2025 arXiv

  30. [30]

    Multimodal chain-of-thought reasoning: A comprehensive survey

    Yaoting Wang, Shengqiong Wu, Yuecheng Zhang, Shuicheng Yan, Ziwei Liu, Jiebo Luo, and Hao Fei. Multimodal chain-of-thought reasoning: A comprehensive survey. arXiv preprint arXiv:2503.12605, 2025

    2025 arXiv

  31. [31]

    Perception, reason, think, and plan: A survey on large multimodal reasoning models, 2025

    Yunxin Li, Zhenyu Liu, Zitao Li, Xuanyu Zhang, Zhenran Xu, Xinyu Chen, Haoyuan Shi, Shenyuan Jiang, Xintong Wang, Jifang Wang, Shouzheng Huang, Xinping Zhao, Borui Jiang, Lanqing Hong, Longyue Wang, Zhuotao Tian, Baoxing Huai, Wenhan Luo, Weihua Luo, Zheng Zhang, Baotian Hu, a...

    2025

  32. [32]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599, 2025

    2025 arXiv

  33. [33]

    Introducing openai o3 and o4-mini

    OpenAI. Introducing openai o3 and o4-mini. https://openai.com/index/ introducing-o3-and-o4-mini/ , 2025

    2025

  34. [34]

    Grok 3 beta — the age of reasoning agents

    xAI. Grok 3 beta — the age of reasoning agents. https://x.ai/grok, 2025

    2025

  35. [35]

    Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model

    Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum. Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model. arXiv preprint arXiv:2503.24290, 2025

    2025 arXiv

  36. [36]

    High- dimensional continuous control using generalized advantage estimation

    John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. High- dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438, 2015

    2015 arXiv

  37. [37]

    What’s behind ppo’s collapse in long-cot? value optimization holds the secret

    Yufeng Yuan, Yu Yue, Ruofei Zhu, Tiantian Fan, and Lin Yan. What’s behind ppo’s collapse in long-cot? value optimization holds the secret. arXiv preprint arXiv:2503.01491, 2025

    2025 arXiv

  38. [38]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  39. [39]

    Approximating kl divergence

    John Schulman. Approximating kl divergence. http://joschu.net/blog/ kl-approx.html, 2020

    2020

  40. [40]

    Dapo: An open-source llm reinforcement learning system at scale

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476, 2025

    2025 arXiv

  41. [41]

    Understanding r1-zero-like training: A critical perspective

    Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding r1-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783, 2025

    2025 arXiv

  42. [42]

    Light-r1: Curriculum sft, dpo and rl for long cot from scratch and beyond

    Liang Wen, Yunke Cai, Fenrui Xiao, Xin He, Qi An, Zhenyu Duan, Yimin Du, Junchen Liu, Lifu Tang, Xiaowei Lv, et al. Light-r1: Curriculum sft, dpo and rl for long cot from scratch and beyond. arXiv preprint arXiv:2503.10460, 2025

    2025 arXiv

  43. [43]

    Tinyzero

    Jiayi Pan, Junjie Zhang, Xingyao Wang, Lifan Yuan, Hao Peng, and Alane Suhr. Tinyzero. https://github.com/Jiayi-Pan/TinyZero, 2025. Accessed: 2025-01-24

    2025

  44. [44]

    Can 1b llm surpass 405b llm? rethinking compute-optimal test-time scaling

    Runze Liu, Junqi Gao, Jian Zhao, Kaiyan Zhang, Xiu Li, Biqing Qi, Wanli Ouyang, and Bowen Zhou. Can 1b llm surpass 405b llm? rethinking compute-optimal test-time scaling. arXiv preprint arXiv:2502.06703, 2025

    2025 arXiv

  45. [45]

    Visualprm: An effective process reward model for multimodal reasoning

    Weiyun Wang, Zhangwei Gao, Lianjie Chen, Zhe Chen, Jinguo Zhu, Xiangyu Zhao, Yangzhou Liu, Yue Cao, Shenglong Ye, Xizhou Zhu, et al. Visualprm: An effective process reward model for multimodal reasoning. arXiv preprint arXiv:2503.10291, 2025

    2025 arXiv

  46. [46]

    Can we generate images with cot? let’s verify and reinforce image generation step by step

    Ziyu Guo, Renrui Zhang, Chengzhuo Tong, Zhizheng Zhao, Peng Gao, Hongsheng Li, and Pheng-Ann Heng. Can we generate images with cot? let’s verify and reinforce image generation step by step. arXiv preprint arXiv:2501.13926, 2025

    2025 arXiv

  47. [47]

    Audio-reasoner: Improving reasoning capability in large audio language models

    Zhifei Xie, Mingbao Lin, Zihang Liu, Pengcheng Wu, Shuicheng Yan, and Chunyan Miao. Audio-reasoner: Improving reasoning capability in large audio language models. arXiv preprint arXiv:2503.02318, 2025

    2025

  48. [48]

    Reinforce- ment learning outperforms supervised fine-tuning: A case study on audio question answering

    Gang Li, Jizhong Liu, Heinrich Dinkel, Yadong Niu, Junbo Zhang, and Jian Luan. Reinforce- ment learning outperforms supervised fine-tuning: A case study on audio question answering. arXiv preprint arXiv:2503.11197, 2025

    2025 arXiv

  49. [49]

    Sari: Structured audio reasoning via curriculum-guided reinforcement learning

    Cheng Wen, Tingwei Guo, Shuaijiang Zhao, Wei Zou, and Xiangang Li. Sari: Structured audio reasoning via curriculum-guided reinforcement learning. arXiv preprint arXiv:2504.15900, 2025. 12

    2025 arXiv

  50. [50]

    R1-omni: Explainable omni-multimodal emotion recognition with reinforcement learning

    Jiaxing Zhao, Xihan Wei, and Liefeng Bo. R1-omni: Explainable omni-multimodal emotion recognition with reinforcement learning. arXiv preprint arXiv:2503.05379, 2025

    2025 arXiv

  51. [51]

    Echoink-r1: Exploring audio-visual reasoning in multimodal llms via reinforcement learning

    Zhenghao Xing, Xiaowei Hu, Chi-Wing Fu, Wenhai Wang, Jifeng Dai, and Pheng-Ann Heng. Echoink-r1: Exploring audio-visual reasoning in multimodal llms via reinforcement learning. arXiv preprint arXiv:2505.04623, 2025

    2025 arXiv

  52. [52]

    Ui-r1: Enhancing action prediction of gui agents by reinforcement learning

    Zhengxi Lu, Yuxiang Chai, Yaxuan Guo, Xi Yin, Liang Liu, Hao Wang, Guanjing Xiong, and Hongsheng Li. Ui-r1: Enhancing action prediction of gui agents by reinforcement learning. arXiv preprint arXiv:2503.21620, 2025

    2025 arXiv

  53. [53]

    Gui-r1: A generalist r1-style vision-language action model for gui agents

    Run Luo, Lu Wang, Wanwei He, and Xiaobo Xia. Gui-r1: A generalist r1-style vision-language action model for gui agents. arXiv preprint arXiv:2504.10458, 2025

    2025 arXiv

  54. [54]

    Infigui-r1: Advancing multimodal gui agents from reactive actors to deliberative reasoners

    Yuhang Liu, Pengxiang Li, Congkai Xie, Xavier Hu, Xiaotian Han, Shengyu Zhang, Hongxia Yang, and Fei Wu. Infigui-r1: Advancing multimodal gui agents from reactive actors to deliberative reasoners. arXiv preprint arXiv:2504.14239, 2025

    2025 arXiv

  55. [55]

    Metaspatial: Reinforcing 3d spatial reasoning in vlms for the metaverse

    Zhenyu Pan and Han Liu. Metaspatial: Reinforcing 3d spatial reasoning in vlms for the metaverse. arXiv preprint arXiv:2503.18470, 2025

    2025

  56. [57]

    Inter-gps: Interpretable geometry problem solving with formal language and symbolic reasoning

    Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiaodan Liang, and Song-Chun Zhu. Inter-gps: Interpretable geometry problem solving with formal language and symbolic reasoning. arXiv preprint arXiv:2105.04165, 2021

    2021 arXiv

  57. [58]

    Measuring multimodal mathematical reasoning with math-vision dataset

    Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with math-vision dataset. Advances in Neural Information Processing Systems, 37:95095–95169, 2024

    2024

  58. [59]

    Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? In European Conference on Computer Vision, pages 169–186

    Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Yu Qiao, et al. Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? In European Conference on Computer Vision, pages 169–186. Sprin...

    2024

  59. [60]

    Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255, 2023

    arXiv

  60. [61]

    Mm-math: Advancing multimodal math evaluation with process evaluation and fine-grained classification

    Kai Sun, Yushi Bai, Ji Qi, Lei Hou, and Juanzi Li. Mm-math: Advancing multimodal math evaluation with process evaluation and fine-grained classification. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 1358–1375, 2024

    2024

  61. [62]

    We-math: Does your large multi- modal model achieve human-like mathematical reasoning? arXiv preprint arXiv:2407.01284, 2024

    Runqi Qiao, Qiuna Tan, Guanting Dong, Minhui Wu, Chong Sun, Xiaoshuai Song, Zhuoma GongQue, Shanglin Lei, Zhe Wei, Miaoxuan Zhang, et al. We-math: Does your large multi- modal model achieve human-like mathematical reasoning? arXiv preprint arXiv:2407.01284, 2024

    2024 arXiv

  62. [63]

    Dynamath: A dynamic visual benchmark for evaluating mathematical reasoning robustness of vision language models

    Chengke Zou, Xingang Guo, Rui Yang, Junyu Zhang, Bin Hu, and Huan Zhang. Dynamath: A dynamic visual benchmark for evaluating mathematical reasoning robustness of vision language models. In The Thirteenth International Conference on Learning Representations,

  63. [64]

    Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems

    Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, et al. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. arXiv preprint arXiv:2402....

    2024 arXiv

  64. [65]

    Mmmu-pro: A more robust multi-discipline multimodal understanding benchmark

    Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, et al. Mmmu-pro: A more robust multi-discipline multimodal understanding benchmark. arXiv preprint arXiv:2409.02813, 2024

    2024 arXiv

  65. [66]

    Can mllms reason in multimodality? emma: An enhanced multimodal reasoning benchmark

    Yunzhuo Hao, Jiawei Gu, Huichen Will Wang, Linjie Li, Zhengyuan Yang, Lijuan Wang, and Yu Cheng. Can mllms reason in multimodality? emma: An enhanced multimodal reasoning benchmark. arXiv preprint arXiv:2501.05444, 2025

    2025 arXiv

  66. [67]

    Enhancing the reasoning ability of multimodal large language models via mixed preference optimization

    Weiyun Wang, Zhe Chen, Wenhai Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Jinguo Zhu, Xizhou Zhu, Lewei Lu, Yu Qiao, et al. Enhancing the reasoning ability of multimodal large language models via mixed preference optimization. arXiv preprint arXiv:2411.10442, 2024

    2024 arXiv

  67. [68]

    Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search

    Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, et al. Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search. arXiv preprint arXiv:2412.18319, 2024

    2024 arXiv

  68. [70]

    Mm- eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning

    F Meng, L Du, Z Liu, Z Zhou, Q Lu, D Fu, B Shi, W Wang, J He, K Zhang, et al. Mm- eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning. arXiv preprint arXiv:2503.07365, 2025

    2025 arXiv

  69. [71]

    Vision-r1: Incentivizing reasoning capability in multimodal large language models

    Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-r1: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749, 2025

    2025 arXiv

  70. [74]

    R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization

    Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, et al. R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization. arXiv preprint arXiv:2503.10615, 2025

    2025 arXiv

  71. [75]

    Skywork r1v: Pioneering multimodal reasoning with chain-of-thought

    Yi Peng, Xiaokun Wang, Yichen Wei, Jiangbo Pei, Weijie Qiu, Ai Jian, Yunzhuo Hao, Jiachun Pan, Tianyidan Xie, Li Ge, et al. Skywork r1v: Pioneering multimodal reasoning with chain-of-thought. arXiv preprint arXiv:2504.05599, 2025

    2025 arXiv

  72. [78]

    Vl- rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning

    Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. Vl- rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning. arXiv preprint arXiv:2504.08837, 2025

    2025 arXiv

  73. [79]

    Noisyrollout: Reinforcing visual reasoning with data augmentation

    Xiangyan Liu, Jinjie Ni, Zijian Wu, Chao Du, Longxu Dou, Haonan Wang, Tianyu Pang, and Michael Qizhe Shieh. Noisyrollout: Reinforcing visual reasoning with data augmentation. arXiv preprint arXiv:2504.13055, 2025. 14

    2025

  74. [80]

    Skywork r1v2: Multimodal hybrid reinforcement learning for reasoning

    Yichen Wei, Yi Peng, Xiaokun Wang, Weijie Qiu, Wei Shen, Tianyidan Xie, Jiangbo Pei, Jianhao Zhang, Yunzhuo Hao, Xuchen Song, et al. Skywork r1v2: Multimodal hybrid reinforcement learning for reasoning. arXiv preprint arXiv:2504.16656, 2025

    2025 arXiv

  75. [81]

    Fast-slow thinking for large vision-language model reasoning

    Wenyi Xiao, Leilei Gan, Weilong Dai, Wanggui He, Ziwei Huang, Haoyuan Li, Fangxun Shu, Zhelun Yu, Peng Zhang, Hao Jiang, et al. Fast-slow thinking for large vision-language model reasoning. arXiv preprint arXiv:2504.18458, 2025

    2025

  76. [82]

    Vlm-r1: A stable and generalizable r1-style large vision-language model

    Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, et al. Vlm-r1: A stable and generalizable r1-style large vision-language model. arXiv preprint arXiv:2504.07615, 2025

    2025 arXiv

  77. [83]

    Crowdvlm-r1: Expanding r1 ability to vision language model for crowd counting using fuzzy group relative policy reward

    Zhiqiang Wang, Pengbin Feng, Yanbin Lin, Shuzhang Cai, Zongao Bian, Jinghua Yan, and Xingquan Zhu. Crowdvlm-r1: Expanding r1 ability to vision language model for crowd counting using fuzzy group relative policy reward. arXiv preprint arXiv:2504.03724, 2025

    2025

  78. [84]

    aha moment

    Hengguang Zhou, Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsieh. R1-zero’s" aha moment" in visual reasoning on a 2b non-sft model. arXiv preprint arXiv:2503.05132, 2025

    2025 arXiv

  79. [85]

    Think or not think: A study of explicit thinking in rule-based visual reinforcement fine-tuning

    Ming Li, Jike Zhong, Shitian Zhao, Yuxiang Lai, and Kaipeng Zhang. Think or not think: A study of explicit thinking in rule-based visual reinforcement fine-tuning. arXiv preprint arXiv:2503.16188, 2025

    2025

  80. [86]

    Seg- zero: Reasoning-chain guided segmentation via cognitive reinforcement

    Yuqi Liu, Bohao Peng, Zhisheng Zhong, Zihao Yue, Fanbin Lu, Bei Yu, and Jiaya Jia. Seg- zero: Reasoning-chain guided segmentation via cognitive reinforcement. arXiv preprint arXiv:2503.06520, 2025

    2025 arXiv

  81. [87]

    Q-insight: Understanding image quality via visual reinforcement learning

    Weiqi Li, Xuanyu Zhang, Shijie Zhao, Yabin Zhang, Junlin Li, Li Zhang, and Jian Zhang. Q-insight: Understanding image quality via visual reinforcement learning. arXiv preprint arXiv:2503.22679, 2025

    2025 arXiv

  82. [88]

    Perception-r1: Pioneering perception policy with reinforcement learning

    En Yu, Kangheng Lin, Liang Zhao, Jisheng Yin, Yana Wei, Yuang Peng, Haoran Wei, Jian- jian Sun, Chunrui Han, Zheng Ge, et al. Perception-r1: Pioneering perception policy with reinforcement learning. arXiv preprint arXiv:2504.07954, 2025

    2025 arXiv

  83. [89]

    Compile scene graphs with reinforcement learning

    Zuyao Chen, Jinlin Wu, Zhen Lei, Marc Pollefeys, and Chang Wen Chen. Compile scene graphs with reinforcement learning. arXiv preprint arXiv:2504.13617, 2025

    2025 arXiv

  84. [90]

    Relation-r1: Cognitive chain-of-thought guided reinforcement learning for unified relational comprehension

    Lin Li, Wei Chen, Jiahui Li, and Long Chen. Relation-r1: Cognitive chain-of-thought guided reinforcement learning for unified relational comprehension. arXiv preprint arXiv:2504.14642, 2025

    2025

  85. [92]

    SeekWorld: Geolocation is a natural RL task for o3-like visual clue-tracking

    Kaibin Tian, Zijie Xin, and Jiazhen Liu. SeekWorld: Geolocation is a natural RL task for o3-like visual clue-tracking. https://github.com/TheEighthDay/SeekWorld,

  86. [93]

    Openthinkimg: Learning to think with images via visual tool reinforcement learning

    Zhaochen Su, Linjie Li, Mingyang Song, Yunzhuo Hao, Zhengyuan Yang, Jun Zhang, Guanjie Chen, Jiawei Gu, Juntao Li, Xiaoye Qu, et al. Openthinkimg: Learning to think with images via visual tool reinforcement learning. arXiv preprint arXiv:2505.08617, 2025

    2025 arXiv

  87. [94]

    Insight-v: Exploring long-chain visual reasoning with multimodal large language models

    Yuhao Dong, Zuyan Liu, Hai-Long Sun, Jingkang Yang, Winston Hu, Yongming Rao, and Ziwei Liu. Insight-v: Exploring long-chain visual reasoning with multimodal large language models. arXiv preprint arXiv:2411.14432, 2024

    2024 arXiv

  88. [95]

    Visual-rft: Visual reinforcement fine-tuning

    Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement fine-tuning. arXiv preprint arXiv:2503.01785, 2025

    2025 arXiv

  89. [96]

    Reason-rft: Reinforcement fine-tuning for visual reasoning

    Huajie Tan, Yuheng Ji, Xiaoshuai Hao, Minglan Lin, Pengwei Wang, Zhongyuan Wang, and Shanghang Zhang. Reason-rft: Reinforcement fine-tuning for visual reasoning. arXiv preprint arXiv:2503.20752, 2025. 15

    2025

  90. [99]

    Kimi-vl technical report

    Kimi Team, Angang Du, Bohong Yin, Bowei Xing, Bowen Qu, Bowen Wang, Cheng Chen, Chenlin Zhang, Chenzhuang Du, Chu Wei, et al. Kimi-vl technical report. arXiv preprint arXiv:2504.07491, 2025

    2025 arXiv

  91. [100]

    R1-vision: Let’s first take a look at the image

    Ya-Qi Yu, Minghui Liao, , Feilong Chen, Jihao Wu, and Chao Weng. R1-vision: Let’s first take a look at the image. https://github.com/yuyq96/R1-Vision, 2025. Accessed: 2025-02-08

    2025

  92. [101]

    Ocean- r1: An open and generalizable large vision-language model enhanced by reinforcement learning

    Lingfeng Ming, Yadong Li, Song Chen, Jianhua Xu, Zenan Zhou, and Weipeng Chen. Ocean- r1: An open and generalizable large vision-language model enhanced by reinforcement learning. https://github.com/VLM-RL/Ocean-R1, 2025. Accessed: 2025-04-03

    2025

  93. [103]

    Timezero: Temporal video grounding with reasoning-guided lvlm

    Ye Wang, Boshen Xu, Zihao Yue, Zihan Xiao, Ziheng Wang, Liang Zhang, Dingyi Yang, Wenxuan Wang, and Qin Jin. Timezero: Temporal video grounding with reasoning-guided lvlm. arXiv preprint arXiv:2503.13377, 2025

    2025 arXiv

  94. [104]

    Envolving temporal rea- soning capability into lmms via temporal consistent reward

    Hongyu Li, Songhao Han, Yue Liao, Jialin Gao, and Si Liu. Envolving temporal rea- soning capability into lmms via temporal consistent reward. https://github.com/ appletea233/Temporal-R1, 2025

    2025

  95. [105]

    Open llava-video-r1

    Canhui Tang. Open llava-video-r1. https://github.com/Hui-design/ Open-LLaVA-Video-R1 , 2025. Accessed: 2025-03-18

    2025

  96. [106]

    Video-r1: Reinforcing video reasoning in mllms

    Kaituo Feng, Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang, Tianshuo Peng, Benyou Wang, and Xiangyu Yue. Video-r1: Reinforcing video reasoning in mllms. arXiv preprint arXiv:2503.21776, 2025

    2025 arXiv

  97. [107]

    Exploring the effect of reinforcement learning on video understanding: Insights from seed-bench-r1

    Yi Chen, Yuying Ge, Rui Wang, Yixiao Ge, Lu Qiu, Ying Shan, and Xihui Liu. Exploring the effect of reinforcement learning on video understanding: Insights from seed-bench-r1. arXiv preprint arXiv:2503.24376, 2025

    2025 arXiv

  98. [108]

    Improved visual-spatial reasoning via r1-zero-like training

    Zhenyi Liao, Qingsong Xie, Yanhao Zhang, Zijian Kong, Haonan Lu, Zhenyu Yang, and Zhijie Deng. Improved visual-spatial reasoning via r1-zero-like training. arXiv preprint arXiv:2504.00883, 2025

    2025 arXiv

  99. [109]

    Spatial-r1: Enhancing mllms in video spatial reasoning

    Kun Ouyang. Spatial-r1: Enhancing mllms in video spatial reasoning. arXiv preprint arXiv:2504.01805, 2025

    2025 arXiv

  100. [110]

    Videochat-r1: Enhancing spatio-temporal perception via reinforce- ment fine-tuning

    Xinhao Li, Ziang Yan, Desen Meng, Lu Dong, Xiangyu Zeng, Yinan He, Yali Wang, Yu Qiao, Yi Wang, and Limin Wang. Videochat-r1: Enhancing spatio-temporal perception via reinforce- ment fine-tuning. arXiv preprint arXiv:2504.06958, 2025

    2025 arXiv

  101. [111]

    Tinyllava-video-r1: Towards smaller lmms for video reasoning

    Xingjian Zhang, Siwei Wen, Wenjun Wu, and Lei Huang. Tinyllava-video-r1: Towards smaller lmms for video reasoning. arXiv preprint arXiv:2504.09641, 2025

    2025 arXiv

  102. [112]

    Medvlm-r1: Incentivizing medical reasoning capability of vision-language models (vlms) via reinforcement learning.arXiv preprint arXiv:2502.19634, 2025

    Jiazhen Pan, Che Liu, Junde Wu, Fenglin Liu, Jiayuan Zhu, Hongwei Bran Li, Chen Chen, Cheng Ouyang, and Daniel Rueckert. Medvlm-r1: Incentivizing medical reasoning capability of vision-language models (vlms) via reinforcement learning.arXiv preprint arXiv:2502.19634, 2025

    2025 arXiv

  103. [113]

    Med-r1: Reinforcement learning for generalizable medical reasoning in vision-language models

    Yuxiang Lai, Jike Zhong, Ming Li, Shitian Zhao, and Xiaofeng Yang. Med-r1: Reinforcement learning for generalizable medical reasoning in vision-language models. arXiv preprint arXiv:2503.13939, 2025. 16

    2025

  104. [114]

    Chestx- reasoner: Advancing radiology foundation models with reasoning through step-by-step verifi- cation

    Ziqing Fan, Cheng Liang, Chaoyi Wu, Ya Zhang, Yanfeng Wang, and Weidi Xie. Chestx- reasoner: Advancing radiology foundation models with reasoning through step-by-step verifi- cation. arXiv preprint arXiv:2504.20930, 2025

    2025 arXiv

  105. [115]

    Embodied-reasoner: Synergizing visual search, reasoning, and action for embodied interactive tasks

    Wenqi Zhang, Mengna Wang, Gangao Liu, Xu Huixin, Yiwei Jiang, Yongliang Shen, Guiyang Hou, Zhe Zheng, Hang Zhang, Xin Li, et al. Embodied-reasoner: Synergizing visual search, reasoning, and action for embodied interactive tasks. arXiv preprint arXiv:2503.21696, 2025

    2025 arXiv

  106. [116]

    Embodied-r: Collaborative framework for activating embodied spatial reasoning in foundation models via reinforcement learning

    Baining Zhao, Ziyou Wang, Jianjie Fang, Chen Gao, Fanhang Man, Jinqiang Cui, Xin Wang, Xinlei Chen, Yong Li, and Wenwu Zhu. Embodied-r: Collaborative framework for activating embodied spatial reasoning in foundation models via reinforcement learning. arXiv preprint arXiv:2504....

    2025 arXiv

  107. [117]

    Simplear: Pushing the frontier of autoregressive visual generation through pretraining, sft, and rl

    Junke Wang, Zhi Tian, Xun Wang, Xinyu Zhang, Weilin Huang, Zuxuan Wu, and Yu-Gang Jiang. Simplear: Pushing the frontier of autoregressive visual generation through pretraining, sft, and rl. arXiv preprint arXiv:2504.11455, 2025

    2025 arXiv

  108. [118]

    T2i-r1: Reinforcing image generation with collaborative semantic-level and token-level cot

    Dongzhi Jiang, Ziyu Guo, Renrui Zhang, Zhuofan Zong, Hao Li, Le Zhuo, Shilin Yan, Pheng- Ann Heng, and Hongsheng Li. T2i-r1: Reinforcing image generation with collaborative semantic-level and token-level cot. arXiv preprint arXiv:2505.00703, 2025

    2025 arXiv

  109. [119]

    Flow-grpo: Training flow matching models via online rl

    Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Wanli Ouyang. Flow-grpo: Training flow matching models via online rl. arXiv preprint arXiv:2505.05470, 2025

    2025 arXiv

  110. [120]

    Dancegrpo: Unleashing grpo on visual generation

    Zeyue Xue, Jie Wu, Yu Gao, Fangyuan Kong, Lingting Zhu, Mengzhao Chen, Zhiheng Liu, Wei Liu, Qiushan Guo, Weilin Huang, and Ping Luo. Dancegrpo: Unleashing grpo on visual generation. arXiv preprint arXiv:2505.07818, 2025

    2025 arXiv

  111. [121]

    Boosting the generalization and reasoning of vision language models with curriculum reinforcement learning

    Huilin Deng, Ding Zou, Rui Ma, Hongchen Luo, Yang Cao, and Yu Kang. Boosting the generalization and reasoning of vision language models with curriculum reinforcement learning. arXiv preprint arXiv:2503.07065, 2025

    2025 arXiv

  112. [122]

    Othink- mr1: Stimulating multimodal generalized reasoning capabilities via dynamic reinforcement learning

    Zhiyuan Liu, Yuting Zhang, Feng Liu, Changwang Zhang, Ying Sun, and Jun Wang. Othink- mr1: Stimulating multimodal generalized reasoning capabilities via dynamic reinforcement learning. arXiv preprint arXiv:2503.16081, 2025

    2025 arXiv

  113. [123]

    Training reasoning model with dynamic advantage estimation on reinforcement learning

    Lujun Gui and Qingnan Ren. Training reasoning model with dynamic advantage estimation on reinforcement learning. https://github.com/ShadeCloak/ADORA, 2025

    2025

  114. [124]

    The second half

    Shunyu Yao. The second half. https://ysymyth.github.io/ The-Second-Half/, 2025

    2025

  115. [125]

    Zerobench: An impossible visual benchmark for contemporary large multimodal models

    Jonathan Roberts, Mohammad Reza Taesiri, Ansh Sharma, Akash Gupta, Samuel Roberts, Ioana Croitoru, Simion-Vlad Bogolin, Jialu Tang, Florian Langer, Vyas Raina, et al. Zerobench: An impossible visual benchmark for contemporary large multimodal models. arXiv preprint arXiv:2502....

    2025 arXiv

  116. [126]

    V1: Toward multimodal reasoning by designing auxiliary tasks, 2025

    Tianyu Pang Haonan Wang, Chao Du. V1: Toward multimodal reasoning by designing auxiliary tasks, 2025. URL https://v1-videoreasoning.notion.site

    2025

  117. [127]

    Geosense: Evaluating identification and application of geometric principles in multimodal reasoning

    Liangyu Xu, Yingxiu Zhao, Jingyun Wang, Yingyao Wang, Bu Pi, Chen Wang, Mingliang Zhang, Jihao Gu, Xiang Li, Xiaoyong Zhu, et al. Geosense: Evaluating identification and application of geometric principles in multimodal reasoning. arXiv preprint arXiv:2504.12597, 2025

    2025 arXiv

  118. [128]

    Mm-iq: Benchmarking human-like abstraction and reasoning in multimodal models

    Huanqia Cai, Yijun Yang, and Winston Hu. Mm-iq: Benchmarking human-like abstraction and reasoning in multimodal models. arXiv preprint arXiv:2502.00698, 2025

    2025 arXiv

  119. [129]

    Mdk12-bench: A multi-discipline benchmark for evaluating reasoning in multimodal large language models

    Pengfei Zhou, Fanrui Zhang, Xiaopeng Peng, Zhaopan Xu, Jiaxin Ai, Yansheng Qiu, Chuanhao Li, Zhen Li, Ming Li, Yukang Feng, et al. Mdk12-bench: A multi-discipline benchmark for evaluating reasoning in multimodal large language models. arXiv preprint arXiv:2504.05782, 2025. 17

    2025 arXiv

  120. [130]

    Mv- math: Evaluating multimodal math reasoning in multi-visual contexts

    Peijie Wang, Zhong-Zhi Li, Fei Yin, Xin Yang, Dekang Ran, and Cheng-Lin Liu. Mv- math: Evaluating multimodal math reasoning in multi-visual contexts. arXiv preprint arXiv:2502.20808, 2025

    2025 arXiv

  121. [131]

    Pulsecheck457: A diagnostic benchmark for 6d spatial reasoning of large multimodal models

    Xingrui Wang, Wufei Ma, Tiezheng Zhang, Celso M de Melo, Jieneng Chen, and Alan Yuille. Pulsecheck457: A diagnostic benchmark for 6d spatial reasoning of large multimodal models. arXiv e-prints, pages arXiv–2502, 2025

    2025

  122. [132]

    Vcr-bench: A comprehensive evaluation framework for video chain-of-thought reasoning

    Yukun Qi, Yiming Zhao, Yu Zeng, Xikun Bao, Wenxuan Huang, Lin Chen, Zehui Chen, Jie Zhao, Zhongang Qi, and Feng Zhao. Vcr-bench: A comprehensive evaluation framework for video chain-of-thought reasoning. arXiv preprint arXiv:2504.07956, 2025

    2025 arXiv

  123. [133]

    Mme-cot: Benchmarking chain-of-thought in large multimodal models for reasoning quality, robustness, and efficiency

    Dongzhi Jiang, Renrui Zhang, Ziyu Guo, Yanwei Li, Yu Qi, Xinyan Chen, Liuhui Wang, Jianhan Jin, Claire Guo, Shen Yan, et al. Mme-cot: Benchmarking chain-of-thought in large multimodal models for reasoning quality, robustness, and efficiency. arXiv preprint arXiv:2502.09621, 2025

    2025 arXiv

  124. [134]

    Video-mmlu: A massive multi-discipline lecture understanding benchmark

    Enxin Song, Wenhao Chai, Weili Xu, Jianwen Xie, Yuxuan Liu, and Gaoang Wang. Video-mmlu: A massive multi-discipline lecture understanding benchmark. arXiv preprint arXiv:2504.14693, 2025

    2025 arXiv

  125. [135]

    Gdi-bench: A benchmark for general document intelligence with vision and reasoning decoupling

    Siqi Li, Yufan Shen, Xiangnan Chen, Jiayi Chen, Hengwei Ju, Haodong Duan, Song Mao, Hongbin Zhou, Bo Zhang, Pinlong Cai, et al. Gdi-bench: A benchmark for general document intelligence with vision and reasoning decoupling. arXiv preprint arXiv:2505.00063, 2025

    2025 arXiv

  126. [136]

    Visulogic: A benchmark for evaluating visual reasoning in multi-modal large language models

    Weiye Xu, Jiahao Wang, Weiyun Wang, Zhe Chen, Wengang Zhou, Aijun Yang, Lewei Lu, Houqiang Li, Xiaohua Wang, Xizhou Zhu, et al. Visulogic: A benchmark for evaluating visual reasoning in multi-modal large language models. arXiv preprint arXiv:2504.15279, 2025

    2025 arXiv

  127. [137]

    ivispar–an interactive visual-spatial reasoning benchmark for vlms

    Julius Mayer, Mohamad Ballout, Serwan Jassim, Farbod Nosrat Nezami, and Elia Bruni. ivispar–an interactive visual-spatial reasoning benchmark for vlms. arXiv preprint arXiv:2502.03214, 2025

    2025

  128. [138]

    Open r1: A fully open reproduction of deepseek-r1, January 2025

    Hugging Face. Open r1: A fully open reproduction of deepseek-r1, January 2025. URL https://github.com/huggingface/open-r1

    2025

  129. [139]

    Trl: Transformer reinforcement learning

    Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallouédec. Trl: Transformer reinforcement learning. https://github.com/huggingface/trl, 2020

    2020

  130. [140]

    R1-v: Reinforcing super gener- alization ability in vision-language models with less than $3

    Liang Chen, Lei Li, Haozhe Zhao, Yifan Song, and Vinci. R1-v: Reinforcing super gener- alization ability in vision-language models with less than $3. https://github.com/ Deep-Agent/R1-V, 2025. Accessed: 2025-02-02

    2025

  131. [141]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems...

    2023

  132. [142]

    Easyr1: An efficient, scalable, multi-modality rl training framework

    Yaowei Zheng, Junting Lu, Shenzhi Wang, Zhangchi Feng, Dongdong Kuang, and Yuwen Xiong. Easyr1: An efficient, scalable, multi-modality rl training framework. https:// github.com/hiyouga/EasyR1, 2025

    2025

  133. [143]

    Hybridflow: A flexible and efficient rlhf framework

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256, 2024

    2024 arXiv

  134. [144]

    Rethinking rl scaling for vision language models: A transparent, from-scratch framework and comprehensive evaluation scheme

    Yan Ma, Steffi Chern, Xuyang Shen, Yiran Zhong, and Pengfei Liu. Rethinking rl scaling for vision language models: A transparent, from-scratch framework and comprehensive evaluation scheme. arXiv preprint arXiv:2504.02587, 2025

    2025 arXiv

  135. [145]

    X-reasoner: Towards generalizable reasoning across modalities and domains

    Qianchu Liu, Sheng Zhang, Guanghui Qin, Timothy Ossowski, Yu Gu, Ying Jin, Sid Kiblawi, Sam Preston, Mu Wei, Paul V ozila, et al. X-reasoner: Towards generalizable reasoning across modalities and domains. arXiv preprint arXiv:2505.03981, 2025. 18

    2025 arXiv

  136. [146]

    R1-reward: Training multimodal reward model through stable reinforcement learning

    Yi-Fan Zhang, Xingyu Lu, Xiao Hu, Chaoyou Fu, Bin Wen, Tianke Zhang, Changyi Liu, Kaiyu Jiang, Kaibing Chen, Kaiyu Tang, et al. R1-reward: Training multimodal reward model through stable reinforcement learning. arXiv preprint arXiv:2505.02835, 2025

    2025 arXiv

  137. [147]

    Unified multimodal chain-of-thought reward model through reinforcement fine-tuning

    Yibin Wang, Zhimin Li, Yuhang Zang, Chunyu Wang, Qinglin Lu, Cheng Jin, and Jiaqi Wang. Unified multimodal chain-of-thought reward model through reinforcement fine-tuning. arXiv preprint arXiv:2505.03318, 2025

    2025

  138. [148]

    Skywork-vl reward: An effective reward model for multimodal understanding and reasoning

    Xiaokun Wang, Jiangbo Pei, Wei Shen, Yi Peng, Yunzhuo Hao, Weijie Qiu, Ai Jian, Tianyidan Xie, Xuchen Song, Yang Liu, et al. Skywork-vl reward: An effective reward model for multimodal understanding and reasoning. arXiv preprint arXiv:2505.07263, 2025

    2025 arXiv

  139. [149]

    Mllm-protector: Ensuring mllm’s safety without hurting performance

    Renjie Pi, Tianyang Han, Jianshu Zhang, Yueqi Xie, Rui Pan, Qing Lian, Hanze Dong, Jipeng Zhang, and Tong Zhang. Mllm-protector: Ensuring mllm’s safety without hurting performance. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages...

    2024

  140. [150]

    Eyes closed, safety on: Protecting multimodal llms via image- to-text transformation

    Yunhao Gou, Kai Chen, Zhili Liu, Lanqing Hong, Hang Xu, Zhenguo Li, Dit-Yan Yeung, James T Kwok, and Yu Zhang. Eyes closed, safety on: Protecting multimodal llms via image- to-text transformation. In European Conference on Computer Vision, pages 388–404. Springer, 2024

    2024

  141. [151]

    Mllmguard: A multi-dimensional safety evaluation suite for multimodal large language models

    Tianle Gu, Zeyang Zhou, Kexin Huang, Liang Dandan, Yixu Wang, Haiquan Zhao, Yuanqi Yao, Yujiu Yang, Yan Teng, Yu Qiao, et al. Mllmguard: A multi-dimensional safety evaluation suite for multimodal large language models. Advances in Neural Information Processing Systems, 37:7256...

    2024

  142. [152]

    Safety of multimodal large language models on images and text

    Xin Liu, Yichen Zhu, Yunshi Lan, Chao Yang, and Yu Qiao. Safety of multimodal large language models on images and text. In IJCAI, 2024

    2024

  143. [153]

    The hidden risks of large reasoning models: A safety assessment of r1

    Kaiwen Zhou, Chengzhi Liu, Xuandong Zhao, Shreedhar Jangam, Jayanth Srinivasa, Gaowen Liu, Dawn Song, and Xin Eric Wang. The hidden risks of large reasoning models: A safety assessment of r1. arXiv preprint arXiv:2502.12659, 2025

    2025

  144. [154]

    Safechain: Safety of language models with long chain-of-thought reasoning capabilities

    Fengqing Jiang, Zhangchen Xu, Yuetai Li, Luyao Niu, Zhen Xiang, Bo Li, Bill Yuchen Lin, and Radha Poovendran. Safechain: Safety of language models with long chain-of-thought reasoning capabilities. arXiv preprint arXiv:2502.12025, 2025

    2025 arXiv

  145. [155]

    A mousetrap: Fooling large reasoning models for jailbreak with chain of iterative chaos

    Yang Yao, Xuan Tong, Ruofan Wang, Yixu Wang, Lujundong Li, Liang Liu, Yan Teng, and Yingchun Wang. A mousetrap: Fooling large reasoning models for jailbreak with chain of iterative chaos. arXiv preprint arXiv:2502.15806, 2025

    2025 arXiv

  146. [156]

    Concrete problems in ai safety

    Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Concrete problems in ai safety. arXiv preprint arXiv:1606.06565, 2016

    2016 arXiv

  147. [157]

    Reward hacking in reinforcement learning

    Lilian Weng. Reward hacking in reinforcement learning. lilianweng.github.io, Nov 2024. URL https://lilianweng.github.io/posts/2024-11-28-reward-hacking/

    2024

  148. [158]

    Aligning large multimodal models with factually augmented rlhf

    Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang-Yan Gui, Yu-Xiong Wang, Yiming Yang, et al. Aligning large multimodal models with factually augmented rlhf. arXiv preprint arXiv:2309.14525, 2023

    2023 arXiv

  149. [159]

    Self-supervised visual preference alignment

    Ke Zhu, Liang Zhao, Zheng Ge, and Xiangyu Zhang. Self-supervised visual preference alignment. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 291–300, 2024

    2024

  150. [160]

    R1V-Free: Advancing Open-World Visual Reasoning with Label-Free AI Feedback,

    Xize Cheng, Zhengzhou Cai, Zehan Wang, Shengpeng Ji, Ziyue Jiang, Tao Jin, and Zhou Zhao. R1V-Free: Advancing Open-World Visual Reasoning with Label-Free AI Feedback,

  151. [162]

    19 A Summary of works done on RFT for MLLMs A.1 Vision (Image) Papers

    URL https://github.com/Exgc/R1V-Free. 19 A Summary of works done on RFT for MLLMs A.1 Vision (Image) Papers

  152. [164]

    [DanceGRPO (Gen) [120] ] DanceGRPO: Unleashing GRPO on Visual Generation [ Project] [ Code]

  153. [165]

    [Flow-GRPO (Gen) [119] ] Flow-GRPO: Training Flow Matching Models via Online RL [ Models] [ Code]

  154. [166]

    [X-Reasoner [145] ] X-Reasoner: Towards Generalizable Reasoning Across Modalities and Domains [ Code]

  155. [167]

    [T2I-R1 (Gen) [118] ] T2I-R1: Reinforcing Image Generation with Collaborative Semantic- level and Token-level CoT [ Code]

  156. [169]

    [Skywork R1V2 [ 80] ] Skywork R1V2: Multimodal Hybrid Reinforcement Learning for Reasoning [ Models] [ Code]

  157. [170]

    [Relation-R1 [90] ] Relation-R1: Cognitive Chain-of-Thought Guided Reinforcement Learn- ing for Unified Relational Comprehension [ Code]

  158. [171]

    [R1-SGG [89] ] Compile Scene Graphs with Reinforcement Learning [ Code]

  159. [172]

    [NoisyRollout [79] ] Reinforcing Visual Reasoning with Data Augmentation [ Models] [ Datasets] [ Code]

  160. [173]

    [SimpleAR (Gen) [117] ] SimpleAR: Pushing the Frontier of Autoregressive Visual Generation through Pretraining, SFT, and RL [ Models] [ Code]

  161. [174]

    [VL-Rethinker [78] ] VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning [ Project] [ Models] [ Dataset] [ Code]

  162. [175]

    [Kimi-VL [99] ] Kimi-VL Technical Report [ Project] [ Models] [ Demo] [ Code]

  163. [176]

    [VLAA-Thinking [98] ] SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models [ Models] [ Dataset] [ Code]

  164. [177]

    [Perception-R1 [ 88] ] Perception-R1: Pioneering Perception Policy with Reinforcement Learning [ Models] [ Datasets] [ Code]

  165. [178]

    [SoTA with Less [97] ] SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement [ Model] [ Datasets] [ Code]

  166. [179]

    [VLM-R1 [82] ] VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model [ Model] [ Dataset] [ Demo] [ Code]

  167. [180]

    [CrowdVLM-R1 [83] ] CrowdVLM-R1: Expanding R1 Ability to Vision Language Model for Crowd Counting using Fuzzy Group Relative Policy Reward [ Dataset] [ Code]

  168. [181]

    [MAYE [144] ] Rethinking RL Scaling for Vision Language Models: A Transparent, From- Scratch Framework and Comprehensive Evaluation Scheme [ Dataset] [ Code]

  169. [183]

    [Reason-RFT [ 96] ] Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning [ Project] [ Dataset] [ Code]

  170. [184]

    [OpenVLThinker [77] ] OpenVLThinker: An Early Exploration to Vision-Language Reasoning via Iterative Self-Improvement [ Model] [ Code]

  171. [185]

    [Think or Not Think [85] ] Think or Not Think: A Study of Explicit Thinking in Rule-Based Visual Reinforcement Fine-Tuning [ Models] [ Datasets] [ Code] 20

  172. [186]

    [OThink-MR1 [122] ] OThink-MR1: Stimulating multimodal generalized reasoning capabili- ties via dynamic reinforcement learning

  173. [187]

    [R1-VL [76] ] R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization [ Model] [ Code]

  174. [188]

    [Skywork R1V [ 80] ] Skywork R1V: Pioneering Multimodal Reasoning with Chain-of- Thought [ Model] [ Code]

  175. [189]

    [R1-Onevision [74] ] R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization [ Model] [ Dataset] [ Demo] [ Code]

  176. [190]

    [VisualPRM [45] ] VisualPRM: An Effective Process Reward Model for Multimodal Reason- ing [ Project] [ Model] [ Dataset] [ Benchmark]

  177. [191]

    [LMM-R1 [72] ] LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL [ Code]

  178. [192]

    [Curr-ReFT [121] ] Boosting the Generalization and Reasoning of Vision Language Models with Curriculum Reinforcement Learning [ Models] [ Dataset] [ Code]

  179. [193]

    Aha Moment

    [VisualThinker-R1-Zero [ 84] ] R1-Zero’s "Aha Moment" in Visual Reasoning on a 2B Non-SFT Model [ Code]

  180. [194]

    [Vision-R1 [ 71] ] Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models [ Code]

  181. [195]

    [Seg-Zero [86] ] Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforce- ment [ Model] [ Dataset] [ Code]

  182. [196]

    [MM-Eureka [70] ] MM-Eureka: Exploring Visual Aha Moment with Rule-based Large-scale Reinforcement Learning [ Models] [ Dataset] [ Code]

  183. [197]

    [Visual-RFT [ 95] ] Visual-RFT: Visual Reinforcement Fine-Tuning [ Project] [ Data sets][ Code]

  184. [199]

    [Kimi k1.5 [32] ] Kimi k1.5: Scaling Reinforcement Learning with LLMs [ Project]

  185. [200]

    [Virgo [69] ] Virgo: A Preliminary Exploration on Reproducing o1-like MLLM [ Model] [ Code]

  186. [203]

    [InternVL2-MPO [67] ] Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization [ Project] [ Model] [ Code] Open-Source Projects (Repository without Paper) [R1-V [140] ] [ Code] [ Datasets] [ Blog] [Multimodal Open R1 [138] ] [ Code]...

  187. [204]

    [TinyLLaV A-Video-R1 [111] ] TinyLLaV A-Video-R1: Towards Smaller LMMs for Video Reasoning [ Model] [ Code]

  188. [205]

    [VideoChat-R1 [110] ] VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforce- ment Fine-Tuning [ Model] [ Code]

  189. [206]

    [Spatial-R1 [109] ] Spatial-R1: Enhancing MLLMs in Video Spatial Reasoning [ Model] [ Datasets] [ Code]

  190. [207]

    [R1-Zero-VSI [108] ] Improved Visual-Spatial Reasoning via R1-Zero-Like Training [ Code]

  191. [208]

    [SEED-Bench-R1 [107] ] Exploring the Effect of Reinforcement Learning on Video Under- standing: Insights from SEED-Bench-R1 [ Dataset] [ Code]

  192. [209]

    [Video-R1 [106] ] Video-R1: Reinforcing Video Reasoning in MLLMs [ Model] [ Data set][ Code]

  193. [210]

    [TimeZero [103] ] TimeZero: Temporal Video Grounding with Reasoning-Guided LVLM [ Model] [ Code] Open-Source Projects (Repository without Paper) [Open R1 Video [102] ] [ Code] [ Model] [ Dataset] [Temporal-R1 [104] ] [ Code] [ Models] [Open-LLaV A-Video-R1 [105] ] [ Code] A.3 ...

  194. [211]

    [ChestX-Reasoner [114] ] ChestX-Reasoner: Advancing Radiology Foundation Models with Reasoning through Step-by-Step Verification

  195. [212]

    [Med-R1 [113] ] Med-R1: Reinforcement Learning for Generalizable Medical Reasoning in Vision-Language Models [ Model] [ Code]

  196. [214]

    [Embodied-R [116] ] Embodied-R: Collaborative Framework for Activating Embodied Spatial Reasoning in Foundation Models via Reinforcement Learning [ Code]

  197. [215]

    [Embodied-Reasoner [95] ] Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Tasks [ Project] [ Dataset][ Code] A.5 Multimodal Reward Model Papers

  198. [216]

    [Skywork-VL Reward [148] ] Skywork-VL Reward: An Effective Reward Model for Multi- modal Understanding and Reasoning [ Model] [ Code]

  199. [217]

    [UnifiedReward-Think [147] ] Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuning [ Project] [ Models] [ Datasets] [ Code]

  200. [218]

    [R1-Reward [146] ] R1-Reward: Training Multimodal Reward Model Through Stable Rein- forcement Learning [ Model] [ Dataset] [ Code] 22 A.6 Audio Papers

  201. [219]

    [SARI [ 49] ] SARI: Structured Audio Reasoning via Curriculum-Guided Reinforcement Learning

  202. [220]

    [R1-AQA [48] ] Reinforcement Learning Outperforms Supervised Fine-Tuning: A Case Study on Audio Question Answering [ Model] [ Code]

  203. [221]

    [Audio-Reasoner [47] ] Audio-Reasoner: Improving Reasoning Capability in Large Audio Language Models [ Project] [ Model] [ Code] A.7 Omni Papers

  204. [222]

    [EchoInk-R1 [51] ] EchoInk-R1: Exploring Audio-Visual Reasoning in Multimodal LLMs via Reinforcement Learning [ Model] [ Dataset] [ Code]

  205. [223]

    [R1-Omni [50] ] R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Rein- forcement Learning [ Model] [ Code] A.8 GUI Papers

  206. [224]

    [InfiGUI-R1 [54] ] InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasoners [ Model] [ Code]

  207. [225]

    [GUI-R1 [52] ] GUI-R1: A Generalist R1-Style Vision-Language Action Model For GUI Agents [ Model] [ Dataset] [ Code]

  208. [226]

    [UI-R1 [53] ] UI-R1: Enhancing Action Prediction of GUI Agents by Reinforcement Learning A.9 Framework Open-Source Project (Repository without Paper) [EasyR1 [142] ] [ Code] A.10 Metaverse Paper

  209. [227]

    [MetaSpatial [55] ] MetaSpatial: Reinforcing 3D Spatial Reasoning in VLMs for the Metaverse [ Dataset] [ Code] A.11 Agents Open-Source Project (Repository without Paper) [V AGEN [56] ] [ Code] 23

  210. [2025]

    URL https://openreview.net/forum?id=VOAMTA8jKu

  211. [2411]

    [Insight-V [94] ] Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models [ Model] [ Code]

  212. [2412]

    [Mulberry [68] ] Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Search [ Model] [ Code]

  213. [2501]

    [PARM++ (Gen) [46] ] Can We Generate Images with CoT? Let’s Verify and Reinforce Image Generation Step by Step [ Project] [ Model] [ Code]

  214. [2502]

    [MedVLM-R1 [112] ] MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision- Language Models (VLMs) via Reinforcement Learning [ Model] A.4 Embodied Vision Papers

  215. [2503]

    [Q-Insight [87] ] Q-Insight: Understanding Image Quality via Visual Reinforcement Learning [ Code]

  216. [2504]

    [FAST [81] ] Fast-Slow Thinking for Large Vision-Language Model Reasoning [ Code]

  217. [2505]

    [OpenThinkIMG [93] ] OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning [ Model] [ Datasets] [ Code]

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.