Pith. sign in

REVIEW 4 major objections 7 minor 26 references

The Synergy Dilemma of Long-CoT SFT and RL: Investigating Post-Training Techniques for Reasoning VLMs

T0 review · 4 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Long-CoT SFT and RL do not combine synergistically in vision-language reasoning models: all five tested fusion methods underperform standalone RL.

desk verdict Useful difficulty-level analysis of SFT vs RL for reasoning VLMs, but the 'synergy dilemma' headline is not supported by the reported statistics. read the letter →

arxiv 2507.07562 v1 pith:VSNBATHH submitted 2025-07-10 cs.CL

Jierun Chen , Tiezheng Yu , Haoli Bai , Lewei Yao , Jiannan Wu , Kaican Li , Fei Mi , Chaofan Tao
show 6 more authors
Lei Zhu Manyi Zhang Xiaohui Li Lu Hou Lifeng Shang Qun Liu
This is my paper
classification cs.CL
keywords vision-languagemodelslongchain-of-thoughtsupervisedfine-tuningreinforcementlearningmultimodalreasoningsynergydilemmadifficultystratificationGRPO
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether long chain-of-thought supervised fine-tuning (SFT) and reinforcement learning (RL), two post-training techniques that reportedly work well together in language-only models, can be combined to improve reasoning in vision-language models. Analyzing benchmark questions by difficulty, it finds that SFT teaches deep, structured reasoning that helps the hardest questions but hurts easy ones by making the model verbose, while RL gives broad, consistent gains with concise responses but helps the hardest questions less. The paper then tests five ways to combine the two—two-stage, interleaved, and progressive training, data mixing, and model merging—and reports that none beats standalone RL on average accuracy across five multimodal benchmarks. This "synergy dilemma" is the paper's central claim: SFT and RL behave like a trade-off, not a complement, under the recipes tested.

What carries the argument

The load-bearing mechanism is difficulty-stratified analysis: every benchmark question is placed into one of five difficulty levels using the baseline model's pass rate over 16 runs, so gains can be separated by question type rather than averaged away. On top of that, the paper uses token-level KL divergence before and after fine-tuning to expose "reasoning pivotal tokens"—words like "first", "then", "check", and "maybe"—that SFT injects at sentence beginnings, and it tracks response lengths and RL training dynamics under the GRPO objective with a KL regularization term. These tools let the authors attribute SFT's gains to hard questions and verbosity, RL's gains to concise generalization, and the failure of hybrids to unresolved trade-offs.

What would settle it

Replace Eureka-Distill with long-CoT traces distilled from the target model itself, keep every other setting identical, and rerun the five combination strategies on the same five benchmarks; if any variant averages above 55.1 percent across the five, the dilemma is a property of the tested recipe, not of reasoning VLMs generally.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that long-CoT SFT and RL play distinct, difficulty-dependent roles in reasoning VLMs, and naive ways of stacking them do not combine those roles. SFT raises accuracy on the hardest questions (levels 4-5) by injecting structured, step-by-step reasoning traces, but makes responses roughly an order of magnitude longer than baseline and lowers accuracy on easy questions. RL produces concise responses with stable token distributions and improves accuracy across all five difficulty levels, yet its gains on the hardest questions are smaller than SFT's. Five integration strategies—two-stage, interleaved, and progressive training, data mixing, and model merging—all land between SFT and RL or below RL on average accuracy (51.4, 53.6, 54.9, 54.0, and 54.3 percent versus RL's 55.1), showing trade-offs in accuracy, reasoning style, and response length.

Load-bearing premise

The generalization of the dilemma rests on one base model, one distilled long-CoT training set, one RL algorithm with one KL setting, and the authors' specific choices for each combination; if those choices are not representative, the trade-off could be an artifact of the recipe rather than a property of reasoning VLMs.

Editorial extensions

If this is right

  • For average accuracy on current multimodal reasoning benchmarks, standalone RL with KL regularization is the strongest recipe tested; stacking SFT first, interleaving it, mixing data, or merging models does not add value.
  • SFT has a measurable strength on the hardest questions, so any future combination should target hard questions rather than apply SFT uniformly.
  • Retaining easy questions in RL training, even though their advantages normalize to zero, preserves accuracy on easy problems through the KL term.
  • High-quality reasoning traces matter more than volume or length: 1k textual traces outperformed a 34k multimodal set, and longer traces of lower quality hurt performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: apply SFT loss only to difficulty levels 4-5 and RL loss elsewhere, instead of the paper's zero-pass-rate threshold; the paper's own difficulty curves suggest this could recover parts of both gains.
  • The authors' data-mixing recipe collects RL-generated correct responses and then runs SFT on them; the resulting 10x response-length increase hints that SFT imitates format and verbosity from the distilled traces, which a format-reward ablation could confirm.
  • If the dilemma is a general property, inference-time routing—send hard questions to the SFT model and easy ones to the RL model—would sidestep training incompatibility entirely and could beat either model alone, a consequence the paper does not test.
  • Language-model synergy reported elsewhere may depend on SFT data being self-distilled or model-compatible; reproducing this study with self-distilled traces would clarify whether the dilemma is multimodal or a data-mismatch artifact.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. This manuscript studies post-training of Qwen2.5-VL-7B with long-CoT SFT and GRPO-based RL. It reports that SFT improves the most difficult questions but hurts easy ones and produces verbose outputs, whereas RL produces concise, broad gains. It then evaluates five combination strategies—two-stage, interleaved, progressive training, data mixing, and model merging—and concludes on the basis of Table 6 that none surpasses standalone RL, terming this a 'synergy dilemma.' The difficulty-level analyses in Figures 1 and 6 are used to support the complementary-but-incompatible characterization of SFT and RL.

Significance. If the negative result were robust, the paper would be a useful counterpoint to the prevailing assumption that SFT and RL automatically complement each other in reasoning VLMs. The study is systematic in its coverage of combination strategies and benchmarks, and it reports training/evaluation details and a code repository link. However, the central quantitative claim currently rests on point estimates without measures of uncertainty, and the evaluation protocol's best-checkpoint selection compounds the issue. The headline 'synergy dilemma' is therefore not yet supported at the strength claimed.

major comments (4)
  1. [Sec. 2.1, Table 6] Table 6 is the load-bearing evidence for the 'synergy dilemma,' but no variance or significance testing is reported despite the statement in Sec. 2.1 that results are averaged over 4 runs. The closest combination, Progressive SFT & RL, averages 54.9 versus RL's 55.1, and per-benchmark deltas such as MathVerse -1.1 and MMStar -0.6 are within one standard error of a binomial proportion on benchmarks of only hundreds to about a thousand items. The conclusion that all five combination strategies 'fail to produce additive benefits' is statistically indistinguishable from a null result under sampling noise. Please report standard deviations or confidence intervals and run paired or bootstrap significance tests across the four runs, and soften the 'fundamental' framing until these analyses support the point estimates.
  2. [Sec. 2.1, Evaluation settings] The evaluation protocol states 'For fine-tuned models, we report their best checkpoint results.' Because each method has a different training curve and a different number of saved checkpoints, selecting the best checkpoint on the evaluation benchmarks can differentially inflate each method's score and makes the comparisons in Table 6 non-comparable. This is especially problematic given the small margins (e.g., 54.9 versus 55.1). Please either evaluate a fixed checkpoint per training budget, report full training curves or the sensitivity of rankings to checkpoint selection, or perform selection on a held-out split and then report the resulting test numbers.
  3. [Sec. 2.3, Fig. 6] The difficulty-level accuracy gains in Fig. 6 (and the corresponding gains in Fig. 1) are shown without error bars, despite the four-run protocol. The claim that 'SFT degrades easier questions and is strongest on the hardest questions' depends on these per-level deltas, which can be small and are based on small per-level item counts. Please provide per-level intervals or raw per-level counts; without them, the qualitative difficulty-dependent claims are not quantitatively supported.
  4. [Sec. 3.1, Sec. 3.3] The paper presents the 'synergy dilemma' as a general property of reasoning VLMs ('fundamental,' 'often behave more like a trade-off'), but the study uses one base model, one training set (Eureka-Distill, distilled from the authors' own SFT model), one RL algorithm with one KL value, and method-specific choices that are known to be consequential—for example, interleaved SFT applied only to pass-rate-zero questions and progressive prefix weight 0.2 in Sec. 3.1. These choices may be representative, but the current evidence cannot rule out that the dilemma is an artifact of this particular recipe. Please add supporting variation (e.g., a second base model or data source, or a sensitivity analysis over the method-specific hyperparameters) or explicitly restrict the claim to 'under the configurations tested here.'
minor comments (7)
  1. [Throughout] There are numerous typos and misspellings, for example 'contorling' in Sec. 2.2, 'Abalation' in the Table 2 caption, 'Acuracy' in the Table 6 caption, 'varing' in Sec. 2.1, and 'thourgh' in Sec. 3.1; these should be corrected.
  2. [Sec. 2.1, Data] The data section refers to 's1.1-R1' while the training settings refer to 's1.1'; please clarify whether these are the same dataset and use one consistent name.
  3. [Sec. 2.3, Fig. 7] The token-level KL divergence used to identify 'reasoning pivotal tokens' is not defined in the text; please specify how tokens are aligned and what reference distribution is used.
  4. [Sec. 2.3, Difficulty levels] The difficulty levels are defined using 16 independent runs of the baseline model, while the evaluation protocol in Sec. 2.1 says results are averaged over 4 runs; please clarify the relation between the two settings.
  5. [Sec. 3.1, Two-stage SFT & RL] The statement 'We have also attempted reducing the number of preliminary SFT epochs to 1' is not accompanied by any result or table; please add a sentence summarizing the outcome or remove the claim.
  6. [References] The reference list contains duplicates: Chen et al. 2025a and 2025b are the same arXiv paper, and Yang et al. 2025a and 2025b are also the same; please merge or distinguish them correctly.
  7. [Table 5] The SFT count of 249,091 for the word 'wait' appears implausibly large for a single benchmark unless counting across many generated responses; please clarify the counting procedure and verify the reported numbers.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the 'synergy dilemma' is an empirical measurement directly reported in Table 6, not a quantity fitted from or defined by its own inputs.

full rationale

This paper is an empirical measurement study. The central claim, that combining long-CoT SFT and RL produces trade-offs rather than additive benefits, is a direct experimental comparison in Table 6: each listed method is trained and evaluated under the same protocol, and the reported averages are measurements, not predictions derived from fitted parameters. No equation in the paper defines a target quantity in terms of the inputs, and no fitted parameter is renamed as a prediction. The difficulty levels are defined from the baseline model's pass rate (Sec. 2.3), independently of the SFT/RL gains, so the difficulty-stratified analysis is a measurement convention rather than a self-definitional result. The Eureka-Distill dataset is distilled from the authors' own SFT model, but both the SFT and RL arms use the same data, making the comparison fair rather than forced by construction. The absence of error bars and the use of best-checkpoint selection are statistical robustness concerns, not circularity. None of the load-bearing conclusions reduces to a self-citation or to an imported uniqueness theorem. Therefore, no specific circular step can be exhibited, and the appropriate score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper's claims rest on empirical measurements and training choices rather than a formal derivation. The key free parameters are training hyperparameters and difficulty thresholds chosen by hand. The axioms are domain assumptions about representativeness of the base model, data, and benchmarks; none are introduced ad hoc to force a result.

free parameters (5)
  • KL coefficient beta = 0.005
    GRPO KL regularization coefficient in Sec. 2.2; only two values (0 and 0.005) are compared, and 0.005 is used throughout.
  • SFT prefix loss weight = 0.2
    Used in progressive SFT and RL (Sec. 3.1); described as 'empirically set'.
  • Training epochs = 15 for s1, 5 for Eureka-Distill SFT, 2 for RL/data mixing
    Epoch counts are chosen without a reported sweep and affect the resulting models.
  • Max new generation length N = 4k for RL, 16k for two-stage
    The two-stage setup requires 16k to avoid clipping collapse (Sec. 3.1, Fig. 11).
  • Difficulty thresholds = 12/16, 8/16, 5/16, 2/16
    Used to define the five difficulty levels in Sec. 2.3; different thresholds would change the per-level gains.
assumptions (5)
  • domain assumption Eureka-Distill queries and answers are a representative training distribution for multimodal reasoning
    Both SFT and RL are trained on Eureka-Distill (Sec. 2.1); the conclusions about SFT vs RL and their combinations are conditioned on this data.
  • domain assumption Baseline pass rate over 16 runs is a stable measure of question difficulty
    Difficulty levels in Sec. 2.3 are defined by the baseline model's pass rate; instability would add noise to the per-level accuracy gains.
  • domain assumption Qwen2.5-VL-7B is representative of current reasoning VLMs
    All experiments use one base model; the paper generalizes from this model to reasoning VLMs as a class.
  • domain assumption The evaluation benchmarks are valid measures of multimodal reasoning
    The five benchmarks (MathVision, MathVerse, MathVista, MMMU, MMStar) are used without questioning their validity beyond noting they mix simple and hard questions.
  • standard math GRPO is an appropriate RL algorithm
    The RL objective in Sec. 2.2 is taken from Shao et al. (2024) and used as-is.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Synergy Dilemma of Long-CoT SFT and RL: Investigating Post-Training Techniques for Reasoning VLMs." pith.science (2026). https://pith.science/paper/VSNBATHH

@misc{pith2026250707562,
  author       = {Pith},
  title        = {Pith review of: The Synergy Dilemma of Long-CoT SFT and RL: Investigating Post-Training Techniques for Reasoning VLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VSNBATHH}},
  note         = {Machine review of arXiv:2507.07562}
}
read the original abstract

Large vision-language models (VLMs) increasingly adopt post-training techniques such as long chain-of-thought (CoT) supervised fine-tuning (SFT) and reinforcement learning (RL) to elicit sophisticated reasoning. While these methods exhibit synergy in language-only models, their joint effectiveness in VLMs remains uncertain. We present a systematic investigation into the distinct roles and interplay of long-CoT SFT and RL across multiple multimodal reasoning benchmarks. We find that SFT improves performance on difficult questions by in-depth, structured reasoning, but introduces verbosity and degrades performance on simpler ones. In contrast, RL promotes generalization and brevity, yielding consistent improvements across all difficulty levels, though the improvements on the hardest questions are less prominent compared to SFT. Surprisingly, combining them through two-staged, interleaved, or progressive training strategies, as well as data mixing and model merging, all fails to produce additive benefits, instead leading to trade-offs in accuracy, reasoning style, and response length. This ``synergy dilemma'' highlights the need for more seamless and adaptive approaches to unlock the full potential of combined post-training techniques for reasoning VLMs.

Figures

Figures reproduced from arXiv: 2507.07562 by the authors.

Figure 1
Figure 1. Accuracy gains from various post-training techniques across five difficulty levels (L1, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Response length comparison. 10 2 10 3 10 4 Avg Length (Tokens) 30 40 50 60 Accuracy (%) MathVision MathVerse Baseline SFT w/ s1.1-R1 SFT shifted [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Training dynamics comparisons. Without KL regularization, RL training suffers from [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figures from the paper (10 more)
Figure 5
Figure 5. Figure 5: Accuracy gap differs across questions of varying difficulty lev￾els on MathVision. 2.3 COMPARISONS OF LONG-COT SFT AND RL With optimized data sources and training configurations, both long-CoT SFT and RL yield partial or full improvements in model accuracy. However, th…
Figure 6
Figure 6. Figure 6: Accuracy gain of SFT and RL across 5 difficulty levels on 5 benchmarks. [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Illustration of token-level KL divergence, where darker tokens indicate larger divergence. [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: Response length comparison [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]
Figure 9
Figure 9. Figure 9: Attempts to integrate SFT and RL, including training alternation, data mixing, and model [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 10
Figure 10. Figure 10: Frequency of reasoning words across fine-tuning methods, evaluated on the MathVision. [PITH_FULL_IMAGE:figures/full_fig_p007_10.png]
Figure 11
Figure 11. Figure 11: Training dynamics of RL following SFT. The model demonstrates a higher clip ratio [PITH_FULL_IMAGE:figures/full_fig_p008_11.png]
Figure 12
Figure 12. Figure 12: Training dynamics of Interleaved SFT and RL (ISR) applied to all samples or with SFT [PITH_FULL_IMAGE:figures/full_fig_p008_12.png]
Figure 13
Figure 13. Figure 13: Training dynamics of Progressive SFT and RL, compared to pure RL training. [PITH_FULL_IMAGE:figures/full_fig_p008_13.png]
Figure 14
Figure 14. Figure 14: Accuracy gains of model merging methods (Linear, TIES, SLERP) by merging ratio. [PITH_FULL_IMAGE:figures/full_fig_p009_14.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

26 extracted references · 2 canonical work pages

  1. [1]

    Qwen2.5-vl technical report

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. a...

    arXiv

  2. [4]

    Cogni- tive behaviors that enable self-improving reasoners, or, four habits of highly effective stars.arXiv preprint arXiv:2503.01307,

    Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. Cogni- tive behaviors that enable self-improving reasoners, or, four habits of highly effective stars.arXiv preprint arXiv:2503.01307,

    arXiv

  3. [5]

    Arcee’s mergekit: A toolkit for merging large language models

    Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vladimir Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. Arcee’s mergekit: A toolkit for merging large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pp. 477–485,

    work page 2024

  4. [6]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al

    URL https://cloud.google.com/vertex-ai/generative-ai/docs/ thinking-mode. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948,

    arXiv

  5. [7]

    Mammoth-vl: Eliciting multimodal reasoning with instruction tuning at scale

    Jarvis Guo, Tuney Zheng, Yuelin Bai, Bo Li, Yubo Wang, King Zhu, Yizhi Li, Graham Neubig, Wenhu Chen, and Xiang Yue. Mammoth-vl: Eliciting multimodal reasoning with instruction tuning at scale. arXiv preprint arXiv:2412.05237,

    arXiv

  6. [8]

    Measuring mathematical problem solving with the math dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874,

    arXiv

  7. [10]

    Openai o1 system card

    11 Technical Report on Reasoning VLMs Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720,

    arXiv

  8. [11]

    Acereason-nemotron 1.1: Advancing math and code reasoning through sft and rl synergy

    Zihan Liu, Zhuolin Yang, Yang Chen, Chankyu Lee, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. Acereason-nemotron 1.1: Advancing math and code reasoning through sft and rl synergy. arXiv preprint arXiv:2506.13284,

    arXiv

Show all 26 references
  1. [13]

    Yi Peng, Xiaokun Wang, Yichen Wei, Jiangbo Pei, Weijie Qiu, Ai Jian, Yunzhuo Hao, Jiachun Pan, Tianyidan Xie, Li Ge, et al

    URL https://arxiv.org/abs/2501.19393. Yi Peng, Xiaokun Wang, Yichen Wei, Jiangbo Pei, Weijie Qiu, Ai Jian, Yunzhuo Hao, Jiachun Pan, Tianyidan Xie, Li Ge, et al. Skywork r1v: Pioneering multimodal reasoning with chain-of- thought. arXiv preprint arXiv:2504.05599,

    arXiv

  2. [14]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300,

    arXiv

  3. [15]

    Hybridflow: A flexible and efficient rlhf framework

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256,

    arXiv

  4. [17]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599,

    arXiv

  5. [18]

    Vl- rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning

    Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. Vl- rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning. arXiv preprint arXiv:2504.08837, 2025a. Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun...

    arXiv

  6. [19]

    Skywork r1v2: Multimodal hybrid reinforcement learn- ing for reasoning

    Peiyu Wang, Yichen Wei, Yi Peng, Xiaokun Wang, Weijie Qiu, Wei Shen, Tianyidan Xie, Jiangbo Pei, Jianhao Zhang, Yunzhuo Hao, et al. Skywork r1v2: Multimodal hybrid reinforcement learn- ing for reasoning. arXiv preprint arXiv:2504.16656, 2025b. 12 Technical Report on Reasoning ...

    arXiv

  7. [21]

    R1-onevision: Advancing general- ized multimodal reasoning through cross-modal formalization

    Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, Bo Zhang, and Wei Chen. R1-onevision: Advancing general- ized multimodal reasoning through cross-modal formalization. arXiv preprint arXiv:2503.10615, 2025a...

    arXiv

  8. [22]

    Demystifying long chain- of-thought reasoning in llms

    Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying long chain- of-thought reasoning in llms. arXiv preprint arXiv:2502.03373,

    arXiv

  9. [23]

    Dapo: An open-source llm reinforcement learning system at scale

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476,

    arXiv

  10. [24]

    R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization

    Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization. arXiv preprint arXiv:2503.12937,

    arXiv

  11. [25]

    Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? In European Conference on Computer Vision , pp

    Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Yu Qiao, et al. Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? In European Conference on Computer Vision , pp. 169–186. Spring...

    arXiv

  12. [26]

    URL http://arxiv.org/abs/2403.13372

    Association for Computational Linguis- tics. URL http://arxiv.org/abs/2403.13372. Hengguang Zhou, Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsieh. R1- zero’s” aha moment” in visual reasoning on a 2b non-sft model.arXiv preprint arXiv:2503.05132,

    arXiv

  13. [1985]

    Reason-rft: Reinforcement fine-tuning for visual reasoning

    Huajie Tan, Yuheng Ji, Xiaoshuai Hao, Minglan Lin, Pengwei Wang, Zhongyuan Wang, and Shanghang Zhang. Reason-rft: Reinforcement fine-tuning for visual reasoning. arXiv preprint arXiv:2503.20752,

  14. [2021]

    Vision-r1: Incentivizing reasoning capability in multimodal large language models

    Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-r1: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749,

    arXiv

  15. [2022]

    Llava-o1: Let vision language models reason step-by-step

    Guowei Xu, Peng Jin, Li Hao, Yibing Song, Lichao Sun, and Li Yuan. Llava-o1: Let vision language models reason step-by-step. arXiv preprint arXiv:2411.10440,

    arXiv

  16. [2023]

    Mm-eureka: Exploring the frontiers of multi- modal reasoning with rule-based reinforcement learning

    Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Tiancheng Han, Botian Shi, Wenhai Wang, Junjun He, et al. Mm-eureka: Exploring the frontiers of multi- modal reasoning with rule-based reinforcement learning. arXiv preprint arXiv:2503.07365,

    arXiv

  17. [2024]

    Unlocking the potential of difficulty prior in rl-based multimodal reasoning

    Mingrui Chen, Haogeng Liu, Hao Liang, Huaibo Huang, Wentao Zhang, and Ran He. Unlocking the potential of difficulty prior in rl-based multimodal reasoning. arXiv preprint arXiv:2505.13261 , 2025c. Yifan Du, Zikang Liu, Yifan Li, Wayne Xin Zhao, Yuqi Huo, Bingning Wang, Weipeng...

  18. [2025]

    Sft or rl? an early investigation into training r1-like reasoning large vision-language models

    Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. Sft or rl? an early investigation into training r1-like reasoning large vision-language models. arXiv preprint arXiv:2504.11468, 2025a. Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xi...

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.