Pith. sign in

REVIEW 4 major objections 5 minor 23 references

Unveiling the Compositional Ability Gap in Vision-Language Reasoning Model

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Reinforcement learning, not supervised fine-tuning, is what lets vision-language models compose separately learned skills, and captioning before thinking plus step-level rewards closes much of the remaining gap.

desk verdict ComPABench is a genuinely useful diagnostic and RL-Ground looks promising, but the paper's central SFT-versus-RL comparison is confounded by RL's 8x sampling budget and needs rework before the headline claim can be trusted. read the letter →

arxiv 2505.19406 v1 pith:Z3HEOA7W submitted 2025-05-26 cs.AI

Tianle Li , Jihai Zhang , Yongming Rao , Yu Cheng This is my paper
classification cs.AI
keywords compositionalgeneralizationvision-languagemodelsreinforcementlearningsupervisedfine-tuningcross-modalreasoningcross-taskout-of-distributionvisualgrounding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to measure, rather than assume, whether vision-language models can combine skills they learned separately. It builds a benchmark of paired pure-text and image questions about shape areas and grid positions, trains models with either supervised fine-tuning or reinforcement learning, and then tests them on tasks that require both skills at once and on out-of-distribution variants. The central claim is that reinforcement learning with a verifiable answer reward integrates independently learned skills far better than supervised fine-tuning, while supervised fine-tuning can even erase a model's existing compositional ability. The paper further claims that asking the model to caption the image before reasoning, and rewarding correct intermediate steps, substantially reduces the remaining cross-modal gap. A careful reader should care because these are concrete, reusable training choices, not just benchmark scores.

What carries the argument

The carrying object is ComPABench, a synthetic diagnostic benchmark in which every task exists in a pure-text form and a matched image form. The two atomic skills are shape-area computation and grid-position retrieval; the compositional task asks for the combined area of a target shape and its nearest neighbour, and the out-of-distribution variants swap the objectives (largest area, farthest shape, and their composition). Paired forms let the benchmark separate failure to read an image from failure to combine skills. The proposed remedy, RL-Ground, works on the same benchmark: it forces a `<caption>` block that verbalizes the visual scene before thinking, and it apportions reward across intermediate vision-grounded steps rather than only the final answer. The benchmark supplies the measurement; RL-Ground supplies the intervention.

What would settle it

Match the amount of per-prompt training signal: give the supervised model eight gold or reward-ranked completions per question, or give RL only one sampled completion per question, then rerun the compositional and out-of-distribution tests. If the supervised model then matches RL's accuracy, the claim that RL is the better post-training strategy for composition is falsified.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is a gap and a fix. Vision-language models post-trained on isolated tasks look strong on each task, yet fail when those tasks are composed: supervised fine-tuning on geometric and spatial tasks separately can take a 3-billion-parameter model's compositional accuracy from 49.4% to 0.6% on text inputs, and reinforcement learning with a final-answer reward raises it back to 93%. The same pattern appears with images, but at a much lower level, which the paper reads as a distinct cross-modal composition gap. The fix, called RL-Ground, inserts an explicit caption block before the reasoning trace and adds a progress reward for correct intermediate visual grounding, lifting multimodal compositional accuracy from 31.2% to 52.8% on the 7-billion-parameter model while also improving out-of-distribution generalization. The paper's conclusions are that RL better supports compositional generalization than SFT, that visual-to-text alignment and accurate visual grounding are the two ingredients that matter, and that current post-training leaves a large unused compositional margin.

Load-bearing premise

The weakest link is the assumption that supervised and reinforcement training are being compared on equal footing: the reinforcement runs draw eight candidate answers per prompt and get a reward signal on all of them, while the supervised runs see one gold answer, so part of RL's compositional advantage could come from more practice rather than from the learning rule itself.

Editorial extensions

If this is right

  • Pure-text reasoning does not transfer to images under SFT: models trained only on text can score 99.2% on a text grid-position task and 4.8% on the identical question asked from an image.
  • SFT on sub-skills can be actively harmful to composition, reducing pure-text compositional accuracy from 49.4% to 0.6% on the smaller model despite near-perfect sub-skill scores.
  • RL with final-answer reward recovers composition in text (93% on the smaller model) but leaves a large multimodal gap (17.4%), so the cross-modal bottleneck is separate from the cross-task one.
  • Caption-before-thinking plus progress reward, the RL-Ground recipe, raises multimodal compositional accuracy to 52.8% on the larger model and matches or beats plain RL on out-of-distribution compositional tasks.
  • The out-of-distribution results suggest RL's advantage grows with scale: the larger model generalizes to changed task objectives much better than the smaller one.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A fairer test of the RL-versus-SFT claim would match the number of completion samples seen per prompt: the RL runs here see eight generated completions per question and their rewards, while SFT sees one gold answer, so the advantage credited to RL may partly reflect extra sampling.
  • If the caption-before-thinking effect generalizes, it offers a way to improve compositional reasoning on real images without retraining visual encoders, simply by making the model's text interface the bottleneck; this is testable on natural-image benchmarks.
  • Progress rewards require intermediate labels, so scaling RL-Ground to open-ended tasks would need automatic or self-supervised progress estimation; the paper's synthetic benchmark provides labels that real tasks will not.
  • The scale-dependent out-of-distribution trend suggests compositional generalization under RL may be a scaling law; probing additional model sizes would show whether the gap closes smoothly or abruptly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces ComPABench, a synthetic diagnostic benchmark for probing compositional generalization in vision-language models (VLMs) across three axes: cross-modal transfer, cross-task composition, and out-of-distribution generalization. Using Qwen2.5-VL 3B/7B models, the authors compare supervised fine-tuning (SFT), reinforcement learning with verifiable rewards (GRPO), SFT-initialized RL, and a proposed RL-Ground method that combines caption-before-thinking prompting with a fine-grained progress reward. The main claims are that RL-trained models outperform SFT on compositional generalization, that VLMs still struggle with multimodal compositional reasoning, and that RL-Ground improves compositional performance through visual-to-text alignment and intermediate grounding supervision. Experiments are reported for pure-text and multimodal variants, with OOD versions of the tasks.

Significance. If the findings hold, ComPABench would be a useful controlled testbed for studying how post-training strategies affect compositional reasoning in VLMs, and RL-Ground would provide a concrete, if narrow, recipe for improving such reasoning. The paper's strengths include the parallel pure-text/multimodal task design, the evaluation across two model scales, the OOD variants, and an ablation of the two RL-Ground components. However, the present evidence is limited by a confounded SFT/RL comparison, a task-specific progress reward that matches the evaluation metrics, a methodology gap in the SFT-init RL description, and the absence of any variance estimation. These issues prevent the strongest claims from being fully supported as stated.

major comments (4)
  1. [Sec. 4.2] The central SFT-versus-RL comparison is confounded by the training budgets. Section 4.2 fixes 'a total of 1 training epoch' for all runs but states that RL generates '8 completions per prompt in training,' whereas SFT is teacher-forced on a single gold output per prompt. Thus RL sees eight on-policy samples per prompt and performs reward-filtered updates, while SFT sees one gold trace and no exploration. The large RL advantage on compositional tasks (e.g., 93% vs. 0.6% for PT-Comp in Fig. 5) could therefore reflect the extra sampling budget or reward-based data selection rather than an intrinsic property of the policy-gradient objective. To support the abstract's claim that 'RL-trained models consistently outperform SFT' on compositional generalization, the authors should either equalize the number of training outputs per prompt (e.g., SFT on multiple gold traces, or RL with one completion), or carefully rephrase the claim as 'RL training with sampling and reward filtering better supports compositional generalization.'
  2. [Sec. 4.3.2, Fig. 6] The RL-Ground progress reward is defined on the exact subgoals used in the evaluation tasks: the paper describes rewarding 'correct shape area computation or distance estimation,' which are precisely the components of the Shape Area, Grid Position, and Compositional benchmark tasks. This means RL-Ground is partly designed to optimize the same intermediate quantities on which it is evaluated, so its compositional gains (28.6% for 3B and 52.8% for 7B in Fig. 5) may reflect metric-specific reward shaping rather than a general principle for compositionality. The paper should either evaluate RL-Ground on held-out subgoal types or explicitly discuss this limitation alongside the existing Limitations paragraph; the current Limitations section only notes reliance on intermediate signals, not the overlap between the reward function and the evaluation metrics.
  3. [Sec. 4.3.2 vs. Sec. 3.3 and Sec. 4.2] The description of the SFT-initialized RL method is internally inconsistent. Section 3.3 formally defines SFT-init RL as initializing the policy and reference model from an SFT checkpoint, with no mention of alternating updates. In contrast, Section 4.3.2 attributes the method's poor performance to 'our hybrid training strategy, which alternates evenly between SFT and RL updates (half-half step attribution).' The experimental settings in Section 4.2 do not list any alternating schedule. Without a precise, consistent description of the training procedure, the reported SFT-init RL results (2.6% for 3B and 1.0% for 7B) are difficult to reproduce and interpret. The authors should correct the methods description to match the implemented schedule, or vice versa.
  4. [All experiments, especially Table 3] No error bars, seeds, or multiple runs are reported for any result, and the evaluation sets contain only 500 samples. As a result, some differences that are central to the RL-Ground claim, such as the Shape Area score of 73.8% versus the baseline RL's 74.6% in Table 3, are within sampling noise, while even the larger improvements could in principle be affected by run-to-run variance. The authors should report at least three independent training runs with means and standard deviations for the headline comparisons in Figures 3, 4, 5, and Table 3, or clearly state if each result is from a single deterministic run and justify that assumption.
minor comments (5)
  1. [Sec. 3.2, Eq. (2)] Equation (2) has a typo in the probability ratio: both numerator and denominator are πθ, making the ratio identically 1. The denominator should presumably be the old policy or the reference policy; please correct the equation to match the standard GRPO form used in the implementation.
  2. [Throughout] There are several typographical errors: 'Firgure' in Figure 1, 'strategie' in the abstract, 'Y ongming' in the author list, 'PathXu' missing a space in Section 2.1, and 'Grig Position' in Section 4.3.1. A careful proofread is needed.
  3. [Sec. 4.2] The phrase 'we set the scale before KL divergence constraints to 0' is ambiguous; it should state explicitly that the KL coefficient β in Eq. (2) is set to 0, and briefly discuss why this removal was necessary and what implications it has for reward hacking, especially since RL-Ground introduces additional reward components.
  4. [Table 1] In Table 1, the abbreviation 'Cross-Model' should be 'Cross-Modal' to match the terminology used elsewhere.
  5. [Appendix A.1] The appendix states that SFT data includes 'thinking path' traces, but does not explain how these traces were generated (e.g., templated LaTeX formulas or model-generated rationales). This detail is important for reproducing the SFT results and for understanding the SFT/RL comparison.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's claims are empirical benchmark results, not derivations that reduce to their inputs; the RL/SFT sample-budget asymmetry and the RL-Ground reward/evaluation overlap are experimental-design concerns, not circular steps.

full rationale

The paper's central claims are empirical comparisons on ComPABench, and no formal derivation chain is offered that could collapse into its own assumptions. The SFT-versus-RL comparison is not circular: RL is trained with 8 completions per prompt while SFT sees one gold trace, but this is an uncontrolled variable that weakens causal attribution, not a definitional equivalence or a fitted parameter renamed as a prediction. The RL-Ground progress reward is defined on intermediate quantities, such as correct shape area computation or distance estimation, that also appear in the compositional evaluation; this makes the reported gains unsurprising, but the paper presents them as an empirical ablation rather than deriving them from the task definition, so it does not meet the standard of Eq. X = Eq. Y by construction. The one author-overlapping citation, Hao et al. (2025) in related work, is not load-bearing for the paper's conclusions. No uniqueness theorem is imported from the authors' prior work, no ansatz is smuggled in via citation, and no known result is merely renamed. The appended Limitations section explicitly acknowledges the reliance on synthetic tasks and intermediate reward signals, consistent with a bounded empirical study rather than a tautological derivation. Thus no circular step is identified.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities. Its central claims rest on benchmark design choices and training hyperparameters rather than fitted parameters.

free parameters (4)
  • learning rate = 1e-6
    Uniform learning rate used for SFT and RL; no sweep reported (Section 4.2).
  • number of RL completions per prompt = 8
    RL samples 8 completions per prompt while SFT has a single gold output, making the comparison potentially unequal (Section 4.2).
  • KL divergence scale = 0
    Authors set KL regularization to 0 after observing performance degradation, a data-dependent hyperparameter choice (Section 4.2).
  • number of training epochs = 1
    All training uses a single epoch, which may limit SFT convergence and bias the comparison.
assumptions (4)
  • domain assumption Synthetic shape-area and grid-position tasks are valid measures of compositional reasoning.
    The entire benchmark rests on the assumption that these controlled tasks capture the intended compositional abilities (Section 4.1).
  • domain assumption The base Qwen2.5-VL models have sufficient visual perception, so observed gaps are due to post-training.
    The paper does not measure perception error separately; pure-text SFT gaps on multimodal inputs are attributed to compositional transfer rather than visual grounding.
  • domain assumption GRPO with KL=0 is a representative RL setup.
    The authors deviate from standard GRPO by removing KL regularization, citing performance gains; this choice may not generalize to other settings.
  • standard math Manhattan distance and geometric area formulas are correct and unambiguous.
    The evaluation relies on these formulas; no issue.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unveiling the Compositional Ability Gap in Vision-Language Reasoning Model." pith.science (2026). https://pith.science/paper/Z3HEOA7W

@misc{pith2026250519406,
  author       = {Pith},
  title        = {Pith review of: Unveiling the Compositional Ability Gap in Vision-Language Reasoning Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z3HEOA7W}},
  note         = {Machine review of arXiv:2505.19406}
}
read the original abstract

While large language models (LLMs) demonstrate strong reasoning capabilities utilizing reinforcement learning (RL) with verifiable reward, whether large vision-language models (VLMs) can directly inherit such capabilities through similar post-training strategies remains underexplored. In this work, we conduct a systematic compositional probing study to evaluate whether current VLMs trained with RL or other post-training strategies can compose capabilities across modalities or tasks under out-of-distribution conditions. We design a suite of diagnostic tasks that train models on unimodal tasks or isolated reasoning skills, and evaluate them on multimodal, compositional variants requiring skill integration. Through comparisons between supervised fine-tuning (SFT) and RL-trained models, we identify three key findings: (1) RL-trained models consistently outperform SFT on compositional generalization, demonstrating better integration of learned skills; (2) although VLMs achieve strong performance on individual tasks, they struggle to generalize compositionally under cross-modal and cross-task scenario, revealing a significant gap in current training strategies; (3) enforcing models to explicitly describe visual content before reasoning (e.g., caption-before-thinking), along with rewarding progressive vision-to-text grounding, yields notable gains. It highlights two essential ingredients for improving compositionality in VLMs: visual-to-text alignment and accurate visual grounding. Our findings shed light on the current limitations of RL-based reasoning VLM training and provide actionable insights toward building models that reason compositionally across modalities and tasks.

Figures

Figures reproduced from arXiv: 2505.19406 by the authors.

Figure 1
Figure 1. Demonstration of the tasks and partial results for probing of cross-modality and cross-task [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Demonstration of proposed ComPABench for RQ1, RQ2, and RQ3. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Performance comparison when post-trained with pure-text and evaluated with either [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Trend of multi-modal performance without and with pure-text training initialization. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Comparison of Results for compositional reasoning from independently acquired skills. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Illustration of RL-Ground framework. As shown in [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Comparison of Results for generalization to OOD independent and compositional tasks. [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

23 extracted references · 22 linked inside Pith

  1. [1]

    Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture-of-loras

    Abdelrahman Abouelenin, Atabak Ashfaq, Adam Atkinson, Hany Awadalla, Nguyen Bach, Jianmin Bao, Alon Benhaim, Martin Cai, Vishrav Chaudhary, Congcong Chen, et al. Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture-of-loras. arXiv preprint arXiv:2503.01743,

    arXiv

  2. [4]

    Virgo: A preliminary exploration on reproducing o1-like mllm

    Yifan Du, Zikang Liu, Yifan Li, Wayne Xin Zhao, Yuqi Huo, Bingning Wang, Weipeng Chen, Zheng Liu, Zhongyuan Wang, and Ji-Rong Wen. Virgo: A preliminary exploration on reproducing o1-like mllm. arXiv preprint arXiv:2501.01904,

    arXiv

  3. [5]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948,

    arXiv

  4. [6]

    Can mllms reason in multimodality? emma: An enhanced multimodal reasoning benchmark

    Yunzhuo Hao, Jiawei Gu, Huichen Will Wang, Linjie Li, Zhengyuan Yang, Lijuan Wang, and Yu Cheng. Can mllms reason in multimodality? emma: An enhanced multimodal reasoning benchmark. arXiv preprint arXiv:2501.05444,

    arXiv

  5. [7]

    Advancing language model reasoning through reinforcement learning and inference scaling

    Zhenyu Hou, Xin Lv, Rui Lu, Jiajie Zhang, Yujiang Li, Zijun Yao, Juanzi Li, Jie Tang, and Yuxiao Dong. Advancing language model reasoning through reinforcement learning and inference scaling. arXiv preprint arXiv:2501.11651,

    arXiv

  6. [8]

    Boosting mllm reasoning with text-debiased hint-grpo

    Qihan Huang, Long Chan, Jinlong Liu, Wanggui He, Hao Jiang, Mingli Song, Jingyuan Chen, Chang Yao, and Jie Song. Boosting mllm reasoning with text-debiased hint-grpo. arXiv preprint arXiv:2503.23905,

    arXiv

  7. [9]

    Openai o1 system card

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720,

    arXiv

  8. [10]

    Understanding the effects of rlhf on llm generalisation and diversity

    Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu. Understanding the effects of rlhf on llm generalisation and diversity. ArXiv, abs/2310.06452,

    arXiv

Show all 23 references
  1. [12]

    Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning

    Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, et al. Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning. arXiv preprint arXiv:2503.07365,

    arXiv

  2. [13]

    Metaspatial: Reinforcing 3d spatial reasoning in vlms for the metaverse

    10 Zhenyu Pan and Han Liu. Metaspatial: Reinforcing 3d spatial reasoning in vlms for the metaverse. arXiv preprint arXiv:2503.18470,

  3. [14]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300,

    arXiv

  4. [15]

    Vlm-r1: A stable and generalizable r1-style large vision-language model

    Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, Ruochen Xu, and Tiancheng Zhao. Vlm-r1: A stable and generalizable r1-style large vision-language model. arXiv preprint arXiv:2504.07615, 2025a. Ma...

    arXiv

  5. [16]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191, 2024a. Weiyun Wang, Zhangwei Gao...

    arXiv

  6. [17]

    Generalization vs memorization: Tracing language models’ capabilities back to pretraining data

    Xinyi Wang, Antonis Antoniades, Yanai Elazar, Alfonso Amayuelas, Alon Albalak, Kexun Zhang, and William Yang Wang. Generalization vs memorization: Tracing language models’ capabilities back to pretraining data. arXiv preprint arXiv:2407.14985, 2024b. Zhiyu Wu, Xiaokang Chen, Z...

    arXiv

  7. [18]

    Llava-o1: Let vision language models reason step-by-step

    Guowei Xu, Peng Jin, Li Hao, Yibing Song, Lichao Sun, and Li Yuan. Llava-o1: Let vision language models reason step-by-step. arXiv preprint arXiv:2411.10440, 2024a. Zhuoyan Xu, Zhenmei Shi, and Yingyu Liang. Do large language models have compositional ability? an investigation...

    arXiv

  8. [19]

    Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search

    Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, et al. Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search. arXiv preprint arXiv:2412.18319,

    arXiv

  9. [20]

    Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? arXiv preprint arXiv:2504.13837,

    Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, and Gao Huang. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? arXiv preprint arXiv:2504.13837,

    arXiv

  10. [21]

    Vision-r1: Evolving human-free alignment in large vision-language models via vision-guided reinforcement learning

    Yufei Zhan, Yousong Zhu, Shurong Zheng, Hongyin Zhao, Fan Yang, Ming Tang, and Jinqiao Wang. Vision-r1: Evolving human-free alignment in large vision-language models via vision-guided reinforcement learning. arXiv preprint arXiv:2503.18013,

    arXiv

  11. [22]

    Exploring the compositional deficiency of large language models in mathematical reasoning

    Jun Zhao, Jingqi Tong, Yurong Mou, Ming Zhang, Qi Zhang, and Xuanjing Huang. Exploring the compositional deficiency of large language models in mathematical reasoning. arXiv preprint arXiv:2405.06680,

    arXiv

  12. [23]

    Echo chamber: Rl post-training amplifies behaviors learned in pretraining

    Rosie Zhao, Alexandru Meterez, Sham Kakade, Cengiz Pehlevan, Samy Jelassi, and Eran Malach. Echo chamber: Rl post-training amplifies behaviors learned in pretraining. arXiv preprint arXiv:2504.07912,

    arXiv

  13. [2023]

    Liangchen Luo, Yinxiao Liu, Rosanne Liu, Samrat Phatale, Meiqi Guo, Harsh Lara, Yunxuan Li, Lei Shu, Yun Zhu, Lei Meng, et al

    URL https://api.semanticscholar.org/ CorpusID:263830929. Liangchen Luo, Yinxiao Liu, Rosanne Liu, Samrat Phatale, Meiqi Guo, Harsh Lara, Yunxuan Li, Lei Shu, Yun Zhu, Lei Meng, et al. Improve mathematical reasoning in language models by automated process supervision. arXiv pre...

    arXiv

  14. [2024]

    Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma

    URL https: //api.semanticscholar.org/CorpusID:274581884. Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. Sft memorizes, rl generalizes: A comparative study of foundation model post-training. arXiv prepr...

    arXiv

  15. [2025]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, Lixin Gu, Xuehui Wang, Qingyun Li, Yiming Ren, Zixuan Chen, Jiapeng Luo, Jiahao Wang, Tan Jiang, Bo Wang, Conghui He, Botian Shi, Xingcheng Zhang, Han Lv, Y...

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.