Pith. sign in

REVIEW 5 major objections 6 minor 3 cited by

G1: Bootstrapping Perception and Reasoning Abilities of Vision-Language Model via Reinforcement Learning

T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Reinforcement learning with a perception-enhanced cold start lets a 7-billion-parameter vision-language model beat its teacher and frontier proprietary models on four visual games.

desk verdict A genuinely useful VLM RL testbed and a clever perception-enhanced cold start, but the headline 'mutual bootstrapping' claim is an interpretation that the paper's own ablation partially undermines. read the letter →

arxiv 2505.13426 v1 pith:XKUS2V6K submitted 2025-05-19 cs.CV

Liang Chen , Hongcheng Gao , Tianyu Liu , Zhiqi Huang , Flood Sung , Xinyu Zhou , Yuxin Wu , Baobao Chang This is my paper · ORCID
classification cs.CV
keywords vision-languagemodelsreinforcementlearningvisualgamesGRPOperceptionandreasoningbootstrappingknowledgedistillationVLM-Gyminteractivedecision-making
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to close the 'knowing-doing' gap in vision-language models (VLMs) by training them to play visual games with reinforcement learning. It introduces VLM-Gym, a suite of four visual games with unified interfaces and adjustable difficulty, and trains a 7-billion-parameter VLM in two ways: G0 by pure RL self-evolution, and G1 after a perception-enhanced cold start of 1,000 teacher-distilled responses that include ground-truth perception. The central claim is that G1 consistently outperforms its teacher and strong proprietary models on all four games, and that perception and reasoning abilities bootstrap each other during RL, with structured localization patterns appearing in rollouts before game rewards rise. If that is right, a modest open model with access to privileged perception data plus outcome-reward RL can beat frontier proprietary agents on these games, and reward-driven training alone can induce structured perception and reasoning.

What carries the argument

The load-bearing machinery has two parts. The first is VLM-Gym itself: a set of four visual games (2048, Shisen-Sho, Shisen-Sho with CIFAR-10 tiles, and Swap) built on a unified observation/action interface, with ground-truth perception available from the environment, adjustable compositional difficulty, and parallel action sampling so that GRPO's group-relative advantages can be computed. The second is the training recipe: a perception-enhanced cold start—supervised fine-tuning on teacher-distilled responses in which the prompt contains the ground-truth board description—followed by GRPO with a final reward that is the sum of game reward, a format reward, and an optional perception reward. The paper also uses the emergence of two textual patterns, localization and enumeration, as measurable evidence that perception and reasoning are improving in the model's own rollouts.

What would settle it

Reproduce the G0 Shisen-Sho run with a small action space (four macro moves instead of full coordinate pairs) or with rewards that sometimes give points for wrong perception; if localization patterns no longer appear before reward gains and scores stop improving, the bootstrapping claim is an artifact of this game set rather than a general property of VLM reinforcement learning.

Watch

Extended reading notes

Core claim

At the center of the paper is a claim about how a vision-language model learns to act: when a VLM is placed in an interactive visual environment and optimized with GRPO on a per-step game reward, its perception and reasoning abilities improve together, each enabling the other. In G0, pure RL produces an 'aha' moment in Shisen-Sho: the model learns to localize every tile with coordinates, and this localization pattern appears before game reward rises. In G1, the authors add a perception-enhanced cold start—1,000 teacher-distilled responses that include ground-truth perception together with the teacher's thinking and actions—and then run the same RL. The result is that the G1-7B model surpasses the teacher and proprietary models on all games (for example, 1070 versus 892 in 2048, 17.5 versus 15.3 in Shisen-Sho, 14.1 versus 8.7 in Shisen-Sho-Cifar10, and 0.78 versus 0.43 in Swap). The paper's central discovery is the bootstrapping loop: correct perception is rewarded only when reasoning converts it into a valid action, and correct reasoning is impossible without accurate perception, so RL pressure improves both together.

Load-bearing premise

The load-bearing premise is that the per-step game reward is informative about perception quality, so a model cannot earn positive rewards while perceiving incorrectly; the paper's own 2048 result shows this premise fails when the action space is small.

Editorial extensions

If this is right

  • A 7-billion-parameter open VLM can outperform its teacher and leading proprietary models on all four VLM-Gym games when perception-enhanced cold start is followed by GRPO.
  • Pure reward-driven RL can induce structured perception and reasoning in Shisen-Sho and its CIFAR-10 variant, with localization patterns appearing before reward gains.
  • The cold start removes the perception prior gap and addresses both the inaccurate reward credit problem seen in 2048 and the sparse reward problem seen in Swap.
  • Perception and reasoning co-evolve under a final verifiable reward; adding an explicit perception process reward speeds perception accuracy but does not improve final game reward.
  • VLM-Gym's unified, parallel, compositionally difficult environment makes scalable multi-game RL with GRPO practical for vision-language models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The bootstrapping mechanism is likely sensitive to action-space size and reward informativeness: the paper's own 2048 result shows the mechanism fails when four actions let wrong perception still score, so the claim should be read as a property of tasks where perception is the bottleneck, not of VLM RL in general.
  • If the recipe generalizes beyond synthetic games, the same combination—outcome reward plus a cold start that injects ground-truth perception from a privileged simulator—would transfer to embodied tasks where instrumentation supplies perception labels; the paper does not test this transfer.
  • A direct next experiment would be to add a perception process reward to Swap, since the paper's Section 4.3 shows perception reward accelerates perception accuracy without changing game reward, suggesting it may not be enough to overcome sparse reward on its own.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper introduces VLM-Gym, a suite of four visual games (2048, Shisen-Sho, Shisen-Sho-Cifar10, Swap) with unified interfaces, parallel action sampling, and adjustable difficulty, intended for GRPO-style reinforcement learning of vision-language models. The authors train Qwen2.5-VL-7B in two regimes: G0 via RL from game rewards with random-step exploration, and G1 via a cold-start SFT on 1,000 teacher (Claude-3.7-Sonnet-Thinking) responses that are prompted with ground-truth perception, followed by GRPO. They report that G1 surpasses G0, the teacher, and several proprietary baselines on all games, and they analyze perception accuracy (P_acc) and reasoning accuracy (R_acc) to argue that perception and reasoning abilities mutually bootstrap during RL training.

Significance. If the reported results are reproducible, the paper makes a useful contribution: it provides a lightweight, parallelizable RL benchmark for VLMs, shows that outcome-reward RL can induce structured perception outputs (localization patterns) and reasoning patterns in a small open VLM, and reports the non-obvious claim that a 7B model with a privileged cold start can beat frontier proprietary models on these synthetic games. Strengths include the open-source release of VLM-Gym and training code, the use of external game scores as rewards (which avoids circular reward design), and the explicit P_acc/R_acc decomposition for studying training dynamics. The significance is real but conditional on resolving the statistical, confounding, and mechanism-evidence issues below.

major comments (5)
  1. [Table 1 / Sec. 3.4] The headline claim that G1 'consistently surpasses' the teacher is not supported with statistical evidence. Table 1 reports one mean per model per game, with no standard deviation, confidence interval, or number of training seeds, and the text does not specify how many independent runs contribute to each mean. The note explains '100 steps×10' but not '1 step×100' used for Swap, and a single-step protocol is inconsistent with the stated multi-turn evaluation setting in Sec. 3.4. Please report means with variance or bootstrap confidence intervals over at least 5 independent evaluation runs (ideally multiple training seeds) and use a consistent multi-turn protocol for all games, or explicitly justify why Swap is evaluated differently.
  2. [Sec. 3.2 / Table 1] The comparison between G1 and Claude-3.7-Sonnet-Thinking is confounded by privileged perception. The cold-start data are generated by prompting Claude with the environment's ground-truth perception, while G1 is evaluated without such privileged information; the teacher is therefore at a perceptual disadvantage at test time. To support 'surpass their teacher across all games,' the authors should either evaluate Claude with the same ground-truth-perception prompts at test time, or train a cold-start variant that uses the teacher's own non-privileged perception outputs, so that the comparison isolates model capability rather than access to ground truth.
  3. [Sec. 4.1 / Figs. 5, 8, 10 / Eq. (1)] The central 'mutual bootstrapping' claim rests on an untested premise: that per-step game reward is informative about perception quality, so incorrect perception cannot easily obtain positive reward. The 2048 result (Sec. 4.1, Fig. 10) is exactly the regime where this premise fails, and it is presented as post-hoc evidence rather than a controlled test. The paper should either manipulate action-space size or reward informativeness across otherwise matched games, or quantify per-state the association between perception correctness and reward (e.g., mutual information between P_acc and positive reward), and it should support the 'precedes' claim in Fig. 5 with a statistical test across seeds rather than a single rollout count. In addition, Sec. 4.3 (Fig. 8) shows that an explicit perception reward accelerates P_acc without improving game reward; the paper should explain how this observation is consistent with the claim that outcome reward drives perception improvement through a bootstrapping mechanism.
  4. [Sec. 3.1 / Eq. (6)] The GRPO advantage in Eq. (6) normalizes group rewards by their standard deviation. With group size G=5 and rewards that are sums of a few binary components (game reward, format reward, perception reward), many groups will have zero variance, making the update undefined unless zero-variance groups are skipped or a variance floor is introduced. This detail is not stated and is material to reproducibility. Please specify the exact handling of zero-variance groups in the implementation.
  5. [Sec. 3.2 / Sec. 4.3] The design does not isolate the effect of the ground-truth-perception cold start from the effect of SFT in general. The cold start simultaneously teaches the output format, game mechanics, action syntax, and perception content, so the observed improvements in G1 relative to G0 could be due to any of these factors. An ablation with a cold start on the same 1,000 states but without the ground-truth perception field (for example, using teacher-generated perception instead) is needed to support the claim that 'perception-enhanced cold start' specifically addresses the inaccurate-credit and sparse-reward problems.
minor comments (6)
  1. [Sec. 1] The word 'boarder' should be 'broader', and the phrase 'knowing-doing' gap is used repeatedly; please standardize the terminology.
  2. [Figure 5] The y-axis and the regex rule used to count 'localization patterns' are not defined; please specify what counts as a localization pattern and report variance across runs.
  3. [Sec. 3.1 / Eq. (4)] It is not stated whether the Format Reward and Perception Reward are included in R(q_s, o_i) when computing the group advantage in Eq. (6); please clarify the reward composition used for advantage estimation.
  4. [Sec. 3.2 / Fig. 9] Figure 9 shows ground-truth perception in the distillation prompt, but G1 at test time receives only the screenshot; please state explicitly that this asymmetry exists and discuss any distribution shift it may cause.
  5. [Sec. 4.4] The phrase 'existed experiments' should read 'existing experiments'.
  6. [References] In Section 1, reference [1] (the Claude 3.7 system card) is cited for the claim about Pokémon, which appears to be the wrong source.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: game scores are external objective benchmarks and no fitted quantity is relabeled as a prediction.

full rationale

The paper's central results—G0/G1 game scores and the mutual-bootstrapping observation—are not derived from the inputs by definition. Game rewards (Eqs. 1–3) are external environment signals independent of the model's perception text; GRPO optimization (Eqs. 5–6) uses these rewards, not any fitted predictor. The G1 cold start uses 1,000 teacher-distilled responses with ground-truth perception, but the subsequent claims are evaluated against held-out game scores (Table 1) and training dynamics; no fitted parameter is later renamed as a prediction. The bootstrapping claim is an interpretation of measured localization-pattern counts (Figure 5) preceding game-reward improvements, not a consequence of the definitions of P_acc and R_acc. The only self-referential element is that Claude-3.7-Sonnet-Thinking serves both as distillation teacher and as a benchmark; this makes the comparison favorable to G1 but does not make G1's score equal to the teacher's by construction. Self-citations to PCA-Bench, MMEvalPro, and R1-V are background or methodological and are not load-bearing. Therefore no circular step is exhibited.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The ledger shows the paper's contribution is empirical and engineering-driven: no hidden fitted parameters are relabeled as predictions. The free parameters are reward weights, exploration budgets, SFT size, and training length, all stated in the text; the key domain assumptions are that the GRPO framework works in interactive settings, that the random-warm-start training distribution is representative of evaluation, and that the binary per-step rewards align with cumulative scores, the last two of which the paper never verifies.

free parameters (6)
  • Format reward weight alpha = 1 (default, Eq. 4)
    Final reward = GR + alpha*FR + beta*PR; alpha=1 adds the format reward to the game reward and changes the reward scale seen by GRPO, chosen by hand.
  • Perception reward weight beta = 0 in main runs (Eq. 4); enabled in Sec 4.3 experiment
    Hand-chosen; the discussion experiment sets a nonzero value to test perception as a process reward and reports it does not change game reward.
  • Random exploration steps per game = 100 (2048), 250 (Shisen-Sho, Shisen-Sho-Cifar10, Swap)
    Hand-chosen (Sec 3.1); defines the training state distribution (random-warm-started boards), which differs from evaluation's fresh-board starts; the shift is never quantified.
  • Cold-start SFT dataset size = 1,000 observations per game
    Hand-chosen (Sec 3.3); the amount of distilled teacher data used before RL; the paper does not ablate this size.
  • Binary per-step game reward design = +1/-1 per step (Eqs. 1-3)
    Reward-shaping choice; for 2048 the RL signal is 'did any tile merge' rather than the cumulative score used for evaluation (Sec 3.4), a proxy the paper itself links to the G0-2048 collapse.
  • Number of RL steps per game = 500 (convergence narratives reference ~400)
    Hand-chosen training budget (Sec 3.3); the reward-curve claims in Sec 4.1 (e.g., Shisen-Sho rises to 0.8 over 400 steps) depend on this budget.
assumptions (6)
  • domain assumption GRPO implemented in EasyR1/VeRL computes valid advantages for multi-turn game rollouts
    Sec 3.3 builds the RL system on EasyR1/VeRL; a framework bug would affect all training results, and no verification of the framework for interactive stateful environments is provided.
  • domain assumption Environment-provided ground-truth perception is complete and exact
    The cold-start data (Sec 3.2) and P_acc computation (Sec 4.2) assume the board descriptions generated by VLM-Gym are correct; this holds by construction for synthetic games.
  • domain assumption Qwen2.5-VL-7B's visual encoder can acquire the localization patterns from 640x840 screenshots via RL
    The G0 result presupposes sufficient base perception; the paper itself notes a 'perception prior gap' that slows Shisen-Sho-Cifar10 learning (Sec 4.1).
  • domain assumption Per-step outcome rewards plus group normalization are an aligned training signal for the cumulative-score benchmark
    GRPO optimizes Eqs. 1-6 while evaluation reports cumulative scores (Sec 3.4); no alignment analysis is given, and the G0-2048 collapse is attributed to reward misalignment (Sec 4.1).
  • domain assumption Policies trained on random-warm-started boards transfer to fresh-board evaluation
    Training episodes begin after 100-250 random steps (Sec 3.1), while evaluation starts fresh (Sec 3.4); the training reward (~0.8/step Shisen-Sho) is inconsistent with the evaluation score (12.8/36 steps) under matched distributions, indicating an unquantified shift.
  • standard math Standard GRPO objective, clipping, and KL penalty (Eqs. 5-6) are correct as given
    Taken from DeepSeekMath/DeepSeek-R1 and applied without modification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of G1: Bootstrapping Perception and Reasoning Abilities of Vision-Language Model via Reinforcement Learning." pith.science (2026). https://pith.science/paper/XKUS2V6K

@misc{pith2026250513426,
  author       = {Pith},
  title        = {Pith review of: G1: Bootstrapping Perception and Reasoning Abilities of Vision-Language Model via Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XKUS2V6K}},
  note         = {Machine review of arXiv:2505.13426}
}
read the original abstract

Vision-Language Models (VLMs) excel in many direct multimodal tasks but struggle to translate this prowess into effective decision-making within interactive, visually rich environments like games. This ``knowing-doing'' gap significantly limits their potential as autonomous agents, as leading VLMs often performing badly in simple games. To address this, we introduce VLM-Gym, a curated reinforcement learning (RL) environment featuring diverse visual games with unified interfaces and adjustable, compositional difficulty, specifically designed for scalable multi-game parallel training. Leveraging VLM-Gym, we train G0 models using pure RL-driven self-evolution, which demonstrate emergent perception and reasoning patterns. To further mitigate challenges arising from game diversity, we develop G1 models. G1 incorporates a perception-enhanced cold start prior to RL fine-tuning. Our resulting G1 models consistently surpass their teacher across all games and outperform leading proprietary models like Claude-3.7-Sonnet-Thinking. Systematic analysis reveals an intriguing finding: perception and reasoning abilities mutually bootstrap each other throughout the RL training process. Source code including VLM-Gym and RL training are released at https://github.com/chenllliang/G1 to foster future research in advancing VLMs as capable interactive agents.

Figures

Figures reproduced from arXiv: 2505.13426 by the authors.

Figure 1
Figure 1. Comparison of different models on games from VLM-Gym. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Key features of VLM-Gym. We illustrate them using the Shisen-Sho game as an example. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Average game reward curves of different games for G0 models during RL process. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: The explored perception and reasoning patterns during G0 RL training in Shisen-Sho Game [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Localization patterns count during G0 RL training for different games. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Training curves of G1 [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Comparisons of game reward between G0 and G1 across different games during RL. [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: RL training curves exploring Percep￾tion Accuracy as process reward. SFT+RL Jointly Helps the Base Model Outperform the Supervi￾sor Across all games, G1 RL training significantly improves game reward compared to the base model after the cold start SFT. This SFT process…
Figure 9
Figure 9. Figure 9: The process of constructing perception-enhanced cold start data via knowledge distillation and programmable [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: G0 2048 case studies. The case shows that before RL training, the model produced incorrect perception and [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: Rules of 2048. C.1 2048 Rule In the 2048 game, the player swipes the entire 4×4 grid up, down, left, or right. All tiles slide as far as possible in the chosen direction, and any two tiles of equal value that collide merge immediately into a single tile whose value is…
Figure 12
Figure 12. Figure 12: Rules of Shisen-Sho. Connect the two dog image Result: Matched shape disappeared (a) Correct Action Connect the dog with the car Result: No matched shapes (b) Wrong Action [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]
Figure 13
Figure 13. Figure 13: Rules of Shisen-Sho-CIFAR10. no unobstructed path with ≤ 2 turns exists (Fig. 13b), the move fails and the board remains unchanged. As shown in [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]
Figure 14
Figure 14. Figure 14: Rules of Swap. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_14.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. CAST: Game Solvers as Turn-Level Teachers for LLM Agents

    cs.CL 2026-07 conditional novelty 6.0 of 10

    CAST converts a game solver's per-action cost-to-go changes into turn-level RL credits for LLM agents and reports gains over outcome-only RLVR on three games plus zero-shot transfer to ALFWorld and WebShop.

  2. Explain Before You Answer: A Survey on Compositional Visual Reasoning

    cs.CV 2025-08 conditional novelty 5.0 of 10

    A survey that classifies compositional visual reasoning methods into five stages, from prompt-based pipelines to unified agentic vision-language models, and catalogs associated benchmarks.

  3. Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Models

    cs.CV 2025-05 conditional novelty 4.0 of 10

    A survey organizes multimodal reasoning research into a staged roadmap and proposes native large multimodal reasoning models that unify perception, generation, and agentic planning.

Reference graph

Works this paper leans on

48 extracted references · 18 canonical work pages · cited by 3 Pith papers

  1. [1]

    Claude 3.7 sonnet system card

    work page

  2. [2]

    Qwen2.5-vl technical report.arXiv preprint arXiv:2502.13923, 2025

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report.ar...

    arXiv 2025

  3. [3]

    Deep blue.Artificial intelligence, 134(1-2):57–83, 2002

    Murray Campbell, A Joseph Hoane Jr, and Feng-hsiung Hsu. Deep blue.Artificial intelligence, 134(1-2):57–83, 2002

    2002

  4. [4]

    R1-v: Reinforcing super generalization ability in vision-language models with less than $3

    Liang Chen, Lei Li, Haozhe Zhao, Yifan Song, and Vinci. R1-v: Reinforcing super generalization ability in vision-language models with less than $3. https://github.com/Deep-Agent/R1-V, 2025. Accessed: 2025-02-02

    work page 2025

  5. [5]

    Next token prediction towards multimodal intelligence: A comprehensive survey, 2024

    Liang Chen, Zekun Wang, Shuhuai Ren, Lei Li, Haozhe Zhao, Yunshui Li, Zefan Cai, Hongcheng Guo, Lei Zhang, Yizhe Xiong, Yichi Zhang, Ruoyu Wu, Qingxiu Dong, Ge Zhang, Jian Yang, Lingwei Meng, Shujie Hu, Yulong Chen, Junyang Lin, Shuai Bai, Andreas Vlachos, Xu Tan, Minjia Zhang, Wen Xiao, Aaron Yee, Tianyu Liu, and Baobao Chang. Next token prediction towar...

    work page 2024

  6. [6]

    Pca-bench: Evaluating multimodal large language models in perception-cognition-action chain, 2024

    Liang Chen, Yichi Zhang, Shuhuai Ren, Haozhe Zhao, Zefan Cai, Yuchi Wang, Peiyi Wang, Xiangdi Meng, Tianyu Liu, and Baobao Chang. Pca-bench: Evaluating multimodal large language models in perception-cognition-action chain, 2024

    work page 2024

  7. [7]

    Can vlms play action role-playing games? take black myth wukong as a study case.arXiv preprint arXiv:2409.12889, 2024

    Peng Chen, Pi Bu, Jun Song, Yuan Gao, and Bo Zheng. Can vlms play action role-playing games? take black myth wukong as a study case.arXiv preprint arXiv:2409.12889, 2024

    arXiv 2024

  8. [8]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai D...

    work page 2025

Show all 48 references
  1. [9]

    Gtbench: Uncovering the strategic reasoning limitations of llms via game-theoretic evaluations.arXiv preprint arXiv:2402.12348, 2024

    Jinhao Duan, Renming Zhang, James Diffenderfer, Bhavya Kailkhura, Lichao Sun, Elias Stengel-Eskin, Mohit Bansal, Tianlong Chen, and Kaidi Xu. Gtbench: Uncovering the strategic reasoning limitations of llms via game-theoretic evaluations.arXiv preprint arXiv:2402.12348, 2024

    2024 arXiv

  2. [10]

    Flowreasoner: Reinforcing query-level meta-agents.arXiv preprint arXiv:2504.15257, 2025

    Hongcheng Gao, Yue Liu, Yufei He, Longxu Dou, Chao Du, Zhijie Deng, Bryan Hooi, Min Lin, and Tianyu Pang. Flowreasoner: Reinforcing query-level meta-agents.arXiv preprint arXiv:2504.15257, 2025

    2025 arXiv

  3. [11]

    Vlm q-learning: Aligning vision-language models for interactive decision-making.arXiv preprint arXiv:2505.03181, 2025

    Jake Grigsby, Yuke Zhu, Michael Ryoo, and Juan Carlos Niebles. Vlm q-learning: Aligning vision-language models for interactive decision-making.arXiv preprint arXiv:2505.03181, 2025

    2025 arXiv

  4. [12]

    Mmevalpro: Calibrating multimodal benchmarks towards trustworthy and efficient evaluation, 2025

    Jinsheng Huang, Liang Chen, Taian Guo, Fu Zeng, Yusheng Zhao, Bohan Wu, Ye Yuan, Haozhe Zhao, Zhihui Guo, Yichi Zhang, Jingyang Yuan, Wei Ju, Luchen Liu, Tianyu Liu, Baobao Chang, and Ming Zhang. Mmevalpro: Calibrating multimodal benchmarks towards trustworthy and efficient ev...

    2025

  5. [13]

    Vision-r1: Incentivizing reasoning capability in multimodal large language models.arXiv preprint arXiv:2503.06749, 2025

    Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-r1: Incentivizing reasoning capability in multimodal large language models.arXiv preprint arXiv:2503.06749, 2025

    2025 arXiv

  6. [14]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009

    2009

  7. [15]

    Jarvis-vla: Post-training large-scale vision language models to play visual games with keyboards and mouse.arXiv preprint arXiv:2503.16365, 2025

    Muyao Li, Zihao Wang, Kaichen He, Xiaojian Ma, and Yitao Liang. Jarvis-vla: Post-training large-scale vision language models to play visual games with keyboards and mouse.arXiv preprint arXiv:2503.16365, 2025

    2025 arXiv

  8. [16]

    Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning

    Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Tiancheng Han, Botian Shi, Wenhai Wang, Junjun He, et al. Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning. arXiv preprint arXiv:2503.07365, 2025

    2025 arXiv

  9. [17]

    Playing atari with deep reinforcement learning, 2013

    V olodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning, 2013

    2013

  10. [18]

    OpenAI, :, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, Ally Bennet...

    2024

  11. [19]

    Gpt-4v(ision) system card

    OpenAI. Gpt-4v(ision) system card. 2023

    2023

  12. [20]

    hello-gpt-4o, 2024

    OpenAI. hello-gpt-4o, 2024

    2024

  13. [21]

    Benchmarking agentic llm and vlm reasoning on games.arXiv preprint arXiv:2411.13543, 2024

    Davide Paglieri, Bartłomiej Cupiał, Sam Coward, Ulyana Piterbarg, Maciej Wołczyk, Akbir Khan, Eduardo Pignatelli, Łukasz Kuci´nski, Lerrel Pinto, Rob Fergus, Jakob Nicolaus Foerster, Jack Parker-Holder, and Tim Rocktäschel. Benchmarking agentic llm and vlm reasoning on games.a...

    2024 arXiv

  14. [22]

    Generative agents: Interactive simulacra of human behavior

    Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. InProceedings of the 36th annual acm symposium on user interface software and technology, pages 1–22, 2023. 12 G1

    2023

  15. [23]

    Llms are greedy agents: Effects of rl fine-tuning on decision-making abilities, 2025

    Thomas Schmied, Jörg Bornschein, Jordi Grau-Moya, Markus Wulfmeier, and Razvan Pascanu. Llms are greedy agents: Effects of rl fine-tuning on decision-making abilities, 2025

    2025

  16. [24]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024

    2024

  17. [25]

    Vlm-r1: A stable and generalizable r1-style large vision-language model.arXiv preprint arXiv:2504.07615, 2025

    Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, et al. Vlm-r1: A stable and generalizable r1-style large vision-language model.arXiv preprint arXiv:2504.07615, 2025

    2025 arXiv

  18. [26]

    Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256, 2024

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256, 2024

    2024 arXiv

  19. [27]

    Maddison, Arthur Guez, L

    David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, L. Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Vedavyas Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy P. Lillicrap, Made...

    2016

  20. [28]

    Mastering the game of go with deep neural networks and tree search.nature, 529(7587):484–489, 2016

    David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. Mastering the game of go with deep neural networks and tree search.nature, 529(7587):484–489, 2016

    2016

  21. [29]

    Cradle: Empowering foundation agents towards general computer control.arXiv preprint arXiv:2403.03186, 2024

    Weihao Tan, Wentao Zhang, Xinrun Xu, Haochong Xia, Ziluo Ding, Boyu Li, Bohan Zhou, Junpeng Yue, Jiechuan Jiang, Yewen Li, et al. Cradle: Empowering foundation agents towards general computer control.arXiv preprint arXiv:2403.03186, 2024

    2024 arXiv

  22. [30]

    Gemini Team, Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry, Lepikhin, Timothy Lillicrap, Jean baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, Ioannis Antonoglou, Rohan Anil, Sebastian Borgeaud, Andrew Dai, Katie Millican, Ethan D...

    2024

  23. [31]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms.arXiv preprint arXiv:2501.12599, 2025

    2025 arXiv

  24. [32]

    Kimi Team, Angang Du, Bohong Yin, Bowei Xing, Bowen Qu, Bowen Wang, Cheng Chen, Chenlin Zhang, Chenzhuang Du, Chu Wei, Congcong Wang, Dehao Zhang, Dikang Du, Dongliang Wang, Enming Yuan, Enzhe Lu, Fang Li, Flood Sung, Guangda Wei, Guokun Lai, Han Zhu, Hao Ding, Hao Hu, Hao Yan...

    2025

  25. [33]

    Mark Towers, Ariel Kwiatkowski, Jordan Terry, John U. Balis, Gianluca De Cola, Tristan Deleu, Manuel Goulão, Andreas Kallinteris, Markus Krimmel, Arjun KG, Rodrigo Perez-Vicente, Andrea Pierré, Sander Schulhoff, Jun Jet Tai, Hannah Tan, and Omar G. Younis. Gymnasium: A standar...

    2024

  26. [34]

    Can large language models play text games well? current state-of-the-art and open questions.arXiv preprint arXiv:2304.02868, 2023

    Chen Feng Tsai, Xiaochen Zhou, Sierra S Liu, Jing Li, Mo Yu, and Hongyuan Mei. Can large language models play text games well? current state-of-the-art and open questions.arXiv preprint arXiv:2304.02868, 2023

    2023 arXiv

  27. [35]

    Are large vision language models good game players?, 2025

    Xinyu Wang, Bohan Zhuang, and Qi Wu. Are large vision language models good game players?, 2025

    2025

  28. [36]

    Are large vision language models good game players?arXiv preprint arXiv:2503.02358, 2025

    Xinyu Wang, Bohan Zhuang, and Qi Wu. Are large vision language models good game players?arXiv preprint arXiv:2503.02358, 2025

    2025 arXiv

  29. [37]

    Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents.arXiv preprint arXiv:2302.01560, 2023

    Zihao Wang, Shaofei Cai, Guanzhou Chen, Anji Liu, Xiaojian Ma, and Yitao Liang. Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents.arXiv preprint arXiv:2302.01560, 2023

    2023 arXiv

  30. [38]

    Waytowich, Devin White, MD Sunbeam, and Vinicius G

    Nicholas R. Waytowich, Devin White, MD Sunbeam, and Vinicius G. Goecks. Atari-gpt: Benchmarking multimodal large language models as low-level policies in atari games, 2024

    2024

  31. [39]

    Smartplay: A benchmark for llms as intelligent agents.arXiv preprint arXiv:2310.01557, 2023

    Yue Wu, Xuan Tang, Tom M Mitchell, and Yuanzhi Li. Smartplay: A benchmark for llms as intelligent agents.arXiv preprint arXiv:2310.01557, 2023

    2023 arXiv

  32. [40]

    R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization

    Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, Bo Zhang, and Wei Chen. R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization. arXiv preprint arXiv:2503.10615, 2025

    2025 arXiv

  33. [41]

    Fine-tuning large vision-language models as decision-making agents via reinforcement learning

    Yuexiang Zhai, Hao Bai, Zipeng Lin, Jiayi Pan, Shengbang Tong, Yifei Zhou, Alane Suhr, Saining Xie, Yann LeCun, Yi Ma, and Sergey Levine. Fine-tuning large vision-language models as decision-making agents via reinforcement learning. InThe Thirty-eighth Annual Conference on Neu...

    2024

  34. [42]

    Easyr1: An efficient, scalable, multi-modality rl training framework

    Yaowei Zheng, Shenzhi Wang Junting Lu, Zhangchi Feng, Dongdong Kuang, and Yuwen Xiong. Easyr1: An efficient, scalable, multi-modality rl training framework. https://github.com/hiyouga/EasyR1, 2025

    2025

  35. [43]

    Llamafactory: Unified efficient fine-tuning of 100+ language models

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System D...

    2024

  36. [44]

    aha moment

    Hengguang Zhou, Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsieh. R1-zero’s "aha moment" in visual reasoning on a 2b non-sft model, 2025. 15 G1 A Perception-Enhanced Cold Start Data Construction Game Observation DistillationPrompt:Yo u a r e p l a y i n g a...

    2025

  37. [46]

    Output Format Description First describe the board in <perception></perception>

    Second coordinate: (row2, col2) Where row and col are 0-indexed numbers such as (0, 1), starting from the top-left of the board. Output Format Description First describe the board in <perception></perception>. Then output your thinking process in <think></think> and final acti...

  38. [48]

    (row1,col1) (row2,col2)

    Second coordinate: (row2, col2) Where row and col are 0-indexed numbers such as (0, 1), starting from the top-left of the board. Output Format Description First describe the board in <perception></perception>. Then output your thinking process in <think></think> and final acti...

  39. [49]

    First coordinate: (row1, col1)

  40. [50]

    Output Format Description First describe the board in <perception></perception>

    Second coordinate: (row2, col2) Where row and col are 0-indexed numbers starting from the top-left of the board. Output Format Description First describe the board in <perception></perception>. Then output your thinking process in <think></think> and final action in <answer>(r...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.