Pith. sign in

REVIEW 5 major objections 6 minor 6 cited by

Learning Only with Images: Visual Reinforcement Learning with Reasoning, Rendering, and Visual Feedback

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Comparing renders lets a 7B model beat its 72B teacher

desk verdict A useful RL-with-rendering recipe that overstates its 'only images' claim on charts due to a GPT-4o cold-start and a favorable train/test split. read the letter →

arxiv 2507.20766 v4 pith:XXAJHR7W submitted 2025-07-28 cs.CV

Yang Chen , Yufan Shen , Wenxuan Huang , Sheng Zhou , Qunshu Lin , Xinyu Cai , Zhi Yu , Jiajun Bu
show 2 more authors
Botian Shi Yu Qiao
This is my paper
classification cs.CV
keywords visualreinforcementlearningmultimodallargelanguagemodelsimage-to-codegenerationfeedbackasymmetryofverificationGRPOchart-to-codeweb-to-code
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

RRVF claims that a multimodal language model can learn complex visual reasoning from raw images alone, without paired image-text supervision. The method makes the model generate rendering code, execute that code to produce an image, and then use a larger, more capable model to compare the render with the original input; that comparison, together with format and tool-use checks, becomes the reward in a GRPO reinforcement-learning loop. The paper reports that a 7-billion-parameter model trained this way outperforms similarly sized open-source models and supervised fine-tuning baselines on chart-to-code and web-to-code benchmarks, and on ChartMimic it also surpasses the 72-billion-parameter model that supplied the visual feedback. If correct, this turns the 'asymmetry of verification'—checking whether an output matches the source is easier than generating the output—into a scalable training signal for deep visual reasoning without human-labeled text.

What carries the argument

The load-bearing mechanism is the reasoning-rendering-visual-feedback loop. At each turn the policy model emits <think> reasoning and a <tool_call> containing executable code; the tool renders that code into an image; and a separate, stronger MLLM compares the render to the original image, producing textual feedback for the next turn and a final similarity score. This score, combined with a format-validity reward and a capped tool-use reward, becomes the advantage signal in GRPO, which optimizes the whole loop without ground-truth code or text labels. The design draws its force from making verification—an easier problem than generation—the source of the learning signal. For charts, a cold-start set of generated dialogues first teaches the tool-call format, but the RL reward itself still comes from visual comparison.

What would settle it

Render a set of chart images whose data values have been changed while colors and layout are kept identical; if the 72-billion-parameter judge still rates those renders highly, or if the RRVF-trained model reproduces the altered values while matching the surface style, then the reward signal has been gamed and the claim that verification drives true visual reasoning is falsified.

Watch

Extended reading notes

Core claim

The paper's central discovery is that image-to-code tasks exhibit an 'asymmetry of verification': verifying whether a rendered image matches a source image is substantially easier than generating the rendering code from scratch, and this easier check can be used as a dense reward for reinforcement learning. RRVF operationalizes this with a closed loop: the model iteratively writes code, a domain-specific tool renders it (matplotlib for charts, Playwright for web pages), and a stronger MLLM judge returns natural-language feedback plus a numeric similarity score. The loop is optimized end-to-end with GRPO under a hybrid reward combining visual similarity, format correctness, and tool-use incentive. In single-turn evaluation without tools, the trained 7B model reaches a 97.83% execution rate on ChartMimic and higher overall scores than comparable open-source models and SFT baselines, and it surpasses the 72B judge model on the same benchmark; on the unseen Plot2Code dataset its execution rate stays near 96%, which the paper takes as evidence of transferable visual reasoning rather than memorized code templates.

Load-bearing premise

The load-bearing premise is that the larger model's visual-similarity score, when the rendered image is compared with the source, is a trustworthy and non-gameable measure of real visual fidelity, so that maximizing that score teaches genuine image understanding rather than tricks that please the judge.

Editorial extensions

If this is right

  • If the claim holds, curated image-text pairs are not required to train visual code-generation skills; a renderer and a visual comparator can supply the learning signal.
  • Optimizing against rendered-image verification produces higher code execution rates and more robust zero-shot generalization than supervised fine-tuning on ground-truth code, as shown by the near-flat execution rate from ChartMimic to Plot2Code.
  • A small policy can surpass the larger model that provides its feedback, implying the training loop transfers self-correction ability rather than simple imitation of the teacher.
  • The recipe is not limited to the rigid syntax of chart libraries, since the web-to-code results show it also works for highly flexible webpage layouts with no ground-truth HTML.
  • The multi-turn training loop instills single-turn competence: at evaluation the model runs without tools or iteration and still outperforms baselines, suggesting the correction skill has been internalized.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's two domains, the same loop should apply to any task with a deterministic renderer—diagrams, slide decks, CAD sketches, or even simulated environments—where a render can be compared with a reference image.
  • The paper does not isolate how much of the gain comes from the textual feedback versus the scalar similarity score; ablating these two channels would pinpoint the mechanism and could make the reward design simpler and cheaper.
  • A testable question the paper leaves open is judge gaming: if the larger model's similarity score rewards surface style over semantic content, the policy could overfit to colors and layout while ignoring data values; using independent judges or explicit data-value checks would detect this.
  • The chart-task cold start uses generated dialogues, so the pure 'images only' claim is strongest for web-to-code; measuring how much the cold start contributes would clarify how label-free the method really is.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper introduces RRVF (Reasoning-Rendering-Visual-Feedback), a reinforcement learning framework that trains a 7B MLLM to generate rendering code (matplotlib for charts, HTML for web pages) from raw images. The model iteratively produces code, renders it, receives natural-language visual feedback from a larger judge model (Qwen2.5-VL-72B-Instruct), and is optimized with GRPO using a hybrid reward combining visual similarity, format correctness, and tool-use efficiency. The authors report state-of-the-art results among open-source sub-10B models on ChartMimic, Plot2Code, and WebSight, and claim that the trained 7B model outperforms the 72B model that provided training feedback. The central claim is that RRVF enables learning 'from only raw images' without text-based ground truth, based on the asymmetry between verifying a rendered image against a source image and generating the code from scratch.

Significance. If the claims were fully supported, the framework would be a valuable contribution: it demonstrates a general mechanism for converting a stronger model's visual-comparison ability into a training signal for image-to-code tasks, without human-annotated text labels. The idea of using rendering-based verification as an RL reward is timely and relevant to the growing interest in visual RL and tool-augmented reasoning. The paper also ships code and performs evaluations across two structurally distinct domains. However, the significance is currently undercut by two load-bearing problems: the chart-to-code pipeline relies on a text-supervised cold-start with GPT-4o-generated dialogues, contradicting the 'only images' framing, and the ChartMimic evaluation trains on 75% of the official test set, making the comparisons with zero-shot baselines favorable by construction. These issues must be resolved before the contributions can be assessed.

major comments (5)
  1. [Appendix A; Abstract; Section 3.2] The central claim that RRVF learns 'from only raw images' and requires 'no text-based ground truth' is contradicted by the cold-start procedure in Appendix A. For the flagship chart-to-code task, the model is initialized with 1,913 multi-turn dialogues generated by GPT-4o from ChartQA images. These dialogues contain full Python code, reasoning text, and visual feedback. The paper only excludes human-annotated ground-truth code, but it still uses substantial text supervision from a proprietary MLLM. Without this cold-start, the authors report that the model cannot reliably learn tool calling on charts. Therefore the ChartMimic/Plot2Code results are conditioned on a text-supervised warm start, and the claim of learning 'solely from raw images' is not supported. The authors should either remove the cold-start and show that RL alone works, or substantially revise the claims and title to reflect that the method reduces but does not eliminate text supervision.
  2. [Section 4.1 (Datasets and Metrics); Table 1] The ChartMimic evaluation protocol is unfairly favorable to RRVF. The paper repurposes 1,800 of the 2,400 official ChartMimic test images for training and evaluates on the remaining 600. All baselines, including the closed-source models and the Qwen2.5-VL-72B-Instruct teacher, are evaluated zero-shot on those 600 images, whereas RRVF has trained on 75% of the official test set. The reported superiority over Qwen2.5-VL-7B-Instruct, and especially the claim of surpassing the 72B teacher, cannot be attributed to the learning framework rather than to train/test contamination. The authors should train on a genuinely disjoint set (e.g., ChartQA training images) and evaluate on the full 2,400-image test set, or at least compare against baselines that are also fine-tuned on the same 1,800-image training split.
  3. [Section 4.2 (Ablation Study); Table 1] The 'beats the teacher' claim (Overall 64.36 vs. 47.30) is not a valid bootstrapping demonstration. The 72B teacher is evaluated zero-shot without any training on the 1,800-image chart distribution, while RRVF is trained on those images. A proper test would fine-tune Qwen2.5-VL-72B-Instruct on the same data (or give it tool-use and multi-turn feedback during evaluation) before comparing. As presented, the result simply reflects that training on 75% of the test distribution beats zero-shot inference on a held-out slice of that distribution.
  4. [Section 3.3 (Hybrid Reward Design); Section 3.2 (Visual Feedback); Appendix A] The reward signal and the cold-start data selection both rely on the same judge model (Qwen2.5-VL-72B-Instruct). The cold-start dialogues are filtered by 'turn-by-turn reward scores' that are presumably produced by this same judge, and the RL objective optimizes the policy against the judge's visual-similarity scores. This creates a circularity risk: the policy may learn to exploit the judge's preferences rather than achieve genuine visual fidelity. The final evaluation partially mitigates this by using independent metrics (ChartMimic official metrics and CLIP/GPT-4o for web), but the paper does not report any analysis of judge bias, reward hacking, or inter-judge agreement. I recommend adding a validation of the judge's reliability against human ratings and a qualitative analysis of cases where the judge's score increases but independent metrics do not.
  5. [Section 4.1 (Datasets and Metrics); Table 3] The web-to-code evaluation is more sound than the chart evaluation because the 2,000 training screenshots are sampled from WebSight and the 500 test images are disjoint. However, the only reported metrics are CLIP similarity and a GPT-4o judge score, not any established HTML-match metric. The prompt in Figure 10 instructs the judge to ignore missing images and icons, which may inflate scores. The authors should either report a standard metric for HTML generation (e.g., block-level matching on the WebSight test set) or justify why visual similarity alone is the appropriate metric and show that the GPT-4o judge correlates with human judgments.
minor comments (6)
  1. [Table 1] The table contains a typo: 'LLaV A-OneVision' should be 'LLaVA-OneVision'.
  2. [Figure 1 caption] The caption states RRVF 'trains solely on raw images,' which is inconsistent with the cold-start described in Appendix A. The caption should be revised to match the actual training pipeline.
  3. [Section 4.1 (Datasets and Metrics)] The paper says 'The official chart-to-code subset of the ChartMimic dataset comprises 2,400 test images' and then repurposes 1,800 for training. This should be flagged as a deviation from the benchmark's intended split, even if the smaller 600-image set is the official 'test' split. Please cite the original ChartMimic paper to clarify the intended usage.
  4. [Section 4.3 (RRVF vs. SFT)] The SFT baseline is not described in sufficient detail: training data, number of epochs, learning rate, and whether it uses the same cold-start dialogues or ground-truth ChartMimic code are not reported. This makes the ablation hard to interpret.
  5. [Section 3.3 (Reinforcement Learning Optimization)] Equation (1) is the standard GRPO objective, but the KL coefficient is set to 0.0 in the experimental settings. The text should explain why the KL term is omitted and whether this affects stability, since the objective function in Eq. (1) includes a KL term.
  6. [Appendix A] The cold-start curriculum thresholds (top 10%, top 30%, top 40%) are described, but the exact reward scores used for partitioning are not defined. Please specify which reward (e.g., visual similarity score) and what thresholds were used.

Circularity Check

1 steps flagged · score 4.0 of 10

Chart-domain 'only images' claim is conditioned on a GPT-4o text cold-start; the RL/evaluation loop is otherwise not closed circularly.

  1. fitted input called prediction [Table 1 caption and Appendix A (Cold Start Details)]
    "'RRVF (Ours) [without text labels]' (Table 1); 'To address tool-calling failures in the chart to code task, a cold-start dataset is synthesized using a score-guided approach... Multi-turn dialogues are generated for these images using GPT-4o... This stratification process results in a final cold-start dataset of 1,913 dialogues.'"

    The chart-domain headline result is advertised as trained 'without text labels,' and the abstract claims RRVF 'enables MLLMs to learn complex visual reasoning from only raw images.' But the training pipeline for charts begins with 1,913 GPT-4o-generated multi-turn text dialogues containing code, reasoning, and feedback, and the paper states that without this cold-start the model cannot learn tool-calling on charts. The ChartMimic result is therefore produced by a text-warm-started model, not derived from image-only inputs; the '[without text labels]' label holds only by excluding this text supervision from the definition. This is a partial construction of the claimed result from a text input that the paper simultaneously claims is absent.

full rationale

The only material circularity concern is concentrated in the chart-to-code domain. The abstract and Table 1 present RRVF as learning 'from only raw images' and '[without text labels],' but Appendix A discloses a mandatory GPT-4o cold-start of 1,913 multi-turn text dialogues, and the paper states that without it the model cannot learn tool-calling on charts. Thus the ChartMimic result is not a pure image-only derivation; it is a text-warm-started result relabeled as unsupervised. I do not find additional circular steps: the RL reward comes from a fixed Qwen2.5-VL-72B judge, while the final evaluation uses external, held-out metrics (official ChartMimic metrics, CLIP, GPT-4o), so the evaluation loop is not closed by the training reward. The web-to-code experiments do not require the cold-start and provide some independent support for the framework. There are no load-bearing self-citations or imported uniqueness theorems. The cold-start overclaim is significant but partial, so the score is 4 rather than 6-8.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The ledger shows that the method depends on several hand-set hyperparameters and on the reliability of a learned judge model. No new physical entities are introduced.

free parameters (6)
  • reward weights wv, wf, wt = 0.2, 0.8, 1.0
    Chosen by hand to balance visual similarity, format, and tool use; no sensitivity analysis is reported.
  • max interaction turns Tmax = 4
    Limits the closed-loop refinement; no ablation on turn count is shown.
  • GRPO group size G = 8
    Number of sampled trajectories per prompt for advantage normalization.
  • tool reward cap threshold = 0.95 visual similarity
    Tool-use reward is capped at 1.0 when the judge similarity exceeds 0.95; this threshold is a hand-set hyperparameter.
  • cold-start score filters = 10%, 30%, 40% for stages 1-3
    The chart cold-start dataset is filtered by percentile reward scores, biasing the initial policy toward examples the judge scores highly.
  • KL coefficient = 0.0
    No KL regularization to reference policy; could affect stability.
assumptions (6)
  • domain assumption Verification is easier than generation for image-to-code (Asymmetry of Verification)
    This is the paper's core premise; it is invoked in the Introduction and Method to justify using visual comparison as a reward. It is plausible but not proven for all chart and web cases.
  • domain assumption The Qwen2.5-VL-72B-Instruct judge provides a faithful and non-gameable visual similarity reward
    The reward signal used for RL comes from this model's comparison; if the judge can be fooled, the policy will overfit to it.
  • domain assumption Rendering engines (matplotlib for charts, Playwright for web) faithfully reproduce the code's visual output
    The comparison step relies on the rendered image being an accurate realization of the code; this is stated in the Visual Feedback section but not validated.
  • domain assumption The pretrained Qwen2.5-VL-7B-Instruct model already has sufficient visual perception and code-generation ability to benefit from RL
    Training starts from an instruction-tuned checkpoint, so the 'only images' contribution is on top of existing image-text supervision.
  • standard math GRPO with KL coefficient 0 remains stable for this multimodal tool-use setting
    The GRPO objective in Equation 1 is standard; however, setting KL to 0 is a choice that the paper does not justify and that can lead to policy collapse in some settings.
  • ad hoc to paper GPT-4o-generated cold-start dialogues and score filtering provide a valid initial policy for chart-to-code
    Appendix A introduces a 1,913-dialogue cold-start set generated by GPT-4o and filtered by reward scores; this injects text supervision and is specific to this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning Only with Images: Visual Reinforcement Learning with Reasoning, Rendering, and Visual Feedback." pith.science (2026). https://pith.science/paper/XXAJHR7W

@misc{pith2026250720766,
  author       = {Pith},
  title        = {Pith review of: Learning Only with Images: Visual Reinforcement Learning with Reasoning, Rendering, and Visual Feedback},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XXAJHR7W}},
  note         = {Machine review of arXiv:2507.20766}
}
read the original abstract

Multimodal Large Language Models (MLLMs) exhibit impressive performance across various visual tasks. Subsequent investigations into enhancing their visual reasoning abilities have significantly expanded their performance envelope. However, a critical bottleneck in the advancement of MLLMs toward deep visual reasoning is their heavy reliance on curated image-text supervision. To solve this problem, we introduce a novel framework, ``Reasoning-Rendering-Visual-Feedback'' (RRVF), that enables MLLMs to learn complex visual reasoning from only raw images. This framework builds on the ``Asymmetry of Verification'' principle, i.e., verifying the rendered output against the source image is substantially easier than performing deep visual reasoning to generate a faithful, structured representation such as code. We demonstrate that this relative ease provides an ideal reward signal for optimization via Reinforcement Learning (RL), thereby reducing reliance on image-text supervision. RRVF implements a closed-loop iterative process encompassing reasoning, rendering, and visual feedback components, enabling the model to perform complex reasoning, including self-correction through multi-turn interactions. This process is optimized end-to-end using the GRPO algorithm. Extensive evaluations are conducted on image-to-code generation across two diverse domains: data charts and web interfaces. The RRVF-trained model not only outperforms existing similarly sized open-source MLLMs and supervised fine-tuning baselines but also exhibits superior generalization. Notably, the model outperforms the more advanced MLLM used to generate visual feedback during training. Code is available at https://github.com/L-O-I/RRVF.

Figures

Figures reproduced from arXiv: 2507.20766 by the authors.

Figure 1
Figure 1. Comparison of training paradigms. Conventional [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. RRVF is a training framework that improves visual reasoning ability using only image inputs. Given an input image [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Training curves for reward score and tool usage. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: An illustration of the RRVF training process. The code for a pie chart is iteratively refined based on visual feedback. [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Training examples illustrating the generalizability of the RRVF framework. The framework demonstrates its capability [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: System prompt template for the ‘chart to code‘ task User prompt template for the chart to code task 1 Please generate the python code of this chart. Think first, call ** chart_render_and_compare_tool** if needed, then answer. Format strictly as: < think>...</think> <to…
Figure 7
Figure 7. Figure 7: User prompt template for the ‘chart to code‘ task [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: System prompt template for the ‘web to code‘ task. User prompt template for the web to code task 1 Please generate the HTML code of this webpage, only HTML code, including necessary CSS. The HTML code should be as concise as possible. The HTML code should be as concise…
Figure 9
Figure 9. Figure 9: User prompt template for the ‘web to code‘ task. Evaluation prompt template for the ‘web to code‘ task using GPT-4o as an automated judge 1 Score the similarity between the AI-generated image(Image 2) and the reference image( Image 1). 2 NOTE: Per instructions, missing…
Figure 10
Figure 10. Figure 10: Evaluation prompt template for the ‘web to code‘ task using GPT-4o as an automated judge [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    ChartArena unifies eight chart families across three real-world visual scenarios and two languages under a format-agnostic triple/graph evaluation protocol, revealing clear gaps among 26 MLLMs.

  2. From Pixels to PCells: A Neurosymbolic Approach to Photonic Component Creation

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Multimodal AI agents can convert images of photonic components into executable parametric programs with mean IoU above 0.9, and these programs support cross-stack retargeting and verifier-driven training.

  3. Iterative Visual Thinking and the Self-Correction Mirage in VLM Grounding

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    IVT teaches VLMs iterative spatial self-correction via visual feedback from rendered bounding boxes, improving Acc@0.5 by 2.4pp on referring expression benchmarks using 2400 samples and GRPO.

  4. HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Better

    cs.CV 2026-07 conditional novelty 5.0 of 10

    HunyuanOCR-1.5 makes a lightweight end-to-end OCR VLM the fastest among peers via DFlash speculative decoding while expanding long-tail capabilities through agent-driven data construction.

  5. Imagination Helps Visual Reasoning, But Not Yet in Latent Space

    cs.CL 2026-02 conditional novelty 5.0 of 10

    Intervening on latent 'imagination' tokens in three visual-reasoning models changes almost nothing, while replacing them with explicit text descriptions (CapImagine) improves benchmark scores.

  6. Reinforced Visual Perception with Tools

    cs.CV 2025-09 conditional novelty 5.0 of 10

    ReVPT uses GRPO reinforcement learning with a cold-start SFT phase to make Qwen2.5-VL models call visual tools, improving perception benchmarks over SFT and text-only RL baselines.

Reference graph

Works this paper leans on

60 extracted references · 14 canonical work pages · cited by 6 Pith papers

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Anthropic . 2025. Claude 4 . https://www.anthropic.com/news/claude-4/. Accessed: 2025-07-24

    work page 2025

  4. [4]

    Bai, S.; Chen, K.; Liu, X.; Wang, J.; Ge, W.; Song, S.; Dang, K.; Wang, P.; Wang, S.; Tang, J.; Zhong, H.; Zhu, Y.; Yang, M.; Li, Z.; Wan, J.; Wang, P.; Ding, W.; Fu, Z.; Xu, Y.; Ye, J.; Zhang, X.; Xie, T.; Cheng, Z.; Zhang, H.; Yang, Z.; Xu, H.; and Lin, J. 2025. Qwen2.5-VL Technical Report. arXiv:2502.13923

    arXiv 2025

  5. [5]

    Bai, Y.; Kadavath, S.; Kundu, S.; Askell, A.; Kernion, J.; Jones, A.; Chen, A.; Goldie, A.; Mirhoseini, A.; McKinnon, C.; et al. 2022. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073

    arXiv 2022

  6. [6]

    Beltramelli, T. 2018. pix2code: Generating code from a graphical user interface screenshot. In Proceedings of the ACM SIGCHI symposium on engineering interactive computing systems, 1--6

    work page 2018

  7. [7]

    Chen, L.; Lu, K.; Rajeswaran, A.; Lee, K.; Grover, A.; Laskin, M.; Abbeel, P.; Srinivas, A.; and Mordatch, I. 2021. Decision transformer: Reinforcement learning via sequence modeling. Advances in neural information processing systems, 34: 15084--15097

    2021

  8. [8]

    V.; Levine, S.; and Ma, Y

    Chu, T.; Zhai, Y.; Yang, J.; Tong, S.; Xie, S.; Schuurmans, D.; Le, Q. V.; Levine, S.; and Ma, Y. 2025. SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training. arXiv:2501.17161

    arXiv 2025

Show all 60 references
  1. [9]

    DeepSeek-AI; Guo, D.; Yang, D.; Zhang, H.; Song, J.; Zhang, R.; and et al. 2025. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948

    2025 arXiv

  2. [10]

    Fu, X.; Liu, M.; Yang, Z.; Corring, J.; Lu, Y.; Yang, J.; Roth, D.; Florencio, D.; and Zhang, C. 2025. Refocus: Visual editing as a chain of thought for structured image understanding. arXiv preprint arXiv:2501.05452

    2025 arXiv

  3. [11]

    Google . 2025. Gemini-2-5-model-family . https://blog.google/products/gemini/gemini-2-5-model-family-expands/. Accessed: 2025-07-15

    2025

  4. [12]

    Guo, D.; Wu, F.; Zhu, F.; Leng, F.; Shi, G.; Chen, H.; Fan, H.; Wang, J.; Jiang, J.; and et al. 2025. Seed1.5-VL Technical Report. arXiv:2505.07062

    2025 arXiv

  5. [13]

    Hu, J.; Zhang, Y.; Han, Q.; Jiang, D.; Zhang, X.; and Shum, H.-Y. 2025. Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model. arXiv:2503.24290

    2025 arXiv

  6. [14]

    Huang, W.; Jia, B.; Zhai, Z.; Cao, S.; Ye, Z.; Zhao, F.; Xu, Z.; Hu, Y.; and Lin, S. 2025. Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models. arXiv:2503.06749

    2025 arXiv

  7. [15]

    H.; Gonzalez, J

    Kwon, W.; Li, Z.; Zhuang, S.; Sheng, Y.; Zheng, L.; Yu, C. H.; Gonzalez, J. E.; Zhang, H.; and Stoica, I. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles

    2023

  8. [16]

    Lauren c on, H.; Tronchon, L.; and Sanh, V. 2024. Unlocking the conversion of web screenshots into html code with the websight dataset. arXiv preprint arXiv:2403.09029

    2024 arXiv

  9. [17]

    Li, B.; Zhang, Y.; Guo, D.; Zhang, R.; Li, F.; Zhang, H.; Zhang, K.; Zhang, P.; Li, Y.; Liu, Z.; et al. 2024. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326

    2024 arXiv

  10. [18]

    Liu, H.; Li, C.; Wu, Q.; and Lee, Y. J. 2023. Visual instruction tuning. Advances in neural information processing systems, 36: 34892--34916

    2023

  11. [19]

    Liu, Z.; Sun, Z.; Zang, Y.; Dong, X.; Cao, Y.; Duan, H.; Lin, D.; and Wang, J. 2025 a . Visual-RFT: Visual Reinforcement Fine-Tuning. arXiv:2503.01785

    2025 arXiv

  12. [20]

    Liu, Z.; Zang, Y.; Zou, Y.; Liang, Z.; Dong, X.; Cao, Y.; Duan, H.; Lin, D.; and Wang, J. 2025 b . Visual Agentic Reinforcement Fine-Tuning. arXiv:2505.14246

    2025 arXiv

  13. [21]

    Luo, C.; Shen, Y.; Zhu, Z.; Zheng, Q.; Yu, Z.; and Yao, C. 2024. Layoutllm: Layout instruction tuning with large language models for document understanding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 15630--15640

    2024

  14. [22]

    Madaan, A.; Tandon, N.; Gupta, P.; Hallinan, S.; Gao, L.; Wiegreffe, S.; Alon, U.; Dziri, N.; Prabhumoye, S.; Yang, Y.; et al. 2023. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36: 46534--46594

    2023

  15. [23]

    Mao, S.; Chen, Y.; Cai, P.; Wang, D.; Yan, G.; Yu, Z.; and Shi, B. 2025. Investigating Redundancy in Multimodal Large Language Models with Multiple Vision Encoders. arXiv:2507.03262

    2025

  16. [24]

    OpenAI . 2024 a . gpt-4o-and-more-tools-to-chatgpt-free . https://openai.com/zh-Hans-CN/index/gpt-4o-and-more-tools-to-chatgpt-free/. Accessed: 2025-07-24

    2024

  17. [25]

    OpenAI . 2024 b . introducing-openai-o1-preview . https://openai.com/zh-Hans-CN/index/introducing-openai-o1-preview/. Accessed: 2025-07-24

    2024

  18. [26]

    OpenAI . 2025 a . Introducing-o3-and-o4-mini . https://openai.com/zh-Hans-CN/index/introducing-o3-and-o4-mini/. Accessed: 2025-07-15

    2025

  19. [27]

    OpenAI . 2025 b . Thinking with images . https://openai.com/index/thinking-with-images/. Accessed: 2025-07-15

    2025

  20. [28]

    Ouyang, L.; Wu, J.; Jiang, X.; Almeida, D.; Wainwright, C.; Mishkin, P.; Zhang, C.; Agarwal, S.; Slama, K.; Ray, A.; et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35: 27730--27744

    2022

  21. [29]

    Qi, J.; Ding, M.; Wang, W.; Bai, Y.; Lv, Q.; Hong, W.; Xu, B.; Hou, L.; Li, J.; Dong, Y.; et al. 2024. Cogcom: Train large vision-language models diving into details through chain of manipulations

    2024

  22. [30]

    W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

    Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, 8748--8763. PmLR

    2021

  23. [31]

    Schick, T.; Dwivedi-Yu, J.; Dess \` , R.; Raileanu, R.; Lomeli, M.; Zettlemoyer, L.; Cancedda, N.; and Scialom, T. 2023. Toolformer: Language models can teach themselves to use tools, 2023. arXiv preprint arXiv:2302.04761

    2023 arXiv

  24. [32]

    Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; and Klimov, O. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347

    2017 arXiv

  25. [33]

    Shao, H.; Qian, S.; Xiao, H.; Song, G.; Zong, Z.; Wang, L.; Liu, Y.; and Li, H. 2024 a . Visual cot: Advancing multi-modal language models with a comprehensive dataset and benchmark for chain-of-thought reasoning. Advances in Neural Information Processing Systems, 37: 8612--8642

    2024

  26. [34]

    K.; Wu, Y.; and Guo, D

    Shao, Z.; Wang, P.; Zhu, Q.; Xu, R.; Song, J.; Bi, X.; Zhang, H.; Zhang, M.; Li, Y. K.; Wu, Y.; and Guo, D. 2024 b . DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv:2402.03300

    2024 arXiv

  27. [35]

    Shen, H.; Liu, P.; Li, J.; Fang, C.; Ma, Y.; Liao, J.; Shen, Q.; Zhang, Z.; Zhao, K.; Zhang, Q.; Xu, R.; and Zhao, T. 2025 a . VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model. arXiv:2504.07615

    2025 arXiv

  28. [36]

    Shen, Y.; Luo, C.; Zhu, Z.; Chen, Y.; Zheng, Q.; Yu, Z.; Bu, J.; and Yao, C. 2025 b . Proctag: Process tagging for assessing the efficacy of document instruction data. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, 6851--6859

    2025

  29. [37]

    Su, A.; Wang, H.; Ren, W.; Lin, F.; and Chen, W. 2025 a . Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning. arXiv:2505.15966

    2025 arXiv

  30. [38]

    Su, Z.; Li, L.; Song, M.; Hao, Y.; Yang, Z.; Zhang, J.; Chen, G.; Gu, J.; Li, J.; Qu, X.; et al. 2025 b . Openthinkimg: Learning to think with images via visual tool reinforcement learning. arXiv preprint arXiv:2505.08617

    2025 arXiv

  31. [39]

    Su, Z.; Xia, P.; Guo, H.; Liu, Z.; Ma, Y.; Qu, X.; Liu, J.; Li, Y.; Zeng, K.; Yang, Z.; Li, L.; Cheng, Y.; Ji, H.; He, J.; and Fung, Y. R. 2025 c . Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers. arXiv:2506.23918

    2025 arXiv

  32. [40]

    S.; Barto, A

    Sutton, R. S.; Barto, A. G.; et al. 1998. Reinforcement learning: An introduction, volume 1. MIT press Cambridge

    1998

  33. [41]

    Team, C.; Yue, Z.; Lin, Z.; Song, Y.; Wang, W.; Ren, S.; Gu, S.; Li, S.; Li, P.; Zhao, L.; Li, L.; Bao, K.; Tian, H.; Zhang, H.; Wang, G.; Zhu, D.; Cici; He, C.; Ye, B.; Shen, B.; Zhang, Z.; Jiang, Z.; Zheng, Z.; Song, Z.; Luo, Z.; Yu, Y.; Wang, Y.; Tian, Y.; Tu, Y.; Yan, Y.; ...

    2025 arXiv

  34. [42]

    Team, K.; Du, A.; Gao, B.; Xing, B.; Jiang, C.; Chen, C.; Li, C.; and et al. 2025 b . Kimi k1.5: Scaling Reinforcement Learning with LLMs. arXiv:2501.12599

    2025 arXiv

  35. [43]

    Team, V.; Hong, W.; Yu, W.; Gu, X.; Wang, G.; Gan, G.; Tang, H.; Cheng, J.; Qi, J.; and et al. 2025 c . GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning. arXiv:2507.01006

    2025 arXiv

  36. [44]

    Wang, K.; Pan, J.; Wei, L.; Zhou, A.; Shi, W.; Lu, Z.; Xiao, H.; Yang, Y.; Ren, H.; Zhan, M.; et al. 2025 a . MathCoder-VL: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoning. arXiv preprint arXiv:2505.10557

    2025 arXiv

  37. [45]

    Wang, Q.; Ding, R.; Zeng, Y.; Chen, Z.; Chen, L.; Wang, S.; Xie, P.; Huang, F.; and Zhao, F. 2025 b . VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learning. arXiv:2505.22019

    2025 arXiv

  38. [46]

    Wang, Y.; Wu, S.; Zhang, Y.; Yan, S.; Liu, Z.; Luo, J.; and Fei, H. 2025 c . Multimodal chain-of-thought reasoning: A comprehensive survey. arXiv preprint arXiv:2503.12605

    2025 arXiv

  39. [47]

    Wei, J. 2025. The Asymmetry of Verification, and Verifier's Law . https://www.jasonwei.net/blog/asymmetry-of-verification-and-verifiers-law. Accessed: 2025-07-15

    2025

  40. [48]

    Wu, C.; Ge, Y.; Guo, Q.; Wang, J.; Liang, Z.; Lu, Z.; Shan, Y.; and Luo, P. 2024 a . Plot2code: A comprehensive benchmark for evaluating multi-modal large language models in code generation from scientific plots. arXiv preprint arXiv:2405.07990

    2024 arXiv

  41. [49]

    Wu, C.; Yin, S.; Qi, W.; Wang, X.; Tang, Z.; and Duan, N. 2023. Visual chatgpt: Talking, drawing and editing with visual foundation models. arXiv preprint arXiv:2303.04671

    2023 arXiv

  42. [50]

    Wu, M.; Yang, J.; Jiang, J.; Li, M.; Yan, K.; Yu, H.; Zhang, M.; Zhai, C.; and Nahrstedt, K. 2025. VTool-R1: VLMs Learn to Think with Images via Reinforcement Learning on Multimodal Tool Use. arXiv:2505.19255

    2025

  43. [51]

    Wu, Y.; Wang, Y.; Tang, S.; Wu, W.; He, T.; Ouyang, W.; Torr, P.; and Wu, J. 2024 b . Dettoolchain: A new prompting paradigm to unleash detection ability of mllm. In European Conference on Computer Vision, 164--182. Springer

    2024

  44. [52]

    Xiao, T.; Xu, X.; Huang, Z.; Gao, H.; Liu, Q.; Liu, Q.; and Chen, E. 2025. Advancing Multimodal Reasoning Capabilities of Multimodal Large Language Models via Visual Perception Reward. arXiv:2506.07218

    2025

  45. [53]

    Xu, C.; Wang, Y.; Wei, L.; Sun, L.; and Huang, W. 2025. Improved Iterative Refinement for Chart-to-Code Generation via Structured Instruction. arXiv:2506.14837

    2025

  46. [54]

    Yang, C.; Shi, C.; Liu, Y.; Shui, B.; Wang, J.; Jing, M.; Xu, L.; Zhu, X.; Li, S.; Zhang, Y.; et al. 2024. Chartmimic: Evaluating lmm's cross-modal reasoning capability via chart-to-code generation. arXiv preprint arXiv:2406.09961

    2024 arXiv

  47. [55]

    Yao, S.; Zhao, J.; Yu, D.; Du, N.; Shafran, I.; Narasimhan, K.; and Cao, Y. 2023. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR)

    2023

  48. [56]

    Zhang, Z.; Zhang, A.; Li, M.; Zhao, H.; Karypis, G.; and Smola, A. 2023. Multimodal chain-of-thought reasoning in language models. arXiv preprint arXiv:2302.00923

    2023 arXiv

  49. [57]

    Zhao, X.; Luo, X.; Shi, Q.; Chen, C.; Wang, S.; Liu, Z.; and Sun, M. 2025. Chartcoder: Advancing multimodal large language model for chart-to-code generation. arXiv preprint arXiv:2501.06598

    2025 arXiv

  50. [58]

    Zheng, L.; Chiang, W.-L.; Sheng, Y.; Zhuang, S.; Wu, Z.; Zhuang, Y.; Lin, Z.; Li, Z.; Li, D.; Xing, E.; et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems, 36: 46595--46623

    2023

  51. [59]

    Zheng, Z.; Yang, M.; Hong, J.; Zhao, C.; Xu, G.; Yang, L.; Shen, C.; and Yu, X. 2025. DeepEyes: Incentivizing ``Thinking with Images" via Reinforcement Learning. arXiv:2505.14362

    2025 arXiv

  52. [60]

    Zhu, J.; Wang, W.; Chen, Z.; Liu, Z.; Ye, S.; Gu, L.; Tian, H.; Duan, Y.; Su, W.; Shao, J.; et al. 2025. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479

    2025 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.