Pith. sign in

REVIEW 4 major objections 4 minor 17 cited by

ReVPT claims that group-relative reinforcement learning with four visual tools—object detection, zoom-in, edge detection, depth estimation—lets small multimodal models beat supervised fine-tuning and text-only RL on perception-heavy benchma

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

ReVPT uses GRPO reinforcement learning with a cold-start SFT phase to make Qwen2.5-VL models call visual tools, improving perception benchmarks over SFT and text-only RL baselines.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A promising RL-for-visual-tool-use idea undercut by sloppy reporting—the headline numbers don't match the tables and the key SAT result is missing. the 4 major comments →

arxiv 2509.01656 v1 pith:6PYB7QHF submitted 2025-09-01 cs.CV cs.CL

Reinforced Visual Perception with Tools

Zetong Zhou , Dongping Chen , Zixian Ma , Zhihan Hu , Mingyang Fu , Sinan Wang , Yao Wan , Zhou Zhao
show 1 more author
Ranjay Krishna
This is my paper
classification cs.CV cs.CL
keywords reinforcement learningvisual tool useGRPOmultimodal language modelsvisual perceptioncold-start fine-tuningperception benchmarksdepth estimation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that reinforcement learning, not more supervised data, is the missing ingredient for teaching multimodal language models to use visual tools. The proposed two-stage method, ReVPT, first cold-starts the model with a small set of verified tool-use traces, then runs GRPO—group-relative policy optimization—on 20k questions the base model answers incorrectly. The reward is binary and rule-based: format plus final-answer correctness, with no learned reward model. On perception-heavy benchmarks (CV-Bench, BLINK, BLINK-Hard, MMVP, MMStar), ReVPT-3B and ReVPT-7B outperform SFT and text-only RL baselines, including 9.03% and 9.44% gains over the instruct models on CV-Bench. If this is right, a small model can acquire generalizable tool-using perception without expensive, carefully filtered trajectory data.

Core claim

The central claim is that rewarding the model for correct final answers while letting it freely choose among object detection, zoom-in, edge detection, and depth estimation is enough to induce strategic tool use. ReVPT uses a binary rule-based reward: +1 if the response matches the required <think>/<tool_call>/<answer> format and the final answer is correct, −1 otherwise; GRPO converts these scores into group-relative advantages and updates the policy toward better tool-selection strategies. A cold-start supervised phase on GPT-4.1-synthesized traces prevents the tool-call collapse observed when RL starts from scratch. The paper reports state-of-the-art results on perception-heavy benchmarks

What carries the argument

The load-bearing machinery is the GRPO training loop with rule-based rewards over a tool-augmented rollout. In each rollout the model may answer directly or call one of four tools—open-vocabulary object detection, region zoom-in, edge detection, and monocular depth estimation—and then reason over the rendered tool output. The policy is updated by normalizing each rollout's reward against its group, so the model learns which tools help without a learned reward model. Two stabilizers carry the argument: the cold-start SFT phase that teaches the tool-calling format and prevents tool-use collapse, and the error-filtered 20k-question RL set that provides a dense signal of hard perceptual failures

Load-bearing premise

The load-bearing premise is that the 20k questions the base model answers incorrectly are representative of visual perception failures in general, so training on them transfers to held-out benchmarks; if that filtered set merely captures the base model's idiosyncratic blind spots, the reported gains could be artifacts of data selection rather than a general tool-use ability.

What would settle it

Run ReVPT's pipeline unchanged but replace the error-filtered RL set with a random sample of SAT and TACO questions the base model answers correctly, keeping everything else fixed. If CV-Bench and BLINK gains disappear or reverse, the claim that RL on hard, error-filtered perception questions produces general tool-using visual reasoning is falsified; the gains would instead trace to the specific filtered distribution.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • ReVPT-3B and ReVPT-7B outperform their instruct baselines by 9.03% and 9.44% on CV-Bench, and beat SFT and text-only RL baselines across BLINK, BLINK-Hard, MMVP, and MMStar.
  • RL changes tool-selection behavior: after training, the model calls tools less often but with a higher share of correct answers, indicating it is learning when tools help.
  • Object detection and depth estimation are the load-bearing tools: removing object detection costs 5 points on BLINK relation and 12.33 points on MMVP.
  • Cold-start data composition is decisive: combining synthesized tool-use traces with general TACO questions preserves general capabilities while adding tool-use skill.
  • Perception-specialized training brings a small general-capability tradeoff on MMMU and MMBench even as perception benchmarks improve substantially.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Extension: the same pipeline should work with a different tool suite (e.g., segmentation or OCR); because cold-start data fixes the tool repertoire, swapping tools is a direct way to test whether the RL benefit is tool-agnostic.
  • Extension: adding a small penalty for tool calls, or a separate reward for correct tool choice when the final answer is wrong, would test whether the binary reward is the binding constraint; the case studies show distinct failure modes the binary reward cannot tell apart.
  • The paper's discussion implies tool benefit is non-monotonic with model scale: small models gain most, mid-size models gain less, and the largest models may use tools differently. Testing ReVPT on a range of base-model sizes would map where the benefit crosses zero.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces ReVPT, a two-stage training procedure (cold-start SFT followed by GRPO-style RL) that teaches Qwen2.5-VL models to invoke four visual tools (object detection, zoom-in, edge detection, depth estimation) in interleaved reasoning traces. The authors claim state-of-the-art results on perception-heavy benchmarks, including SAT, CV-Bench, BLINK, and MMStar, with ReVPT-3B/7B outperforming instruct models by 9.03%/9.44% on CV-Bench. They also provide ablations on cold-start data construction and tool selection, and release code and data.

Significance. If the reported gains hold, ReVPT would be a useful, low-cost recipe for adding tool-use to small multimodal models; the use of a rule-based binary reward (Section 2.4) is a strength because it avoids learned reward models and reward hacking. The paper is empirical and does not make circular claims: the rewards are based on ground-truth answers, and evaluations are external benchmarks (though the SAT evaluation itself is missing). The main obstacle is that the headline numbers are internally inconsistent and the SAT evidence is absent; until those are fixed, the SOTA claim is not verifiable.

major comments (4)
  1. [Abstract / §3.2 / Table 2] The headline CV-Bench gains are mutually inconsistent. The abstract reports ReVPT-3B and ReVPT-7B outperform instruct models by 9.03% and 9.44% on CV-Bench. Table 2 shows +1.78 for ReVPT-3B (70.43 vs 68.65) and +9.14 for ReVPT-7B (74.11 vs 64.97). Section 3.2 instead states '+8.65% and +9.82%' for CV-Bench, and also cites overall gains of +5.81% and +5.73% while Table 2's AVG delta row shows +6.69 for the 3B model. The same comparison is reported with three different numbers. The central empirical claim is not reproducible from the paper's own tables and must be reconciled.
  2. [§3.1 / Appendix B.1 / Table 2] SAT is named in the abstract and introduction as a key benchmark where ReVPT achieves state-of-the-art performance, but Table 2 has no SAT column. Section 3.1 says eight benchmarks are evaluated, while Appendix B.1 says the evaluation covers 'five benchmarks' including a 1,000-question SAT subset. Moreover, the RL training data are filtered from the SAT training set (Section 2.3 and Appendix A), so a held-out SAT evaluation is essential both to support the abstract's claim and to rule out train/eval overlap. Please add the SAT subset results, specify its provenance, and clarify the benchmark count.
  3. [Section 2.3 / Appendix A] The RL dataset is constructed by retaining only SAT and TACO examples that Qwen2.5-VL-7B-Instruct answers incorrectly. Because this selection is tied to one base model's specific error patterns, the reported transfer to held-out benchmarks could be an artifact of self-selected training data rather than a general improvement in visual tool-use. The paper provides no analysis of the filtered distribution or an ablation with unfiltered or differently filtered data. Please add such an ablation or an error-category analysis to justify the claim that the filtering is representative of general perception failures.
  4. [Section 3.1 / Table 2] The paper states that all results are reported as averages of three runs, but no variance, per-run values, or significance tests are provided anywhere in Table 2 or the text. Several headline differences are small (e.g., ReVPT-3B vs Qwen2.5-VL-3B-Instruct on CV-Bench is +1.78), so without variance information it is impossible to tell whether the claimed improvements are reliable. Please report standard deviations or the three individual run scores.
minor comments (4)
  1. [Figure 12 / Figure 13] Some case-study annotations are internally inconsistent: Figure 12 lists 'Answer: (E)' while the choices only include (A) and (B), and the helper model boxes (A); Figure 13 lists 'Answer: (E)' but the assistant concludes with \(\boxed{C}\). These errors reduce confidence in the qualitative examples and should be corrected.
  2. [Figure 6] The figure or its caption contains garbled text ('/uni00000026/...'), apparently a rendering artifact. Please replace it with a clean figure.
  3. [Table 1 / text] There are several typos and formatting inconsistencies: 'core funtions' in Table 1, inconsistent capitalization of 'ReVPT' (e.g., 'REVPT' in Appendix B.2 and GitHub URL), and malformed LaTeX in the case studies (e.g., '<//think>', 'Let's').
  4. [Section 2.3 / Appendix A] The exact filtering criterion for the cold-start data is stated loosely: Section 2.3 says 'we filter out rollout trajectories generated by the GPT model that contain errors,' while Appendix A says 'filtering out any rollouts that lead to incorrect answers.' Please specify whether this is answer-correctness only or also includes format-based filtering.

Circularity Check

0 steps flagged

No meaningful circularity: ReVPT is an empirical RL/tool-use pipeline whose reported benchmark gains are held-out measurements, not constructions from their own inputs.

full rationale

The paper's load-bearing claim is empirical: GRPO with four visual tools improves performance on external benchmarks. No equation or derivation feeds a quantity back into itself. The reward (Sec. 2.4) is rule-based correctness on training questions (“If the answer match the ground truth, it gets through checking”) and the evaluation benchmarks in Sec. 3.1 / Table 2 are separate from the SAT/TACO RL training data. The training data are filtered by base-model errors (“we filter the SAT dataset and Taco training set with Qwen2.5-VL-7B-Instruct and retain the samples it answers incorrectly”), which is self-referential but not circular: the model is optimized on those errors, then evaluated on held-out CV-Bench/BLINK/MMVP/MMStar/MMMU/MMBench/MathVista numbers. Self-citations (TACO, SAT, BLINK-Hard, VisualSketchPad) are external datasets/methods used as benchmarks or baselines, not as proof of the method's correctness, so they do not raise the circularity score under the stated hard rules. There are, however, serious internal verifiability problems that are correctness issues rather than circularity: the abstract's CV-Bench deltas (9.03% / 9.44%) do not match Table 2 (+1.78 / +9.14); the abstract names SAT as a key benchmark but Table 2 has no SAT column; Sec. 3.1 says “We select 8 multi-modal benchmarks” while Appendix B.1 says “we evaluate five multi-modal benchmarks” plus a 1,000-sample SAT subset; and the SAT evaluation subset's separation from the SAT-derived training filter is not documented. These inconsistencies make the SOTA claim unverifiable from the paper itself, but they are not examples of a conclusion being equivalent to its input by construction.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The paper introduces no new theoretical entities. Its central claim rests on a handful of hand-chosen design decisions: cold-start/RL dataset sizes, the four-tool repertoire, binary reward weights, and the hard-example filtering rule. The GRPO algorithm and tool models (LLMDet, DepthAnything v2) are taken from prior work.

free parameters (4)
  • cold-start dataset size = 1.5k
    Hand-chosen number of GPT-4.1-synthesized tool-use trajectories used for SFT (Section 2.3).
  • RL dataset size = 20k
    Hand-chosen number of filtered SAT/TACO examples used for GRPO training (Section 2.3, Appendix A).
  • GRPO group size G = 8
    Number of sampled responses per prompt in the GRPO objective (Table 6).
  • RL training steps = 200
    Training stops when the reward curve converges (Section 3.1, Figure 8).
axioms (4)
  • standard math GRPO update rule (Eq. 2) with clipping and KL penalty produces stable policy improvement
    The paper relies on the GRPO framework from Shao et al. without proving its convergence properties.
  • domain assumption The four visual tools (object detection, depth estimation, edge detection, zoom-in) return outputs that preserve task-relevant information
    Section 2.2 assumes bounding boxes, depth maps, edge maps, and crops are usable by the model; the paper's own failure cases (Figure 7) show this is not always true.
  • domain assumption Binary reward based on exact answer match and format is a sufficient training signal without reward hacking
    Section 2.4 intentionally avoids learned reward models, assuming the rule-based check is reliable.
  • domain assumption Training on examples the base model gets wrong transfers to held-out benchmarks
    Appendix A filters SAT/TACO to retain only Qwen2.5-VL-7B-Instruct errors, implicitly assuming these hard examples are representative of the evaluation distribution.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Reinforced Visual Perception with Tools." pith.science (2026). https://pith.science/paper/6PYB7QHF

@misc{pith2026250901656,
  author       = {Pith},
  title        = {Pith review of: Reinforced Visual Perception with Tools},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6PYB7QHF}},
  note         = {Machine review of arXiv:2509.01656}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Visual reasoning, a cornerstone of human intelligence, encompasses complex perceptual and logical processes essential for solving diverse visual problems. While advances in computer vision have produced powerful models for various perceptual tasks, leveraging these for general visual reasoning remains challenging. Prior work demonstrates that augmenting LLMs with vision models via supervised finetuning improves performance, but faces key limitations such as expensive data generation, reliance on careful data filtering, and poor generalization. To address these issues, we propose ReVPT to enhance multi-modal LLMs' abilities to reason about and use visual tools through reinforcement learning. We introduce a novel RL algorithm based on GRPO, designed to train models to reason with a suite of four visual tools. Through extensive experiments, we show that our method achieves state-of-the-art performance on several perception-heavy benchmarks, including SAT, CV-Bench, BLINK and MMStar, significantly outperforming the supervised and text-based RL finetuning baselines. Notably, Our ReVPT-3B and ReVPT-7B outperform the instruct models by 9.03% and 9.44% on CV-Bench. Finally, we bring to the community new insights on RL-based visual tool-usage through extensive ablations. Our code is available at https://github.com/ls-kelvin/REVPT.

Figures

Figures reproduced from arXiv: 2509.01656 by Dongping Chen, Mingyang Fu, Ranjay Krishna, Sinan Wang, Yao Wan, Zetong Zhou, Zhihan Hu, Zhou Zhao, Zixian Ma.

Figure 1
Figure 1. Figure 1: Our ReV PT-3B and 7B models outperform their instruct and text-only GRPO counterparts on perception￾centric tasks while maintaining strong general capabilities across multiple benchmarks. Dongping Chen is the project leader. Corresponding author(s): Yao Wan: wanyao@hust.edu.cn, Ranjay Krishna: ranjay@cs.washington.edu, Dongping Chen: dongping@umd.edu. arXiv:2509.01656v1 [cs.CV] 1 Sep 2025 [PITH_FULL_IMAGE… view at source ↗
Figure 2
Figure 2. Figure 2: ReV PT-3B and 7B are able to solve problems through thinking and selectively employing visual tools to reach strong visual perception. We illustrate some step-by-step visual reasoning breakdowns for challenging examples that model leveraging visual tools to solve complex perception tasks and outperform commercial models GPT-4.1. detection, depth estimation, etc. Due to the complexity of visual perception, … view at source ↗
Figure 3
Figure 3. Figure 3: An overall pipeline of our ReV PT. (Top): Model-generated tool requests are managed by a local environment￾based Tool Controller, which independently deploys visual tool services (e.g., Depth, Object Detection). These tools’ outputs are then fed back to the LVLM for iterative reasoning. (Bottom): When processing a visual reasoning problem, ReV PT employs K-turn rollouts where the model interacts with the t… view at source ↗
Figure 4
Figure 4. Figure 4: Reinforced visual tool-usage training requires high-quality and verified data. We transform two datasets into multiple-choice question, then filter out too easy or hard questions for Qwen2.5-VL-7B by multiple inference. 2.3. Cold Start Initially, our objective is to train a multimodal agent using the R1-Zero method. However, during the training process, we observe a progressive decline in the agent’s prope… view at source ↗
Figure 5
Figure 5. Figure 5: ReV PT-3B effectively learns tool utilization after cold-start process. ReV PT-3B shows bias toward object detec￾tion and depth estimation over zoom and edge detection tools due to cold-start data construction. Future work will address data balance and generalized perception objectives. Correct (58%) Incorrect (42%) Correct (68%) Incorrect (32%) 49% 9% 19% 23% Cold-start CVBench RL Correct (47%) Incorrect … view at source ↗
Figure 7
Figure 7. Figure 7: Case studies illustrating various failure modes of ReV PT when using visual tools.Examples include incorrect tool output (top left) , misinterpretation of a correct tool output (top right) , inappropriate tool usage that interferes with perception (bottom left) , and the selection of an unhelpful tool for the given task (bottom right). Tool usage analysis. We analyze the frequency of tool usage by cold-sta… view at source ↗
Figure 8
Figure 8. Figure 8: Our reward gradually upgrade and reach converge. The hyperparameters used for the agent are detailed in [PITH_FULL_IMAGE:figures/full_fig_p020_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Case study for the tool type: Edge Detection. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Case study for the tool type: Zoom In. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Case study for the tool type: Object Detection. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Case study for the tool type: Depth Estimation. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Case study for the error:Model’s Misunderstanding of Tool Results (Object Detection). 25 [PITH_FULL_IMAGE:figures/full_fig_p025_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Case study for the error: Model’s Misunderstanding of Tool Results (Depth Estimation). 26 [PITH_FULL_IMAGE:figures/full_fig_p026_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Case study for the error: Flawed Tool Results (Object Detection). 27 [PITH_FULL_IMAGE:figures/full_fig_p027_15.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 17 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Diversity Over Frequency: Rethinking Tool Use in Visual Chain-of-Thought Agents

    cs.CV 2026-05 unverdicted novelty 7.0

    Visual CoT agents exhibit tool-use collapse where tool usage declines but task accuracy rises, and adding entropy regularization for rollout diversity produces the strongest performance.

  2. DepthAgent: Towards Better Universal Depth Estimation via Sample-wise Expert Selection

    cs.CV 2026-05 unverdicted novelty 7.0

    A reinforcement-learned vision-language agent adaptively selects and fuses monocular depth experts per sample for better performance across camera geometries.

  3. Towards Camera-Robust 3D Localization: Equation-Anchored Tool-Use for MLLMs

    cs.CV 2026-05 unverdicted novelty 7.0

    Proposes an equation-anchored tool-use method for MLLMs that writes the pinhole back-projection equation in Chain-of-Thought and substitutes retrieved camera intrinsics and depths to achieve robustness in 3D object de...

  4. Code-in-the-Loop Forensics: Agentic Tool Use for Image Forgery Detection

    cs.AI 2025-12 unverdicted novelty 7.0

    ForenAgent lets MLLMs create and iteratively improve low-level Python tools for image forgery detection via a two-stage training pipeline and a new 100k-image benchmark dataset.

  5. Beyond the Eye: Efficient Multimodal Reasoning via Self-Regulated Implicit Visual Tools

    cs.CV 2026-07 accept novelty 6.5

    Self-regulated implicit visual tools let MLLMs match or beat external-tool and latent-reasoning methods on high-resolution perception while cutting inference latency by up to 86%.

  6. DeltaV: Thinking with Visual State Updates in Unified Large Multimodal Models

    cs.CV 2026-07 conditional novelty 6.0

    Modeling sparse visual state updates instead of full images cuts generated visual tokens ~55.6% and improves interleaved multimodal reasoning over full-image ULMMs.

  7. Dense Reward for Multi-View 3D Reasoning with Global Maps and Local Views

    cs.CV 2026-06 unverdicted novelty 6.0

    DR-MV3D decomposes MV3D-VQA into global map construction, question-conditioned view planning, and egocentric grounding, supervised by global consistency and local trajectory rewards optimized via GRPO.

  8. VESTA: Visual Exploration with Statistical Tool Agents

    cs.AI 2026-05 unverdicted novelty 6.0

    VESTA introduces dynamic tool creation for VLMs that outperforms static-tool and no-tool baselines on distribution fitting, time series, and astronomy tasks in the new DAWN benchmark.

  9. InterSketch: An Interleaved Reasoning Model with Self-correcting Visual Sketch and Stepwise Reward

    cs.CV 2026-05 unverdicted novelty 6.0

    InterSketch improves long-horizon visual-textual chain-of-thought in VLMs by dynamically generating and interleaving self-correcting visual sketches with text, using a synthesized dataset plus reflection in cold-start...

  10. Leveraging Latent Visual Reasoning in Silence

    cs.CV 2026-05 conditional novelty 6.0

    Latent visual reasoning improves multimodal models via training effects even without using latent tokens at inference, enabled by an attention-based RL reward that promotes interaction with text tokens.

  11. Self-Consistent Latent Reasoning: Long Latent Sequence Reasoning for Vision-Language Model

    cs.CV 2026-05 unverdicted novelty 6.0

    SCOLAR addresses information gain collapse in latent visual reasoning by generating independent auxiliary visual tokens from LLM hidden states, extending acceptable CoT length over 30x and achieving +14.12% gains on b...

  12. Self-Consistent Latent Reasoning: Long Latent Sequence Reasoning for Vision-Language Model

    cs.CV 2026-05 unverdicted novelty 6.0

    SCOLAR fixes information gain collapse in latent visual reasoning by generating independent auxiliary visual tokens via a detransformer, extending acceptable CoT length over 30x and delivering +14.12% gains on reasoni...

  13. Visual Reasoning through Tool-supervised Reinforcement Learning

    cs.CV 2026-04 unverdicted novelty 6.0

    ToolsRL trains MLLMs via a tool-specific then accuracy-focused RL curriculum to master visual tools for complex reasoning tasks.

  14. Don't Show Pixels, Show Cues: Unlocking Visual Tool Reasoning in Language Models via Perception Programs

    cs.CV 2026-04 unverdicted novelty 6.0

    Perception Programs rewrite dense visual tool outputs into language-native summaries, boosting MLLM accuracy by 15-45% absolute on BLINK perception tasks and setting new state-of-the-art results.

  15. LAST: Leveraging Tools as Hints to Enhance Spatial Reasoning for Multimodal Large Language Models

    cs.CV 2026-04 unverdicted novelty 6.0

    LAST augments MLLMs with a tool-abstraction sandbox and three-stage training to deliver around 20% gains on spatial reasoning tasks, outperforming closed-source models.

  16. OmniView-Space: Reinforcing Spatial Reasoning via Multi-Perspective Spatial Mapping

    cs.CV 2026-07 unverdicted novelty 5.0

    OmniView-Space framework with MPSM, tool-guided reasoning, and distillation achieves SOTA on spatial reasoning benchmarks for MLLMs while reducing external geometry dependencies.

  17. Code-in-the-Loop Forensics: Agentic Tool Use for Image Forgery Detection

    cs.AI 2025-12 conditional novelty 5.0

    ForenAgent, an MLLM that writes and runs Python forensics tools over multiple turns, beats prior image-forgery detectors on its new FABench benchmark and on SIDA-Test.

Reference graph

Works this paper leans on

68 extracted references · 10 canonical work pages · cited by 15 Pith papers

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  4. [4]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  5. [5]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  6. [6]

    Openflamingo: An open-source framework for training large autoregressive vision-language models, 2023

    Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, Jenia Jitsev, Simon Kornblith, Pang Wei Koh, Gabriel Ilharco, Mitchell Wortsman, and Ludwig Schmidt. Openflamingo: An open-source framework for training large autoregressive vision-language models, 2023

    2023

  7. [7]

    Perception tokens enhance visual reasoning in multimodal language models

    Mahtab Bigverdi, Zelun Luo, Cheng-Yu Hsieh, Ethan Shen, Dongping Chen, Linda G Shapiro, and Ranjay Krishna. Perception tokens enhance visual reasoning in multimodal language models. arXiv preprint arXiv:2412.03548, 2024

    Pith/arXiv arXiv 2024

  8. [8]

    Omni3D : A large benchmark and model for 3D object detection in the wild

    Garrick Brazil, Abhinav Kumar, Julian Straub, Nikhila Ravi, Justin Johnson, and Georgia Gkioxari. Omni3D : A large benchmark and model for 3D object detection in the wild. In CVPR, Vancouver, Canada, June 2023. IEEE

    work page 2023

  9. [9]

    Sft or rl? an early investigation into training r1-like reasoning large vision-language models

    Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. Sft or rl? an early investigation into training r1-like reasoning large vision-language models. arXiv preprint arXiv:2504.11468, 2025 a

    Pith/arXiv arXiv 2025

  10. [10]

    Are we on the right way for evaluating large vision-language models? arXiv preprint arXiv:2403.20330, 2024

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. Are we on the right way for evaluating large vision-language models? arXiv preprint arXiv:2403.20330, 2024

    Pith/arXiv arXiv 2024

  11. [11]

    Learning only with images: Visual reinforcement learning with reasoning, rendering, and visual feedback

    Yang Chen, Yufan Shen, Wenxuan Huang, Shen Zhou, Qunshu Lin, Xinyu Cai, Zhi Yu, Botian Shi, and Yu Qiao. Learning only with images: Visual reinforcement learning with reasoning, rendering, and visual feedback. arXiv preprint arXiv:2507.20766, 2025 b

    Pith/arXiv arXiv 2025

  12. [12]

    Visrl: Intention-driven visual perception via reinforced reasoning

    Zhangquan Chen, Xufang Luo, and Dongsheng Li. Visrl: Intention-driven visual perception via reinforced reasoning. arXiv preprint arXiv:2503.07523, 2025 c

    Pith/arXiv arXiv 2025

  13. [13]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, Bin Li, Ping Luo, Tong Lu, Yu Qiao, and Jifeng Dai. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. arXiv preprint arXiv:2312.14238, 2023

    Pith/arXiv arXiv 2023

  14. [14]

    Instructblip: Towards general-purpose vision-language models with instruction tuning

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning. Advances in Neural Information Processing Systems, 36, 2024

    2024

  15. [15]

    Molmo and pixmo: Open weights and open data for state-of-the-art multimodal models

    Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. Molmo and pixmo: Open weights and open data for state-of-the-art multimodal models. arXiv preprint arXiv:2409.17146, 2024

    Pith/arXiv arXiv 2024

  16. [16]

    Video-r1: Reinforcing video reasoning in mllms

    Kaituo Feng, Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang, Tianshuo Peng, Benyou Wang, and Xiangyu Yue. Video-r1: Reinforcing video reasoning in mllms. arXiv preprint arXiv:2503.21776, 2025

    Pith/arXiv arXiv 2025

  17. [17]

    Llmdet: Learning strong open-vocabulary object detectors under the supervision of large language models

    Shenghao Fu, Qize Yang, Qijie Mo, Junkai Yan, Xihan Wei, Jingke Meng, Xiaohua Xie, and Wei-Shi Zheng. Llmdet: Learning strong open-vocabulary object detectors under the supervision of large language models. arXiv preprint arXiv:2501.18954, 2025

    Pith/arXiv arXiv 2025

  18. [18]

    Blink: Multimodal large language models can see but not perceive

    Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive. arXiv preprint arXiv:2404.12390, 2024

    Pith/arXiv arXiv 2024

  19. [19]

    Gemini 2.5 flash

    Google. Gemini 2.5 flash. https://deepmind.google/technologies/gemini/flash/, 2024

    work page 2024

  20. [20]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    Pith/arXiv arXiv 2025

  21. [21]

    Visual programming: Compositional visual reasoning without training

    Tanmay Gupta and Aniruddha Kembhavi. Visual programming: Compositional visual reasoning without training. ArXiv, abs/2211.11559, 2022

    Pith/arXiv arXiv 2022

  22. [22]

    Visual sketchpad: Sketching as a visual chain of thought for multimodal language models, 2024 a

    Yushi Hu, Weijia Shi, Xingyu Fu, Dan Roth, Mari Ostendorf, Luke Zettlemoyer, Noah A Smith, and Ranjay Krishna. Visual sketchpad: Sketching as a visual chain of thought for multimodal language models, 2024 a . URL https://arxiv.org/abs/2406.09403

    Pith/arXiv arXiv 2024

  23. [23]

    Visual program distillation: Distilling tools and programmatic reasoning into vision-language models

    Yushi Hu, Otilia Stretcu, Chun-Ta Lu, Krishnamurthy Viswanathan, Kenji Hata, Enming Luo, Ranjay Krishna, and Ariel Fuxman. Visual program distillation: Distilling tools and programmatic reasoning into vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9590--9601, 2024 b

    work page 2024

  24. [24]

    Vision-r1: Incentivizing reasoning capability in multimodal large language models

    Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-r1: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749, 2025

    Pith/arXiv arXiv 2025

  25. [25]

    Mantis: Interleaved multi-image instruction tuning

    Dongfu Jiang, Xuan He, Huaye Zeng, Con Wei, Max Ku, Qian Liu, and Wenhu Chen. Mantis: Interleaved multi-image instruction tuning. arXiv preprint arXiv:2405.01483, 2024

    Pith/arXiv arXiv 2024

  26. [26]

    Llava-onevision: Easy visual task transfer

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024

    Pith/arXiv arXiv 2024

  27. [27]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In ICML, 2022

    2022

  28. [28]

    BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models. In ICML, 2023

    2023

  29. [29]

    Videochat-r1: Enhancing spatio-temporal perception via reinforcement fine-tuning

    Xinhao Li, Ziang Yan, Desen Meng, Lu Dong, Xiangyu Zeng, Yinan He, Yali Wang, Yu Qiao, Yi Wang, and Limin Wang. Videochat-r1: Enhancing spatio-temporal perception via reinforcement fine-tuning. arXiv preprint arXiv:2504.06958, 2025

    Pith/arXiv arXiv 2025

  30. [30]

    Let's verify step by step

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2023

    2023

  31. [31]

    Belongie, Lubomir D

    Tsung - Yi Lin, Michael Maire, Serge J. Belongie, Lubomir D. Bourdev, Ross B. Girshick, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll \' a r, and C. Lawrence Zitnick. Microsoft COCO: common objects in context. CoRR, abs/1405.0312, 2014. URL http://arxiv.org/abs/1405.0312

    Pith/arXiv arXiv 2014

  32. [32]

    Improved baselines with visual instruction tuning, 2023 a

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning, 2023 a

    2023

  33. [33]

    Visual instruction tuning, 2023 b

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023 b

    2023

  34. [34]

    Llava-next: Improved reasoning, ocr, and world knowledge, January 2024 a

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024 a . URL https://llava-vl.github.io/blog/2024-01-30-llava-next/

    2024

  35. [35]

    Llava-plus: Learning to use tools for creating multimodal agents, 2023 c

    Shilong Liu, Hao Cheng, Haotian Liu, Hao Zhang, Feng Li, Tianhe Ren, Xueyan Zou, Jianwei Yang, Hang Su, Jun Zhu, Lei Zhang, Jianfeng Gao, and Chunyuan Li. Llava-plus: Learning to use tools for creating multimodal agents, 2023 c . URL https://arxiv.org/abs/2311.05437

    Pith/arXiv arXiv 2023

  36. [36]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499, 2023 d

    Pith/arXiv arXiv 2023

  37. [37]

    Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, pages 216--233

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, pages 216--233. Springer, 2024 b

    work page 2024

  38. [38]

    Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In International Conference on Learning Representations (ICLR), 2024

    work page 2024

  39. [39]

    Ui-r1: Enhancing action prediction of gui agents by reinforcement learning

    Zhengxi Lu, Yuxiang Chai, Yaxuan Guo, Xi Yin, Liang Liu, Hao Wang, Guanjing Xiong, and Hongsheng Li. Ui-r1: Enhancing action prediction of gui agents by reinforcement learning. arXiv preprint arXiv:2503.21620, 2025

    Pith/arXiv arXiv 2025

  40. [40]

    Taco: Learning multi-modal action models with synthetic chains-of-thought-and-action

    Zixian Ma, Jianguo Zhang, Zhiwei Liu, Jieyu Zhang, Juntao Tan, Manli Shu, Juan Carlos Niebles, Shelby Heinecke, Huan Wang, Caiming Xiong, et al. Taco: Learning multi-modal action models with synthetic chains-of-thought-and-action. arXiv preprint arXiv:2412.05479, 2024

    arXiv 2024

  41. [41]

    OpenAI. Gpt-4.1. https://openai.com/index/gpt-4-1/, 2025 a . Large language model. Accessed: 2025-05-11

    work page 2025

  42. [42]

    Openai o3-mini, 2025 b

    OpenAI. Openai o3-mini, 2025 b . URL https://openai.com/index/openai-o3-mini/

    work page 2025

  43. [43]

    Cogcom: Train large vision-language models diving into details through chain of manipulations, 2024

    Ji Qi, Ming Ding, Weihan Wang, Yushi Bai, Qingsong Lv, Wenyi Hong, Bin Xu, Lei Hou, Juanzi Li, Yuxiao Dong, and Jie Tang. Cogcom: Train large vision-language models diving into details through chain of manipulations, 2024. URL https://arxiv.org/abs/2402.04236

    Pith/arXiv arXiv 2024

  44. [44]

    Sat: Spatial aptitude training for multimodal language models

    Arijit Ray, Jiafei Duan, Reuben Tan, Dina Bashkirova, Rose Hendrix, Kiana Ehsani, Aniruddha Kembhavi, Bryan A Plummer, Ranjay Krishna, Kuo-Hao Zeng, et al. Sat: Spatial aptitude training for multimodal language models. arXiv preprint arXiv:2412.07755, 3, 2024

    arXiv 2024

  45. [45]

    Visual cot: Unleashing chain-of-thought reasoning in multi-modal language models, 2024 a

    Hao Shao, Shengju Qian, Han Xiao, Guanglu Song, Zhuofan Zong, Letian Wang, Yu Liu, and Hongsheng Li. Visual cot: Unleashing chain-of-thought reasoning in multi-modal language models, 2024 a

    work page 2024

  46. [46]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024 b

    Pith/arXiv arXiv 2024

  47. [47]

    Vlm-r1: A stable and generalizable r1-style large vision-language model

    Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, et al. Vlm-r1: A stable and generalizable r1-style large vision-language model. arXiv preprint arXiv:2504.07615, 2025

    Pith/arXiv arXiv 2025

  48. [48]

    Hybridflow: A flexible and efficient rlhf framework

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256, 2024

    Pith/arXiv arXiv 2024

  49. [49]

    Cambrian-1: A fully open, vision-centric exploration of multimodal llms, 2024 a

    Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai Charitha Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, Austin Wang, Rob Fergus, Yann LeCun, and Saining Xie. Cambrian-1: A fully open, vision-centric exploration of multimodal llms, 2024 a

    work page 2024

  50. [50]

    Eyes wide shut? exploring the visual shortcomings of multimodal llms, 2024 b

    Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal llms, 2024 b

    work page 2024

  51. [51]

    Vgr: Visual grounded reasoning

    Jiacong Wang, Zijian Kang, Haochen Wang, Haiyong Jiang, Jiawen Li, Bohong Wu, Ya Wang, Jiao Ran, Xiao Liang, Chao Feng, et al. Vgr: Visual grounded reasoning. arXiv preprint arXiv:2506.11991, 2025 a

    Pith/arXiv arXiv 2025

  52. [52]

    Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191, 2024 a

    Pith/arXiv arXiv 2024

  53. [53]

    Visualprm: An effective process reward model for multimodal reasoning

    Weiyun Wang, Zhangwei Gao, Lianjie Chen, Zhe Chen, Jinguo Zhu, Xiangyu Zhao, Yangzhou Liu, Yue Cao, Shenglong Ye, Xizhou Zhu, et al. Visualprm: An effective process reward model for multimodal reasoning. arXiv preprint arXiv:2503.10291, 2025 b

    Pith/arXiv arXiv 2025

  54. [54]

    Simple o3: Towards interleaved vision-language reasoning

    Ye Wang, Qianglong Chen, Zejun Li, Siyuan Wang, Shijie Guo, Zhirui Zhang, and Zhongyu Wei. Simple o3: Towards interleaved vision-language reasoning. arXiv preprint arXiv:2508.12109, 2025 c

    Pith/arXiv arXiv 2025

  55. [55]

    A theoretical understanding of self-correction through in-context alignment

    Yifei Wang, Yuyang Wu, Zeming Wei, Stefanie Jegelka, and Yisen Wang. A theoretical understanding of self-correction through in-context alignment. arXiv preprint arXiv:2405.18634, 2024 b

    Pith/arXiv arXiv 2024

  56. [56]

    Llava-cot: Let vision language models reason step-by-step, 2025

    Guowei Xu, Peng Jin, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step, 2025. URL https://arxiv.org/abs/2411.10440

    Pith/arXiv arXiv 2025

  57. [57]

    Depth anything v2

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. arXiv:2406.09414, 2024

    Pith/arXiv arXiv 2024

  58. [58]

    Dapo: An open-source llm reinforcement learning system at scale

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476, 2025

    Pith/arXiv arXiv 2025

  59. [59]

    Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for exp...

    2024

  60. [60]

    Vision-r1: Evolving human-free alignment in large vision-language models via vision-guided reinforcement learning

    Yufei Zhan, Yousong Zhu, Shurong Zheng, Hongyin Zhao, Fan Yang, Ming Tang, and Jinqiao Wang. Vision-r1: Evolving human-free alignment in large vision-language models via vision-guided reinforcement learning. arXiv preprint arXiv:2503.18013, 2025

    Pith/arXiv arXiv 2025

  61. [61]

    Nemotron-research-tool-n1: Tool-using language models with reinforced reasoning

    Shaokun Zhang, Yi Dong, Jieyu Zhang, Jan Kautz, Bryan Catanzaro, Andrew Tao, Qingyun Wu, Zhiding Yu, and Guilin Liu. Nemotron-research-tool-n1: Tool-using language models with reinforced reasoning. arXiv preprint arXiv:2505.00024, 2025 a

    Pith/arXiv arXiv 2025

  62. [62]

    Thyme: Think beyond images

    Yi-Fan Zhang, Xingyu Lu, Shukang Yin, Chaoyou Fu, Wei Chen, Xiao Hu, Bin Wen, Kaiyu Jiang, Changyi Liu, Tianke Zhang, et al. Thyme: Think beyond images. arXiv preprint arXiv:2508.11630, 2025 b

    Pith/arXiv arXiv 2025

  63. [63]

    Recognize anything: A strong image tagging model

    Youcai Zhang, Xinyu Huang, Jinyu Ma, Zhaoyang Li, Zhaochuan Luo, Yanchun Xie, Yuzhuo Qin, Tong Luo, Yaqian Li, Shilong Liu, et al. Recognize anything: A strong image tagging model. arXiv preprint arXiv:2306.03514, 2023

    Pith/arXiv arXiv 2023

  64. [64]

    Pyvision: Agentic vision with dynamic tooling

    Shitian Zhao, Haoquan Zhang, Shaoheng Lin, Ming Li, Qilong Wu, Kaipeng Zhang, and Chen Wei. Pyvision: Agentic vision with dynamic tooling. arXiv preprint arXiv:2507.07998, 2025

    Pith/arXiv arXiv 2025

  65. [65]

    Llamafactory: Unified efficient fine-tuning of 100+ language models

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), Bangkok, Thailand, 2024. Association for Computational Linguist...

    Pith/arXiv arXiv 2024

  66. [66]

    thinking with images

    Ziwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang, Chao Shen, and Xing Yu. Deepeyes: Incentivizing" thinking with images" via reinforcement learning. arXiv preprint arXiv:2505.14362, 2025

    Pith/arXiv arXiv 2025

  67. [67]

    Scene parsing through ade20k dataset

    Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ade20k dataset. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017

    2017

  68. [68]

    aha moment

    Hengguang Zhou, Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsieh. R1-zero's "aha moment" in visual reasoning on a 2b non-sft model, 2025. URL https://arxiv.org/abs/2503.05132

    Pith/arXiv arXiv 2025

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.