Pith. sign in

REVIEW 27 cited by

VisualPRM: An Effective Process Reward Model for Multimodal Reasoning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2503.10291 v1 pith:YV2S2RHP submitted 2025-03-13 cs.CV cs.CL

Weiyun Wang , Zhangwei Gao , Lianjie Chen , Zhe Chen , Jinguo Zhu , Xiangyu Zhao , Yangzhou Liu , Yue Cao
show 7 more authors
Shenglong Ye Xizhou Zhu Lewei Lu Haodong Duan Yu Qiao Jifeng Dai Wenhai Wang
This is my paper
classification cs.CVcs.CL
keywords multimodalmodelreasoningevaluationmllmsprmsprocessreward
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We introduce VisualPRM, an advanced multimodal Process Reward Model (PRM) with 8B parameters, which improves the reasoning abilities of existing Multimodal Large Language Models (MLLMs) across different model scales and families with Best-of-N (BoN) evaluation strategies. Specifically, our model improves the reasoning performance of three types of MLLMs and four different model scales. Even when applied to the highly capable InternVL2.5-78B, it achieves a 5.9-point improvement across seven multimodal reasoning benchmarks. Experimental results show that our model exhibits superior performance compared to Outcome Reward Models and Self-Consistency during BoN evaluation. To facilitate the training of multimodal PRMs, we construct a multimodal process supervision dataset VisualPRM400K using an automated data pipeline. For the evaluation of multimodal PRMs, we propose VisualProcessBench, a benchmark with human-annotated step-wise correctness labels, to measure the abilities of PRMs to detect erroneous steps in multimodal reasoning tasks. We hope that our work can inspire more future research and contribute to the development of MLLMs. Our model, data, and benchmark are released in https://internvl.github.io/blog/2025-03-13-VisualPRM/.

Discussion (0). Sign in to comment.

Forward citations

Cited by 27 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Evaluating MLLMs with Multimodal Multi-image Reasoning Benchmark

    cs.CV 2025-06 conditional novelty 7.0 of 10

    MMRB is the first benchmark combining multi-image inputs with chain-of-thought reasoning annotations, and its evaluation shows open-source MLLMs trail commercial models while multi-image reward models are unstable.

  2. More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models

    cs.CL 2025-05 conditional novelty 7.0 of 10

    Reasoning models trade visual grounding for language-based inference, and this paper measures that trade-off with a new metric and benchmark.

  3. Correcting What You Cannot See: Credit Assignment for Perception Distillation in Multimodal Reasoners

    cs.AI 2026-07 conditional novelty 6.0 of 10

    Perception-Correction Distillation up-weights teacher supervision on perception spans only when low downstream success and high teacher–student disagreement co-occur, lifting 8B→2B and 32B→8B multimodal averages over ...

  4. PAIR: Prefix-Aware Internal Reward Model for Multi-Turn Agent Optimization

    cs.AI 2026-05 unverdicted novelty 6.0 of 10

    PAIR combines a hidden-state probe with an attention correction to deliver robust step-level rewards for GRPO-based optimization of multi-turn LLM agents, achieving high AUROC on contaminated trajectories at low cost.

  5. CoVR-R:Reason-Aware Composed Video Retrieval

    cs.CV 2026-03 conditional novelty 6.0 of 10

    Zero-shot LMM reasoning over edit after-effects (states, phases, camera, tempo) plus a new CoVR-R benchmark yields large recall gains on implicit-effect composed video retrieval without task-specific training.

  6. EvoLMM: Self-Evolving Large Multimodal Models with Continuous Rewards

    cs.CV 2025-11 conditional novelty 6.0 of 10

    A self-evolving multimodal model using continuous self-consistency rewards improves math reasoning by about 2–3% using only raw images, without labels or external reward models.

  7. LLaVA-Critic-R1: Your Critic Model is Secretly a Strong Policy Model

    cs.CV 2025-08 conditional novelty 6.0 of 10

    RL training on preference-labeled critic data transforms a 7B vision-language model into both a stronger critic and a stronger generative policy, improving average benchmark accuracy by 5.7% and enabling self-critique...

  8. GM-PRM: A Generative Multimodal Process Reward Model for Multimodal Mathematical Reasoning

    cs.CL 2025-08 conditional novelty 6.0 of 10

    A generative multimodal process reward model that produces step-level critiques and corrections improves average math accuracy for six multimodal LLMs by 2.9 to 5.9 points under a refinement-based Best-of-N strategy.

  9. VL-Cogito: Progressive Curriculum Reinforcement Learning for Advanced Multimodal Reasoning

    cs.CV 2025-07 conditional novelty 6.0 of 10

    VL-Cogito, trained with progressive curriculum RL, online difficulty weighting, and dynamic length rewards, matches or beats prior reasoning MLLMs on ten multimodal benchmarks.

  10. BMMR: A Large-Scale Bilingual Multimodal Multi-Discipline Reasoning Dataset

    cs.CL 2025-07 conditional novelty 6.0 of 10

    BMMR provides a 110k-question bilingual, multimodal, college-level dataset across 300 subjects where state-of-the-art models score at most about 50%.

  11. MMRefine: Unveiling the Obstacles to Robust Refinement in Multimodal Large Language Models

    cs.CL 2025-06 conditional novelty 6.0 of 10

    MMRefine introduces a six-scenario, six-error-type benchmark for multimodal math refinement, and its evaluation of 17 models shows open-source models largely lag closed ones, with spatial reasoning errors the hardest to fix.

  12. K12Vista: Exploring the Boundaries of MLLMs in K-12 Education

    cs.AI 2025-06 conditional novelty 6.0 of 10

    A new Chinese K-12 multimodal benchmark with 33,660 questions, an 840K process-evaluation dataset, and a fine-tuned step-level evaluator shows current MLLMs solve under 60% of questions and make frequent reasoning errors.

  13. GThinker: Towards General Multimodal Reasoning via Cue-Guided Rethinking

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A cue-tagging and rethinking training recipe lifts a 7B multimodal model to 81.5% on M3CoT, though the gain may reflect training on the same benchmark.

  14. PixelThink: Towards Efficient Chain-of-Pixel Reasoning

    cs.CV 2025-05 conditional novelty 6.0 of 10

    A soft token-budget penalty driven by task difficulty and model confidence roughly halves the reasoning tokens used by segmentation MLLMs while slightly improving mask accuracy on ReasonSeg-derived benchmarks.

  15. UI-Genie: A Self-Improving Approach for Iteratively Boosting MLLM-based Mobile GUI Agents

    cs.CL 2025-05 conditional novelty 6.0 of 10

    UI-Genie uses a specialized reward model and iterative self-improvement to generate synthetic training trajectories, achieving state-of-the-art results for mobile GUI agents on AndroidControl, AndroidLab, and Android Arena.

  16. OCR-Reasoning Benchmark: Unveiling the True Capabilities of MLLMs in Complex Text-Rich Image Reasoning

    cs.LG 2025-05 conditional novelty 6.0 of 10

    OCR-Reasoning, a 1,069-question benchmark with reasoning-chain annotations for text-rich images, finds that no evaluated multimodal model surpasses 50% accuracy.

  17. UniGen: Enhanced Training & Test-Time Strategies for Unified Multimodal Understanding and Generation

    cs.CV 2025-05 conditional novelty 6.0 of 10

    UniGen shows a 1.5B model trained on open data can beat larger systems on image understanding and generation once it verifies its own outputs with chain-of-thought and Best-of-N selection.

  18. Test-Time Scaling for Small VLMs on Multilingual Visual MCQ

    cs.CL 2026-07 accept novelty 5.5 of 10

    On EXAMS-V, token budget and parseability dominate chain count, PRM-guided search, and selectors for small VLMs; the policy model itself yields the largest gain.

  19. PaLMR: Towards Faithful Visual Reasoning via Multimodal Process Alignment

    cs.CV 2026-02 conditional novelty 5.0 of 10

    By gating the reinforcement-learning reward on a binary visual-fidelity score, PaLMR lifts Qwen2.5-VL-7B to 70.9 on HallusionBench and 59.3 on MMMU, surpassing its GRPO baseline.

  20. Physical Plausibility Reasoning via HCM-GRPO: Empowering Compact Model for Superior Performance

    cs.CV 2025-11 conditional novelty 5.0 of 10

    A 2B multimodal model trained with HCM-GRPO, a GRPO variant with partial-credit rewards and hard-case oversampling, outperforms larger models on the authors' private physical-plausibility test set.

  21. Reinforced Visual Perception with Tools

    cs.CV 2025-09 conditional novelty 5.0 of 10

    ReVPT uses GRPO reinforcement learning with a cold-start SFT phase to make Qwen2.5-VL models call visual tools, improving perception benchmarks over SFT and text-only RL baselines.

  22. VRPRM: Process Reward Modeling via Visual Reasoning

    cs.LG 2025-08 unverdicted novelty 5.0 of 10

    VRPRM combines visual reasoning with a two-stage SFT-plus-RL strategy to deliver higher-quality process reward modeling using far less annotated data than prior non-thinking PRMs.

  23. EduFlow: Advancing MLLMs' Problem-Solving Proficiency through Multi-Stage, Multi-Perspective Critique

    cs.AI 2025-07 conditional novelty 5.0 of 10

    A multi-stage framework (data filtering, MCTS-guided trajectory construction, PRM-based reranking) improves Qwen MLLMs' accuracy on K-12 multimodal science benchmarks.

  24. FinLMM-R1: Enhancing Financial Reasoning in LMM through Scalable Data and Reward Design

    cs.CL 2025-06 conditional novelty 5.0 of 10

    A two-stage RL framework with length, image-selection, and adversarial rewards, trained on 89,378 ASP-built financial image-question pairs, improves multimodal reasoning over LMM-R1.

  25. WeThink: Toward General-purpose Vision-Language Reasoning via Reinforcement Learning

    cs.CV 2025-06 conditional novelty 5.0 of 10

    WeThink, a 120K-image QA dataset with AI-generated reasoning paths, combined with hybrid-reward reinforcement learning, improves a 7B vision-language model across 14 benchmarks.

  26. Improving Medical Reasoning with Curriculum-Aware Reinforcement Learning

    cs.AI 2025-05 conditional novelty 4.0 of 10

    A curriculum-based GRPO schedule that first trains on close-ended medical VQA and then on open-ended VQA improves benchmark scores over joint training and vanilla RL, though the open-ended metric is the training objec...

  27. Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Models

    cs.CL 2025-05 conditional novelty 2.0 of 10

    A survey-style position paper claims that reinforcement fine-tuning powers reasoning in multimodal LLMs, summarizing over a hundred recent works and proposing five future research directions.

Pith tools