Pith. sign in

REVIEW 8 cited by

R-PRM: Reasoning-Driven Process Reward Modeling

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2503.21295 v1 pith:DLM7TVE7 submitted 2025-03-27 cs.CL

Shuaijie She , Junxiao Liu , Yifeng Liu , Jiajun Chen , Xin Huang , Shujian Huang This is my paper · ORCID
classification cs.CL
keywords r-prmreasoningevaluationdatafurtherprocessrewardaccuracy
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large language models (LLMs) inevitably make mistakes when performing step-by-step mathematical reasoning. Process Reward Models (PRMs) have emerged as a promising solution by evaluating each reasoning step. However, existing PRMs typically output evaluation scores directly, limiting both learning efficiency and evaluation accuracy, which is further exacerbated by the scarcity of annotated data. To address these issues, we propose Reasoning-Driven Process Reward Modeling (R-PRM). First, we leverage stronger LLMs to generate seed data from limited annotations, effectively bootstrapping our model's reasoning capabilities and enabling comprehensive step-by-step evaluation. Second, we further enhance performance through preference optimization, without requiring additional annotated data. Third, we introduce inference-time scaling to fully harness the model's reasoning potential. Extensive experiments demonstrate R-PRM's effectiveness: on ProcessBench and PRMBench, it surpasses strong baselines by 11.9 and 8.5 points in F1 scores, respectively. When applied to guide mathematical reasoning, R-PRM achieves consistent accuracy improvements of over 8.5 points across six challenging datasets. Further analysis reveals that R-PRM exhibits more comprehensive evaluation and stronger generalization capabilities, thereby highlighting its significant potential.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. StructVRM: Aligning Multimodal Reasoning with Structured and Verifiable Reward Models

    cs.AI 2025-08 conditional novelty 6.0 of 10

    A model-based verifier that grades sub-question-level correctness supplies fine-grained RL rewards, and training with it yields a VLM that tops several multimodal reasoning benchmarks, including the authors' new STEM-Bench.

  2. GM-PRM: A Generative Multimodal Process Reward Model for Multimodal Mathematical Reasoning

    cs.CL 2025-08 conditional novelty 6.0 of 10

    A generative multimodal process reward model that produces step-level critiques and corrections improves average math accuracy for six multimodal LLMs by 2.9 to 5.9 points under a refinement-based Best-of-N strategy.

  3. Error Typing for Smarter Rewards: Improving Process Reward Models with Error-Aware Hierarchical Supervision

    cs.CL 2025-05 conditional novelty 6.0 of 10

    A hierarchical process reward model that classifies math and consistency errors before scoring steps achieves the best PRMBench result among discriminative PRMs and improves reward-guided math search.

  4. AURA: Affordance-Understanding and Risk-aware Alignment Technique for Large Language Models

    cs.CL 2025-08 unverdicted novelty 5.0 of 10

    AURA uses step-level reward models, self-critique, and safety-aware decoding to reduce affordance-based safety failures in LLM outputs.

  5. PATS: Process-Level Adaptive Thinking Mode Switching

    cs.CL 2025-05 conditional novelty 5.0 of 10

    PATS adapts the number of beam-search candidates per reasoning step using process reward model scores, improving accuracy-efficiency tradeoffs on math benchmarks.

  6. Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation

    cs.LG 2026-02 conditional novelty 4.0 of 10

    A taxonomy-driven survey arguing that reward design is the central mechanism shaping reliable LLM reasoning, with maps of reward paradigms, reward-hacking failure modes, and benchmark pitfalls.

  7. HERMES: Towards Efficient and Verifiable Mathematical Reasoning in LLMs

    cs.AI 2025-11 conditional novelty 4.0 of 10

    Interleaving LLM reasoning with Lean4 verification of individual steps improves math-problem accuracy and cuts token budgets versus reward-model methods.

  8. An Explainable Machine Learning Framework for Railway Predictive Maintenance using Data Streams from the Metro Operator of Portugal

    cs.AI 2025-08 unverdicted novelty 4.0 of 10

    The submitted document is internally inconsistent: the abstract describes railway predictive maintenance while the body presents an unrelated multimodal reward-model paper, leaving the reported fault-prediction result...

Pith tools