REVIEW 7 cited by
PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Process-level Reward Models (PRMs) are crucial for complex reasoning and decision-making tasks, where each intermediate step plays an important role in the reasoning process. Since language models are prone to various types of errors during the reasoning process, PRMs are required to possess nuanced capabilities for detecting various implicit error types in real-world scenarios. However, current benchmarks primarily focus on step correctness, failing to evaluate PRMs' performance systematically. To address this gap, we introduce PRMBench, a process-level benchmark specifically designed to assess the fine-grained error detection capabilities of PRMs. PRMBench comprises 6,216 carefully designed problems and 83,456 step-level labels, evaluating models across multiple dimensions, including simplicity, soundness, and sensitivity. In our experiments on 15 models, spanning both open-source PRMs and closed-source large language models prompted as critic models, we uncover significant weaknesses in current PRMs. These findings underscore the challenges inherent in process-level evaluation and highlight key directions for future research. We hope PRMBench can be a robust bench for advancing research on PRM evaluation and development.
Forward citations
Cited by 7 Pith papers
-
MME-SCI: A Comprehensive and Challenging Science Benchmark for Multimodal Large Language Models
A manually curated, multilingual, multimodal science benchmark shows that even top MLLMs struggle, with fine-grained knowledge-point labels revealing specific weaknesses.
-
GM-PRM: A Generative Multimodal Process Reward Model for Multimodal Mathematical Reasoning
A generative multimodal process reward model that produces step-level critiques and corrections improves average math accuracy for six multimodal LLMs by 2.9 to 5.9 points under a refinement-based Best-of-N strategy.
-
Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Models
Non-reasoning LLMs fail to correct their own errors (64.5% blind spot) while correcting identical external errors, and appending 'Wait' cuts the gap by 89.3%.
-
AURA: Affordance-Understanding and Risk-aware Alignment Technique for Large Language Models
AURA uses step-level reward models, self-critique, and safety-aware decoding to reduce affordance-based safety failures in LLM outputs.
-
Mitigating Think-Answer Mismatch in LLM Reasoning Through Noise-Aware Advantage Reweighting
S-GRPO reweights GRPO advantages by an assumed noise level p to down-weight unbalanced groups, claiming improved and more noise-robust math reasoning training.
-
Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation
A taxonomy-driven survey arguing that reward design is the central mechanism shaping reliable LLM reasoning, with maps of reward paradigms, reward-hacking failure modes, and benchmark pitfalls.
-
Step-level Verifier-guided Hybrid Test-Time Scaling for Large Language Models
A step-level verifier-guided hybrid of Best-of-N sampling, Monte Carlo tree search, and conditional self-refinement improves reasoning in small instruction-tuned LLMs, claiming up to 28.6-point gains.
Discussion (0). Sign in to comment.