REVIEW 12 cited by
PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Process-level Reward Models (PRMs) are crucial for complex reasoning and decision-making tasks, where each intermediate step plays an important role in the reasoning process. Since language models are prone to various types of errors during the reasoning process, PRMs are required to possess nuanced capabilities for detecting various implicit error types in real-world scenarios. However, current benchmarks primarily focus on step correctness, failing to evaluate PRMs' performance systematically. To address this gap, we introduce PRMBench, a process-level benchmark specifically designed to assess the fine-grained error detection capabilities of PRMs. PRMBench comprises 6,216 carefully designed problems and 83,456 step-level labels, evaluating models across multiple dimensions, including simplicity, soundness, and sensitivity. In our experiments on 15 models, spanning both open-source PRMs and closed-source large language models prompted as critic models, we uncover significant weaknesses in current PRMs. These findings underscore the challenges inherent in process-level evaluation and highlight key directions for future research. We hope PRMBench can be a robust bench for advancing research on PRM evaluation and development.
Forward citations
Cited by 12 Pith papers
-
MME-SCI: A Comprehensive and Challenging Science Benchmark for Multimodal Large Language Models
A manually curated, multilingual, multimodal science benchmark shows that even top MLLMs struggle, with fine-grained knowledge-point labels revealing specific weaknesses.
-
GM-PRM: A Generative Multimodal Process Reward Model for Multimodal Mathematical Reasoning
A generative multimodal process reward model that produces step-level critiques and corrections improves average math accuracy for six multimodal LLMs by 2.9 to 5.9 points under a refinement-based Best-of-N strategy.
-
Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Models
Non-reasoning LLMs fail to correct their own errors (64.5% blind spot) while correcting identical external errors, and appending 'Wait' cuts the gap by 89.3%.
-
PixelThink: Towards Efficient Chain-of-Pixel Reasoning
A soft token-budget penalty driven by task difficulty and model confidence roughly halves the reasoning tokens used by segmentation MLLMs while slightly improving mask accuracy on ReasonSeg-derived benchmarks.
-
Error Typing for Smarter Rewards: Improving Process Reward Models with Error-Aware Hierarchical Supervision
A hierarchical process reward model that classifies math and consistency errors before scoring steps achieves the best PRMBench result among discriminative PRMs and improves reward-guided math search.
-
System-1.5 Reasoning: Traversal in Language and Latent Spaces with Dynamic Shortcuts
System-1.5 Reasoning lets LLMs reason in latent space with early exits and step-skipping, matching chain-of-thought accuracy at over 20x speedup on GSM8K and StrategyQA.
-
AURA: Affordance-Understanding and Risk-aware Alignment Technique for Large Language Models
AURA uses step-level reward models, self-critique, and safety-aware decoding to reduce affordance-based safety failures in LLM outputs.
-
Mitigating Think-Answer Mismatch in LLM Reasoning Through Noise-Aware Advantage Reweighting
S-GRPO reweights GRPO advantages by an assumed noise level p to down-weight unbalanced groups, claiming improved and more noise-robust math reasoning training.
-
Speculative Reward Model Boosts Decision Making Ability of LLMs Cost-Effectively
A plug-and-play external reward model with speculative rejection sampling cuts tree-search cost for LLM decision-making to about 1/10 while keeping or slightly improving accuracy on math, planning, and financial reaso...
-
Step-Wise Formal Verification for LLM-Based Mathematical Problem Solving
MATH-VF formalizes LLM math solutions into SimpleMath and uses a tool-augmented critic to verify each reasoning step and offer corrective feedback.
-
Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation
A taxonomy-driven survey arguing that reward design is the central mechanism shaping reliable LLM reasoning, with maps of reward paradigms, reward-hacking failure modes, and benchmark pitfalls.
-
Step-level Verifier-guided Hybrid Test-Time Scaling for Large Language Models
A step-level verifier-guided hybrid of Best-of-N sampling, Monte Carlo tree search, and conditional self-refinement improves reasoning in small instruction-tuned LLMs, claiming up to 28.6-point gains.
Discussion (0). Sign in to comment.