REVIEW 6 cited by
RLPR: Extrapolating RLVR to General Domains without Verifiers
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Reinforcement Learning with Verifiable Rewards (RLVR) demonstrates promising potential in advancing the reasoning capabilities of LLMs. However, its success remains largely confined to mathematical and code domains. This primary limitation stems from the heavy reliance on domain-specific verifiers, which results in prohibitive complexity and limited scalability. To address the challenge, our key observation is that LLM's intrinsic probability of generating a correct free-form answer directly indicates its own evaluation of the reasoning reward (i.e., how well the reasoning process leads to the correct answer). Building on this insight, we propose RLPR, a simple verifier-free framework that extrapolates RLVR to broader general domains. RLPR uses the LLM's own token probability scores for reference answers as the reward signal and maximizes the expected reward during training. We find that addressing the high variance of this noisy probability reward is crucial to make it work, and propose prob-to-reward and stabilizing methods to ensure a precise and stable reward from LLM intrinsic probabilities. Comprehensive experiments in four general-domain benchmarks and three mathematical benchmarks show that RLPR consistently improves reasoning capabilities in both areas for Gemma, Llama, and Qwen based models. Notably, RLPR outperforms concurrent VeriFree by 7.6 points on TheoremQA and 7.5 points on Minerva, and even surpasses strong verifier-model-dependent approaches General-Reasoner by 1.6 average points across seven benchmarks.
Forward citations
Cited by 6 Pith papers
-
Copy Less, Ground More: Overcoming Repetitive Copying in Long-Context Reasoning via Evidence-Aware Reinforcement Learning
GEAR adds a grounding reward for overlap with key evidence and a distractor penalty to the RL accuracy signal, reducing repetitive copying and improving long-context reasoning accuracy by up to 4.6 points over accurac...
-
StructVRM: Aligning Multimodal Reasoning with Structured and Verifiable Reward Models
A model-based verifier that grades sub-question-level correctness supplies fine-grained RL rewards, and training with it yields a VLM that tops several multimodal reasoning benchmarks, including the authors' new STEM-Bench.
-
Can One Domain Help Others? A Data-Centric Study on Multi-Domain Reasoning via Reinforcement Learning
Training a Qwen2.5-7B model with GRPO on math and puzzle data improves both domains, code transfer depends on the starting model, and template or reward mismatches sharply hurt performance.
-
StepX-Edge: An On-Device UI Vision-Language Model via Architecture-Training-Deployment Co-Design
A co-designed 0.9B UI vision-language model tops small-model benchmarks on screen Q&A and Chinese OCR and runs on a Snapdragon 8 Gen5 at 98 tokens/second.
-
Do Ethical AI Principles Matter to Users? A Large-Scale Analysis of User Sentiment and Satisfaction
The abstract claims all seven EU ethical AI dimensions are positively associated with user satisfaction in 100,000+ G2 reviews, but the provided manuscript body is an unrelated RAG paper, so the result could not be verified.
-
An Explainable Machine Learning Framework for Railway Predictive Maintenance using Data Streams from the Metro Operator of Portugal
The submitted document is internally inconsistent: the abstract describes railway predictive maintenance while the body presents an unrelated multimodal reward-model paper, leaving the reported fault-prediction result...
Discussion (0). Sign in to comment.