Pith. sign in

REVIEW 10 cited by

A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Future

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2504.12328 v1 pith:A5IOGR2W submitted 2025-04-12 cs.CL cs.AI

Jialun Zhong , Wei Shen , Yanzeng Li , Songyang Gao , Hua Lu , Yicheng Chen , Yang Zhang , Wei Zhou
show 2 more authors
Jinjie Gu Lei Zou
This is my paper
classification cs.CLcs.AI
keywords comprehensiverewardapplicationschallengesfuturegithubmodelspotential
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Reward Model (RM) has demonstrated impressive potential for enhancing Large Language Models (LLM), as RM can serve as a proxy for human preferences, providing signals to guide LLMs' behavior in various tasks. In this paper, we provide a comprehensive overview of relevant research, exploring RMs from the perspectives of preference collection, reward modeling, and usage. Next, we introduce the applications of RMs and discuss the benchmarks for evaluation. Furthermore, we conduct an in-depth analysis of the challenges existing in the field and dive into the potential research directions. This paper is dedicated to providing beginners with a comprehensive introduction to RMs and facilitating future studies. The resources are publicly available at github\footnote{https://github.com/JLZhong23/awesome-reward-models}.

Discussion (0). Sign in to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Agents Don't Just Agree, They Remember: Benchmarking Persistent Sycophancy in Stateful Personal Agents

    cs.AI 2026-07 conditional novelty 8.0 of 10

    Stateful personal agents convert conversational sycophancy into durable memory commits that raise later failure rates by about 27 points once claims are written to state.

  2. Toward Robust LLM-Based Judges: Taxonomic Bias Evaluation and Debiasing Optimization

    cs.CL 2026-03 conditional novelty 6.0 of 10

    JudgeBiasBench measures 12 LLM-judge bias types under a 4-dimension taxonomy, and bias-aware GRPO/InfoNCE training cuts BSR substantially while preserving general evaluation performance.

  3. GM-PRM: A Generative Multimodal Process Reward Model for Multimodal Mathematical Reasoning

    cs.CL 2025-08 conditional novelty 6.0 of 10

    A generative multimodal process reward model that produces step-level critiques and corrections improves average math accuracy for six multimodal LLMs by 2.9 to 5.9 points under a refinement-based Best-of-N strategy.

  4. The Other Mind: How Language Models Exhibit Human Temporal Cognition

    cs.AI 2025-07 conditional novelty 6.0 of 10

    Larger LLMs develop a subjective 'present' around the current date, and their year similarity judgments follow a logarithmic Weber-Fechner compression, with supporting neural and representational evidence.

  5. Uncertainty-aware Reward Design Process

    cs.LG 2025-07 conditional novelty 6.0 of 10

    URDP couples LLM-based reward component design with uncertainty-weighted Bayesian optimization, reporting better reward quality and efficiency than Eureka and Text2Reward on three benchmarks.

  6. RAG-Zeval: Towards Robust and Interpretable Evaluation on RAG Responses through End-to-End Rule-Guided Reasoning

    cs.CL 2025-05 conditional novelty 6.0 of 10

    RAG-Zeval uses rule-guided RL with ranking rewards on synthetic responses to train a 7B model that evaluates RAG faithfulness and correctness competitively with 70B-class judges.

  7. Guided by Gut: Efficient Test-Time Scaling with Reinforced Intrinsic Confidence

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Guided by Gut lets an LLM guide its own search using token-level confidence and step novelty, with a RL fine-tuning step to calibrate that confidence, achieving PRM-level accuracy at far lower compute and memory.

  8. Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation

    cs.LG 2026-02 conditional novelty 4.0 of 10

    A taxonomy-driven survey arguing that reward design is the central mechanism shaping reliable LLM reasoning, with maps of reward paradigms, reward-hacking failure modes, and benchmark pitfalls.

  9. Reward Reasoning Model

    cs.CL 2025-05 conditional novelty 4.0 of 10

    Reward models that think before judging, trained via reinforcement learning without human-written reasoning traces, outperform standard reward models and improve with more test-time compute.

  10. Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle

    cs.CL 2025-09 conditional novelty 3.0 of 10

    A survey that maps reinforcement learning methods, datasets, benchmarks, and open-source tools across the full training lifecycle of large language models, focusing on verifiable-reward reasoning.

Pith tools