Pith. sign in

REVIEW 3 major objections 4 minor 36 references

Reward Models in Deep Reinforcement Learning: A Survey

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that all reward-modeling methods in deep RL can be organized by a three-axis classification: source, mechanism, and learning paradigm.

desk verdict A useful survey with a solid taxonomy, but Table 1's 'human intrinsic' row is a structural error that undercuts the comprehensiveness claim. read the letter →

arxiv 2506.15421 v1 pith:BWR27SCW submitted 2025-06-18 cs.LG cs.AI

Rui Yu , Shenghua Wan , Yucen Wang , Chen-Xiao Gao , Le Gan , Zongzhang Zhang , De-Chuan Zhan This is my paper · ORCID
classification cs.LGcs.AI
keywords rewardmodelingdeepreinforcementlearningdesigninversepreference-basedintrinsicmotivationfromhumanfeedbackfoundationmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Reward models translate a task designer's intentions into the numerical signals an RL agent optimizes, and this survey treats them as a subject in their own right rather than a side topic inside inverse RL or RLHF. It tries to establish that the field can be systematically mapped by three questions: where the reward comes from (human-provided or AI-generated), what drive makes the agent learn (extrinsic or intrinsic motivation), and how the reward is learned from feedback (demonstrations, goals, or preferences). The survey then uses this map to organize established and emerging methods, to review applications in control and generative-model post-training, and to lay out how reward models should be evaluated. A sympathetic reader would take away a structured route into the literature and a shared vocabulary for comparing methods.

What carries the argument

The carrying mechanism is the taxonomy itself: a three-axis grid of source, mechanism, and learning paradigm, drawn as Figure 1 and populated as Table 1. Each surveyed method is assigned to one cell of this grid, so that, for example, preference-based RLHF occupies the human-extrinsic-preference cell while a vision-language reward method like RoboCLIP sits in the AI-extrinsic-goal cell. This grid is what converts a list of techniques into a map, and it is also the paper's main original contribution. The mathematical preference models (Bradley-Terry, Plackett-Luce, regret-based) serve as the concrete machinery inside the preference cell, connecting observed human comparisons to reward parameters.

What would settle it

Take a random sample of roughly one hundred recently published reinforcement-learning papers that introduce or use a reward model, and have two independent coders assign each to a cell of Table 1 using the survey's definitions; if a substantial fraction cannot be placed uniquely, or if the authors' own representative table contains cells that conflict with those definitions, the partition claim fails.

Watch

Extended reading notes

Core claim

The survey's central claim is that reward modeling deserves a standalone systematic review and that its literature can be cleanly organized by three axes. The source axis distinguishes human-provided rewards, ranging from hand-crafted functions to human-in-the-loop supervision, from AI-generated rewards, where LLMs and VLMs supply reward functions or feedback. The mechanism axis separates extrinsic rewards tied to the task objective from intrinsic motivation, such as curiosity, empowerment, and knowledge-driven bonuses. The learning-paradigm axis groups methods by the feedback they consume: demonstrations, goals, or preferences, with preference learning covering Bradley-Terry models, regret-based models, ordinal feedback, and listwise rankings. The paper presents this framework as covering both established methods and emerging foundation-model-based approaches, and supplements it with a table of representative algorithms and a discussion of evaluation via policy performance, distance metrics, and interpretable representations.

Load-bearing premise

The map's usefulness assumes the three-axis scheme—source, mechanism, learning paradigm—covers the reward-modeling literature without leaving out a major family and without letting methods fall into more than one cell.

Editorial extensions

If this is right

  • Practitioners can position any reward-modeling method by asking three questions—source, mechanism, learning paradigm—which makes method selection and comparison more systematic.
  • Reward-model evaluation is presented as inherently multi-perspective: policy performance, distance-based metrics, and interpretable representations each expose different failure modes and are best used together.
  • Foundation-model-based reward generation is classified as a distinct and growing source, so future reviews and research agendas should treat LLM and VLM rewards as a first-class category alongside human feedback.
  • The named future directions—vectorized rewards, interpretable reward models, ethical constraints, and reward foundation models—follow directly from gaps left open by the taxonomy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the survey leaves implicit: if the taxonomy is truly exhaustive, a random sample of recent reinforcement-learning papers should each fit exactly one cell of Table 1, so coding such a sample would validate or challenge the partition.
  • The source axis suggests a comparative research program the survey does not pursue: reward-hacking failures should cluster differently for human-provided rewards (misspecified objectives) than for AI-generated rewards (inherited model biases), and that difference could be measured empirically.
  • The preference-learning section implicitly orders feedback types by information content—pairwise, ordinal, listwise—hinting at a quantifiable trade-off between annotator effort and statistical efficiency that the survey does not formalize.
  • The taxonomy could be extended with a fourth axis, evaluation regime, by pairing each cell with its most appropriate metric from Section 7; this would turn the map into a decision procedure.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This survey reviews reward modeling in deep reinforcement learning. It proposes a taxonomy on three axes — source (human-provided vs. AI-generated), mechanism (extrinsic vs. intrinsic), and learning paradigm (demonstrations, goals, preferences) — and operationalizes this taxonomy in Table 1. It covers background on MDPs and reward design, describes MaxEnt-IRL, AIRL, goal-conditioned rewards, preference models (Bradley-Terry, Plackett-Luce, regret-based), applications (control, generative model post-training, other domains), and evaluation methods (EPIC, DARD, STARC, interpretable representations). The paper's stated contribution is a comprehensive, systematic map of reward-modeling techniques.

Significance. If the taxonomy were sound, the survey would fill a genuine gap: no prior survey treats reward modeling as a standalone topic, and the paper usefully connects RLHF with older IRL and intrinsic-motivation literature. The mathematical exposition is generally reliable: Eq. (3) (MaxEnt-IRL), Eq. (6) (AIRL), Eq. (9) (BT), and Eq. (13) (PL) are standard and correctly stated, and the evaluation section is a helpful resource. However, the central classification — the reason the survey exists — is not currently a faithful partition of the literature: Table 1 places agent-computed intrinsic rewards under "human", and Section 5's learning paradigms cover only human feedback while Table 1 includes AI-generated variants. These issues are load-bearing for the paper's main claim, although they are fixable by restructuring the taxonomy and the table.

major comments (3)
  1. [Table 1 / Section 4.2] The row "human intrinsic" in Table 1 is internally inconsistent. The methods listed there — Pathak et al. 2017, Houthooft et al. 2016, Burda et al. 2018, Tang et al. 2017, Bellemare et al. 2016, Eysenbach et al. 2018, and Mazzaglia et al. 2022 — generate rewards from agent-environment statistics (prediction error, count-based exploration, density estimation, or mutual information), not from human expertise or supervision, and they do not match the Section 3.1 definition of human-provided rewards. Section 4.2 itself describes intrinsic motivation as the agent's innate motivation to explore. Consequently, the source axis in the proposed taxonomy has no faithful category for a major class of intrinsic rewards, which invalidates the claim that the taxonomy is a partition. I recommend adding an algorithmic/agent-generated source category (or redefining "source" to accommodate these methods) and moving the affected entries from the "human" row.
  2. [Section 5 / Table 1] The learning-paradigm axis is defined and developed only for human feedback: Section 5 begins by saying the paradigms learn "from different kinds of human feedback" and then covers demonstrations, goals, and preferences. However, Section 3.2 and Table 1 explicitly include AI-generated rewards, including "AI extrinsic goal" (Sontakke et al. 2023; Fan et al. 2022; Rocamonde et al. 2023) and "AI extrinsic preference" (Bai et al. 2022; Lee et al. 2024; Wang et al. 2024). These entries have no corresponding learning-paradigm discussion, and the two intrinsic rows have "-" in the Feedback column. The third axis therefore does not classify all methods in the survey. Please extend Section 5 to cover AI-generated feedback and intrinsic-reward mechanisms, or restrict the stated scope of the learning-paradigm axis and adjust the paper's claims accordingly.
  3. [Section 1, Section 3.1, Section 8, Table 1] The paper repeatedly claims a "comprehensive" and "systematic" review, but no inclusion criteria, search protocol, or other selection mechanism is given for Table 1, and Section 8 concedes that "the design and variations of reward models still extend beyond the scope of this discussion." Moreover, manual reward engineering, which is discussed as a human-provided source in Section 3.1, appears nowhere in Table 1. This makes the comprehensiveness claim unverifiable and the categorization non-exhaustive with respect to the paper's own text. Please state the scope and inclusion criteria explicitly, or qualify the claims in the abstract and Section 1.
minor comments (4)
  1. [Figure 1 caption] The caption says the framework categorizes by "sources, feedback types, and mechanisms", but Section 1 and Table 1 use "source, mechanism, and learning paradigm"; align the caption with the actual three axes.
  2. [Section 5.2] The phrase "In the below" should be "In the following".
  3. [Throughout] The text contains many citation spacing errors (e.g., "Silveret al." and "Towerset al."); a careful proofreading pass is needed.
  4. [Table 1] Wan et al. 2024 appears in the "human intrinsic" row but is not discussed in Section 4.2; the caption says the table summarizes algorithms mentioned in Sections 3-5, so either add the corresponding discussion or remove the entry.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the survey's taxonomy is an organizational framework, and its self-citations are illustrative entries, not load-bearing evidence.

full rationale

This is a survey paper, not a derivation with fitted parameters or predictions. The central product is a three-axis taxonomy (source, mechanism, learning paradigm) and a table of representative methods. There is no equation in which an output is constructed from an input of the same form, and no fitted quantity is later renamed as a prediction. The paper's self-citations—Wan et al. 2024, Wang et al. 2025, and Xu et al. 2023—appear as entries in Table 1 and as examples in Sections 4 and 5. They are used to illustrate categories, not to justify the taxonomy's validity or completeness. The taxonomy is an organizational choice, and its consistency (e.g., whether 'human intrinsic' is a faithful cell) is a correctness or coverage concern, not a circularity concern. The paper even concedes in Section 8 that reward-model design extends beyond the scope of the discussion, which further indicates the survey does not claim a closed-form derivation that could be circular. Under the stated rules, self-citation is not circularity when the cited works are merely representative, as is the case here. Therefore the appropriate finding is no significant circularity, with a score of 0.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

This survey contributes no fitted parameters and no invented entities. All quantitative structure is imported from prior work: MDP return, MaxEnt-IRL, Bradley-Terry, Plackett-Luce, EPIC, and STARC. The two domain assumptions concern the taxonomy's exhaustiveness and the representativeness of the cited sample; both are asserted rather than demonstrated, and Table 1's human intrinsic row directly weakens the first one.

assumptions (5)
  • standard math MDP formulation with return J(pi) in Eq. (1) is the correct optimization target.
    Used throughout Section 2 as the background for all reward modeling.
  • standard math MaxEnt-IRL trajectory distribution p(tau) proportional to exp(R_theta(tau)) in Eq. (3) is valid for the covered IRL methods.
    Adopted as the objective for demonstration-based reward learning in Section 5.1.
  • standard math Bradley-Terry and Plackett-Luce models are appropriate preference models for reward learning.
    Equations (9) and (13) frame all preference-based reward learning in Section 5.3.
  • domain assumption The source-mechanism-paradigm grid is an exhaustive and mutually exclusive partition of reward-modeling methods.
    Introduced in Section 1 and used to build Table 1; no formal justification or completeness proof is provided, and Table 1's human intrinsic row undermines mutual exclusivity.
  • domain assumption The cited papers in Table 1 are a representative and comprehensive sample of the literature.
    No search or inclusion criteria are reported; Section 8 concedes coverage limits.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reward Models in Deep Reinforcement Learning: A Survey." pith.science (2026). https://pith.science/paper/BWR27SCW

@misc{pith2026250615421,
  author       = {Pith},
  title        = {Pith review of: Reward Models in Deep Reinforcement Learning: A Survey},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BWR27SCW}},
  note         = {Machine review of arXiv:2506.15421}
}
read the original abstract

In reinforcement learning (RL), agents continually interact with the environment and use the feedback to refine their behavior. To guide policy optimization, reward models are introduced as proxies of the desired objectives, such that when the agent maximizes the accumulated reward, it also fulfills the task designer's intentions. Recently, significant attention from both academic and industrial researchers has focused on developing reward models that not only align closely with the true objectives but also facilitate policy optimization. In this survey, we provide a comprehensive review of reward modeling techniques within the deep RL literature. We begin by outlining the background and preliminaries in reward modeling. Next, we present an overview of recent reward modeling approaches, categorizing them based on the source, the mechanism, and the learning paradigm. Building on this understanding, we discuss various applications of these reward modeling techniques and review methods for evaluating reward models. Finally, we conclude by highlighting promising research directions in reward modeling. Altogether, this survey includes both established and emerging methods, filling the vacancy of a systematic review of reward models in current literature.

Figures

Figures reproduced from arXiv: 2506.15421 by the authors.

Figure 1
Figure 1. A framework for reward modeling in RL, categorizing reward models by their sources, feedback types, and mechanisms to provide [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 5 canonical work pages

  1. [4]

    Vision-language models as a source of rewards.arXiv preprint arXiv:2312.09187,

    Kate Baumli, Satinder Baveja, Feryal Behbahani, Harris Chan, Ghe- orghe Comanici, Sebastian Flennerhag, Maxime Gazeau, Kristian Holsheimer, Dan Horgan, Michael Laskin, et al. Vision-language models as a source of rewards.arXiv preprint arXiv:2312.09187,

    arXiv

  2. [7]

    Diversity is all you need: Learning skills without a re- ward function.arXiv preprint arXiv:1802.06070,

    Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine. Diversity is all you need: Learning skills without a re- ward function.arXiv preprint arXiv:1802.06070,

    arXiv

  3. [10]

    Quantifying differences in reward functions.arXiv preprint arXiv:2006.13900,

    Adam Gleave, Michael Dennis, Shane Legg, Stuart Russell, and Jan Leike. Quantifying differences in reward functions.arXiv preprint arXiv:2006.13900,

    arXiv 2006

  4. [13]

    Preprocessing reward functions for interpretability.arXiv preprint arXiv:2203.13553,

    Erik Jenner and Adam Gleave. Preprocessing reward functions for interpretability.arXiv preprint arXiv:2203.13553,

    arXiv

  5. [14]

    Regularized inverse rein- forcement learning.arXiv preprint arXiv:2010.03691,

    Wonseok Jeon, Chen-Yang Su, Paul Barde, Thang Doan, Derek Nowrouzezahrai, and Joelle Pineau. Regularized inverse rein- forcement learning.arXiv preprint arXiv:2010.03691,

    arXiv 2010

  6. [15]

    A survey of reinforcement learning from human feedback.arXiv preprint arXiv:2312.14925, 10,

    Timo Kaufmann, Paul Weng, Viktor Bengs, and Eyke H ¨ullermeier. A survey of reinforcement learning from human feedback.arXiv preprint arXiv:2312.14925, 10,

    arXiv

  7. [16]

    Preference transformer: Modeling human preferences using transformers for rl.arXiv preprint arXiv:2303.00957,

    Changyeon Kim, Jongjin Park, Jinwoo Shin, Honglak Lee, Pieter Abbeel, and Kimin Lee. Preference transformer: Modeling human preferences using transformers for rl.arXiv preprint arXiv:2303.00957,

    arXiv

  8. [18]

    Empowerment: A universal agent-centric measure of control

    Alexander S Klyubin, Daniel Polani, and Chrystopher L Nehaniv. Empowerment: A universal agent-centric measure of control. In 2005 ieee congress on evolutionary computation, volume 1, pages 128–135. IEEE,

    work page 2005

Show all 36 references
  1. [20]

    Goal- conditioned reinforcement learning: Problems and solutions

    Minghuan Liu, Menghui Zhu, and Weinan Zhang. Goal- conditioned reinforcement learning: Problems and solutions. arXiv:2201.08299,

    arXiv

  2. [21]

    Reward modeling with ordinal feedback: Wisdom of the crowd.arXiv preprint arXiv:2411.12843,

    Shang Liu, Yu Pan, Guanting Chen, and Xiaocheng Li. Reward modeling with ordinal feedback: Wisdom of the crowd.arXiv preprint arXiv:2411.12843,

    arXiv

  3. [23]

    Reft: Reasoning with reinforced fine-tuning

    Trung Quoc Luong, Xinbo Zhang, Zhanming Jie, Peng Sun, Xiaoran Jin, and Hang Li. Reft: Reasoning with reinforced fine-tuning. arXiv preprint arXiv:2401.08967,

    arXiv

  4. [24]

    Choreographer: Learning and adapting skills in imagination.arXiv preprint arXiv:2211.13350,

    Pietro Mazzaglia, Tim Verbelen, Bart Dhoedt, Alexandre Lacoste, and Sai Rajeswar. Choreographer: Learning and adapting skills in imagination.arXiv preprint arXiv:2211.13350,

    arXiv

  5. [25]

    Learning to assist humans without inferring re- wards.arXiv preprint arXiv:2411.02623,

    Vivek Myers, Evan Ellis, Sergey Levine, Benjamin Eysenbach, and Anca Dragan. Learning to assist humans without inferring re- wards.arXiv preprint arXiv:2411.02623,

    arXiv

  6. [26]

    Metra: Scalable unsupervised rl with metric-aware abstraction.arXiv preprint arXiv:2310.08887,

    Seohong Park, Oleh Rybkin, and Sergey Levine. Metra: Scalable unsupervised rl with metric-aware abstraction.arXiv preprint arXiv:2310.08887,

    arXiv

  7. [27]

    Vision-language models are zero-shot reward models for reinforcement learning.arXiv preprint arXiv:2310.12921,

    Juan Rocamonde, Victoriano Montesinos, Elvis Nava, Ethan Perez, and David Lindner. Vision-language models are zero-shot reward models for reinforcement learning.arXiv preprint arXiv:2310.12921,

    arXiv

  8. [28]

    Starc: A general frame- work for quantifying differences between reward functions.arXiv preprint arXiv:2309.15257,

    Joar Skalse, Lucy Farnik, Sumeet Ramesh Motwani, Erik Jenner, Adam Gleave, and Alessandro Abate. Starc: A general frame- work for quantifying differences between reward functions.arXiv preprint arXiv:2309.15257,

    arXiv

  9. [29]

    Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288,

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Am- jad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288,

    arXiv

  10. [30]

    Gymnasium: A standard interface for reinforcement learning environments

    Mark Towers, Ariel Kwiatkowski, Jordan Terry, et al. Gymnasium: A standard interface for reinforcement learning environments. arXiv:2407.17032,

    arXiv

  11. [31]

    Hindsight priors for reward learning from human preferences.arXiv preprint arXiv:2404.08828,

    Mudit Verma and Katherine Metcalf. Hindsight priors for reward learning from human preferences.arXiv preprint arXiv:2404.08828,

    arXiv

  12. [33]

    Text2reward: Auto- mated dense reward function generation for reinforcement learn- ing.arXiv preprint arXiv:2309.11489,

    Tianbao Xie, Siheng Zhao, Chen Henry Wu, Yitao Liu, Qian Luo, Victor Zhong, Yanchao Yang, and Tao Yu. Text2reward: Auto- mated dense reward function generation for reinforcement learn- ing.arXiv preprint arXiv:2309.11489,

    arXiv

  13. [34]

    Deepseek-prover-v1

    Huajian Xin, ZZ Ren, Junxiao Song, Zhihong Shao, Wanjia Zhao, Haocheng Wang, Bo Liu, Liyue Zhang, Xuan Lu, Qiushi Du, et al. Deepseek-prover-v1. 5: Harnessing proof assistant feedback for reinforcement learning and monte-carlo tree search.arXiv preprint arXiv:2408.08152,

    arXiv

  14. [35]

    Generative verifiers: Reward modeling as next-token prediction.arXiv preprint arXiv:2408.15240,

    Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. Generative verifiers: Reward modeling as next-token prediction.arXiv preprint arXiv:2408.15240,

    arXiv

  15. [36]

    Maximum entropy inverse reinforcement learning

    Brian D Ziebart, Andrew Maas, J Andrew Bagnell, and Anind K Dey. Maximum entropy inverse reinforcement learning. In Proceedings of the 23rd national conference on Artificial intelligence-Volume 3, pages 1433–1438, 2008

    2008

  16. [1950]

    Dynamical distance learning for semi-supervised and unsupervised skill discovery.arXiv preprint arXiv:1907.08225,

    Kristian Hartikainen, Xinyang Geng, Tuomas Haarnoja, and Sergey Levine. Dynamical distance learning for semi-supervised and unsupervised skill discovery.arXiv preprint arXiv:1907.08225,

    1907 arXiv

  17. [1952]

    Exploration by random network distillation.arXiv preprint arXiv:1810.12894,

    Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov. Exploration by random network distillation.arXiv preprint arXiv:1810.12894,

    arXiv

  18. [2004]

    Improving multimodal interactive agents with reinforcement learning from human feedback.arXiv preprint arXiv:2211.11602,

    Josh Abramson, Arun Ahuja, Federico Carnevale, Petko Georgiev, Alex Goldin, Alden Hung, Jessica Landon, Jirka Lhotka, Timothy Lillicrap, Alistair Muldal, et al. Improving multimodal interactive agents with reinforcement learning from human feedback.arXiv preprint arXiv:2211.11602,

    arXiv

  19. [2005]

    Models of hu- man preference for learning reward functions.arXiv preprint arXiv:2206.02231,

    W Bradley Knox, Stephane Hatgis-Kessell, Serena Booth, Scott Niekum, Peter Stone, and Alessandro Allievi. Models of hu- man preference for learning reward functions.arXiv preprint arXiv:2206.02231,

    arXiv

  20. [2011]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948,

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948,

    arXiv

  21. [2016]

    Learning robust rewards with adversarial inverse reinforcement learning.arXiv preprint arXiv:1710.11248,

    Justin Fu, Katie Luo, and Sergey Levine. Learning robust rewards with adversarial inverse reinforcement learning.arXiv preprint arXiv:1710.11248,

    arXiv

  22. [2017]

    Safe rlhf: Safe re- inforcement learning from human feedback.arXiv preprint arXiv:2310.12773,

    Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang. Safe rlhf: Safe re- inforcement learning from human feedback.arXiv preprint arXiv:2310.12773,

    arXiv

  23. [2020]

    Constitutional ai: Harm- lessness from ai feedback.arXiv preprint arXiv:2212.08073,

    Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harm- lessness from ai feedback.arXiv preprint arXiv:2212.08073,

    arXiv

  24. [2021]

    Never give up: Learning directed exploration strategies.arXiv preprint arXiv:2002.06038,

    Adri`a Puigdom `enech Badia, Pablo Sprechmann, Alex Vitvitskyi, Daniel Guo, Bilal Piot, Steven Kapturowski, Olivier Tieleman, Mart´ın Arjovsky, Alexander Pritzel, Andew Bolt, et al. Never give up: Learning directed exploration strategies.arXiv preprint arXiv:2002.06038,

    2002 arXiv

  25. [2022]

    A connection between generative adversarial networks, inverse reinforcement learning, and energy-based models.arXiv preprint arXiv:1611.03852,

    Chelsea Finn, Paul Christiano, Pieter Abbeel, and Sergey Levine. A connection between generative adversarial networks, inverse reinforcement learning, and energy-based models.arXiv preprint arXiv:1611.03852,

    arXiv

  26. [2023]

    Motif: Intrinsic motivation from artificial intelli- gence feedback.arXiv preprint arXiv:2310.00166,

    Martin Klissarov, Pierluca D’Oro, Shagun Sodhani, Roberta Raileanu, Pierre-Luc Bacon, Pascal Vincent, Amy Zhang, and Mikael Henaff. Motif: Intrinsic motivation from artificial intelli- gence feedback.arXiv preprint arXiv:2310.00166,

    arXiv

  27. [2024]

    Lipo: Listwise preference optimiza- tion through learning-to-rank.arXiv preprint arXiv:2402.01878,

    Tianqi Liu, Zhen Qin, Junru Wu, Jiaming Shen, Misha Khalman, Rishabh Joshi, Yao Zhao, Mohammad Saleh, Simon Baum- gartner, Jialu Liu, et al. Lipo: Listwise preference optimiza- tion through learning-to-rank.arXiv preprint arXiv:2402.01878,

    arXiv

  28. [2025]

    Dynamics-aware comparison of learned reward functions.arXiv preprint arXiv:2201.10081,

    Blake Wulfe, Ashwin Balakrishna, Logan Ellis, Jean Mercat, Rowan McAllister, and Adrien Gaidon. Dynamics-aware comparison of learned reward functions.arXiv preprint arXiv:2201.10081,

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.