REVIEW 8 cited by
Training Agents using Upside-Down Reinforcement Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We develop Upside-Down Reinforcement Learning (UDRL), a method for learning to act using only supervised learning techniques. Unlike traditional algorithms, UDRL does not use reward prediction or search for an optimal policy. Instead, it trains agents to follow commands such as "obtain so much total reward in so much time." Many of its general principles are outlined in a companion report; the goal of this paper is to develop a practical learning algorithm and show that this conceptually simple perspective on agent training can produce a range of rewarding behaviors for multiple episodic environments. Experiments show that on some tasks UDRL's performance can be surprisingly competitive with, and even exceed that of some traditional baseline algorithms developed over decades of research. Based on these results, we suggest that alternative approaches to expected reward maximization have an important role to play in training useful autonomous agents.
Forward citations
Cited by 8 Pith papers
-
Success Conditioning as Policy Improvement: The Optimization Problem Solved by Imitating Success
Success conditioning exactly maximizes linearized policy improvement under a chi-squared divergence trust region whose radius is action-influence, with relative improvement, policy change, and action-influence exactly...
-
A Provable Approach for End-to-End Safe Reinforcement Learning
PLS combines offline return-conditioned policy training with Gaussian-process safe optimization of target returns to provide high-probability safety throughout deployment.
-
Equivariant Goal Conditioned Contrastive Reinforcement Learning
Equivariant Contrastive RL imposes C8 rotation symmetry on the critic and actor, improving sample efficiency and goal generalization in simulated manipulation.
-
Behavioral Exploration: Learning to Explore via In-Context Adaptation
A coverage-conditioned behavioral cloning policy adapts in-context to its own history, making robots explore new expert-like behaviors online without online reinforcement learning.
-
How to Provably Improve Return Conditioned Supervised Learning?
R2CSL provably reaches the in-distribution optimal stitched policy by conditioning on the maximum return-to-go per state, improving on standard RCSL without dynamic programming.
-
LEMUR: Learning to Align with Multi-Objective Reinforcement Learning from Preference Feedback
LEMUR jointly learns a separate reward model for each teacher's preferences and uses them to train a population of multi-objective policies, beating baselines that merge feedback into one reward.
-
GeoExplorer: Active Geo-localization with Curiosity-Driven Exploration
A goal-agnostic curiosity reward is claimed to make active geo-localization agents generalize better to unseen targets and environments than distance-based rewards.
-
Reinforcement Learning: From Algorithms To Foundation Models
A dissertation uniting the author's published results: non-exploitable Nash-DQN policies and the FightLadder benchmark for games, plus diffusion/consistency-model world models for RL — a compilation rather than new results.
Discussion (0). Continue with ORCID to comment.