REVIEW 8 cited by
Reward-Conditioned Policies
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Reinforcement learning offers the promise of automating the acquisition of complex behavioral skills. However, compared to commonly used and well-understood supervised learning methods, reinforcement learning algorithms can be brittle, difficult to use and tune, and sensitive to seemingly innocuous implementation decisions. In contrast, imitation learning utilizes standard and well-understood supervised learning methods, but requires near-optimal expert data. Can we learn effective policies via supervised learning without demonstrations? The main idea that we explore in this work is that non-expert trajectories collected from sub-optimal policies can be viewed as optimal supervision, not for maximizing the reward, but for matching the reward of the given trajectory. By then conditioning the policy on the numerical value of the reward, we can obtain a policy that generalizes to larger returns. We show how such an approach can be derived as a principled method for policy search, discuss several variants, and compare the method experimentally to a variety of current reinforcement learning methods on standard benchmarks.
Forward citations
Cited by 8 Pith papers
-
Diffusion Guidance Is a Controllable Policy Improvement Operator
Diffusion guidance with a tunable weight is a controllable policy improvement operator, improving offline and goal-conditioned policies beyond the data without retraining and often without a value function.
-
A Provable Approach for End-to-End Safe Reinforcement Learning
PLS combines offline return-conditioned policy training with Gaussian-process safe optimization of target returns to provide high-probability safety throughout deployment.
-
Freeform Preference Learning for Robotic Manipulation
Language-conditioned multi-axis human preferences yield denser rewards and steerable robot policies that outperform sparse and binary-preference baselines by 38 points on long-horizon manipulation.
-
SwanTale: Unified Multi-Speaker Speech and Audio Generation for Instruct and Zero-Shot Tasks
SwanTale unifies instruction-driven and zero-shot speech and audio generation in one 48 kHz model, with a large captioning pipeline, and reports leading scores on several expressiveness and instruction-following benchmarks.
-
Generative Sequential Notification Optimization via Multi-Objective Decision Transformers
A Decision Transformer with quantile-regression return prompts improved notification decisions at LinkedIn, boosting sessions by 0.72% over the deployed CQL baseline in a live A/B test.
-
EBaReT: Expert-guided Bag Reward Transformer for Auto Bidding
EBaReT, a Decision Transformer variant with a PU-learning expert discriminator and bag-level reward redistribution, outperforms offline RL and generative baselines on seven auto-bidding benchmark periods.
-
Behavioral Exploration: Learning to Explore via In-Context Adaptation
A coverage-conditioned behavioral cloning policy adapts in-context to its own history, making robots explore new expert-like behaviors online without online reinforcement learning.
-
How to Provably Improve Return Conditioned Supervised Learning?
R2CSL provably reaches the in-distribution optimal stitched policy by conditioning on the maximum return-to-go per state, improving on standard RCSL without dynamic programming.
Discussion (0). Sign in to comment.