REVIEW 5 cited by
Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Deep reinforcement learning algorithms require large amounts of experience to learn an individual task. While in principle meta-reinforcement learning (meta-RL) algorithms enable agents to learn new skills from small amounts of experience, several major challenges preclude their practicality. Current methods rely heavily on on-policy experience, limiting their sample efficiency. The also lack mechanisms to reason about task uncertainty when adapting to new tasks, limiting their effectiveness in sparse reward problems. In this paper, we address these challenges by developing an off-policy meta-RL algorithm that disentangles task inference and control. In our approach, we perform online probabilistic filtering of latent task variables to infer how to solve a new task from small amounts of experience. This probabilistic interpretation enables posterior sampling for structured and efficient exploration. We demonstrate how to integrate these task variables with off-policy RL algorithms to achieve both meta-training and adaptation efficiency. Our method outperforms prior algorithms in sample efficiency by 20-100X as well as in asymptotic performance on several meta-RL benchmarks.
Forward citations
Cited by 5 Pith papers
-
Meta-Learning with Warped Gradient Descent
WarpGrad meta-learns interleaved warp layers that precondition gradients, delivering consistent accuracy gains in few-shot and multi-shot learning, plus promising results in reinforcement and continual learning.
-
Robot Trains Robot: Automatic Real-World Policy Adaptation and Learning for Humanoids
A force-sensing arm acts as teacher for a small humanoid, enabling 20-minute real-world walking speed adaptation and 15-minute swing-up learning from scratch.
-
Unraveling the Hidden Dynamical Structure in Recurrent Neural Policies
Recurrent neural policies trained on episodic tasks converge to stable cyclic attractors in hidden state, and the geometry of these cycles mirrors behavior structure.
-
Learning to Adapt: Representation-Based Reinforcement Learning for Multi-Task Skill Transfer
RepMT-SAC uses spectral MDP decomposition to build a task-agnostic value-function core plus minimal task adjustment, yielding up to 30% better performance than baselines on quadcopter trajectory tasks with zero-shot i...
-
Meta-Learning and Meta-Reinforcement Learning -- Tracing the Path towards DeepMind's Adaptive Agent
A survey provides a task-based formalization of meta-learning and meta-RL while chronicling algorithms that lead to DeepMind's Adaptive Agent.
Discussion (0). Continue with ORCID to comment.