REVIEW 5 cited by
RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environments
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Exploration in sparse reward environments remains one of the key challenges of model-free reinforcement learning. Instead of solely relying on extrinsic rewards provided by the environment, many state-of-the-art methods use intrinsic rewards to encourage exploration. However, we show that existing methods fall short in procedurally-generated environments where an agent is unlikely to visit a state more than once. We propose a novel type of intrinsic reward which encourages the agent to take actions that lead to significant changes in its learned state representation. We evaluate our method on multiple challenging procedurally-generated tasks in MiniGrid, as well as on tasks with high-dimensional observations used in prior work. Our experiments demonstrate that this approach is more sample efficient than existing exploration methods, particularly for procedurally-generated MiniGrid environments. Furthermore, we analyze the learned behavior as well as the intrinsic reward received by our agent. In contrast to previous approaches, our intrinsic reward does not diminish during the course of training and it rewards the agent substantially more for interacting with objects that it can control.
Forward citations
Cited by 5 Pith papers
-
Beyond Single-Model Optimization: Preserving Plasticity in Continual Reinforcement Learning
TeLAPA preserves behaviorally diverse policy neighborhoods in a shared latent space, improving MiniGrid continual RL transfer, revisit recovery, and retention over single-model preservation.
-
Progress-conditioned Group Policy Optimization for Long-Horizon Agentic Tasks
ProGPO adds a first-visit observation-coverage advantage only when an entire rollout group fails, improving group-based RL for long-horizon LLM agents on ALFWorld and WebShop.
-
ManiTaskGen: A Comprehensive Task Generator for Benchmarking and Improving Vision-Language Agents on Embodied Decision-Making
ManiTaskGen automatically generates diverse, feasible mobile manipulation tasks from any input scene, and uses them to benchmark and improve vision-language robot agents.
-
Episodic Novelty Through Temporal Distance
An episodic intrinsic reward based on a contrastively learned temporal distance quasimetric improves exploration in sparse-reward Contextual MDPs.
-
The impact of intrinsic rewards on exploration in Reinforcement Learning
An empirical MiniGrid study shows state-counting is best for low-dimensional observations, maximum entropy is more robust with images, and DIAYN skill learning does not aid exploration.
Discussion (0). Continue with ORCID to comment.