REVIEW 4 cited by
InfoBot: Transfer and Exploration via the Information Bottleneck
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
A central challenge in reinforcement learning is discovering effective policies for tasks where rewards are sparsely distributed. We postulate that in the absence of useful reward signals, an effective exploration strategy should seek out {\it decision states}. These states lie at critical junctions in the state space from where the agent can transition to new, potentially unexplored regions. We propose to learn about decision states from prior experience. By training a goal-conditioned policy with an information bottleneck, we can identify decision states by examining where the model actually leverages the goal state. We find that this simple mechanism effectively identifies decision states, even in partially observed settings. In effect, the model learns the sensory cues that correlate with potential subgoals. In new environments, this model can then identify novel subgoals for further exploration, guiding the agent through a sequence of potential decision states and through new regions of the state space.
Forward citations
Cited by 4 Pith papers
-
Geometry of Neural Reinforcement Learning in Continuous State and Action Spaces
For wide two-layer linearized neural policies in deterministic continuous RL, the locally attainable states concentrate on a manifold of dimension at most 2da+1, independent of the state dimension.
-
Learning Temporal Abstractions via Variational Homomorphisms in Option-Induced Abstract MDPs
A variational option-critic algorithm with latent option embeddings and an implicit chain-of-thought cold-start is presented; the central optimality-preservation proof has a gap and some reported benchmark wins are in...
-
Mixture of Balanced Information Bottlenecks for Long-Tailed Visual Recognition
A balanced information bottleneck loss, extended to a mixture over intermediate layers, improves reported accuracy on three long-tailed visual recognition benchmarks.
-
Active Query Selection for Crowd-Based Reinforcement Learning
Extending the Advise algorithm with variational crowd modelling and entropy-based query selection yields faster learning in small tabular RL tasks, especially highly constrained ones.
Discussion (0). Sign in to comment.