LEO enables efficient all-goals learning in goal-conditioned RL by jointly predicting for all goals in one network pass, yielding >250x speedup over relabelling and better performance on Craftax.
Hyperbolic Discounting and Learning over Multiple Horizons
4 Pith papers cite this work. Polarity classification is still indexing.
4
Pith papers citing it
abstract
Reinforcement learning (RL) typically defines a discount factor as part of the Markov Decision Process. The discount factor values future rewards by an exponential scheme that leads to theoretical convergence guarantees of the Bellman equation. However, evidence from psychology, economics and neuroscience suggests that humans and animals instead have hyperbolic time-preferences. In this work we revisit the fundamentals of discounting in RL and bridge this disconnect by implementing an RL agent that acts via hyperbolic discounting. We demonstrate that a simple approach approximates hyperbolic discount functions while still using familiar temporal-difference learning techniques in RL. Additionally, and independent of hyperbolic discounting, we make a surprising discovery that simultaneously learning value functions over multiple time-horizons is an effective auxiliary task which often improves over a strong value-based RL agent, Rainbow.
representative citing papers
Causal localization via attribution and patching identifies a temporal preference subgraph in mid-to-upper layers of Qwen3-4B-Instruct-2507, with time-horizon geometry in the residual stream and initial evidence for steering-vector control.
Standard discounting in AGI design assumes time-consistent preferences, but time-inconsistent preferences prevent an AGI from trusting its future self to follow current plans.
citing papers explorer
-
An AGI with Time-Inconsistent Preferences
Standard discounting in AGI design assumes time-consistent preferences, but time-inconsistent preferences prevent an AGI from trusting its future self to follow current plans.