Pith. sign in

REVIEW 4 major objections 5 minor 61 references

From Sparse to Dense: Toddler-inspired Reward Transition in Goal-Oriented Reinforcement Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Sparse-to-dense reward schedules make RL agents learn faster, generalize better, and settle in wider minima.

desk verdict A broad empirical study of sparse-to-dense reward shaping whose loss-landscape headline is undercut by a mixed-replay confound and missing significance tests. read the letter →

arxiv 2501.17842 v1 pith:ECO7JWOU submitted 2025-01-29 cs.LG cs.AIcs.RO

Junseok Park , Hyeonseo Yang , Min Whoo Lee , Won-Seok Choi , Minsu Lee , Byoung-Tak Zhang This is my paper · ORCID
classification cs.LGcs.AIcs.RO
keywords reinforcementlearningrewardshapingsparserewardscurriculumpotential-basedlosslandscapewideminimageneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that reinforcement learning agents, like toddlers, learn best when they start with sparse rewards that force free exploration and then switch to dense, goal-shaped rewards. Across robotic arm, navigation, and maze tasks, the authors show that this Sparse-to-Dense (S2D) transition beats using only sparse, only dense, or reverse-order rewards on success rate and sample efficiency. They also provide evidence that the transition smooths the policy loss landscape, guiding the agent into wider, more generalizable minima. If correct, reward scheduling itself becomes a cheap way to improve RL robustness without changing the algorithm or the environment.

What carries the argument

The load-bearing mechanism is the S2D curriculum built on potential-based reward shaping: the agent trains first under a sparse reward, then switches to a denser reward $R_i + F_i$ where $F_i(s,a) = \gamma \Phi_i(s') - \Phi_i(s)$ with a distance-to-goal potential $\Phi(s) = \operatorname{diam}_p(S) - \|s - g\|_p$. PBRS guarantees the dense reward preserves the optimal policy, so the density change can only affect the optimization trajectory, not the task definition. This is what lets the authors attribute the gains to the transition itself. The analysis machinery is the Cross-Density Visualizer, a 3D plot of policy loss over randomly perturbed parameters, and the sharpness metric $\max_{\|\epsilon\|_p \le \rho} [L_\pi(\theta+\epsilon) - L_\pi(\theta)]$, used to measure the width of the final minima.

What would settle it

Train the same S2D schedule but relabel every replay-buffer transition with the current stage's reward before each update, or clear the buffer at the transition point. If the S2D advantage over Only Dense shrinks or disappears under relabeling, the mechanism is the stale mixed rewards, not the sparse exploration phase. A second check: if S2D works by forming a cognitive map, its advantage should be measurable as better transfer when both the goal and spawn are randomized at evaluation, and should shrink when goal positions are fixed and visible.

Watch

Extended reading notes

Core claim

The central claim is that transitioning a goal-oriented RL agent from a sparse reward to a potential-based dense reward over the course of training—rather than fixing one reward density—improves learning performance, sample efficiency, and generalization. The authors define an S2D curriculum as a sequence of MDPs whose reward supports grow monotonically while the set of optimal policies shrinks monotonically, and they use potential-based reward shaping to preserve optimality. Empirically, across LunarLander, CartPole-Reacher, UR5-Reacher, ViZDoom, and Minecraft maze tasks, S2D outperforms Only Sparse, Only Dense, and Dense-to-Sparse baselines. Using a Cross-Density Visualizer of the policy loss landscape and a sharpness metric, they report that S2D smooths the loss landscape and converges to wider minima than other reward strategies. They reinterpret Tolman's latent-learning maze experiments, arguing that early sparse-reward free exploration builds robust initial parameters and faster-converging internal representations that later dense-reward learning exploits.

Load-bearing premise

That reheating the replay buffer with rewards collected under two different reward functions—mostly-zero sparse rewards early, dense shaped rewards later—still gives the optimizer a consistent learning signal, so the S2D advantage comes from the curriculum rather than from accidentally mixing reward scales.

Editorial extensions

If this is right

  • S2D is a broadly applicable, algorithm-agnostic reward schedule: it improved results across SAC, PPO, DQN, and A3C in the paper's experiments.
  • The optimal transition point is early—roughly the first quarter to third of training—aligning with the toddler critical-period analogy and giving practitioners a simple default rule.
  • When generalization to unseen layouts or textures is the bottleneck, starting with a sparse phase before dense shaping may make the learned policy less brittle.
  • S2D does not require changing the environment, the architecture, or the RL algorithm, so it can be layered on existing goal-conditioned pipelines.
  • The sharper RNN feature-convergence and more stable action frequencies after the transition are consistent with the idea that the sparse phase builds a reusable internal map, not just a better reward signal.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The loss-landscape smoothing is presented as a consequence of the sparse phase, but it could equally be caused by the off-policy replay buffer mixing two reward functions; a controlled comparison against relabeled or fresh-buffer variants would isolate the mechanism.
  • The toddler and Tolman analogies suggest that S2D may be most beneficial when the agent must form a spatial or object-centric model; in tasks where the goal is trivially locatable, the advantage should shrink or vanish—this is testable.
  • Because the supported-region inclusion condition compares only support sets, the paper's definition of 'denser' ignores reward magnitudes; two curricula with the same supports but different shaping strengths could in principle behave very differently.
  • If the sparse phase truly builds robust initial parameters, S2D should transfer as a pretraining schedule: an agent that explored under sparse rewards should fine-tune faster on a new goal distribution than one trained only on dense rewards from the start.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper proposes a Sparse-to-Dense (S2D) reward curriculum for goal-oriented reinforcement learning, inspired by toddler development. The curriculum starts with sparse rewards and transitions to potential-based dense rewards while preserving optimal policies via potential-based reward shaping. The authors evaluate S2D against Only Sparse, Only Dense, Dense-to-Sparse, and intrinsic motivation baselines across seven main environments, including robotic arm control (UR5-Reacher), CartPole-Reacher, LunarLander, ViZDoom navigation, and Minecraft mazes. They report improved sample efficiency and success rates, and use a 'Cross-Density Visualizer' and a sharpness metric to argue that S2D smooths the policy loss landscape and leads to wider minima. They also reinterpret Tolman's maze experiments as evidence for the benefit of early free exploration.

Significance. The empirical scope is a clear strength: the paper evaluates across state-based and visual, discrete and continuous domains, and includes generalization tests in ViZDoom-Unseen and Cross Maze. The idea of a temporal reward-density curriculum with PBRS is well-motivated and practically relevant. If the reported effects are real, the loss-landscape analysis would be a useful tool for understanding reward shaping. However, the central comparison is threatened by a replay-buffer confound in Algorithm 1, and the quantitative support is weakened by large variances and absence of significance tests. The visual loss-landscape evidence is mostly qualitative. The manuscript is a reasonable candidate for publication after substantial revision.

major comments (4)
  1. [§4.1, Algorithm 1 (lines 13–21)] The replay buffer is neither cleared nor relabeled when the curriculum advances, so after the sparse-to-dense transition the agent continues training on stale sparse-reward transitions. The Only Dense baseline never sees sparse rewards. The S2D-versus-Only-Dense comparisons in Table 2 therefore conflate a temporal curriculum with mixed-reward off-policy replay. Please run a control that clears or relabels the buffer at the transition point, and report whether the S2D advantage persists.
  2. [§6.1, Table 2] The performance claims are not supported by statistical evidence. Several of the reported means have very large standard deviations (e.g., LunarLander S2D(C2) 63.40±160.55 vs. Only Dense 139.68±14.90; CartPole S2D(C2) 14.61±10.96 vs. Only Dense 3.88±4.63), and no significance tests are reported. Please provide per-seed results, confidence intervals, and significance tests (or effect sizes) for the main pairwise comparisons.
  3. [§6.2–6.3, Figure 5 and Eq. (5)] The loss-landscape smoothing claim is based on visual inspection of 2D slices, and the sharpness metric is only reported at the end of training. Sharpness can be artificially low when the policy barely moves (as acknowledged for Only Sparse in Section 7.3). Please provide quantitative landscape measures (e.g., local Hessian norms or neighborhood loss variance) and report sharpness trajectories across training, not only the endpoint.
  4. [§5.1.2 and Table 2 caption] The optimal transition timing is described as 'within the first third of training' in the caption and 'first quarter' in Section 7.1, but for LunarLander the best S2D result uses C3 (400k of 1M steps, i.e., 40% of training). Please reconcile these statements and specify how N is chosen.
minor comments (5)
  1. [Table 1] The entries in the 'Dense Reward' column (e.g., 'γ Φ(s_{t+1}) − Φ(s_t) < 0.3') are ambiguous; clarify whether the dense reward is the potential-based difference itself or a threshold condition for granting a bonus.
  2. [Table 3] The citations for OpenAI Gym and MuJoCo are both [55]; the Gym reference should be [7].
  3. [Table 2 and §11.4] The number of trials is stated as 'at least six' in Table 2 and 'at least five' in Section 11.4; make the numbers consistent.
  4. [§4.2] The statement that 'the relative position of one landscape over another is not significant' is confusing, since the smoothing comparison relies on their relative relief; please clarify the normalization.
  5. [Figure 9(a)] The reward transition is reported at 3M steps, but the main text does not specify that the RNN analysis uses C3; add a pointer to the transition-timing table.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the PBRS optimality argument is externally sourced, the S2D definition does not entail the empirical results, and the self-citation to the authors' prior work is transparent and non-load-bearing.

full rationale

The paper makes no first-principles derivation of its headline claims; the central theoretical load-bearing step, the PBRS optimality-preservation theorem, is cited to the external standard result of Ng, Harada, and Russell [43] and is reproduced with the usual telescoping argument rather than assumed. The S2D definition (Definition 2) fixes only the support-set nesting and optimal-policy nesting of a curriculum; it does not by itself entail the observed success rates, sample efficiency, sharpness values, or loss-landscape smoothness, all of which are empirical measurements on the trained policies. The statement that the study 'builds upon our earlier work [47]' is a provenance self-citation, but the present paper re-specifies the S2D transition in Definition 2 and Algorithm 1, so the cited prior work is not needed to carry the argument. The stale-replay-buffer mixing of sparse-reward transitions after the stage switch is a potential experimental confound in the S2D-versus-Only-Dense comparison, but it is a correctness concern rather than a definitional circularity: nothing in the S2D definition or algorithm forces the reported advantage to equal the curriculum input by construction. No fitted parameter is renamed as a prediction, and no uniqueness theorem is imported from the authors' own prior work. Under the quoted-reduction standard, no circular step can be exhibited.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The only fitted quantities are the curriculum hyperparameters (transition timings and dense-reward thresholds), which are ablated per environment and influence the reported gains. The theoretical scaffolding relies on the standard PBRS theorem and on a borrowed wide-minima generalization hypothesis. No new physical or theoretical entities are introduced.

free parameters (2)
  • Transition timings C1, C2, C3 = LunarLander: 100k/200k/400k frames; CartPole-Reacher: 1k/2k/3k episodes; UR5-Reacher: 1k/2k/3k episodes; ViZDoom…
    The curriculum stage boundaries are manually chosen hyperparameters, ablated per environment. The best timing per task is selected to demonstrate S2D superiority, and the paper reports that the optimal transition occurs within the first third of training (Section 7.1).
  • Dense reward thresholds = LunarLander 0.3, ViZDoom 0.14, Minecraft Manhattan distance 5, etc.
    The distance thresholds that activate the potential-based dense reward are hand-set per environment in Table 1, and they determine when dense feedback begins; they are not derived from any principle.
assumptions (4)
  • standard math Potential-based reward shaping preserves the optimal policy set (Ng et al. theorem)
    Used in Section 3.2 to justify that the S2D curriculum preserves optimal strategies. Cited from [43] and not re-proved in the paper.
  • ad hoc to paper The S2D curriculum requires nested support sets and nested optimal policy sets (Definition 2)
    This formal definition frames the toddler-inspired transition. It is a modeling choice made by the authors rather than a derived theorem.
  • domain assumption Wide minima in the policy loss landscape correlate with better generalization in RL
    Borrowed from supervised learning literature (Keskar et al., Foret et al.) and applied to RL without a dedicated proof. Used in Sections 4.3 and 6.3 to link sharpness to generalization.
  • domain assumption The replay buffer containing rewards from different curriculum stages remains a valid learning signal
    Algorithm 1 stores rewards at collection time and does not relabel them after the transition. The paper assumes this non-stationary mix does not harm training.

how reviews work

0 comments
Cite this review

Pith. "Pith review of From Sparse to Dense: Toddler-inspired Reward Transition in Goal-Oriented Reinforcement Learning." pith.science (2026). https://pith.science/paper/ECO7JWOU

@misc{pith2026250117842,
  author       = {Pith},
  title        = {Pith review of: From Sparse to Dense: Toddler-inspired Reward Transition in Goal-Oriented Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ECO7JWOU}},
  note         = {Machine review of arXiv:2501.17842}
}
read the original abstract

Reinforcement learning (RL) agents often face challenges in balancing exploration and exploitation, particularly in environments where sparse or dense rewards bias learning. Biological systems, such as human toddlers, naturally navigate this balance by transitioning from free exploration with sparse rewards to goal-directed behavior guided by increasingly dense rewards. Inspired by this natural progression, we investigate the Toddler-Inspired Reward Transition in goal-oriented RL tasks. Our study focuses on transitioning from sparse to potential-based dense (S2D) rewards while preserving optimal strategies. Through experiments on dynamic robotic arm manipulation and egocentric 3D navigation tasks, we demonstrate that effective S2D reward transitions significantly enhance learning performance and sample efficiency. Additionally, using a Cross-Density Visualizer, we show that S2D transitions smooth the policy loss landscape, resulting in wider minima that improve generalization in RL models. In addition, we reinterpret Tolman's maze experiments, underscoring the critical role of early free exploratory learning in the context of S2D rewards.

Figures

Figures reproduced from arXiv: 2501.17842 by the authors.

Figure 1
Figure 1. Analogy of agents’ trajectories to toddlers’ learning. (a) A toddler’s learning trajectory––free exploration of [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Summary of the baseline rewards. 2.4 Potential-Based Reward Shaping (PBRS) In RL, the objective is to maximize cumulative rewards. However, designing optimal reward functions often poses significant challenges, frequently involving intensive reward engineering. Reward Shaping (RS) is a well-established method used to accelerate training by offering supplementary feedback [54]. When reward structures are variable, po… view at source ↗
Figure 3
Figure 3. Experimental environments. (a) ViZDoom environments. (b) Minecraft environments. (c) Additional [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (22 more)
Figure 4
Figure 4. Figure 4: The agent’s performance across different reward baselines in several goal-oriented tasks. (1-3) In LunarLander, [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Analysis of policy loss landscape after reward transition. The 3D visualization depicts the policy loss [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Performance analysis of agents using different reward strategies in the Cross maze environment. (a) Episode [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Performance analysis of agents using different reward strategies in the playroom maze environment. (a) [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Visualizations of the trajectories near the final episode and feature analysis in maze environments. (a) [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Feature analysis in maze environments: RNN Feature and Action Frequency Analysis. (a) The left graph [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Visualization of the overall setup, including the number of stages and the transition times, in Toddler-inspired [PITH_FULL_IMAGE:figures/full_fig_p021_10.png]
Figure 11
Figure 11. Figure 11: Examples of environments where goals randomly spawn. (a) UR5-Reacher. (b) CartPole-Reacher. [PITH_FULL_IMAGE:figures/full_fig_p022_11.png]
Figure 12
Figure 12. Figure 12: Egocentric views of a ViZDoom agent in environments with various walls and objects. (a) Three walls in [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]
Figure 13
Figure 13. Figure 13: Egocentric views of a Minecraft agent in environments with various walls and objects. (a) Scenes in [PITH_FULL_IMAGE:figures/full_fig_p023_13.png]
Figure 14
Figure 14. Figure 14: Data Extraction and Processing for RNN Features and Policy Visualization. (a) The process begins by [PITH_FULL_IMAGE:figures/full_fig_p024_14.png]
Figure 15
Figure 15. Figure 15: This figure offers a detailed look at the 3D policy loss landscape during reward scheme transitions. On the [PITH_FULL_IMAGE:figures/full_fig_p026_15.png]
Figure 16
Figure 16. Figure 16: This 3D visualization of the policy loss landscape for CartPole-Reacher illustrates that notable smoothing [PITH_FULL_IMAGE:figures/full_fig_p027_16.png]
Figure 17
Figure 17. Figure 17: This figure presents the 3D policy loss landscape for the UR5-Reacher task after various reward adjustments, [PITH_FULL_IMAGE:figures/full_fig_p028_17.png]
Figure 18
Figure 18. Figure 18: Overview of three ViZDoom-FourObjects environments: [PITH_FULL_IMAGE:figures/full_fig_p029_18.png]
Figure 19
Figure 19. Figure 19: Comparison of different transition timings [PITH_FULL_IMAGE:figures/full_fig_p030_19.png]
Figure 20
Figure 20. Figure 20: Performance metrics in RWARE tasks. The vertical axis measures the average number of shelves successfully [PITH_FULL_IMAGE:figures/full_fig_p031_20.png]
Figure 21
Figure 21. Figure 21: (a) Visual representation of agent trajectories in RWARE-Level1. (b) The three different levels of RWARE [PITH_FULL_IMAGE:figures/full_fig_p032_21.png]
Figure 22
Figure 22. Figure 22: Gridworld-navigation task. Left: 10 × 10 environment with potential-based dense rewards using PPO. Center: 10 × 10 environment with sparse rewards using PPO. Right-Top: 4 × 4 environment with sparse rewards using DQN. Right-Bottom: 4 × 4 environment with potential-bas…
Figure 23
Figure 23. Figure 23: This visualization showcases the 3D policy loss landscape post-transition from sparse-to-dense ( [PITH_FULL_IMAGE:figures/full_fig_p034_23.png]
Figure 24
Figure 24. Figure 24: This figure presents the 3D policy loss landscape for dense-to-sparse (D2S, [PITH_FULL_IMAGE:figures/full_fig_p035_24.png]
Figure 25
Figure 25. Figure 25: This illustration captures the 3D Q-value loss landscape across different reward transitions, including [PITH_FULL_IMAGE:figures/full_fig_p036_25.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 39 canonical work pages

  1. [1]

    Critical learning periods in deep networks

    Alessandro Achille, Matteo Rovere, and Stefano Soatto. Critical learning periods in deep networks. InInternational Conference on Learning Representations, 2018

    2018

  2. [2]

    Learning dexterous in-hand manipulation

    OpenAI: Marcin Andrychowicz, Bowen Baker, Maciek Chociej, Rafal Jozefowicz, Bob McGrew, Jakub Pachocki, Arthur Petron, Matthias Plappert, Glenn Powell, Alex Ray, et al. Learning dexterous in-hand manipulation. The International Journal of Robotics Research, 39(1):3–20, 2020

    2020

  3. [3]

    A survey on intrinsic motivation in reinforcement learning

    Arthur Aubret, Laetitia Matignon, and Salima Hassas. A survey on intrinsic motivation in reinforcement learning. arXiv preprint arXiv:1908.06976, 2019

    arXiv 1908

  4. [4]

    Never give up: Learning directed exploration strategies

    Adrià Puigdomènech Badia, Pablo Sprechmann, Alex Vitvitskyi, Daniel Guo, Bilal Piot, Steven Kapturowski, Olivier Tieleman, Martín Arjovsky, Alexander Pritzel, Andew Bolt, et al. Never give up: Learning directed exploration strategies. arXiv preprint arXiv:2002.06038, 2020

    arXiv 2002

  5. [5]

    Toddler-inspired visual object learning

    Sven Bambach, David Crandall, Linda Smith, and Chen Yu. Toddler-inspired visual object learning. Advances in neural information processing systems, 31, 2018

    work page 2018

  6. [6]

    Curriculum learning

    Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In ICML ’09, 2009

    work page 2009

  7. [7]

    Openai gym

    Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym. arXiv preprint arXiv:1606.01540, 2016

    arXiv 2016

  8. [8]

    Exploration by random network distillation

    Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov. Exploration by random network distillation. arXiv preprint arXiv:1810.12894, 2018

    arXiv 2018

Show all 61 references
  1. [9]

    A critical period for robust curriculum-based deep reinforcement learning of sequential action in a robot arm

    Roy De Kleijn, Deniz Sen, and George Kachergis. A critical period for robust curriculum-based deep reinforcement learning of sequential action in a robot arm. Topics in Cognitive Science, 2(2):311–326, 2022

    2022

  2. [10]

    Class rectification hard mining for imbalanced deep learning

    Qi Dong, Shaogang Gong, and Xiatian Zhu. Class rectification hard mining for imbalanced deep learning. In Proceedings of the IEEE International Conference on Computer Vision, pages 1851–1860, 2017

    2017

  3. [11]

    Self-contrastive learning with hard negative sampling for self-supervised point cloud learning

    Bi’an Du, Xiang Gao, Wei Hu, and Xin Li. Self-contrastive learning with hard negative sampling for self-supervised point cloud learning. In Proceedings of the 29th ACM International Conference on Multimedia, MM ’21, page 3133–3142, New York, NY , USA, 2021. Association for Com...

    2021

  4. [12]

    Automatic goal generation for reinforcement learning agents

    Carlos Florensa, David Held, Xinyang Geng, and Pieter Abbeel. Automatic goal generation for reinforcement learning agents. In International conference on machine learning, pages 1515–1528. PMLR, 2018

    2018

  5. [13]

    Sharpness-aware minimization for efficiently improving generalization

    Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum? id=6Tm1mposlrM

    2021

  6. [14]

    Exploratory behavior in the development of perceiving, acting, and the acquiring of knowledge

    Eleanor J Gibson. Exploratory behavior in the development of perceiving, acting, and the acquiring of knowledge. Annual review of psychology, 39(1):1–42, 1988

    1988

  7. [15]

    Qualitatively characterizing neural network optimization problems

    Ian J Goodfellow, Oriol Vinyals, and Andrew M Saxe. Qualitatively characterizing neural network optimization problems. arXiv preprint arXiv:1412.6544, 2014

    2014 arXiv

  8. [16]

    The scientist in the crib: Minds, brains, and how children learn

    Alison Gopnik, Andrew N Meltzoff, and Patricia K Kuhl. The scientist in the crib: Minds, brains, and how children learn. William Morrow & Co, 1999

    1999

  9. [17]

    Changes in cognitive flexibility and hypothesis search across human life history from childhood to adolescence to adulthood

    Alison Gopnik, Shaun O’Grady, Christopher G Lucas, Thomas L Griffiths, Adrienne Wente, Sophie Bridgers, Rosie Aboody, Hoki Fung, and Ronald E Dahl. Changes in cognitive flexibility and hypothesis search across human life history from childhood to adolescence to adulthood. Proc...

    2017

  10. [18]

    Automated curriculum learning for neural networks

    Alex Graves, Marc G Bellemare, Jacob Menick, Remi Munos, and Koray Kavukcuoglu. Automated curriculum learning for neural networks. In international conference on machine learning, pages 1311–1320. PMLR, 2017

    2017

  11. [19]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor

    Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, pages 1861–1870. PMLR, 2018

    2018

  12. [20]

    On the power of curriculum learning in training deep networks

    Guy Hacohen and Daphna Weinshall. On the power of curriculum learning in training deep networks. ArXiv, 2, 2019

    2019

  13. [21]

    Dealing with sparse rewards in reinforcement learning

    Joshua Hare. Dealing with sparse rewards in reinforcement learning. arXiv preprint arXiv:1910.09281, 2019. 17 A PREPRINT - JANUARY 30, 2025

    1910 arXiv

  14. [22]

    Expressing arbitrary reward functions as potential-based advice

    Anna Harutyunyan, Sam Devlin, Peter Vrancx, and Ann Nowé. Expressing arbitrary reward functions as potential-based advice. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 29, 2015

    2015

  15. [23]

    Comprehensive overview of reward engineering and shaping in advancing reinforcement learning applications

    Sinan Ibrahim, Mostafa Mostafa, Ali Jnadi, Hadi Salloum, and Pavel Osinenko. Comprehensive overview of reward engineering and shaping in advancing reinforcement learning applications. IEEE Access, 2024

    2024

  16. [24]

    Finding flatter minima with sgd, 2018

    Stanisław Jastrz˛ ebski, Zachary Kenton, Devansh Arpit, Nicolas Ballas, Asja Fischer, Yoshua Bengio, and Amos Storkey. Finding flatter minima with sgd, 2018. URL https://openreview.net/forum?id=r1VF9dCUG

    2018

  17. [25]

    Hard negative mixing for contrastive learning

    Yannis Kalantidis, Mert Bulent Sariyildiz, Noe Pion, Philippe Weinzaepfel, and Diane Larlus. Hard negative mixing for contrastive learning. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors,Advances in Neural Information Processing Systems, volume 33, ...

    2020

  18. [26]

    Vizdoom: A doom-based ai research platform for visual reinforcement learning

    Michał Kempka, Marek Wydmuch, Grzegorz Runc, Jakub Toczek, and Wojciech Ja´skowski. Vizdoom: A doom-based ai research platform for visual reinforcement learning. In 2016 IEEE conference on computational intelligence and games (CIG), pages 1–8. IEEE, 2016

    2016

  19. [27]

    On large-batch training for deep learning: Generalization gap and sharp minima

    Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. On large-batch training for deep learning: Generalization gap and sharp minima. arXiv preprint arXiv:1609.04836, 2016

    2016 arXiv

  20. [28]

    On large-batch training for deep learning: Generalization gap and sharp minima

    Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. On large-batch training for deep learning: Generalization gap and sharp minima. In International Conference on Learning Representations,

  21. [29]

    Goal-aware cross-entropy for multi-target reinforcement learning

    Kibeom Kim, Min Whoo Lee, Yoonsung Kim, JeHwan Ryu, Minsu Lee, and Byoung-Tak Zhang. Goal-aware cross-entropy for multi-target reinforcement learning. Advances in Neural Information Processing Systems, 34:2783–2795, 2021

    2021

  22. [30]

    L-sa: Learning under-explored targets in multi-target reinforcement learning

    Kibeom Kim, Hyundo Lee, Min Whoo Lee, Moonheon Lee, Minsu Lee, and Byoung-Tak Zhang. L-sa: Learning under-explored targets in multi-target reinforcement learning. arXiv preprint arXiv:2305.13741, 2023

    2023 arXiv

  23. [31]

    Visual hindsight self-imitation learning for interactive navigation

    Kibeom Kim, Kisung Shin, Min Whoo Lee, Moonhoen Lee, Minsu Lee, and Byoung-Tak Zhang. Visual hindsight self-imitation learning for interactive navigation. arXiv preprint arXiv:2312.03446, 2023

    2023 arXiv

  24. [32]

    Reward (mis) design for autonomous driving

    W Bradley Knox, Alessandro Allievi, Holger Banzhaf, Felix Schmitt, and Peter Stone. Reward (mis) design for autonomous driving. Artificial Intelligence, 316:103829, 2023

    2023

  25. [33]

    Exploration in deep reinforcement learning: A survey

    Pawel Ladosz, Lilian Weng, Minwoo Kim, and Hyondong Oh. Exploration in deep reinforcement learning: A survey. Information Fusion, 85:1–22, 2022

    2022

  26. [34]

    Theory and application of reward shaping in reinforcement learning

    Adam Daniel Laud. Theory and application of reward shaping in reinforcement learning. University of Illinois at Urbana- Champaign, 2004

    2004

  27. [35]

    Visualizing the loss landscape of neural nets.Advances in neural information processing systems, 31, 2018

    Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets.Advances in neural information processing systems, 31, 2018

    2018

  28. [36]

    Continual reinforcement learning in 3d non- stationary environments

    Vincenzo Lomonaco, Karan Desai, Eugenio Culurciello, and Davide Maltoni. Continual reinforcement learning in 3d non- stationary environments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 248–249, 2020

    2020

  29. [37]

    Information-based objective functions for active data selection

    David JC MacKay. Information-based objective functions for active data selection. Neural computation, 4(4):590–604, 1992

    1992

  30. [38]

    Playing atari with deep reinforcement learning

    V olodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013

    2013 arXiv

  31. [39]

    Rusu, Joel Veness, Marc G

    V olodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra,...

    2015 doi

  32. [40]

    Asynchronous methods for deep reinforcement learning

    V olodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In International conference on machine learning, pages 1928–1937. PMLR, 2016

    1928

  33. [41]

    Generalizing curricula for reinforcement learning

    Sanmit Narvekar and Peter Stone. Generalizing curricula for reinforcement learning. In 4th Lifelong Machine Learning Workshop at ICML 2020, 2020

    2020

  34. [42]

    Ng, Daishi Harada, and Stuart J

    A. Ng, Daishi Harada, and Stuart J. Russell. Policy invariance under reward transformations: Theory and application to reward shaping. In International Conference on Machine Learning, 1999. 18 A PREPRINT - JANUARY 30, 2025

    1999

  35. [43]

    Policy invariance under reward transformations: Theory and application to reward shaping

    Andrew Y Ng, Daishi Harada, and Stuart Russell. Policy invariance under reward transformations: Theory and application to reward shaping. In Icml, volume 99, pages 278–287. Citeseer, 1999

    1999

  36. [44]

    How evolution may work through curiosity-driven developmental process

    Pierre-Yves Oudeyer and Linda B Smith. How evolution may work through curiosity-driven developmental process. Topics in Cognitive Science, 8(2):492–502, 2016

    2016

  37. [45]

    Benchmarking multi-agent deep reinforce- ment learning algorithms in cooperative tasks

    Georgios Papoudakis, Filippos Christianos, Lukas Schäfer, and Stefano V Albrecht. Benchmarking multi-agent deep reinforce- ment learning algorithms in cooperative tasks. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1), 2021

    2021

  38. [46]

    Toddler- guidance learning: Impacts of critical period on multimodal ai agents

    Junseok Park, Kwanyoung Park, Hyunseok Oh, Ganghun Lee, Minsu Lee, Youngki Lee, and Byoung-Tak Zhang. Toddler- guidance learning: Impacts of critical period on multimodal ai agents. In Proceedings of the 2021 International Conference on Multimodal Interaction, pages 212–220, 2021

    2021

  39. [47]

    Unveiling the significance of toddler-inspired reward transition in goal-oriented reinforcement learning

    Junseok Park, Yoonsung Kim, Hee Bin Yoo, Min Whoo Lee, Kibeom Kim, Won-Seok Choi, Minsu Lee, and Byoung-Tak Zhang. Unveiling the significance of toddler-inspired reward transition in goal-oriented reinforcement learning. In Proceedings of the AAAI Conference on Artificial Inte...

    2024

  40. [48]

    Curiosity-driven exploration by self-supervised prediction

    Deepak Pathak, Pulkit Agrawal, Alexei A Efros, and Trevor Darrell. Curiosity-driven exploration by self-supervised prediction. In International conference on machine learning, pages 2778–2787. PMLR, 2017

    2017

  41. [49]

    The origins of intelligence in children, volume 8

    Jean Piaget, Margaret Cook, et al. The origins of intelligence in children, volume 8. International Universities Press New York, 1952

    1952

  42. [50]

    Stable-baselines3: Reliable reinforcement learning implementations

    Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dormann. Stable-baselines3: Reliable reinforcement learning implementations. Journal of Machine Learning Research, 22(268):1–8, 2021. URL http: //jmlr.org/papers/v22/20-1364.html

    2021

  43. [51]

    High-dimensional continuous control using generalized advantage estimation

    John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438, 2015

    2015 arXiv

  44. [52]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  45. [53]

    From neurons to neighborhoods: The science of early childhood development

    JP Shonkoff and DA Phillips. From neurons to neighborhoods: The science of early childhood development. eric. ed. gov. National Academy of Sciences Press: Washington DC. Accessed on May, 8:2015, 2000

    2015

  46. [54]

    Transfer learning for reinforcement learning domains: A survey

    Matthew E Taylor and Peter Stone. Transfer learning for reinforcement learning domains: A survey. Journal of Machine Learning Research, 10(7), 2009

    2009

  47. [55]

    Mujoco: A physics engine for model-based control

    Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ international conference on intelligent robots and systems, pages 5026–5033. IEEE, 2012

    2012

  48. [56]

    Cognitive maps in rats and men

    Edward C Tolman. Cognitive maps in rats and men. Psychological review, 55(4):189, 1948

    1948

  49. [57]

    Safe reinforcement learning via curriculum induction

    Matteo Turchetta, Andrey Kolobov, Shital Shah, Andreas Krause, and Alekh Agarwal. Safe reinforcement learning via curriculum induction. Advances in Neural Information Processing Systems, 33:12151–12162, 2020

    2020

  50. [58]

    Curriculum learning by transfer learning: Theory and experiments with deep networks

    Daphna Weinshall, Gad Cohen, and Dan Amir. Curriculum learning by transfer learning: Theory and experiments with deep networks. In International Conference on Machine Learning, pages 5238–5246. PMLR, 2018

    2018

  51. [59]

    Fresh: Interactive reward shaping in high-dimensional state spaces using human feedback

    Baicen Xiao, Qifan Lu, Bhaskar Ramasubramanian, Andrew Clark, Linda Bushnell, and Radha Poovendran. Fresh: Interactive reward shaping in high-dimensional state spaces using human feedback. arXiv preprint arXiv:2001.06781, 2020

    2001 arXiv

  52. [60]

    requested

    Byoung-Tak Zhang. Selecting a critical subset of given examples during learning. In International Conference on Artificial Neural Networks, pages 517–520. Springer, 1994. 19 A PREPRINT - JANUARY 30, 2025 Supplementary Material: Insights into Toddler-Inspired Reward Transitions...

    1994

  53. [2017]

    URL https://openreview.net/forum?id=H1oyRlYgg

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.