Pith. sign in

REVIEW 3 major objections 5 minor 107 references

Syllabus: Portable Curricula for Reinforcement Learning Agents

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that curriculum learning can be separated from RL training code through a portable API, and that existing automatic curricula fail to transfer to NetHack and Neural MMO.

desk verdict The library and reproductions are a genuine contribution; the negative-transfer conclusions rest on a return-based success proxy that can saturate, and the paper itself flags this. read the letter →

arxiv 2411.11318 v2 pith:KVLIYC7Q submitted 2024-11-18 cs.AI

Ryan Sullivan , Ryan Pégoud , Ameen Ur Rehman , Xinchen Yang , Junyun Huang , Aayush Verma , Nistha Mitra , John P. Dickerson This is my paper · ORCID
classification cs.AI
keywords curriculumlearningautomaticunsupervisedenvironmentdesignportableRLinfrastructurereinforcementlibraryNetHackNeuralMMOprioritizedlevelreplay
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that curriculum learning\u2014deciding which tasks an agent practices\u2014can be decoupled from the reinforcement-learning training loop and delivered as a portable layer. It claims that a small API plus a dedicated synchronization channel lets the same automatic-curriculum algorithms attach to almost any RL codebase with only a few lines of change, so researchers do not have to reimplement curricula for each library. To back this, the authors reimplement four popular methods, reproduce the results from the papers that introduced them, then run the same methods in two hard, previously unexplored environments. In those domains, none of the automatic curricula beat random task selection, and the paper reads this as evidence that existing curricula over level seeds do not transfer to complex long-horizon games.

What carries the argument

The object that carries the argument is the Curriculum API together with its synchronization wrappers. A Curriculum maintains a sampling distribution and exposes update methods that can be called per step, per episode, or on demand; a TaskSpace maps arbitrary tasks into a uniform encoding; and a TaskWrapper lets an environment accept a new task at reset. The synchronization layer runs over its own communication channel, batching feedback and buffering task samples, so it does not depend on how the RL library passes messages between processes. That separation is what makes the same algorithm files portable across codebases, and it is also what lets the paper test one implementation of each method in several different training systems without rewriting the curriculum.

What would settle it

Run LP and SFL on NetHack and Neural MMO while replacing the manually scaled, clipped return proxy with the environments' native binary task-success events; if the methods then beat domain randomization or match their original-domain results, the paper's evidence that these methods do not transfer to complex environments would be an artifact of the proxy.

Watch

Extended reading notes

Core claim

The central discovery is a packaging claim with an empirical finding attached. The packaging claim: curriculum logic can live in a standalone Curriculum object that keeps a distribution over tasks, with environment wrappers that accept a new task at reset and a separate sender-receiver pathway that sends tasks outward and feedback back, so the RL training code only needs a few wrapper lines. The authors validate it by reproducing the behavior of Prioritized Level Replay, Learning Progress, and OMNI on their original benchmarks, then applying four automatic-curriculum methods\u2014PLR, LP, OMNI, and Sampling for Learnability\u2014to NetHack and Neural MMO as the first such baselines. The empirical finding: when curricula are restricted to selecting initial level seeds, none of the methods improve over domain randomization in those complex environments, and success-rate-based methods sometimes help early but hurt asymptotic performance. The paper concludes that curricula over initial environment conditions are not the right axis in long-horizon, multi-agent games.

Load-bearing premise

That a scaled and clipped episodic return is a fair stand-in for the binary task-success signal that Learning Progress and Sampling for Learnability are designed to consume.

Editorial extensions

If this is right

  • Reproduction becomes cheap: any open-source training script can be turned into a curriculum experiment by wrapping the environment and attaching a curriculum object, with no changes to the policy optimizer.
  • Research comparisons become fairer: the same code for PLR, LP, OMNI, or SFL runs inside different RL codebases, so differences in results can be attributed to the training system or environment rather than to a reimplementation.
  • The negative NetHack and Neural MMO results challenge the assumption that curricula over level seeds are a general solution; new work should choose task axes like reward functions, objectives, or opponent skill.
  • The full-distribution variant of SFL, which removes two environment-dependent hyperparameters, is proposed as a strong default for exploring a new domain.
  • Syllabus supports sequential and joint task-and-opponent curricula, so a single training loop can graduate through staged tasks or simultaneously vary maps and co-players.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next experiment, not run in the paper, is to apply LP and SFL to Neural MMO using its built-in task predicates as the binary success signal rather than a clipped return; that would separate failure of the methods from failure of the proxy.
  • If the negative results generalize, the productive research direction is to make curricula select mid-episode objectives and opponent strategies rather than starting conditions, which the task-space abstraction in Syllabus is built to support.
  • The matching reproduction on original benchmarks suggests the portability claim is not bought at the price of correctness, so the library could become a neutral comparison layer for future curriculum-learning papers.
  • Because the paper tunes hyperparameters per environment with grid search, a cleaner default story might emerge from measuring how sensitive each method is to its hyperparameters, which the full-distribution variant of SFL already weakens.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces Syllabus, a library that separates curriculum-learning logic from RL training code by providing a minimal API for task spaces, curricula, and environment wrappers, together with a separate multiprocessing synchronization pathway. The authors validate their implementations by reproducing PLR on Procgen and LP/OMNI on Crafter against reference codebases, then present new baselines across Procgen, Crafter, NetHack, and Neural MMO, concluding that popular automatic curriculum methods do not transfer well to complex new environments. The portability/infrastructure claim is central and supported by the reproduction experiments; the empirical negative-transfer claim is the paper's main new scientific conclusion.

Significance. If the portability claims hold, Syllabus fills a genuine gap: it is the first curriculum-learning library that works with multiple RL frameworks without entangling curriculum code into the training loop, and the authors provide machine-checked reproductions against external reference implementations, including exact matches for LP/OMNI on Crafter and close matches for PLR on Procgen. The new baselines and the first ACL experiments on NetHack and Neural MMO are useful resources for the community. However, the negative-transfer conclusion depends critically on a success proxy (scaled, clipped episodic return) that is not validated for LP and SFL, and the paper itself concedes that the proxy may be worse than value predictions. This makes the paper's main empirical contribution, as stated in Contribution 3, currently unsupported, even though the infrastructure contribution remains valuable.

major comments (3)
  1. [§7.3 and Supplementary D.1, D.3, D.4] The success metric used for LP and SFL on all environments except Crafter is defined as scaled, clipped episodic return rather than a binary task-completion signal. In NetHack the metric is min(max(R/1000,0),1), in Neural MMO min(max(max_a(R)/10,0),1), and in Procgen min(max(r_N,0),1). LP and SFL were designed for per-task pass/fail success rates: LP computes learning progress as the absolute difference between fast and slow EMAs of success, and SFL samples according to p(1-p). If the mean episode return exceeds the manually chosen scale, every task's proxy success saturates to 1, so LP reports zero progress and SFL's p(1-p) collapses to zero. The observed NetHack pattern of early acceleration followed by lower asymptotic performance is exactly what this saturation would produce. The paper does not validate that the proxy preserves the ordering of tasks by true learnability, so the negative-transfer conclusion in §8 is not established.
  2. [§8, 'Overall' paragraph] The paper's headline empirical claim—that 'popular curriculum learning methods are far less effective outside of the environments in which they were originally developed'—is drawn from experiments in which LP and SFL use the return-based proxy. The same section concedes: 'In their absence, value predictions might be a better approximation of competence than our return-based success metric defined in subsection 7.3. We leave a thorough investigation of this inconsistency as future work.' This admission applies directly to the central empirical contribution. The portability and reproduction claims are not affected, but the negative-transfer conclusion should be either re-derived using a validated binary success signal (e.g., actual task achievements or a clear task-completion predicate) or explicitly downgraded to a qualified observation about the proxy, not a general statement about the algorithms.
  3. [§7.3, last sentence] The statement that 'each of the libraries used in this paper have different design philosophies, software architectures, and multiprocessing implementations' is used to motivate the portability claim, but the paper does not quantify the integration effort (e.g., lines of code changed or time to integrate) beyond the illustrative code examples in Figure 2 and Supplementary F. A quantitative comparison of integration overhead against existing curriculum libraries (DCD, Minimax, JaxUED, TeachMyAgent) would substantially strengthen Contribution 1. As written, the 'few lines of code' claim is plausible but not directly measured.
minor comments (5)
  1. [§5.5, 'sequential curricula'] There is a typo: 'sequantual' should be 'sequential' in the sentence about the sequential curriculum.
  2. [Supplementary D.2, last paragraph] The phrase 'through it's synchronization wrappers' uses an incorrect apostrophe; it should be 'its synchronization wrappers'.
  3. [Figure 8 caption] The caption reads '95% Stratified Boostrapped Confidence Intervals'; 'Boostrapped' should be 'Bootstrap'.
  4. [§5.5, SFL paragraph] The mixing ratio hyperparameter is denoted p, which collides with the success probability p in the p(1-p) formula; using a distinct symbol such as ρ_mix would prevent confusion, especially since Supplementary D.5 later uses ρ for the SFL top-K sampling ratio.
  5. [Supplementary D.4] The sentence 'where R is the mean episodic return because our agents get close to but do not exceed 1000 mean episodic return' should specify whether R is the mean over all agents, over a single episode, or over a rollout; this is especially relevant because the proxy's saturation threshold determines whether the measured learning progress can ever become nonzero after the agent improves.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the portability claim is anchored to external baselines and the contested success proxy is a validity concern, not a self-referential reduction.

full rationale

The paper's derivation chain is largely self-contained and anchored to external benchmarks. Section 6 validates Syllabus's implementations by reproducing PLR and DR on Procgen against Jiang et al. (2021b) and LP and OMNI on Crafter against Zhang et al. (2023), reporting that normalized returns "almost precisely match" and that implementations "achieve the exact same task success rates throughout training as the reference implementations." The portability claim is demonstrated through integrations with CleanRL, RLLib, moolib, and PufferLib, all external codebases. The only self-citations, such as Neural MMO 2.0 and PettingZoo, are infrastructure and benchmark choices and are not load-bearing for the central argument. The most contestable step is the LP and SFL success proxy in Section 7.3 and Supplementary D.3 and D.4, where scaled, clipped episodic return replaces the binary pass/fail signal these algorithms assume; this is a real validity threat to the negative-transfer conclusion in Section 8, and the paper itself concedes that "value predictions might be a better approximation of competence than our return-based success metric" and leaves the issue to future work. However, this is not circularity: the scale is not fitted to the conclusion, and the conclusion is not equivalent to the proxy by construction. No equation in the paper reduces to its inputs by definition, and no fitted parameter is renamed as a prediction.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central empirical claims rest on three hand-made choices: the pseudo-success metric for LP/SFL, the use of seed/map task spaces in NetHack and Neural MMO, and the transfer of PPO hyperparameters without re-tuning. These choices do not affect the portability of the library, but they condition the negative transfer conclusion.

free parameters (4)
  • NetHack success-scale factor = 1000
    D.4: task reward = min(max(R/1000, 0.0), 1.0). Hand-selected upper bound on mean episodic return; controls LP/SFL prioritization.
  • Neural MMO success-scale factor = 10
    D.3: task reward = min(max(max_a(R)/10, 0.0), 1.0) because individual returns approach but do not exceed 10. Manual choice affects map prioritization.
  • Procgen normalized-return clipping = min(max(r_N, 0), 1)
    D.1: task reward = min(max(r_N, 0.0), 1.0) with r_N normalized by per-environment return ranges. The clipping bounds are hand-chosen.
  • LP/SFL environment-level hyperparameters = See Table 4 (alpha, p_theta, K, rho, T)
    Sec. 7.3: 'We use a grid search to tune every curriculum's hyperparameters in each environment.' Environment-specific fits affect performance comparisons.
assumptions (4)
  • ad hoc to paper Scaled, clipped episodic return is a valid proxy for binary task success when applying LP and SFL to Procgen, NetHack, and Neural MMO.
    Introduced in Sec. 7.3 and detailed in D.1, D.3, D.4. LP and SFL were designed for pass/fail task sets; substituting return-based pseudo-success may change which tasks are prioritized.
  • domain assumption Level seeds and maps are a meaningful task axis that controls difficulty in NetHack and Neural MMO.
    Sec. 8 states seeds can diverge within a few steps in NetHack and maps may have minimal influence on difficulty in Neural MMO. If these axes are not informative, no seed/map curriculum can be expected to help.
  • domain assumption PPO hyperparameters tuned for the original codebases transfer appropriately to the new environment-curriculum combinations.
    Sec. D.5: hyperparameters were taken from the papers that introduced each codebase, with only curriculum hyperparameters grid-searched. Confounding between untuned RL settings and curricula is possible.
  • standard math Stratified bootstrap confidence intervals provide valid estimates of the reported performance differences.
    Used through the paper and supplementary via Open RL Benchmark tools; relies on standard resampling assumptions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Syllabus: Portable Curricula for Reinforcement Learning Agents." pith.science (2026). https://pith.science/paper/KVLIYC7Q

@misc{pith2026241111318,
  author       = {Pith},
  title        = {Pith review of: Syllabus: Portable Curricula for Reinforcement Learning Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KVLIYC7Q}},
  note         = {Machine review of arXiv:2411.11318}
}
read the original abstract

Curriculum learning has been a quiet, yet crucial component of many high-profile successes of reinforcement learning. Despite this, it is still a niche topic that is not directly supported by any of the major reinforcement learning libraries. These methods can improve the capabilities and generalization of RL agents, but often require complex changes to training code. We introduce Syllabus, a portable curriculum learning library, as a solution to this problem. Syllabus provides a universal API for curriculum learning, modular implementations of popular automatic curriculum learning methods, and infrastructure that allows them to be easily integrated with asynchronous training code in nearly any RL library. Syllabus provides a minimal API for core curriculum learning components, making it easier to design new algorithms and adapt existing ones to new environments. We demonstrate this by evaluating the algorithms in Syllabus on several new environments, each using agents written in a different RL library. We present the first examples of automatic curriculum learning in NetHack and Neural MMO, two of the most challenging RL benchmarks, and find evidence that existing methods do not directly transfer to complex new environments. Syllabus can be found at https://github.com/RyanNavillus/Syllabus.

Figures

Figures reproduced from arXiv: 2411.11318 by the authors.

Figure 1
Figure 1. Syllabus with a standard asynchronous RL training setup. [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Using Syllabus for curriculum learning with just a few lines of code. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. (a) Mean normalized test returns for Syllabus’s implementation vs. the original implemen￾tation of Prioritized Level Replay from (Jiang et al., 2021b) on 10 Procgen environments. Domain Randomization is also included for reference. (b) Mean task success rate for Syllabus’s Learning Progress and OMNI implementations vs. the reference implementations from (Zhang et al., 2023) on Crafter. (c) and (d) 95% Stratified boo… view at source ↗
Figures from the paper (16 more)
Figure 4
Figure 4. Figure 4: Automatic curriculum learning results on [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: An abbreviated summary of the Curriculum interface. These represent the main methods [PITH_FULL_IMAGE:figures/full_fig_p022_5.png]
Figure 6
Figure 6. Figure 6: Main features of the Task Space API. A.3 Task Interface API In unsupervised environment design, we study underspecified POMDPs (UPOMDPs), which have free configuration variables that need to be chosen to produce a fully specified POMDP (Dennis et al., 2020). In multi-t…
Figure 7
Figure 7. Figure 7: Main features of the Task Interface. stage. The curriculum passes any updates that it receives to the current curriculum stage, so we can even use automatic curricula sequentially. It is initialized with a set of stages, which can be individual elements of the task spa…
Figure 8
Figure 8. Figure 8: 95% Stratified Boostrapped Confidence Intervals for the Mean, Median, Interquartile [PITH_FULL_IMAGE:figures/full_fig_p026_8.png]
Figure 9
Figure 9. Figure 9: Normalized test returns for Domain Randomization, Prioritized Level Replay, Sampling [PITH_FULL_IMAGE:figures/full_fig_p027_9.png]
Figure 10
Figure 10. Figure 10: 95% Stratified Boostrapped Confidence Intervals for the Mean, Median, Interquartile [PITH_FULL_IMAGE:figures/full_fig_p028_10.png]
Figure 11
Figure 11. Figure 11: 95% Stratified Bootstrapped Confidence Intervals for the Mean, Median, Interquartile [PITH_FULL_IMAGE:figures/full_fig_p029_11.png]
Figure 12
Figure 12. Figure 12: 95% Stratified Bootstrapped Confidence Intervals for the Mean, Median, Interquartile [PITH_FULL_IMAGE:figures/full_fig_p029_12.png]
Figure 14
Figure 14. Figure 14: b. We may need to train for longer to see any benefits to sampling historic agents. (a) (b) [PITH_FULL_IMAGE:figures/full_fig_p032_14.png]
Figure 15
Figure 15. Figure 15: Normalized Test Returns of PPO and PPG with 1 or 3 value epochs when trained with [PITH_FULL_IMAGE:figures/full_fig_p033_15.png]
Figure 16
Figure 16. Figure 16: Training Procgen agents with PLR using stale value predictions. 1 buffer is equivalent to [PITH_FULL_IMAGE:figures/full_fig_p034_16.png]
Figure 17
Figure 17. Figure 17: Mean task success rates for the Full Distribution and Top K implementations of SFL with [PITH_FULL_IMAGE:figures/full_fig_p035_17.png]
Figure 18
Figure 18. Figure 18: Various events and achievements in Neural MMO for Domain randomization and a [PITH_FULL_IMAGE:figures/full_fig_p037_18.png]
Figure 19
Figure 19. Figure 19: Adding curriculum learning with Syllabus to RLLib training code with just a few lines [PITH_FULL_IMAGE:figures/full_fig_p038_19.png]
Figure 20
Figure 20. Figure 20: Quickstart page of Syllabus’s documentation website. [PITH_FULL_IMAGE:figures/full_fig_p040_20.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

107 extracted references · 36 canonical work pages

  1. [1]

    Deep reinforcement learning at the edge of the statistical precipice

    Rishabh Agarwal, Max Schwarzer, Pablo Samuel Castro, Aaron Courville, and Marc G Bellemare. Deep reinforcement learning at the edge of the statistical precipice. Advances in Neural Information Processing Systems, 2021

    2021

  2. [2]

    Griddly: A platform for ai research in games

    Christopher Bamford. Griddly: A platform for ai research in games. Software Impacts, 8: 0 100066, 2021

    2021

  3. [3]

    Barto, Richard S

    Andrew G. Barto, Richard S. Sutton, and Charles W. Anderson. Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Transactions on Systems, Man, and Cybernetics, SMC-13 0 (5): 0 834--846, 1983. doi:10.1109/TSMC.1983.6313077

    arXiv 1983

  4. [4]

    Unifying count-based exploration and intrinsic motivation

    Marc Bellemare, Sriram Srinivasan, Georg Ostrovski, Tom Schaul, David Saxton, and Remi Munos. Unifying count-based exploration and intrinsic motivation. Advances in neural information processing systems, 29, 2016 a

    2016

  5. [5]

    The arcade learning environment: An evaluation platform for general agents

    Marc G Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling. The arcade learning environment: An evaluation platform for general agents. Journal of Artificial Intelligence Research, 47: 0 253--279, 2013

    2013

  6. [6]

    Bellemare, Sriram Srinivasan, Georg Ostrovski, Tom Schaul, David Saxton, and R\' e mi Munos

    Marc G. Bellemare, Sriram Srinivasan, Georg Ostrovski, Tom Schaul, David Saxton, and R\' e mi Munos. Unifying count-based exploration and intrinsic motivation. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS'16, pp.\ 1479–1487, Red Hook, NY, USA, 2016 b . Curran Associates Inc. ISBN 9781510838819

    2016

  7. [7]

    Curriculum learning

    Yoshua Bengio, J\' e r\^ o me Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th Annual International Conference on Machine Learning, ICML '09, pp.\ 41–48, New York, NY, USA, 2009. Association for Computing Machinery. ISBN 9781605585161. doi:10.1145/1553374.1553380. URL https://doi.org/10.1145/1553374.1553380

    arXiv 2009

  8. [8]

    Dota 2 with large scale deep reinforcement learning

    Christopher Berner, Greg Brockman, Brooke Chan, Vicki Cheung, Przemys aw D e biak, Christy Dennison, David Farhi, Quirin Fischer, Shariq Hashme, Chris Hesse, et al. Dota 2 with large scale deep reinforcement learning. arXiv preprint arXiv:1912.06680, 2019

    arXiv 1912

Show all 107 references
  1. [9]

    Refining minimax regret for unsupervised environment design

    Michael Beukman, Samuel Coward, Michael Matthews, Mattie Fellows, Minqi Jiang, Michael Dennis, and Jakob Foerster. Refining minimax regret for unsupervised environment design. arXiv preprint arXiv:2402.12284, 2024

    2024 arXiv

  2. [10]

    JAX : composable transformations of P ython+ N um P y programs, 2018

    James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake Vander P las, Skye Wanderman- M ilne, and Qiao Zhang. JAX : composable transformations of P ython+ N um P y programs, 2018. URL http://github.com/...

    2018

  3. [11]

    Openai gym

    Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym. arXiv preprint arXiv:1606.01540, 2016

    2016 arXiv

  4. [12]

    George W. Brown. Iterative solution of games by fictitious play. In T. C. Koopmans (ed.), Activity Analysis of Production and Allocation. Wiley, New York, 1951

    1951

  5. [13]

    In defense of atari: The ale as a benchmark for autorl

    Pablo Samuel Castro. In defense of atari: The ale as a benchmark for autorl. 2024. URL https://icml.cc/virtual/2024/39301

    2024

  6. [14]

    The Zone of Proximal Development in Vygotsky’s Analysis of Learning and Instruction, pp.\ 39–64

    Seth Chaiklin. The Zone of Proximal Development in Vygotsky’s Analysis of Learning and Instruction, pp.\ 39–64. Learning in Doing: Social, Cognitive and Computational Perspectives. Cambridge University Press, 2003

    2003

  7. [15]

    Fast and accurate deep network learning by exponential linear units (elus)

    Djork-Arn \'e Clevert, Thomas Unterthiner, and Sepp Hochreiter. Fast and accurate deep network learning by exponential linear units (elus). arXiv preprint arXiv:1511.07289, 2015

    2015 arXiv

  8. [16]

    Leveraging procedural generation to benchmark reinforcement learning

    Karl Cobbe, Chris Hesse, Jacob Hilton, and John Schulman. Leveraging procedural generation to benchmark reinforcement learning. In International conference on machine learning, pp.\ 2048--2056. PMLR, 2020 a

    2020

  9. [17]

    Leveraging procedural generation to benchmark reinforcement learning

    Karl Cobbe, Chris Hesse, Jacob Hilton, and John Schulman. Leveraging procedural generation to benchmark reinforcement learning. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine L...

    2020

  10. [18]

    Phasic policy gradient

    Karl W Cobbe, Jacob Hilton, Oleg Klimov, and John Schulman. Phasic policy gradient. In International Conference on Machine Learning, pp.\ 2020--2027. PMLR, 2021

    2020

  11. [19]

    Augmenting autotelic agents with large language models

    C \'e dric Colas, Laetitia Teodorescu, Pierre-Yves Oudeyer, Xingdi Yuan, and Marc-Alexandre C \^o t \'e . Augmenting autotelic agents with large language models. In Conference on Lifelong Learning Agents, pp.\ 205--226. PMLR, 2023

    2023

  12. [20]

    Jaxued: A simple and useable ued library in jax

    Samuel Coward, Michael Beukman, and Jakob Foerster. Jaxued: A simple and useable ued library in jax. arXiv preprint arXiv:2403.13091, 2024

    2024 arXiv

  13. [21]

    Emergent complexity and zero-shot transfer via unsupervised environment design

    Michael Dennis, Natasha Jaques, Eugene Vinitsky, Alexandre Bayen, Stuart Russell, Andrew Critch, and Sergey Levine. Emergent complexity and zero-shot transfer via unsupervised environment design. Advances in neural information processing systems, 33: 0 13049--13061, 2020

    2020

  14. [22]

    Watts: Infrastructure for open-ended learning

    Aaron Dharna, Charlie Summers, Rohin Dasari, Julian Togelius, and Amy K Hoover. Watts: Infrastructure for open-ended learning. In ICLR Workshop on Agent Learning in Open-Endedness, 2022

    2022

  15. [23]

    It takes four to tango: Multiagent self play for automatic curriculum generation

    Yuqing Du, Pieter Abbeel, and Aditya Grover. It takes four to tango: Multiagent self play for automatic curriculum generation. In International Conference on Learning Representations, 2023 a

    2023

  16. [24]

    Guiding pretraining in reinforcement learning with large language models

    Yuqing Du, Olivia Watkins, Zihan Wang, C \'e dric Colas, Trevor Darrell, Pieter Abbeel, Abhishek Gupta, and Jacob Andreas. Guiding pretraining in reinforcement learning with large language models. In International Conference on Machine Learning, pp.\ 8657--8677. PMLR, 2023 b

    2023

  17. [25]

    Jeffrey L. Elman. Learning and development in neural networks: the importance of starting small. Cognition, 48 0 (1): 0 71--99, 1993. ISSN 0010-0277. doi:https://doi.org/10.1016/0010-0277(93)90058-4. URL https://www.sciencedirect.com/science/article/pii/0010027793900584

    1993

  18. [26]

    Omni-epic: Open-endedness via models of human notions of interestingness with environments programmed in code

    Maxence Faldor, Jenny Zhang, Antoine Cully, and Jeff Clune. Omni-epic: Open-endedness via models of human notions of interestingness with environments programmed in code. arXiv preprint arXiv:2405.15568, 2024

    2024 arXiv

  19. [27]

    Minedojo: Building open-ended embodied agents with internet-scale knowledge

    Linxi Fan, Guanzhi Wang, Yunfan Jiang, Ajay Mandlekar, Yuncong Yang, Haoyi Zhu, Andrew Tang, De-An Huang, Yuke Zhu, and Anima Anandkumar. Minedojo: Building open-ended embodied agents with internet-scale knowledge. Advances in Neural Information Processing Systems, 35: 0 18343...

    2022

  20. [28]

    Automatic goal generation for reinforcement learning agents

    Carlos Florensa, David Held, Xinyang Geng, and Pieter Abbeel. Automatic goal generation for reinforcement learning agents. In International conference on machine learning, pp.\ 1515--1528. PMLR, 2018

    2018

  21. [29]

    Automated curriculum learning for neural networks

    Alex Graves, Marc G Bellemare, Jacob Menick, Remi Munos, and Koray Kavukcuoglu. Automated curriculum learning for neural networks. In international conference on machine learning, pp.\ 1311--1320. Pmlr, 2017

    2017

  22. [30]

    Minerl: A large-scale dataset of minecraft demonstrations

    William H Guss, Brandon Houghton, Nicholay Topin, Phillip Wang, Cayden Codel, Manuela Veloso, and Ruslan Salakhutdinov. Minerl: A large-scale dataset of minecraft demonstrations. arXiv preprint arXiv:1907.13440, 2019

    1907 arXiv

  23. [31]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor

    Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, pp.\ 1861--1870. Pmlr, 2018

    2018

  24. [32]

    Benchmarking the spectrum of agent capabilities

    Danijar Hafner. Benchmarking the spectrum of agent capabilities. arXiv preprint arXiv:2109.06780, 2021

    2021 arXiv

  25. [33]

    Insights from the neurips 2021 nethack challenge

    Eric Hambro, Sharada Mohanty, Dmitrii Babaev, Minwoo Byeon, Dipam Chakraborty, Edward Grefenstette, Minqi Jiang, Jo Daejin, Anssi Kanervisto, Jongmin Kim, et al. Insights from the neurips 2021 nethack challenge. In NeurIPS 2021 Competitions and Demonstrations Track, pp.\ 41--5...

    2021

  26. [34]

    a schel, Heinrich K \

    Eric Hambro, Roberta Raileanu, Danielle Rothermel, Vegard Mella, Tim Rockt \"a schel, Heinrich K \"u ttler, and Naila Murray. Dungeons and data: A large-scale nethack dataset. Advances in Neural Information Processing Systems, 35: 0 24864--24878, 2022 b

    2022

  27. [35]

    Deep reinforcement learning from self-play in imperfect-information games

    Johannes Heinrich and David Silver. Deep reinforcement learning from self-play in imperfect-information games. arXiv preprint arXiv:1603.01121, 2016

    2016 arXiv

  28. [36]

    Fictitious self-play in extensive-form games

    Johannes Heinrich, Marc Lanctot, and David Silver. Fictitious self-play in extensive-form games. In Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37, ICML'15, pp.\ 805–813. JMLR.org, 2015 a

    2015

  29. [37]

    Fictitious self-play in extensive-form games

    Johannes Heinrich, Marc Lanctot, and David Silver. Fictitious self-play in extensive-form games. In International Conference on Machine Learning, 2015 b . URL https://api.semanticscholar.org/CorpusID:13937012

    2015

  30. [38]

    Exploration via elliptical episodic bonuses

    Mikael Henaff, Roberta Raileanu, Minqi Jiang, and Tim Rockt \"a schel. Exploration via elliptical episodic bonuses. Advances in Neural Information Processing Systems, 35: 0 37631--37646, 2022

    2022

  31. [39]

    Cleanrl: High-quality single-file implementations of deep reinforcement learning algorithms

    Shengyi Huang, Rousslan Fernand JulienDossa Dossa, Chang Ye, Jeff Braga, Dipam Chakraborty, Kinal Mehta, and Jo \ a o GM Ara \'u jo. Cleanrl: High-quality single-file implementations of deep reinforcement learning algorithms. The Journal of Machine Learning Research, 23 0 (1):...

    2022

  32. [40]

    Open rl benchmark: Comprehensive tracked experiments for reinforcement learning

    Shengyi Huang, Quentin Gallou \'e dec, Florian Felten, Antonin Raffin, Rousslan Fernand Julien Dossa, Yanxiao Zhao, Ryan Sullivan, Viktor Makoviychuk, Denys Makoviichuk, Mohamad H Danesh, et al. Open rl benchmark: Comprehensive tracked experiments for reinforcement learning. a...

    2024 arXiv

  33. [41]

    Discovering general reinforcement learning algorithms with adversarial environment design

    Matthew T Jackson, Minqi Jiang, Jack Parker-Holder, Risto Vuorio, Chris Lu, Greg Farquhar, Shimon Whiteson, and Jakob Foerster. Discovering general reinforcement learning algorithms with adversarial environment design. Advances in Neural Information Processing Systems, 36, 2024

    2024

  34. [42]

    Replay-guided adversarial environment design

    Minqi Jiang, Michael Dennis, Jack Parker-Holder, Jakob Foerster, Edward Grefenstette, and Tim Rockt \"a schel. Replay-guided adversarial environment design. Advances in Neural Information Processing Systems, 34: 0 1884--1897, 2021 a

    2021

  35. [43]

    Prioritized level replay

    Minqi Jiang, Edward Grefenstette, and Tim Rockt \"a schel. Prioritized level replay. In International Conference on Machine Learning, pp.\ 4940--4950. PMLR, 2021 b

    2021

  36. [44]

    Dual curriculum design

    Minqi Jiang, Ishita Mediratta, Mikayel Samvelyan, and Jayden Teoh. Dual curriculum design. https://github.com/facebookresearch/dcd, 2022

    2022

  37. [45]

    minimax: Efficient baselines for autocurricula in jax

    Minqi Jiang, Michael Dennis, Edward Grefenstette, and Tim Rockt \"a schel. minimax: Efficient baselines for autocurricula in jax. arXiv preprint arXiv:2311.12716, 2023

    2023 arXiv

  38. [46]

    Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraft

    Ingmar Kanitscheider, Joost Huizinga, David Farhi, William Hebgen Guss, Brandon Houghton, Raul Sampedro, Peter Zhokhov, Bowen Baker, Adrien Ecoffet, Jie Tang, et al. Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraft. arXiv preprint arXiv:2...

    2021 arXiv

  39. [47]

    Kirkpatrick, C

    S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. Optimization by simulated annealing. Science, 220 0 (4598): 0 671--680, 1983. doi:10.1126/science.220.4598.671. URL https://www.science.org/doi/abs/10.1126/science.220.4598.671

    1983 doi

  40. [48]

    Curriculum reinforcement learning via constrained optimal transport

    Pascal Klink, Haoyi Yang, Carlo D'Eramo, Jan Peters, and Joni Pajarinen. Curriculum reinforcement learning via constrained optimal transport. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th Internat...

    2022

  41. [49]

    u ttler, Nantas Nardelli, Thibaut Lavril, Marco Selvatici, Viswanath Sivakumar, Tim Rockt \

    Heinrich K \"u ttler, Nantas Nardelli, Thibaut Lavril, Marco Selvatici, Viswanath Sivakumar, Tim Rockt \"a schel, and Edward Grefenstette. Torchbeast: A pytorch platform for distributed rl. arXiv preprint arXiv:1910.03552, 2019

    1910 arXiv

  42. [50]

    u ttler, Nantas Nardelli, Alexander Miller, Roberta Raileanu, Marco Selvatici, Edward Grefenstette, and Tim Rockt \

    Heinrich K \"u ttler, Nantas Nardelli, Alexander Miller, Roberta Raileanu, Marco Selvatici, Edward Grefenstette, and Tim Rockt \"a schel. The nethack learning environment. Advances in Neural Information Processing Systems, 33: 0 7671--7684, 2020

    2020

  43. [51]

    A unified game-theoretic approach to multiagent reinforcement learning

    Marc Lanctot, Vinicius Zambaldi, Audrunas Gruslys, Angeliki Lazaridou, Karl Tuyls, Julien P \'e rolat, David Silver, and Thore Graepel. A unified game-theoretic approach to multiagent reinforcement learning. Advances in neural information processing systems, 30, 2017

    2017

  44. [52]

    Autocurricula and the emergence of innovation from social interaction: A manifesto for multi-agent intelligence research

    Joel Z Leibo, Edward Hughes, Marc Lanctot, and Thore Graepel. Autocurricula and the emergence of innovation from social interaction: A manifesto for multi-agent intelligence research. arXiv preprint arXiv:1903.00742, 2019

    1903 arXiv

  45. [53]

    Rllib: Abstractions for distributed reinforcement learning

    Eric Liang, Richard Liaw, Robert Nishihara, Philipp Moritz, Roy Fox, Ken Goldberg, Joseph Gonzalez, Michael Jordan, and Ion Stoica. Rllib: Abstractions for distributed reinforcement learning. In International conference on machine learning, pp.\ 3053--3062. PMLR, 2018

    2018

  46. [54]

    Stabilizing unsupervised environment design with a learned adversary

    Ishita Mediratta, Minqi Jiang, Jack Parker-Holder, Michael Dennis, Eugene Vinitsky, and Tim Rockt \"a schel. Stabilizing unsupervised environment design with a learned adversary. arXiv preprint arXiv:2308.10797, 2023 a

    2023 arXiv

  47. [55]

    Stabilizing unsupervised environment design with a learned adversary

    Ishita Mediratta, Minqi Jiang, Jack Parker-Holder, Michael Dennis, Eugene Vinitsky, and Tim Rockt\"aschel. Stabilizing unsupervised environment design with a learned adversary. In Sarath Chandar, Razvan Pascanu, Hanie Sedghi, and Doina Precup (eds.), Proceedings of The 2nd Con...

    2023

  48. [56]

    Active domain randomization

    Bhairav Mehta, Manfred Diaz, Florian Golemo, Christopher J Pal, and Liam Paull. Active domain randomization. In Conference on Robot Learning, pp.\ 1162--1176. PMLR, 2020

    2020

  49. [57]

    moolib: A Platform for Distributed RL

    Vegard Mella, Eric Hambro, Danielle Rothermel, and Heinrich K \" u ttler. moolib: A Platform for Distributed RL . 2022. URL https://github.com/facebookresearch/moolib

    2022

  50. [58]

    Ray: A distributed framework for emerging \ AI \ applications

    Philipp Moritz, Robert Nishihara, Stephanie Wang, Alexey Tumanov, Richard Liaw, Eric Liang, Melih Elibol, Zongheng Yang, William Paul, Michael I Jordan, et al. Ray: A distributed framework for emerging \ AI \ applications. In 13th USENIX symposium on operating systems design a...

    2018

  51. [59]

    Curriculum learning for reinforcement learning domains: A framework and survey

    Sanmit Narvekar, Bei Peng, Matteo Leonetti, Jivko Sinapov, Matthew E Taylor, and Peter Stone. Curriculum learning for reinforcement learning domains: A framework and survey. The Journal of Machine Learning Research, 21 0 (1): 0 7382--7431, 2020

    2020

  52. [60]

    XL and-minigrid: Scalable meta-reinforcement learning environments in JAX

    Alexander Nikulin, Vladislav Kurenkov, Ilya Zisman, Viacheslav Sinii, Artem Agarkov, and Sergey Kolesnikov. XL and-minigrid: Scalable meta-reinforcement learning environments in JAX . In Intrinsically-Motivated and Open-Ended Learning Workshop, NeurIPS2023, 2023. URL https://o...

    2023

  53. [61]

    Solving rubik's cube with a robot hand

    OpenAI, Ilge Akkaya, Marcin Andrychowicz, Maciek Chociej, Mateusz Litwin, Bob McGrew, Arthur Petron, Alex Paino, Matthias Plappert, Glenn Powell, Raphael Ribas, et al. Solving rubik's cube with a robot hand. arXiv preprint arXiv:1910.07113, 2019

    1910 arXiv

  54. [62]

    Asymmetric self-play for automatic goal discovery in robotic manipulation

    OpenAI OpenAI, Matthias Plappert, Raul Sampedro, Tao Xu, Ilge Akkaya, Vineet Kosaraju, Peter Welinder, Ruben D'Sa, Arthur Petron, Henrique P d O Pinto, et al. Asymmetric self-play for automatic goal discovery in robotic manipulation. arXiv preprint arXiv:2101.04882, 2021

    2021 arXiv

  55. [63]

    Bellemare, A\" a ron van den Oord, and R\' e mi Munos

    Georg Ostrovski, Marc G. Bellemare, A\" a ron van den Oord, and R\' e mi Munos. Count-based exploration with neural density models. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML'17, pp.\ 2721–2730. JMLR.org, 2017

    2017

  56. [64]

    Evolving curricula with regret-based environment design

    Jack Parker-Holder, Minqi Jiang, Michael Dennis, Mikayel Samvelyan, Jakob Foerster, Edward Grefenstette, and Tim Rockt \"a schel. Evolving curricula with regret-based environment design. In International Conference on Machine Learning, pp.\ 17473--17498. PMLR, 2022

    2022

  57. [65]

    Efros, and Trevor Darrell

    Deepak Pathak, Pulkit Agrawal, Alexei A. Efros, and Trevor Darrell. Curiosity-driven exploration by self-supervised prediction. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML'17, pp.\ 2778–2787. JMLR.org, 2017 a

    2017

  58. [66]

    Curiosity-driven exploration by self-supervised prediction

    Deepak Pathak, Pulkit Agrawal, Alexei A Efros, and Trevor Darrell. Curiosity-driven exploration by self-supervised prediction. In International conference on machine learning, pp.\ 2778--2787. PMLR, 2017 b

    2017

  59. [67]

    Sample factory: Egocentric 3d control from pixels at 100000 fps with asynchronous reinforcement learning

    Aleksei Petrenko, Zhehui Huang, Tushar Kumar, Gaurav Sukhatme, and Vladlen Koltun. Sample factory: Egocentric 3d control from pixels at 100000 fps with asynchronous reinforcement learning. In International Conference on Machine Learning, pp.\ 7652--7662. PMLR, 2020

    2020

  60. [68]

    Nethack is hard to hack

    Ulyana Piterbarg, Lerrel Pinto, and Rob Fergus. Nethack is hard to hack. Advances in Neural Information Processing Systems, 36, 2024

    2024

  61. [69]

    Teacher algorithms for curriculum learning of deep rl in continuously parameterized environments

    R \'e my Portelas, C \'e dric Colas, Katja Hofmann, and Pierre-Yves Oudeyer. Teacher algorithms for curriculum learning of deep rl in continuously parameterized environments. In Conference on Robot Learning, pp.\ 835--853. PMLR, 2020 a

    2020

  62. [70]

    Automatic curriculum learning for deep rl: A short survey

    R \'e my Portelas, C \'e dric Colas, Lilian Weng, Katja Hofmann, and Pierre-Yves Oudeyer. Automatic curriculum learning for deep rl: A short survey. arXiv preprint arXiv:2003.04664, 2020 b

    2003 arXiv

  63. [71]

    Automated curricula through setter-solver interactions

    Sebastien Racaniere, Andrew K Lampinen, Adam Santoro, David P Reichert, Vlad Firoiu, and Timothy P Lillicrap. Automated curricula through setter-solver interactions. arXiv preprint arXiv:1909.12892, 2019

    1909 arXiv

  64. [72]

    Stable-baselines3: Reliable reinforcement learning implementations

    Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dormann. Stable-baselines3: Reliable reinforcement learning implementations. Journal of Machine Learning Research, 22 0 (268): 0 1--8, 2021. URL http://jmlr.org/papers/v22/20-1364.html

    2021

  65. [73]

    The infinite gaussian mixture model

    Carl Rasmussen. The infinite gaussian mixture model. In S. Solla, T. Leen, and K. M\" u ller (eds.), Advances in Neural Information Processing Systems, volume 12. MIT Press, 1999. URL https://proceedings.neurips.cc/paper_files/paper/1999/file/97d98119037c5b8a9663cb21fb8ebf47-Paper.pdf

    1999

  66. [74]

    Teachmyagent: a benchmark for automatic curriculum learning in deep rl

    Cl \'e ment Romac, R \'e my Portelas, Katja Hofmann, and Pierre-Yves Oudeyer. Teachmyagent: a benchmark for automatic curriculum learning in deep rl. In International Conference on Machine Learning, pp.\ 9052--9063. PMLR, 2021

    2021

  67. [75]

    Toward evolutionary autocurricula: Emergent sociality from inclusive rewards

    Andries Rosseau, Raphael Avalos Martinez de Escobar, and Ann Nowe. Toward evolutionary autocurricula: Emergent sociality from inclusive rewards. In From Cells to Societies: Collective Learning across Scales, 2022. URL https://openreview.net/forum?id=BcUNSzkT-c

    2022

  68. [76]

    No regrets: Investigating and improving regret approximations for curriculum discovery

    Alexander Rutherford, Michael Beukman, Timon Willi, Bruno Lacerda, Nick Hawes, and Jakob Nicolaus Foerster. No regrets: Investigating and improving regret approximations for curriculum discovery. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  69. [77]

    Learning montezuma's revenge from a single demonstration

    Tim Salimans and Richard Chen. Learning montezuma's revenge from a single demonstration. arXiv preprint arXiv:1812.03381, 2018

    2018 arXiv

  70. [78]

    A. L. Samuel. Some studies in machine learning using the game of checkers. IBM Journal of Research and Development, 3 0 (3): 0 210--229, 1959. doi:10.1147/rd.33.0210

    1959 doi

  71. [79]

    Maestro: Open-ended environment design for multi-agent reinforcement learning

    Mikayel Samvelyan, Akbir Khan, Michael D Dennis, Minqi Jiang, Jack Parker-Holder, Jakob Nicolaus Foerster, Roberta Raileanu, and Tim Rockt \"a schel. Maestro: Open-ended environment design for multi-agent reinforcement learning. In The Eleventh International Conference on Lear...

    2022

  72. [80]

    Measuring intelligence through games

    Tom Schaul, Julian Togelius, and J \"u rgen Schmidhuber. Measuring intelligence through games. arXiv preprint arXiv:1109.1314, 2011

    2011 arXiv

  73. [81]

    Trust region policy optimization

    John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. Trust region policy optimization. In Francis Bach and David Blei (eds.), Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research...

    2015

  74. [82]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  75. [83]

    Mastering the game of go with deep neural networks and tree search

    David Silver, Aja Huang, Christopher Maddison, Arthur Guez, Laurent Sifre, George Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine L...

    2016 doi

  76. [84]

    Mastering chess and shogi by self-play with a general reinforcement learning algorithm

    David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al. Mastering chess and shogi by self-play with a general reinforcement learning algorithm. arXiv preprint arXiv:17...

    2017 arXiv

  77. [85]

    P uffer L ib: Making reinforcement learning libraries and environments play nice

    Joseph Suarez. P uffer L ib: Making reinforcement learning libraries and environments play nice. In Agent Learning in Open-Endedness Workshop at NeurIPS '23, 2023

    2023

  78. [86]

    Neural mmo: A massively multiagent game environment for training and evaluating intelligent agents

    Joseph Suarez, Yilun Du, Phillip Isola, and Igor Mordatch. Neural mmo: A massively multiagent game environment for training and evaluating intelligent agents. arXiv preprint arXiv:1903.00784, 2019

    1903 arXiv

  79. [87]

    Neural mmo 2.0: A massively multi-task addition to massively multi-agent learning

    Joseph Suarez, David Bloomin, Kyoung Whan Choe, Hao Xiang Li, Ryan Sullivan, Nishaanth Kanna, Daniel Scott, Rose Shuman, Herbie Bradley, Louis Castricato, et al. Neural mmo 2.0: A massively multi-task addition to massively multi-agent learning. Advances in Neural Information P...

    2024

  80. [88]

    Intrinsic motivation and automatic curricula via asymmetric self-play

    Sainbayar Sukhbaatar, Zeming Lin, Ilya Kostrikov, Gabriel Synnaeve, Arthur Szlam, and Rob Fergus. Intrinsic motivation and automatic curricula via asymmetric self-play. In 6th International Conference on Learning Representations, ICLR 2018, 2018

    2018

  81. [89]

    On bonus-based exploration methods in the arcade learning environment

    Adrien Ali Taiga, William Fedus, Marlos C Machado, Aaron Courville, and Marc G Bellemare. On bonus-based exploration methods in the arcade learning environment. arXiv preprint arXiv:2109.11052, 2021

    2021 arXiv

  82. [90]

    Taylor and Peter Stone

    Matthew E. Taylor and Peter Stone. Transfer learning for reinforcement learning domains: A survey. J. Mach. Learn. Res., 10: 0 1633–1685, December 2009. ISSN 1532-4435

    2009

  83. [91]

    Human-timescale adaptation in an open-ended task space

    Adaptive Agent Team, Jakob Bauer, Kate Baumli, Satinder Baveja, Feryal Behbahani, Avishkar Bhoopchand, Nathalie Bradley-Schmieg, Michael Chang, Natalie Clay, Adrian Collister, et al. Human-timescale adaptation in an open-ended task space. arXiv preprint arXiv:2301.07608, 2023

    2023 arXiv

  84. [92]

    Pettingzoo: Gym for multi-agent reinforcement learning

    JK Terry, Benjamin J Black, Nathaniel Grammel, Mario Jayakumar, Ananth Hari, Ryan Sullivan, Luis Santos, Clemens Dieffendahl, Caroline Horsch, Rodrigo De Lazcano Perez-Vicente, et al. Pettingzoo: Gym for multi-agent reinforcement learning. In Advances in Neural Information Pro...

    2021

  85. [93]

    Temporal difference learning and td-gammon

    Gerald Tesauro. Temporal difference learning and td-gammon. Commun. ACM, 38 0 (3): 0 58–68, March 1995. ISSN 0001-0782. doi:10.1145/203330.203343. URL https://doi.org/10.1145/203330.203343

    1995

  86. [94]

    Domain randomization for transferring deep neural networks from simulation to the real world

    Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world. In 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp.\ 23--3...

    2017

  87. [95]

    Gymnasium: A standard interface for reinforcement learning environments

    Mark Towers, Ariel Kwiatkowski, Jordan Terry, John U Balis, Gianluca De Cola, Tristan Deleu, Manuel Goulao, Andreas Kallinteris, Markus Krimmel, Arjun KG, et al. Gymnasium: A standard interface for reinforcement learning environments. arXiv preprint arXiv:2407.17032, 2024

    2024 arXiv

  88. [96]

    Scaling laws for imitation learning in nethack

    Jens Tuyls, Dhruv Madeka, Kari Torkkola, Dean Foster, Karthik Narasimhan, and Sham Kakade. Scaling laws for imitation learning in nethack. arXiv preprint arXiv:2307.09423, 2023

    2023 arXiv

  89. [97]

    Proximal curriculum for reinforcement learning agents

    George Tzannetos, B \'a rbara Gomes Ribeiro, Parameswaran Kamalaruban, and Adish Singla. Proximal curriculum for reinforcement learning agents. Transactions on Machine Learning Research, 2023 0 (5): 0 1--21, 2023

    2023

  90. [98]

    Czarnecki, Micha \"e l Mathieu, Andrew Dudzik, Junyoung Chung, David H

    Oriol Vinyals, Igor Babuschkin, Wojciech M. Czarnecki, Micha \"e l Mathieu, Andrew Dudzik, Junyoung Chung, David H. Choi, Richard Powell, Timo Ewalds, Petko Georgiev, Junhyuk Oh, Dan Horgan, Manuel Kroiss, Ivo Danihelka, Aja Huang, L. Sifre, Trevor Cai, John P. Agapiou, Max Ja...

    2019

  91. [99]

    L. S. Vygotsky. Mind in Society: Development of Higher Psychological Processes. Harvard University Press, 1978. ISBN 9780674576285. URL http://www.jstor.org/stable/j.ctvjf9vz4

    1978

  92. [100]

    Paired open-ended trailblazer (poet): Endlessly generating increasingly complex and diverse learning environments and their solutions

    Rui Wang, Joel Lehman, Jeff Clune, and Kenneth O Stanley. Paired open-ended trailblazer (poet): Endlessly generating increasingly complex and diverse learning environments and their solutions. arXiv preprint arXiv:1901.01753, 2019

    1901 arXiv

  93. [101]

    Enhanced poet: open-ended reinforcement learning through unbounded invention of learning challenges and their solutions

    Rui Wang, Joel Lehman, Aditya Rawal, Jiale Zhi, Yulun Li, Jeff Clune, and Kenneth O Stanley. Enhanced poet: open-ended reinforcement learning through unbounded invention of learning challenges and their solutions. In Proceedings of the 37th International Conference on Machine ...

    2020

  94. [102]

    Outracing champion gran turismo drivers with deep reinforcement learning

    Peter Wurman, Samuel Barrett, Kenta Kawamoto, James MacGlashan, Kaushik Subramanian, Thomas Walsh, Roberto Capobianco, Alisa Devlic, Franziska Eckert, Florian Fuchs, Leilani Gilpin, Piyush Khandelwal, Varun Kompella, HaoChih Lin, Patrick MacAlpine, Declan Oller, Takuma Seno, C...

    2022 doi

  95. [103]

    Rlexplore: Accelerating research in intrinsically-motivated reinforcement learning

    Mingqi Yuan, Roger Creus Castanyer, Bo Li, Xin Jin, Glen Berseth, and Wenjun Zeng. Rlexplore: Accelerating research in intrinsically-motivated reinforcement learning. arXiv preprint arXiv:2405.19548, 2024

    2024 arXiv

  96. [104]

    Omni: Open-endedness via models of human notions of interestingness

    Jenny Zhang, Joel Lehman, Kenneth Stanley, and Jeff Clune. Omni: Open-endedness via models of human notions of interestingness. arXiv preprint arXiv:2306.01711, 2023

    2023 arXiv

  97. [105]

    A survey on self-play methods in reinforcement learning

    Ruize Zhang, Zelai Xu, Chengdong Ma, Chao Yu, Wei-Wei Tu, Shiyu Huang, Deheng Ye, Wenbo Ding, Yaodong Yang, and Yu Wang. A survey on self-play methods in reinforcement learning. arXiv preprint arXiv:2408.01072, 2024

    2024

  98. [106]

    Transfer learning in deep reinforcement learning: A survey

    Zhuangdi Zhu, Kaixiang Lin, Anil K Jain, and Jiayu Zhou. Transfer learning in deep reinforcement learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023

    2023

  99. [107]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.