REVIEW 5 cited by
Evolving Curricula with Regret-Based Environment Design
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
It remains a significant challenge to train generally capable agents with reinforcement learning (RL). A promising avenue for improving the robustness of RL agents is through the use of curricula. One such class of methods frames environment design as a game between a student and a teacher, using regret-based objectives to produce environment instantiations (or levels) at the frontier of the student agent's capabilities. These methods benefit from their generality, with theoretical guarantees at equilibrium, yet they often struggle to find effective levels in challenging design spaces. By contrast, evolutionary approaches seek to incrementally alter environment complexity, resulting in potentially open-ended learning, but often rely on domain-specific heuristics and vast amounts of computational resources. In this paper we propose to harness the power of evolution in a principled, regret-based curriculum. Our approach, which we call Adversarially Compounding Complexity by Editing Levels (ACCEL), seeks to constantly produce levels at the frontier of an agent's capabilities, resulting in curricula that start simple but become increasingly complex. ACCEL maintains the theoretical benefits of prior regret-based methods, while providing significant empirical gains in a diverse set of environments. An interactive version of the paper is available at accelagent.github.io.
Forward citations
Cited by 5 Pith papers
-
Anchored Self-Play for Code Repair
Anchored Self-Play uses a small reference bug set and code-embedding similarity to stop generator–fixer self-play from drifting into unrealistic bugs, raising average fix rate by 7.0 points on BugSourceBench.
-
Mitigating Goal Misgeneralization via Minimax Regret
Minimax expected regret training provably prevents goal misgeneralization in fully observable level-conditioned environments, while maximum expected value training provably permits it when distinguishing levels are rare.
-
An Optimisation Framework for Unsupervised Environment Design
UED is reframed as an entropy-regularized minimax problem with two-timescale gradient convergence guarantees for zero-sum scores, and a generalized learnability score improves robustness on three benchmarks, with the ...
-
Improving Environment Novelty Quantification for Effective Unsupervised Environment Design
CENIE augments regret-based unsupervised environment design with a GMM-based novelty score derived from the student's state-action coverage, improving zero-shot transfer in Minigrid, BipedalWalker, and CarRacing.
-
cMALC-D: Contextual Multi-Agent LLM-Guided Curriculum Learning with Diversity-Based Context Blending
cMALC-D uses an LLM to generate training contexts for multi-agent RL and a diversity-blending mechanism to avoid mode collapse, claiming improved generalization on traffic signal control.
Discussion (0). Continue with ORCID to comment.