PettingZoo: Gym for Multi-Agent Reinforcement Learning
read the original abstract
This paper introduces the PettingZoo library and the accompanying Agent Environment Cycle ("AEC") games model. PettingZoo is a library of diverse sets of multi-agent environments with a universal, elegant Python API. PettingZoo was developed with the goal of accelerating research in Multi-Agent Reinforcement Learning ("MARL"), by making work more interchangeable, accessible and reproducible akin to what OpenAI's Gym library did for single-agent reinforcement learning. PettingZoo's API, while inheriting many features of Gym, is unique amongst MARL APIs in that it's based around the novel AEC games model. We argue, in part through case studies on major problems in popular MARL environments, that the popular game models are poor conceptual models of games commonly used in MARL and accordingly can promote confusing bugs that are hard to detect, and that the AEC games model addresses these problems.
This paper has not been read by Pith yet.
Forward citations
Cited by 5 Pith papers
-
NePPO: Near-Potential Policy Optimization for General-Sum Multi-Agent Reinforcement Learning
NePPO learns a player-independent potential function via a novel objective whose minimization yields an approximate Nash equilibrium for general-sum multi-agent games.
-
Reward hacking in physical reinforcement learning revealed by turbulent drag reduction
Reward hacking in RL for wall-turbulence control is exposed through mass-conserving projection, memoryless policies, and pressure-gradient rewards; fixes yield honest 17% drag reduction.
-
Scalable Neighborhood-Based Multi-Agent Actor-Critic
MADDPG-K scales centralized critics in multi-agent RL by limiting each critic to k-nearest neighbors under Euclidean distance, yielding constant input size and competitive performance.
-
Physics-guided surrogate learning enables zero-shot control of turbulent wings
Zero-shot RL control trained on matched channel flows reduces skin-friction drag 28.7% and total drag 10.7% on a NACA4412 wing, outperforming opposition control.
-
Enhancing the MADDPG Algorithm for Multi-Agent Learning via Action Inference and Importance Sampling
Action inference and geometric importance sampling enhance MADDPG, yielding better stability, cooperation, and exploration efficiency on the discrete Predator-Prey benchmark.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.