Pith. sign in

REVIEW 3 major objections 5 minor 33 references

Promoting Coordination through Policy Regularization in Multi-Agent Deep Reinforcement Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Coordinating exploration through policy regularizers improves sparse-reward multi-agent reinforcement learning, with synchronized sub-policy selection the most consistent.

desk verdict CoachReg is a genuinely new coordination mechanism with useful results, but the headline margins are undercut by an asymmetric tuning budget and the absence of significance tests. read the letter →

arxiv 1908.02269 v4 pith:JMZWHW6A submitted 2019-08-06 cs.LG cs.MAstat.ML

Julien Roy , Paul Barde , Félix G. Harvey , Derek Nowrouzezahrai , Christopher Pal This is my paper · ORCID
classification cs.LGcs.MAstat.ML
keywords multi-agentreinforcementlearningcoordinationpolicyregularizationMADDPGcentralizedtrainingdecentralizedexecutionsparserewardsmasksagentmodelling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that coordinated exploration, not just centralized critics, is what makes cooperative multi-agent reinforcement learning work under sparse rewards, and that coordination can be injected as a policy regularizer. It proposes TeamReg, which trains each agent to predict teammates' actions and to act predictably, and CoachReg, which trains a central coach to pick a shared sub-policy mask that agents learn to match from their local observations. On four continuous-control tasks and a discrete-action football task, CoachReg consistently improves final returns over MADDPG and ablations, while TeamReg helps on purely cooperative tasks but can destabilize mixed tasks. If right, this means hard cooperative tasks can be made learnable without task-specific reward shaping.

What carries the argument

The carrier of CoachReg is the policy mask: a one-hot vector of size K (fixed at 4) that modulates the first hidden layer of an agent's policy network through structured dropout, with only units whose index is congruent to the selected mask kept. A coach network, used only in training, outputs a mask distribution from joint observations; each agent has a linear layer producing its own mask distribution from local observations. The coach and agents are tied by two objectives, a KL divergence that makes agents' masks match the coach's, and a policy-gradient objective evaluated under the coach's mask, and the sampled masks are made differentiable via Gumbel-softmax. TeamReg's carrier is the team-spirit objective pair: each agent predicts teammates' actions from its own observation, and is regularized to be predictable in return. These mechanisms are what inject the coordination inductive bias into MADDPG.

What would settle it

A decisive check would rerun the comparison with every baseline given the same number of free tuning parameters, and would replace CoachReg's coach with a random but shared mask schedule; if the margin disappears or the random schedule matches CoachReg, the claimed mechanism is not what drives the result.

Watch

Extended reading notes

Core claim

Adding coordination-promoting regularizers to MADDPG biases exploration toward joint behaviors and raises final returns on several sparse-reward cooperative tasks. CoachReg, the strongest method, enforces synchronous sub-policy selection: a training-only coach chooses a policy mask from the joint observation, agents are regularized to predict the same mask from their own observations, and both coach and agents are optimized so that following the coach's mask yields high returns. At test time the coach is removed and agents still switch masks synchronously. The paper reports, for example, 0.088±0.017 average return on the 3v2 football task versus 0.004–0.006 for baselines, and shows that unsynchronized policy masks alone do not match CoachReg. TeamReg's gains are presented as task-dependent, with a failure mode in the only task with a competitive component.

Load-bearing premise

The load-bearing premise is that choosing each algorithm's best of 50 random hyperparameter settings isolates algorithmic merit, even though CoachReg and TeamReg have three extra tuning weights, and that CoachReg's gains come from mask synchronization rather than from the extra coach network itself.

Editorial extensions

If this is right

  • Sparse-reward cooperative tasks that currently require hand-designed reward shaping or curricula can be approached by adding a synchronization regularizer to an off-the-shelf CTDE algorithm.
  • Decentralized execution is preserved: the coach and extra prediction heads are used only during training and can be removed at deployment.
  • Unsynchronized policy masks are not a substitute; the paper's ablation shows mask structure alone rarely beats plain MADDPG, so the synchronous selection is the operative ingredient.
  • TeamReg should be restricted to fully cooperative settings; on tasks with an adversarial component it can produce a dominant agent and lower average return.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The tuning-flexibility caveat means the reported margins could shrink if baselines were given the same number of free hyperparameters; a matched-tuning re-run would settle this.
  • The discrete mask mechanism suggests CoachReg could plausibly compose with hierarchical or options-based RL to learn temporally extended joint strategies rather than per-step switches.
  • A random-shared-mask control would test whether synchronization itself, rather than situation recognition by the coach, is the active ingredient in CoachReg's gains.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper argues that coordinating agents' policies can guide exploration in multi-agent reinforcement learning and proposes two policy regularization methods, TeamReg and CoachReg, that augment CTDE algorithms (exemplified by MADDPG). TeamReg adds an inter-agent action predictability objective (Eqs. 3-5); CoachReg adds a centralized coach that selects a shared policy mask that agents are regularized to predict and follow (Eqs. 6-10). The methods are evaluated on four sparse-reward continuous-control tasks and on Google Research Football 3v2, with hyperparameter search (50 configurations, 3 seeds; 10 seeds for final comparison), ablations, and behavioral analysis of masks and predictability. The paper reports that CoachReg consistently improves performance and TeamReg improves performance on two tasks but is harmful on the adversarial COMPROMISE task.

Significance. The contribution is potentially significant: if the reported gains are real, the paper provides simple, generally applicable regularizers that can be plugged into existing MARL algorithms, with careful behavioral validation that the proxies (predictability, mask synchronization) are indeed enforced. The experimental effort is substantial for the era (3 CPU-years, reproducibility-oriented appendices, scaling experiments to 6 agents). The toy Markov game in Section 3 provides an intuitive motivating result. However, the headline empirical claims require stronger statistical support and a tuning-budget-neutral comparison to rule out that the gains are an artifact of the extra hyperparameters searched for the proposed methods.

major comments (3)
  1. [Appendix E.1-E.2, Table 1] The comparison between methods is not neutral to tuning budget: TeamReg and CoachReg have three additional hyperparameters (lambda_1, lambda_2, lambda_3) that are included in the per-environment best-of-50 search, while the ablations and baselines either fix or lack these coefficients. The headline numbers in Table 1 and Table 2 are therefore selected from a larger effective configuration space, and the reported margins may reflect tuning flexibility rather than a robust algorithmic advantage. I request a nested validation protocol (e.g., selecting hyperparameters on a separate subset of seeds, or an equal-query budget per algorithm) or an explicit demonstration that the conclusion is insensitive to the number of tuning dimensions.
  2. [Section 7.1, Table 1; Section 7.4, Table 2] The paper makes repeated claims of 'significant' improvements without providing significance tests or effect-size statistics. For example, on SPREAD the CoachReg return (210±12) is statistically indistinguishable from MADDPG+policy mask (221±11) and TeamReg (216±12); on CHASE the CoachReg and MADDPG+sharing intervals overlap; and on BOUNCE the gap between CoachReg (7.4±1.2) and MADDPG (4.0±1.6) is underpowered with only 10 seeds. The football result (Table 2) similarly reports one nonzero mean with no test across seeds. Please add per-task significance tests (e.g., paired bootstrap over the 10 seeds) and adjust the wording of the claims accordingly.
  3. [Section 7.4, Table 2] The football evaluation does not include the two ablation variants (MADDPG+agent modelling and MADDPG+policy mask), so it is not possible to determine whether the CoachReg gain comes from synchronous mask selection rather than the coach network or the mask architecture alone. The paper's own Section 7.3 shows the policy-mask ablation is an important control; without it on football, the discrete-action generalization claim is incomplete.
minor comments (5)
  1. [Equations (5), (9)] The symbols lambda_1, lambda_2, and lambda_3 are reused for different objectives in TeamReg and CoachReg; please use distinct symbols or clearly state in the text that the coefficients play analogous but not identical roles.
  2. [Section 4.2, Equation (6)] The paper states that the categorical mask distribution is reparameterized with the Gumbel-softmax, but the precise estimator (e.g., straight-through versus soft relaxation) is not specified; this matters for the gradient of the mask sampling step.
  3. [Appendix E.3, Tables 3-6] The hyperparameter tables contain formatting glitches (e.g., '0 .71', '0 .083') and should be cleaned; additionally, the large relative critic learning rates (omega_phi values such as 53 and 79) deserve a brief explanation in the training details.
  4. [Figure 1] In Figure 1, the notation 'a=(1,1)' and the dashed versus solid line legend are not explained in the caption; the reader must infer the coordinated versus unconstrained action spaces.
  5. [Appendix G.3] The Hamming proximity for 'MADDPG + policy mask' is maximized over all mask equivalences, which can inflate the reported proximity; this procedure should be stated in the main text where the ablation comparison is mentioned.

Circularity Check

1 steps flagged · score 2.0 of 10

Minor self-confirmatory proxy validation; central claims rest on external comparisons.

  1. self definitional [Section 7.2 ('Effects of enforcing predictable behavior'); Section 7.3 ('Analysis of synchronous sub-policy selection'); Eqs. (3)-(5) and (8)]
    "Here we validate that enforcing predictability makes the agent-modelling task more successful. To this end, we compare, on the SPREAD environment, the team-spirit losses between TeamReg and its ablated versions."

    The 'team-spirit loss' is exactly the objective J_{i,j}^{TS} defined in Eq. (3) and added to the total objective in Eq. (5) with weights lambda_1 and lambda_2. Comparing this loss between TeamReg and versions where those weights are zero shows only that the regularizer can reduce its own training objective; it does not independently establish that predictability causes coordination or higher returns. Section 7.3 repeats the pattern differently: mask entropy and Hamming proximity measure the synchronization that CoachReg's KL objective in Eq. (8) explicitly trains. These analyses are sanity checks rather than fitted predictions, and the paper's headline conclusions are supported by external return comparisons, so this circularity is minor and not load-bearing.

full rationale

The central derivation chain is not circular. TeamReg and CoachReg are auxiliary policy objectives added to MADDPG (Eqs. (3)-(5) and (7)-(10)); the paper does not define final returns as those objectives. Its primary evidence is an external comparison against vanilla MADDPG, DDPG, parameter sharing, and two ablations on four sparse-reward particle tasks and Google Research Football, with learning curves and final returns in Figure 5 and Tables 1-2. The ablations (agent modelling with lambda_2 = 0; policy mask with lambda_{1,2,3} = 0) control for the main architectural additions. The paper uses no load-bearing self-citation chain and imports no uniqueness theorem from the authors' prior work; MADDPG and GRF are cited as external baselines. The one mild self-confirmatory element is the proxy-validation analysis in Sections 7.2-7.3, where the metrics reported are the same quantities the regularizers optimize; that is a sanity check, not a fitted prediction, and the final-return evaluation is independent of it. The per-environment best-of-50 hyperparameter selection is a model-selection fairness concern rather than a circularity: TeamReg and CoachReg tune extra lambda coefficients, but the reported returns are not by construction equal to those fitted values. Overall, no significant circularity; score 2 reflects only the minor self-confirmatory framing of the proxy analyses.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The ledger contains four free hyperparameters (lambda_1, lambda_2, lambda_3, K) that are tuned per environment or fixed by hand and that affect the strength of the proposed regularizers. The axioms are the standard MARL framework assumption plus three domain-specific premises: MADDPG as a valid baseline, the paper's own coordination proxies as meaningful inductive biases, the transferability of the toy-game motivation to deep RL, and the practical validity of Gumbel-softmax gradients. No new physical or ontological entities are introduced; the coach network is an architectural component, not an invented entity with an external falsifiable handle.

free parameters (4)
  • lambda_1 (TeamReg prediction weight, CoachReg mask-matching weight) = Range 0.0013-0.65 across environments (Tables 3-7)
    Selected per environment by random hyperparameter search; weighs how well an agent predicts teammates (TeamReg) or matches the coach mask (CoachReg).
  • lambda_2 (predictability / mask-following weight) = Range 0.0026-0.86 across environments
    Controls how predictable an agent must be for teammates (TeamReg) or how strongly the agent follows the coach-selected mask (CoachReg); tuned per environment.
  • lambda_3 (coach KL objective weight) = Range 0.34-8.4 across environments
    Weights the coach's own KL divergence objective in CoachReg; tuned per environment.
  • K (number of policy masks) = 4 (fixed)
    Hand-chosen fixed hyperparameter for CoachReg, stated in Appendix C; the results depend on K being adequate for each task.
assumptions (4)
  • domain assumption MADDPG is a sound baseline whose centralized critics can in principle identify coordinated joint actions.
    The methods are implemented as regularizers on top of MADDPG; if MADDPG's centralized critics were fundamentally broken, the comparison would not isolate the effect of coordination biases. Section 2.2.
  • ad hoc to paper Predictability of teammate actions (TeamReg) and synchronous switching between shared policy masks (CoachReg) are valid proxies for coordination.
    This is the paper's central inductive-bias hypothesis, introduced in Sections 4.1 and 4.2. The regularizers enforce these proxies by construction; the experiments interpret improved benchmarks as supporting the proxies.
  • domain assumption The variance-reduction motivation demonstrated in the tabular toy game (Section 3) transfers to deep function approximators and continuous control.
    Section 3 shows faster Q-learning in a tabular chain game with a hard-coded coordination module; the deep RL experiments assume the same coordination bias helps with neural policies, which is not proven analytically.
  • domain assumption Gumbel-softmax with temperature 1 provides a usable reparameterized gradient for the sampled one-hot policy masks.
    Section 4.2 uses Gumbel-softmax to backpropagate through mask sampling; no analysis of estimator bias or variance is provided, so the training dynamics rest on this approximation working in practice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Promoting Coordination through Policy Regularization in Multi-Agent Deep Reinforcement Learning." pith.science (2026). https://pith.science/paper/JMZWHW6A

@misc{pith2026190802269,
  author       = {Pith},
  title        = {Pith review of: Promoting Coordination through Policy Regularization in Multi-Agent Deep Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JMZWHW6A}},
  note         = {Machine review of arXiv:1908.02269}
}
read the original abstract

In multi-agent reinforcement learning, discovering successful collective behaviors is challenging as it requires exploring a joint action space that grows exponentially with the number of agents. While the tractability of independent agent-wise exploration is appealing, this approach fails on tasks that require elaborate group strategies. We argue that coordinating the agents' policies can guide their exploration and we investigate techniques to promote such an inductive bias. We propose two policy regularization methods: TeamReg, which is based on inter-agent action predictability and CoachReg that relies on synchronized behavior selection. We evaluate each approach on four challenging continuous control tasks with sparse rewards that require varying levels of coordination as well as on the discrete action Google Research Football environment. Our experiments show improved performance across many cooperative multi-agent problems. Finally, we analyze the effects of our proposed methods on the policies that our agents learn and show that our methods successfully enforce the qualities that we propose as proxies for coordinated behaviors.

Figures

Figures reproduced from arXiv: 1908.02269 by the authors.

Figure 1
Figure 1. (Top) The tabular Q-learning agents learn much more efficiently when constrained to the space of coordinated policies (solid lines) than in the original action space (dashed lines). (Bottom) Simple Markov Game consisting of a chain of length L leading to a terminal state (in grey). Agents can be seen as the two wheels of a vehicle so that their actions need to be in agreement for the vehicle to move. The detailed ex… view at source ↗
Figure 2
Figure 2. Illustration of TeamReg with two agents. Each agent’s policy is equipped with additional heads that are trained to predict other agents’ actions and every agent is regularized to produce actions that its teammates correctly predict. The method is depicted for agent 1 only to avoid cluttering. This first approach aims at exploiting the structure present in the joint action space of coordinated policies to attain a ce… view at source ↗
Figure 3
Figure 3. Illustration of CoachReg with two agents. A central model, the coach, takes all agents’ observations as input and outputs the current mode (policy mask). Agents are regularized to predict the same mask from their local observations and optimize the corresponding sub-policy. Sub-policy selection Firstly, to enable explicit sub￾behavior selection, we propose the use of policy masks as a means to modulate the agents’ p… view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Multi-agent tasks we employ. (a) SPREAD: Agents must spread out and cover a set of [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Learning curves (mean return over agents) for our two proposed algorithms, two ablations [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 7
Figure 7. Figure 7: (Left) Average entropy of the policy mask [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Snapshot of the google research football 3vs1-with-keeper. We evaluate our techniques on the more challenging task of 3vs2 Google Research football environment [20]. In this environment, each agent controls an offensive player and tries to score against a defensive pla…
Figure 9
Figure 9. Figure 9: Hyper-parameter tuning results for all algorithms. There is one distribution per [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Average performance differ￾ence (∆perf ) between the two agents in COMPROMISE for each 150 runs of the hyper-parameter searches (left). All occurrences of abnormally high per￾formance difference are associated with high values of λ2 (right). The results presented in …
Figure 11
Figure 11. Figure 11: Learning curves for TeamReg and the three baselines on COMPROMISE. We see that [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]
Figure 12
Figure 12. Figure 12: Agent’s policy mask distributions. For each [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]
Figure 13
Figure 13. Figure 13: Visualization of two different BOUNCE evaluation episodes. Note that here, the agents’ [PITH_FULL_IMAGE:figures/full_fig_p021_13.png]
Figure 14
Figure 14. Figure 14: Visualization of sequences on two different environments. An agent’s color represent its [PITH_FULL_IMAGE:figures/full_fig_p021_14.png]
Figure 15
Figure 15. Figure 15: (Left) Entropy of the policy mask distributions for each task and method, averaged over [PITH_FULL_IMAGE:figures/full_fig_p022_15.png]
Figure 16
Figure 16. Figure 16: Learning curves (mean return over agents) for all algorithms on the SPREAD environment [PITH_FULL_IMAGE:figures/full_fig_p023_16.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 21 canonical work pages

  1. [1]

    Feudal multi-agent hierarchies for cooperative reinforcement learning

    Sanjeevan Ahilan and Peter Dayan. Feudal multi-agent hierarchies for cooperative reinforcement learning. arXiv preprint arXiv:1901.08492, 2019

    arXiv 1901

  2. [2]

    Layer normalization

    Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450, 2016

    arXiv 2016

  3. [3]

    The option-critic architecture

    Pierre-Luc Bacon, Jean Harb, and Doina Precup. The option-critic architecture. In Thirty-First AAAI Conference on Artificial Intelligence, 2017

    work page 2017

  4. [4]

    Measuring collaborative emergent behavior in multi-agent reinforcement learning

    Sean L Barton, Nicholas R Waytowich, Erin Zaroukian, and Derrik E Asher. Measuring collaborative emergent behavior in multi-agent reinforcement learning. In International Conference on Human Systems Engineering and Design: Future Trends and Applications , pages 422–427. Springer, 2018

    work page 2018

  5. [5]

    Intrinsically motivated reinforcement learning

    Nuttapong Chentanez, Andrew G Barto, and Satinder P Singh. Intrinsically motivated reinforcement learning. In Advances in neural information processing systems, pages 1281– 1288, 2005

    work page 2005

  6. [6]

    Learning to communicate with deep multi-agent reinforcement learning

    Jakob Foerster, Ioannis Alexandros Assael, Nando de Freitas, and Shimon Whiteson. Learning to communicate with deep multi-agent reinforcement learning. In Advances in Neural Information Processing Systems, pages 2137–2145, 2016

    2016

  7. [7]

    Bayesian action decoder for deep multi-agent reinforcement learning

    Jakob Foerster, Francis Song, Edward Hughes, Neil Burch, Iain Dunning, Shimon Whiteson, Matthew Botvinick, and Michael Bowling. Bayesian action decoder for deep multi-agent reinforcement learning. International Conference on Machine Learning, 2019

    work page 2019

  8. [8]

    Counterfactual multi-agent policy gradients

    Jakob N Foerster, Gregory Farquhar, Triantafyllos Afouras, Nantas Nardelli, and Shimon Whiteson. Counterfactual multi-agent policy gradients. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018

    work page 2018

Show all 33 references
  1. [9]

    Gupta, Maxim Egorov, and Mykel J

    Jayesh K. Gupta, Maxim Egorov, and Mykel J. Kochenderfer. Cooperative multi-agent control using deep reinforcement learning. In AAMAS Workshops, 2017. 10

    2017

  2. [10]

    Opponent modeling in deep reinforcement learning

    He He, Jordan Boyd-Graber, Kevin Kwok, and Hal Daumé III. Opponent modeling in deep reinforcement learning. In International Conference on Machine Learning, pages 1804–1813, 2016

    2016

  3. [11]

    Is multiagent deep reinforcement learning the answer or the question? a brief survey

    Pablo Hernandez-Leal, Bilal Kartal, and Matthew E Taylor. Is multiagent deep reinforcement learning the answer or the question? a brief survey. arXiv preprint arXiv:1810.05587, 2018

    2018 arXiv

  4. [12]

    Pablo Hernandez-Leal, Bilal Kartal, and Matthew E. Taylor. Agent Modeling as Auxiliary Task for Deep Reinforcement Learning. In AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 2019

    2019

  5. [13]

    A deep policy inference q-network for multi-agent systems

    Zhang-Wei Hong, Shih-Yang Su, Tzu-Yun Shann, Yi-Hsiang Chang, and Chun-Yi Lee. A deep policy inference q-network for multi-agent systems. arXiv preprint arXiv:1712.07893, 2017

    2017 arXiv

  6. [14]

    Actor-attention-critic for multi-agent reinforcement learning

    Shariq Iqbal and Fei Sha. Actor-attention-critic for multi-agent reinforcement learning. In International Conference on Machine Learning, pages 2961–2970, 2019

    2019

  7. [15]

    Categorical reparametrization with gumble-softmax

    Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparametrization with gumble-softmax. In International Conference on Learning Representations (ICLR 2017). OpenReview. net, 2017

    2017

  8. [16]

    Social influence as intrinsic motivation for multi-agent deep reinforcement learning

    Natasha Jaques, Angeliki Lazaridou, Edward Hughes, Caglar Gulcehre, Pedro Ortega, Dj Strouse, Joel Z Leibo, and Nando De Freitas. Social influence as intrinsic motivation for multi-agent deep reinforcement learning. In International Conference on Machine Learning, pages 3040–3049, 2019

    2019

  9. [17]

    Learning attentional communication for multi-agent cooperation

    Jiechuan Jiang and Zongqing Lu. Learning attentional communication for multi-agent cooperation. In Advances in Neural Information Processing Systems, pages 7254–7264, 2018

    2018

  10. [18]

    Reinforcement learning: A survey

    Leslie Pack Kaelbling, Michael L Littman, and Andrew W Moore. Reinforcement learning: A survey. Journal of artificial intelligence research, 4:237–285, 1996

    1996

  11. [19]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  12. [20]

    Google research football: A novel reinforcement learning environment

    Karol Kurach, Anton Raichuk, Piotr Sta´nczyk, Michał Zajac, Olivier Bachem, Lasse Espeholt, Carlos Riquelme, Damien Vincent, Marcin Michalski, Olivier Bousquet, et al. Google research football: A novel reinforcement learning environment. arXiv preprint arXiv:1907.11180, 2019

    1907 arXiv

  13. [21]

    Multi-agent cooperation and the emergence of (natural) language

    Angeliki Lazaridou, Alexander Peysakhovich, and Marco Baroni. Multi-agent cooperation and the emergence of (natural) language. arXiv preprint arXiv:1612.07182, 2016

    2016 arXiv

  14. [22]

    Continuous control with deep reinforcement learning

    Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971, 2015

    2015 arXiv

  15. [23]

    Markov games as a framework for multi-agent reinforcement learning

    Michael L Littman. Markov games as a framework for multi-agent reinforcement learning. In Machine learning proceedings 1994, pages 157–163. Elsevier, 1994

    1994

  16. [24]

    Multi- agent actor-critic for mixed cooperative-competitive environments

    Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, OpenAI Pieter Abbeel, and Igor Mordatch. Multi- agent actor-critic for mixed cooperative-competitive environments. In Advances in Neural Information Processing Systems, pages 6379–6390, 2017

    2017

  17. [25]

    Maven: Multi- agent variational exploration

    Anuj Mahajan, Tabish Rashid, Mikayel Samvelyan, and Shimon Whiteson. Maven: Multi- agent variational exploration. In Advances in Neural Information Processing Systems, pages 7613–7624, 2019

    2019

  18. [26]

    Emergence of grounded compositional language in multi- agent populations

    Igor Mordatch and Pieter Abbeel. Emergence of grounded compositional language in multi- agent populations. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018

    2018

  19. [27]

    Rectified linear units improve restricted boltzmann machines

    Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th international conference on machine learning (ICML-10) , pages 807–814, 2010

    2010

  20. [28]

    Qmix: Monotonic value function factorisation for deep multi-agent reinforcement learning

    Tabish Rashid, Mikayel Samvelyan, Christian Schroeder Witt, Gregory Farquhar, Jakob Foerster, and Shimon Whiteson. Qmix: Monotonic value function factorisation for deep multi-agent reinforcement learning. In International Conference on Machine Learning, pages 4292–4301, 2018

    2018

  21. [29]

    Opponent modeling in real-time strategy games

    Frederik Schadd, Sander Bakkes, and Pieter Spronck. Opponent modeling in real-time strategy games. In GAMEON, pages 61–70, 2007. 11

    2007

  22. [30]

    Dropout: a simple way to prevent neural networks from overfitting

    Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929–1958, 2014

    1929

  23. [31]

    Learning to share and hide intentions using information regularization

    Daniel Strouse, Max Kleiman-Weiner, Josh Tenenbaum, Matt Botvinick, and David J Schwab. Learning to share and hide intentions using information regularization. In Advances in Neural Information Processing Systems, pages 10270–10281, 2018

    2018

  24. [32]

    On the theory of the brownian motion

    George E Uhlenbeck and Leonard S Ornstein. On the theory of the brownian motion. Physical review, 36(5):823, 1930

    1930

  25. [33]

    MADDPG + policy mask

    Christopher JCH Watkins and Peter Dayan. Q-learning. Machine learning, 8(3-4):279–292, 1992. 12 A Additional details for experiment presented in Section 3 Motivation We trained each agent i with online Q-learning [ 33] on the Qi(ai,s ) table using Boltzmann exploration [18]. T...

    1992

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.