Pith. sign in

REVIEW 3 major objections 5 minor 51 references

A Minimax Approach to Ad Hoc Teamwork

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A minimax-Bayes policy trained against adversarial partner priors achieves optimal worst-case utility in ad hoc teamwork.

desk verdict A genuinely new minimax-Bayes treatment of ad hoc teamwork with solid experiments, but the theoretical guarantees need major repair: the minimax theorem only goes through for c=1 and the ε-net bound as stated is off by a base-case mismatch. read the letter →

arxiv 2502.02377 v1 pith:YE3G7ZVE submitted 2025-02-04 cs.AI

Victor Villin , Thomas Kleine Buening , Christos Dimitrakakis This is my paper · ORCID
classification cs.AI
keywords adhocteamworkminimax-Bayesreinforcementlearningworst-caseutilityminimaxregretmulti-agentdomaingeneralizationzero-shotcoordinationgradientdescent-ascent
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Ad hoc teamwork asks an agent to cooperate with partners it has never coordinated with, and most training methods fix a distribution over partner behaviours and hope it matches deployment. This paper adapts minimax-Bayes reinforcement learning to that setting: nature chooses an adversarial prior over a finite set of partner scenarios, and the learner maximises its expected utility under the worst such prior. The central result is that a policy solving that maximin problem has optimal worst-case utility on the training scenario set, with an analogous optimal worst-case regret version, and the guarantee extends to out-of-distribution partners within a small total-variation radius. The paper backs this with exact experiments on an iterated prisoner's dilemma and deep coordination experiments on a collaborative cooking game, where the adversarial-prior policies fare better on worst-case and average metrics than self-play, fictitious play, and uniform best-response baselines. The practical claim is that selecting the training distribution over partners is itself the main robustness decision.

What carries the argument

The load-bearing object is the maximin game (7), $\max_\pi \min_\beta U(\pi,\beta)$, where $U(\pi,\beta) = \mathbb{E}_{\sigma\sim\beta}[U(\pi,\sigma)]$ is the expected focal-per-capita return under a prior over scenarios. The argument works because any stochastic policy is a mixture over deterministic policies, so $U(\pi,\beta) = \sum_d \sum_\sigma U(d,\sigma)\phi(d)\beta(\sigma)$ is bilinear in the mixing vector $\phi$ and the prior $\beta$; the minimax theorem then gives the game a value. For the regret version, the same bilinear structure runs on $L(\pi,\beta) = \mathbb{E}[U^*(\sigma)-U(\pi,\sigma)]$. Solutions are computed by gradient descent-ascent on policy parameters and the scenario prior, projected onto the simplex, with delayed copies of the focal policy used when a scenario places $c>1$ copies of the same stochastic policy.

What would settle it

Take a finite game with $c=2$ focal copies of the same stochastic policy and at least one background scenario, and compute $\max_\pi \min_\beta U(\pi,\beta)$ and $\min_\beta \max_\pi U(\pi,\beta)$ exactly by enumerating deterministic-policy mixtures and simplex priors. If the two values differ, the value-of-the-game claim fails in the multi-copy case; a small coordination game where the two copies must match actions is enough to test it.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that robust ad hoc teamwork reduces to solving a finite zero-sum game between the focal policy and an adversary who picks the prior over partner scenarios. The paper proves that the maximin utility game $\max_\pi \min_\beta U(\pi,\beta)$ has a value (Corollary 4.1), so the policy $\pi^*_U$ that solves it simultaneously achieves the optimal worst-case utility on the training scenarios, $U_{\min}(\pi^*_U,\Sigma(B)) = \max_\pi U_{\min}(\pi,\Sigma(B))$ (Lemma 4.3), and is a best response to its own worst-case prior. The same construction with regret, $\min_\pi \max_\beta \mathbb{E}[U^*(\sigma)-U(\pi,\sigma)]$, yields a policy with optimal worst-case regret. Under an epsilon-net assumption on the test scenario set, the worst-case utility guarantee degrades by at most $\epsilon T^2 \|\rho\|_\infty / 2$ and the regret guarantee by at most $\epsilon T^2 \|\rho\|_\infty$ (Lemmas 6.4 and 6.5). The value result rests on the utility being bilinear in the policy's deterministic-policy mixture and the scenario prior.

Load-bearing premise

The proof that the maximin game has a value assumes the utility is bilinear in the policy's mixture over deterministic policies and the scenario prior, which holds only when a single focal copy acts; with several independent copies of the same stochastic policy the utility becomes a higher-degree polynomial, so the central worst-case guarantee is not established for those multi-copy scenarios.

Editorial extensions

If this is right

  • Solving the maximin utility problem (7) yields a policy with the best possible worst-case utility on the training scenario set.
  • Solving the minimax regret problem (13) yields the best possible worst-case regret on the training scenario set.
  • When the training scenarios are an epsilon-net of the true scenario set, worst-case utility on the true set is within $\epsilon T^2\|\rho\|_\infty/2$ of the training optimum; for regret, the slack is $\epsilon T^2\|\rho\|_\infty$.
  • Regret-based training is immune to degenerate scenarios because any policy has zero regret on them, so the adversary does not waste training on impossible tasks; utility-based training needs a non-degenerative population to avoid the same deadlock.
  • In the evaluated coordination tasks, adversarial-prior training improved worst-case metrics and often average metrics over self-play, fictitious play, and uniform population best-response baselines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The formal value-of-the-game proof only covers scenarios with a single focal copy; the paper's multi-copy scenarios (including universalisation) are heuristic uses of the same machinery, and repairing the proof would require either correlated copies or a different objective.
  • The epsilon-net transfer bound suggests a practical certificate: if one can certify an upper bound on the total-variation distance between the training background policies and the actual deployment partners, then the worst-case utility guarantee becomes deployable.
  • The observed curriculum-like speedup of adversarial priors could be turned into a scheduling scheme: use the current worst-case partner distribution to prioritise which scenarios to train on next, and compare sample efficiency against uniform sampling.
  • Because the regret guarantee relies on best-response utilities $U^*(\sigma)$, the method's real-world robustness depends on how well those best responses are approximated; a testable check is to measure how worst-case regret on held-out scenarios varies as the best-response approximation budget shrinks.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a minimax-Bayes approach to Ad Hoc Teamwork (AHT). A focal policy is trained against an adversarial prior over scenarios, where each scenario is defined by a number c of focal copies of the policy and a tuple of background policies drawn from a finite training population. Two objectives are formalized: maximin expected utility (Eq. (7)) and minimax Bayesian regret (Eq. (13)). The paper claims (Corollaries 4.1 and 4.2, Lemma 4.3) that both games have values and that the resulting policies attain optimal worst-case utility or regret on the training scenario set, and it extends these guarantees to out-of-distribution scenario sets that are ε-nets in a total-variation distance over background policies (Lemmas 6.2–6.5). A gradient descent-ascent algorithm (Algorithm 1) and a stochastic variant (Algorithm 2) are proposed. Experiments on an iterated Prisoner's Dilemma and on the Melting Pot Collaborative Cooking benchmark compare Maximin Utility, Minimax Regret, Population Best Response, Fictitious Play, and Self-Play.

Significance. The paper addresses a genuinely under-studied design question in AHT—how the choice of training distribution over partners affects worst-case robustness—and formalizes it in a clean minimax-Bayes framework that supports two distinct objectives (utility versus regret). Strengths include the systematic experimental protocol (exact computation on IPD, three-seed deep-RL evaluation on Melting Pot, released code at https://github.com/AkiEl9/minimax_bayes_aht), and the interesting empirical observation that adversarial scenario priors act like a curriculum (Section 8.2, Figure 4). The ε-net OOD analysis is a promising idea for stating quantitative transfer guarantees. However, the theoretical guarantees as stated are not fully established: Corollary 4.1's bilinearity proof fails when c>1 focal copies are deployed (which the paper's own experiments include), and Lemma 6.2's bound is quantitatively false. The core game-theoretic results imported from Buening et al. [8] are properly attributed, but the new claims for c>1 scenarios are unsupported. With the scope restricted and the OOD calculation corrected, the contribution would be solid and novel enough for a strong venue.

major comments (3)
  1. [§4.2, Corollary 4.1 (Eq. (8))] The proof claims that U(π, β) is bilinear in the policy mixture φ over deterministic policies and the scenario prior β, namely U(π, β) = Σ_{d∈Π_D} Σ_{σ∈Σ(B)} U(d, σ) φ(d) β(σ), and then invokes the minimax theorem. This is valid only for c = 1 focal player. For c > 1, the focal copies of a stochastic policy sample independently, so the probability of a focal joint action is a product of c copies of π; when π is expressed through its mixture φ, U(π, β) becomes a degree-c polynomial in φ (U(π, β) = Σ_σ Σ_{d_1,...,d_c} [∏_i φ(d_i)] Û(d_1,...,d_c, σ) β(σ)). For c = 2 the payoff is an indefinite quadratic form in φ, for which the maximin and minimax values generally differ—for example, with two scenarios whose utilities in a scalar mixing weight p are −(p−0.3)² and −(p−0.7)², max_φ min_β U = −0.04 but min_β max_φ U = 0, so equality (8) is not established and is in fact false in general. Because Σ(B) in Eq. (1) includes c > 1 scenarios such as the universalisation scenario σ_SP = (m, ∅), and the IPD and Collaborative Cooking experiments deploy c = 2 scenarios, Lemma 4.3's optimal worst-case training guarantee does not cover the paper's own experimental configurations. Section 7's caveat that the GDA convergence theory applies only to single-agent POMGs is consistent with this gap. The revision should restrict Corollaries 4.1–4.3 to c = 1 (treating c > 1 through the delayed-copy approximation of Section 7) or supply a genuine existence proof for c > 1.
  2. [§6, Lemma 6.2; Appendix B] The induction proving Lemma 6.2 is quantitatively wrong. The claimed bound (28), ε||ρ||∞(T−t+1)(T−t)/2, evaluates to 0 at t = T, whereas the base case (31) establishes ε||ρ||∞; the two are inconsistent. The recurrence used in the induction, formed from (34)–(37), is Δ_t ≤ ε||ρ||∞[1 + (T−t)] + Δ_{t+1}; solving it with the correct base gives Δ_1 ≤ ε||ρ||∞ T(T+1)/2, which exceeds the lemma's stated ε||ρ||∞ T²/2 by ε||ρ||∞ T/2. For T = 1, the lemma's bound ε||ρ||∞/2 is contradicted by the proof's own base case, since two ε-close background policies can induce reward expectations differing by ε||ρ||∞. Consequently, Lemmas 6.3–6.5 inherit incorrect constants, and the OOD robustness guarantees (18)–(19) are not established as stated. The proof strategy is repairable in that the corrected bound still has the same O(εT²) form, but the calculation must be redone and all affected statements updated.
  3. [§6, Eqs. (14)–(15); Appendix B] The scenario distance is defined over states (max_{s∈S} Σ_i Σ_a |π_i(a|s) − π'_i(a|s)|), while policies are history-conditioned per Section 3.1 and the induced POMGs of Section 3.2 are partially observable. The Appendix B proof of Lemma 6.2 compares background action probabilities π^b_i(a^b_i | h_i) and π'^b_i(a^b_i | h_i) at the same history h_i, but the realized histories under σ and σ' diverge because the transition kernels differ; a state-wise TV bound on policies does not control the difference of action distributions at the histories that actually occur. Lemma 6.2, and therefore Lemmas 6.4–6.5, are not established in the partially observable setting, which is exactly the regime of the Collaborative Cooking experiments (local RGB observations, LSTM policies). The proof requires either a coupling argument over histories or an explicit restriction of the OOD claims to Markov state-based policies or fully observable games.
minor comments (5)
  1. [§5, definition of Bayesian regret] The definition of L(π, β) writes R(π, μ) inside the sum, but the second argument should be the scenario σ (as in Eq. (3)) rather than the POMG μ, which is the symbol used for the base game in Section 3.1; this is confusing and should be corrected.
  2. [Table 3 / §8.2] The Melting Pot baseline scores (PP-ACB, PP-OPRE, PP-VMPO) are taken from Agapiou et al. [1] and, as the caption notes, were not obtained on the exact same setting; the body-text conclusion that MU 'outperforms all other evaluated methods' should carry this caveat as well.
  3. [§4.1] The notation for background-policy preferences is inconsistent: the text says each background policy has its own preferences (λ_k, δ_k), while the reward formula ρ_social+risk uses λ_i and δ_i with i a player index; the notation should be aligned.
  4. [Algorithm 2] The stochastic algorithm returns (θ*, β*) uniformly sampled from the trajectory without any convergence statement; since the regret version also estimates U*(σ_i), a sentence clarifying that the stochastic regret update is a heuristic would be appropriate.
  5. [Figure 2] The policy-tree figures would be easier to interpret with a legend and an explicit statement of which history each node conditions on for a three-round simultaneous-move game.

Circularity Check

1 steps flagged · score 3.0 of 10

Training-set worst-case guarantee is a definitional restatement of the maximin objective; otherwise the derivation is independent, with OOD bounds and experiments carrying the central claim.

  1. self definitional [Section 4.2, Corollary 4.2 (Eq. 11) and Lemma 4.3 (Eq. 12)]
    "Corollary 4.2 ... min β∈Δ(Σ(B)) U(π,β) =Umin(π, Σ(B)). (11) ... Lemma 4.3. ... Umin(π∗ U, Σ(B)) = max π∈Π Umin(π, Σ(B)). (12) ... Thus, a policy solving the maximin utility game(7) is guaranteed to have an optimal worst-case utility on its training set."

    By Corollary 4.2, min_β U(π,β)=U_min(π,Σ(B)) for every π. Substituting this identity into the maximin objective (7), max_π min_β U(π,β), yields exactly max_π U_min(π,Σ(B)). Lemma 4.3 then states that the maximizer π*_U attains this value: U_min(π*_U,Σ(B)) = max_π U_min(π,Σ(B)). The 'optimal worst-case utility on the training set' is therefore the very quantity that (7) is defined to optimize; the lemma is a restatement of the objective by construction, not an independent derived guarantee. The paper's nontrivial content lies elsewhere (OOD lemmas, algorithm, experiments), so this is a partial, local circularity.

full rationale

Aside from Lemma 4.3, the derivation chain is not circular. Corollaries 4.1 and 4.2 are credited to Buening et al. [8], which shares two authors with this paper, but Corollary 4.1 is accompanied by a proof sketch invoking the standard minimax theorem and Corollary 4.2 is an immediate consequence of Dirac scenarios in the simplex; these are not unverified uniqueness imports. The OOD Lemmas 6.2–6.5 are new and attempt to deliver nontrivial ε-net bounds; the skeptical concern about the Appendix B induction (base-case mismatch yielding ε∥ρ∥∞ instead of εT^2∥ρ∥∞/2) is a quantitative correctness issue, not circularity. The Section 7 admission that GDA convergence theory covers only single-agent POMGs is a stated limitation, not a circular step. The empirical evaluation on Melting Pot and the OOD extension give the central claim independent content. Hence the circularity burden is limited to the definitional restatement in Lemma 4.3.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No new entities are postulated. Free parameters are experiment-design choices rather than fitted constants in the core derivation. The main axioms are the standard minimax theorem, the non-degenerative population assumption, and the epsilon-net coverage assumption for OOD transfer.

free parameters (4)
  • prosociality lambda_i for background policies = sampled uniformly in [-0.2, 1.2]
    Chosen empirically (Section 8.2, footnote 5) to generate diverse and cooperative background policies; affects the training scenario set.
  • risk aversion delta_i for background policies = sampled uniformly in [0.1, 2]
    Chosen empirically in the same footnote to shape partner diversity.
  • epsilon for IPD test scenario generation = 0.5
    Defines how close test partners are to training partners in the Iterated Prisoner's Dilemma experiment (Section 8.1).
  • prior learning rate eta_beta = 0.4
    Hyperparameter for the GDA update of the scenario distribution (Appendix D).
assumptions (4)
  • standard math Minimax theorem for bilinear zero-sum games
    Invoked in the proof of Corollary 4.1 to assert the game has a value; requires bilinearity of the payoff in the players' mixed strategies.
  • standard math Every stochastic policy can be represented as a mixture over deterministic policies
    Used in the proof of Corollary 4.1 to rewrite U(pi, beta) as a bilinear form.
  • domain assumption Non-degenerative background population (Definition 5.1)
    Assumed for the remainder of the paper so that the utility-minimizing adversary cannot trivially deadlock learning on zero-regret scenarios.
  • domain assumption True test scenarios are close to training scenarios (epsilon-net assumption)
    The out-of-distribution guarantees in Section 6 require the test scenario set S to be covered by an epsilon-net of training scenarios.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Minimax Approach to Ad Hoc Teamwork." pith.science (2026). https://pith.science/paper/YE3G7ZVE

@misc{pith2026250202377,
  author       = {Pith},
  title        = {Pith review of: A Minimax Approach to Ad Hoc Teamwork},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YE3G7ZVE}},
  note         = {Machine review of arXiv:2502.02377}
}
read the original abstract

We propose a minimax-Bayes approach to Ad Hoc Teamwork (AHT) that optimizes policies against an adversarial prior over partners, explicitly accounting for uncertainty about partners at time of deployment. Unlike existing methods that assume a specific distribution over partners, our approach improves worst-case performance guarantees. Extensive experiments, including evaluations on coordinated cooking tasks from the Melting Pot suite, show our method's superior robustness compared to self-play, fictitious play, and best response learning. Our work highlights the importance of selecting an appropriate training distribution over teammates to achieve robustness in AHT.

Figures

Figures reproduced from arXiv: 2502.02377 by the authors.

Figure 1
Figure 1. Illustration of the framework used in this paper. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. The different policies obtained in function of their [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. We consider two kitchen layouts in the Collabo [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Learning curves of the average and worst-case utility [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Average utility, worst-case utility and worst-case regret on the test set, in function of the [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Smoothed learning curves of the distribution over the training scenarios (averaged over 3 runs). [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 40 canonical work pages

  1. [8]

    Thomas Kleine Buening, Christos Dimitrakakis, Hannes Eriksson, Divya Grover, and Emilio Jorge. 2023. Minimax-Bayes Reinforcement Learning. In Proceedings of The 26th International Conference on Artificial Intelligence and Statistics (Pro- ceedings of Machine Learning Research, Vol. 206) , Francisco Ruiz, Jennifer Dy, and Jan-Willem van de Meent (Eds.). PM...

    work page 2023

  2. [1]

    Agapiou, Alexander Sasha Vezhnevets, Edgar A

    John P. Agapiou, Alexander Sasha Vezhnevets, Edgar A. Duéñez-Guzmán, Jayd Matyas, Yiran Mao, Peter Sunehag, Raphael Köster, Udari Madhushani, Kavya Kopparapu, Ramona Comanescu, DJ Strouse, Michael B. Johanson, Sukhdeep Singh, Julia Haas, Igor Mordatch, Dean Mobbs, and Joel Z. Leibo. 2023. Melting Pot 2.0. arXiv:2211.13746 [cs.MA]

    arXiv 2023

  3. [2]

    Stefano Albrecht, Jacob Crandall, and Subramanian Ramamoorthy. 2015. An Empirical Study on the Practical Impact of Prior Beliefs over Policy Types. Pro- ceedings of the AAAI Conference on Artificial Intelligence 29, 1 (Feb. 2015)

    work page 2015

  4. [3]

    Samuel Barrett, Avi Rosenfeld, Sarit Kraus, and Peter Stone. 2017. Making friends on the fly: Cooperating with new teammates. Artificial Intelligence 242 (2017), 132–171

    work page 2017

  5. [4]

    Samuel Barrett and Peter Stone. 2015. Cooperating with Unknown Teammates in Complex Domains: A Robot Soccer Case Study of Ad Hoc Teamwork.Proceedings of the AAAI Conference on Artificial Intelligence 29, 1 (Feb. 2015)

    work page 2015

  6. [5]

    Samuel Barrett, Peter Stone, and Sarit Kraus. 2011. Empirical evaluation of ad hoc teamwork in the pursuit domain. In The 10th International Conference on Autonomous Agents and Multiagent Systems-Volume 2. 567–574

    work page 2011

  7. [6]

    James O Berger. 1985. Statistical decision theory and Bayesian analysis. Springer Series in Statistics (1985)

    work page 1985

  8. [7]

    George W. Brown. 1951. Iterative Solution of Games by Fictitious Play. InActivity Analysis of Production and Allocation , T. C. Koopmans (Ed.). Wiley, New York

    work page 1951

Show all 51 references
  1. [9]

    Rodrigo Canaan, Xianbo Gao, Julian Togelius, Andy Nealen, and Stefan Menzel

  2. [10]

    Micah Carroll, Rohin Shah, Mark K Ho, Tom Griffiths, Sanjit Seshia, Pieter Abbeel, and Anca Dragan. 2019. On the utility of learning about humans for human-ai coordination. Advances in neural information processing systems 32 (2019)

    2019

  3. [11]

    Rujikorn Charakorn, Poramate Manoonpong, and Nat Dilokthanakul. 2020. In- vestigating partner diversification methods in cooperative multi-agent deep reinforcement learning. In Neural Information Processing: 27th International Con- ference, ICONIP 2020, Bangkok, Thailand, Nove...

    2020

  4. [12]

    Shuo Chen, Ewa Andrejczuk, Zhiguang Cao, and Jie Zhang. 2020. Aateam: Achieving the ad hoc teamwork by employing the attention mechanism. In Proceedings of the AAAI conference on artificial intelligence , Vol. 34. 7095–7102

    2020

  5. [13]

    Karl Cobbe, Oleg Klimov, Chris Hesse, Taehoon Kim, and John Schulman. 2019. Quantifying Generalization in Reinforcement Learning. In Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and...

    2019

  6. [14]

    Michael Dennis, Natasha Jaques, Eugene Vinitsky, Alexandre Bayen, Stuart Rus- sell, Andrew Critch, and Sergey Levine. 2020. Emergent complexity and zero-shot transfer via unsupervised environment design. Advances in neural information processing systems 33 (2020), 13049–13061

    2020

  7. [15]

    Hasra Dodampegama and Mohan Sridharan. 2023. Knowledge-based reasoning and learning under partial observability in ad hoc teamwork. Theory and Practice of Logic Programming 23, 4 (2023), 696–714

    2023

  8. [16]

    Samuel Garcin, James Doran, Shangmin Guo, Christopher G Lucas, and Stefano V Albrecht. 2023. How the level sampling process impacts zero-shot generalisation in deep reinforcement learning. arXiv preprint arXiv:2310.03494 (2023)

    2023 arXiv

  9. [17]

    Adam Gleave, Michael Dennis, Cody Wild, Neel Kant, Sergey Levine, and Stuart Russell. 2019. Adversarial policies: Attacking deep reinforcement learning. arXiv preprint arXiv:1905.10615 (2019)

    2019 arXiv

  10. [18]

    Pengjie Gu, Mengchen Zhao, Jianye Hao, and Bo An. 2021. Online ad hoc teamwork under partial observability. In International conference on learning representations

    2021

  11. [19]

    Johannes Heinrich, Marc Lanctot, and David Silver. 2015. Fictitious Self-Play in Extensive-Form Games. In Proceedings of the 32nd International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 37) , Francis Bach and David Blei (Eds.). PMLR, Lille,...

    2015

  12. [20]

    Irina Higgins, Arka Pal, Andrei Rusu, Loic Matthey, Christopher Burgess, Alexan- der Pritzel, Matthew Botvinick, Charles Blundell, and Alexander Lerchner. 2017. DARLA: Improving Zero-Shot Transfer in Reinforcement Learning. In Proceed- ings of the 34th International Conference...

    2017

  13. [21]

    Other- Play

    Hengyuan Hu, Adam Lerer, Alex Peysakhovich, and Jakob Foerster. 2020. “Other- Play” for Zero-Shot Coordination. In Proceedings of the 37th International Confer- ence on Machine Learning (Proceedings of Machine Learning Research, Vol. 119) , Hal Daumé III and Aarti Singh (Eds.)...

    2020

  14. [22]

    Czarnecki, Iain Dunning, Luke Marris, Guy Lever, Antonio Garcia Castañeda, Charles Beattie, Neil C

    Max Jaderberg, Wojciech M. Czarnecki, Iain Dunning, Luke Marris, Guy Lever, Antonio Garcia Castañeda, Charles Beattie, Neil C. Rabinowitz, Ari S. Morcos, Avraham Ruderman, Nicolas Sonnerat, Tim Green, Louise Deason, Joel Z. Leibo, David Silver, Demis Hassabis, Koray Kavukcuogl...

    2019

  15. [23]

    Jiaming Ji, Tianyi Qiu, Boyuan Chen, Borong Zhang, Hantao Lou, Kaile Wang, Yawen Duan, Zhonghao He, Jiayi Zhou, Zhaowei Zhang, et al. 2023. Ai alignment: A comprehensive survey. arXiv preprint arXiv:2310.19852 (2023)

    2023 arXiv

  16. [24]

    Minqi Jiang, Edward Grefenstette, and Tim Rocktäschel. 2021. Prioritized Level Replay. In Proceedings of the 38th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 139) , Marina Meila and Tong Zhang (Eds.). PMLR, 4940–4950

    2021

  17. [25]

    Marc Lanctot, Vinicius Zambaldi, Audrunas Gruslys, Angeliki Lazaridou, Karl Tuyls, Julien Perolat, David Silver, and Thore Graepel. 2017. A Unified Game- Theoretic Approach to Multiagent Reinforcement Learning. InAdvances in Neural Information Processing Systems, I. Guyon, U. ...

    2017

  18. [26]

    Joel Z Leibo, Edgar A Dueñez-Guzman, Alexander Vezhnevets, John P Agapiou, Peter Sunehag, Raphael Koster, Jayd Matyas, Charlie Beattie, Igor Mordatch, and Thore Graepel. 2021. Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Pot. In Proceedings of the 38t...

    2021

  19. [27]

    Jack Li, Zeyu Dong, and Shuo Han. 2024. Bayes-Optimal, Robust, and Distribu- tionally Robust Policies for Uncertain MDPs. (2024)

    2024

  20. [28]

    Shihui Li, Yi Wu, Xinyue Cui, Honghua Dong, Fei Fang, and Stuart Russell. 2019. Robust Multi-Agent Reinforcement Learning via Minimax Deep Deterministic Policy Gradient. Proceedings of the AAAI Conference on Artificial Intelligence 33, 01 (Jul. 2019), 4213–4220

    2019

  21. [29]

    Tianyi Lin, Chi Jin, and Michael Jordan. 2020. On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems. In Proceedings of the 37th Interna- tional Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 119), Hal Daumé III and Aarti Singh (Eds....

    2020

  22. [30]

    Andrei Lupu, Brandon Cui, Hengyuan Hu, and Jakob Foerster. 2021. Trajectory Diversity for Zero-Shot Coordination. InProceedings of the 38th International Con- ference on Machine Learning (Proceedings of Machine Learning Research, Vol. 139) , Marina Meila and Tong Zhang (Eds.)....

    2021

  23. [31]

    Albrecht

    Reuth Mirsky, Ignacio Carlucho, Arrasy Rahman, Elliot Fosong, William Macke, Mohan Sridharan, Peter Stone, and Stefano V. Albrecht. 2022. A Survey of Ad Hoc Teamwork Research. In Multi-Agent Systems, Dorothea Baumeister and Jörg Rothe (Eds.). Springer International Publishing,...

    2022

  24. [32]

    Darius Muglich, Christian Schroeder de Witt, Elise van der Pol, Shimon Whiteson, and Jakob Foerster. 2022. Equivariant Networks for Zero-Shot Coordination. In Advances in Neural Information Processing Systems , S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh ...

    2022

  25. [33]

    Darius Muglich, Luisa M Zintgraf, Christian A Schroeder De Witt, Shimon White- son, and Jakob Foerster. 2022. Generalized Beliefs for Cooperative AI. In Pro- ceedings of the 39th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 162) ...

    2022

  26. [34]

    Frans A Oliehoek. 2012. Decentralized pomdps. In Reinforcement learning: state-of-the-art. Springer, 471–503

    2012

  27. [35]

    Alexander Peysakhovich and Adam Lerer. 2017. Prosocial learning agents solve generalized stag hunts better than selfish ones. arXiv preprint arXiv:1709.02865 (2017)

    2017 arXiv

  28. [36]

    Lerrel Pinto, James Davidson, Rahul Sukthankar, and Abhinav Gupta. 2017. Ro- bust Adversarial Reinforcement Learning. In Proceedings of the 34th Interna- tional Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 70), Doina Precup and Yee Whye Teh (E...

    2017

  29. [37]

    Manish Ravula, Shani Alkoby, and Peter Stone. 2019. Ad Hoc Teamwork With Behavior Switching Agents. InProceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19 . International Joint Conferences on Artificial Intelligence Organization, 550–556

    2019

  30. [38]

    João G Ribeiro, Cassandro Martinho, Alberto Sardinha, and Francisco S Melo

  31. [39]

    Michael Rovatsos and Marco Wolf. 2002. Towards social complexity reduction in multiagent learning: the adhoc approach. In Proceedings of the 2002 AAAI Spring Symposium on Collaborative Learning Agents . 90–97

    2002

  32. [40]

    In ECAI 2023

    Making Friends in the Dark: Ad Hoc Teamwork Under Partial Observability. In ECAI 2023. IOS Press, 1954–1961

    2023

  33. [41]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov

  34. [42]

    Lukas Schäfer. 2022. Task generalisation in multi-agent reinforcement learning. In Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems. 1863–1865

    2022

  35. [43]

    Peter Stone, Gal Kaminka, Sarit Kraus, and Jeffrey Rosenschein. 2010. Ad hoc autonomous agent teams: Collaboration without pre-coordination. InProceedings of the AAAI Conference on Artificial Intelligence , Vol. 24. 1504–1509

    2010

  36. [44]

    DJ Strouse, Kevin McKee, Matt Botvinick, Edward Hughes, and Richard Everett

  37. [45]

    H Francis Song, Abbas Abdolmaleki, Jost Tobias Springenberg, Aidan Clark, Hubert Soyer, Jack W Rae, Seb Noury, Arun Ahuja, Siqi Liu, Dhruva Tirumala, et al. 2019. V-mpo: On-policy maximum a posteriori policy optimization for discrete and continuous control. arXiv preprint arXi...

    2019 arXiv

  38. [46]

    Caroline Wang, Arrasy Rahman, Ishan Durugkar, Elad Liebman, and Peter Stone

  39. [49]

    Alexander Vezhnevets, Yuhuai Wu, Maria Eckstein, Rémi Leblond, and Joel Z Leibo. 2020. OPtions as REsponses: Grounding behavioural hierarchies in multi- agent reinforcement learning. In Proceedings of the 37th International Confer- ence on Machine Learning (Proceedings of Mach...

    2020

  40. [2017]

    arXiv preprint arXiv:1707.06347 (2017)

    Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)

    2017 arXiv

  41. [2021]

    Advances in Neural Information Processing Systems 34 (2021), 14502–14515

    Collaborating with humans without human data. Advances in Neural Information Processing Systems 34 (2021), 14502–14515

    2021

  42. [2023]

    IEEE Transactions on Games 15, 2 (2023), 228–241

    Generating and Adapting to Diverse Ad Hoc Partners in Hanabi. IEEE Transactions on Games 15, 2 (2023), 228–241

    2023

  43. [2024]

    arXiv:2404.10740 [cs.AI] APPENDIX A ALGORITHMS We provide an implementation for solving the maximin utility/minimax regret problems when the game is unknown, in Algorithm 2

    N-Agent Ad Hoc Teamwork. arXiv:2404.10740 [cs.AI] APPENDIX A ALGORITHMS We provide an implementation for solving the maximin utility/minimax regret problems when the game is unknown, in Algorithm 2. Algorithm 2 Background-Focal SGDA 1: Input set of background policiesB, batch ...

    arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.