Pith. sign in

REVIEW 52 cited by

Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2011.09533 v1 pith:P7TTZ2I5 submitted 2020-11-18 cs.AI

Christian Schroeder de Witt , Tarun Gupta , Denys Makoviichuk , Viktor Makoviychuk , Philip H.S. Torr , Mingfei Sun , Shimon Whiteson This is my paper · ORCID
classification cs.AI
keywords learningindependentippomulti-agentapproachescentralizedfunctionjoint
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Most recently developed approaches to cooperative multi-agent reinforcement learning in the \emph{centralized training with decentralized execution} setting involve estimating a centralized, joint value function. In this paper, we demonstrate that, despite its various theoretical shortcomings, Independent PPO (IPPO), a form of independent learning in which each agent simply estimates its local value function, can perform just as well as or better than state-of-the-art joint learning approaches on popular multi-agent benchmark suite SMAC with little hyperparameter tuning. We also compare IPPO to several variants; the results suggest that IPPO's strong performance may be due to its robustness to some forms of environment non-stationarity.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 52 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 185 citations worldwide. Full citation record

  1. Aggregate in the Advantage, Not the Ratio: A Canonical-Form Analysis of Cooperative Multi-Agent Policy Optimization

    cs.MA 2026-07 conditional novelty 7.0 of 10

    For cooperative PPO, the expected gradient at the on-policy point depends on advantage and ratio aggregation supports only through their matrix product, and the variance-optimal design keeps the ratio per-agent and ag...

  2. ReCoDe: Reinforcement Learning-based Dynamic Constraint Design for Multi-Agent Coordination

    cs.RO 2025-07 conditional novelty 7.0 of 10

    ReCoDe improves handcrafted multi-agent controllers by learning a dynamic quadratic constraint that controls how tightly each robot follows a learned reference action, outperforming baselines across navigation and con...

  3. MEAL: A Benchmark for Continual Multi-Agent Reinforcement Learning

    cs.AI 2025-06 conditional novelty 7.0 of 10

    MEAL is a GPU-accelerated, procedurally generated Overcooked benchmark for continual multi-agent reinforcement learning, revealing that existing continual learning methods struggle to retain cooperation on harder layo...

  4. SelectLight: Learning to Select Signal Plans Generated by Distributed Model Predictive Control for Urban Traffic Networks

    eess.SY 2026-08 conditional novelty 6.0 of 10

    SelectLight lets a multi-agent RL policy select unmodified signal plans generated online by distributed MPC, improving delay metrics in simulated urban networks and transferring across demand and topology.

  5. Dreamer-CPC: Message Learning with World Models for Decentralized Multi-agent Reinforcement Learning

    cs.MA 2026-07 conditional novelty 6.0 of 10

    Dreamer-CPC has each decentralized agent send messages drawn from its learned world-model memory, outperforming current-observation messaging baselines, especially when key observations are temporarily missing.

  6. Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents

    cs.CL 2026-07 conditional novelty 6.0 of 10

    A tabular UCB controller trained on task success improves LLM-agent memory use over fixed heuristics, without extra LLM calls.

  7. Action-Factored Multi-Agent Reinforcement Learning for Scalable Quantum Device Tuning

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Online action-space factorization via Kalman-refined cross-capacitance lets shared multi-agent policies zero-shot tune larger quantum-dot arrays with near-constant steps.

  8. Contract-Based Compositional Shielding for Safe Multi-Agent Reinforcement Learning

    cs.LG 2026-06 conditional novelty 6.0 of 10

    Certified local LTL_safe contracts, jointly fixed-point checked and selected by a bandit, recover coordinated safe team policies under decentralised multi-agent RL execution.

  9. TABX: A High-Throughput Sandbox Battle Simulator for Multi-Agent Reinforcement Learning

    cs.MA 2026-02 unverdicted novelty 6.0 of 10

    TABX is a JAX-based, GPU-accelerated, configurable multi-agent battle simulator that lets researchers vary units, terrain, and physics to benchmark cooperative MARL algorithms.

  10. Learning Decentralized LLM Collaboration with Multi-Agent Actor Critic

    cs.AI 2026-01 unverdicted novelty 6.0 of 10

    Multi-agent actor-critic methods with a centralized critic improve decentralized LLM collaboration over Monte Carlo baselines in long-horizon and sparse-reward settings.

  11. Automata-Conditioned Cooperative Multi-Agent Reinforcement Learning

    cs.MA 2025-11 conditional novelty 6.0 of 10

    ACC-MARL trains decentralized multi-agent policies that solve many automaton-specified cooperative tasks at once, with a proof of optimality for the Markovian reformulation and value-based task assignment.

  12. CrazyMARL: Decentralized Direct Motor Control Policies for Cooperative Aerial Transport of Cable-Suspended Payloads

    cs.RO 2025-09 conditional novelty 6.0 of 10

    A decentralized reinforcement learning controller with direct motor commands lets teams of drones carry cable-suspended payloads, recover from harsh disturbances, and transfer from simulation to real Crazyflie hardware.

  13. Hierarchical Message-Passing Policies for Multi-Agent Reinforcement Learning

    cs.LG 2025-07 conditional novelty 6.0 of 10

    A feudal hierarchical MARL method where lower-level policies are rewarded with the upper level's advantage function, with theoretical alignment guarantees and strong benchmark results.

  14. Hierarchical Multi-Agent Reinforcement Learning with Control Barrier Functions for Safety-Critical Autonomous Systems

    cs.LG 2025-07 conditional novelty 6.0 of 10

    A hierarchical multi-agent RL method that selects skills at a high level and enforces pointwise safety with learned CBF-QP policies achieves about 99 percent success in simulated traffic scenarios.

  15. Toward Adaptable Multi-Agent Reinforcement Learning: An Assumption-Aware Review

    cs.AI 2025-07 conditional novelty 6.0 of 10

    A survey proposing adaptability as a three-part taxonomy (learning, policy, scenario-driven) for organizing and evaluating MARL under changing conditions.

  16. Modeling Latent Partner Strategies for Adaptive Zero-Shot Human-Agent Collaboration

    cs.AI 2025-07 conditional novelty 6.0 of 10

    An agent that represents teammate styles as latent clusters and updates its belief with fixed-share regret minimization outperformed baselines with unfamiliar human partners in Overcooked.

  17. Multi-Agent Reinforcement Learning for Inverse Design in Photonic Integrated Circuits

    cs.LG 2025-06 conditional novelty 6.0 of 10

    A multi-agent bandit RL formulation for binary photonic topology optimization produces designs that outperform gradient-based optimization on eight simulated 2D and 3D tasks.

  18. Learned Controllers for Agile Quadrotors in Pursuit-Evasion Games

    cs.RO 2025-06 conditional novelty 6.0 of 10

    Population-based opponent sampling during drone pursuit-evasion training improves robustness against older and unseen strategies, with rate-based control outperforming velocity-based control in simulation.

  19. Credit Assignment and Efficient Exploration based on Influence Scope in Multi-agent Reinforcement Learning

    cs.LG 2025-05 conditional novelty 6.0 of 10

    ISA uses mutual information between actions and state changes to define each agent's influence scope, then uses it for credit assignment and count-based exploration in sparse-reward MARL.

  20. Cross-environment Cooperation Enables Zero-shot Multi-agent Coordination

    cs.MA 2025-04 conditional novelty 6.0 of 10

    Training a self-play agent across many procedurally generated cooperative tasks yields better zero-shot coordination with novel partners and novel layouts than training on one task with many partners.

  21. ColorGrid: A Multi-Agent Non-Stationary Environment for Goal Inference and Assistance

    cs.AI 2025-01 conditional novelty 6.0 of 10

    A new benchmark shows that IPPO, a standard MARL algorithm, cannot reliably infer a partner's changing goal from its actions alone.

  22. Dynamic Graph Communication for Decentralised Multi-Agent Reinforcement Learning

    cs.MA 2024-12 conditional novelty 6.0 of 10

    A GAT-based aggregator and a learned iteration controller improve NetMon's decentralized packet routing in simulated dynamic networks by 9.5% reward while using 6.4% less communication.

  23. SMAC-Hard: Enabling Mixed Opponent Strategy Script and Self-play on SMAC

    cs.AI 2024-12 conditional novelty 6.0 of 10

    Mixed opponent scripts in SMAC-HARD make state-of-the-art MARL methods collapse to near-zero win rates, revealing overfitting and poor transfer to unseen adversary behaviors.

  24. Multi Agent Reinforcement Learning for Sequential Satellite Assignment Problems

    cs.MA 2024-12 conditional novelty 6.0 of 10

    REDA learns per-agent Q-values and uses them as benefit inputs to an optimal assignment mechanism, outperforming IQL, IPPO, COMA, and HAAL on sequential satellite assignment.

  25. The impact of behavioral diversity in multi-agent reinforcement learning

    cs.AI 2024-12 conditional novelty 6.0 of 10

    Controlling behavioral diversity in multi-agent reinforcement learning produces better team play, exploration, and resilience, including emergent roles such as a goalkeeper.

  26. GTDE: Grouped Training with Decentralized Execution for Multi-agent Actor-Critic

    cs.MA 2024-12 conditional novelty 6.0 of 10

    GTDE dynamically groups agents during training using Gumbel-Sigmoid sampling, enabling decentralized execution that outperforms CTDE and DTDE on large-scale multi-agent benchmarks.

  27. InvestESG: A multi-agent reinforcement learning benchmark for studying climate investment as a social dilemma

    cs.LG 2024-11 conditional novelty 6.0 of 10

    InvestESG is a MARL benchmark showing that ESG-conscious investors, not the disclosure mandate itself, drive corporate mitigation in long-run simulated markets.

  28. Is Inter-Seed Cross-Play Enough? Evaluating the Robustness of Zero-Shot Coordination Algorithms to Implementation Details

    cs.AI 2026-08 conditional novelty 5.0 of 10

    For Other-Play in Yokai, agents trained with different implementation details coordinate across implementations about as well as across seeds, supporting inter-seed cross-play as a proxy for cross-implementation evaluation.

  29. Diffusing to Coordinate: Efficient Online Multi-Agent Diffusion Policies

    cs.AI 2026-02 conditional novelty 5.0 of 10

    A diffusion-policy multi-agent RL framework substitutes an ELBO for intractable joint entropy and reports 2.5–5× sample-efficiency gains on 10 MPE/MAMuJoCo tasks.

  30. CoRL-MPPI: Enhancing MPPI With Learnable Behaviours For Efficient And Provably-Safe Multi-Robot Collision Avoidance

    cs.RO 2025-11 conditional novelty 5.0 of 10

    CoRL-MPPI injects a learned cooperative policy into MPPI's sampling distribution to speed up and make safer multi-robot navigation in dense simulations.

  31. Self-Supervised Goal-Reaching Results in Multi-Agent Cooperation and Exploration

    cs.LG 2025-09 conditional novelty 5.0 of 10

    Self-supervised multi-agent goal-reaching, where each agent independently learns a contrastive critic of its own observations, achieves cooperation and exploration in sparse-reward MARL tasks where standard baselines fail.

  32. cMALC-D: Contextual Multi-Agent LLM-Guided Curriculum Learning with Diversity-Based Context Blending

    cs.LG 2025-08 reject novelty 5.0 of 10

    cMALC-D uses an LLM to generate training contexts for multi-agent RL and a diversity-blending mechanism to avoid mode collapse, claiming improved generalization on traffic signal control.

  33. Multi-Agent Trust Region Policy Optimisation: A Joint Constraint Approach

    cs.AI 2025-08 unverdicted novelty 5.0 of 10

    Adaptive per-agent KL-threshold allocation via KKT (HATRPO-W) and greedy (HATRPO-G) improves HATRPO's final reward by over 22.5% in MARL benchmarks.

  34. Remembering the Markov Property in Cooperative MARL

    cs.LG 2025-07 conditional novelty 5.0 of 10

    Cooperative MARL agents solve many benchmarks with brittle conventions that bypass observations and memory, so the field needs tasks that genuinely require grounded, history-based reasoning.

  35. From General Relation Patterns to Task-Specific Decision-Making in Continual Multi-Agent Coordination

    cs.MA 2025-07 conditional novelty 5.0 of 10

    RPG uses a relation capturer plus a task-conditioned hypernetwork to reduce catastrophic forgetting and enable zero-shot transfer in continual multi-agent coordination.

  36. Light Aircraft Game : Basic Implementation and training results analysis

    cs.LG 2025-06 reject novelty 5.0 of 10

    In the new LAG air-combat environment, HASAC scores higher than HAPPO in no-weapon coordination tasks while HAPPO scores higher in missile combat, but the results come from single runs without error bars.

  37. Shapley Machine: A Game-Theoretic Framework for N-Agent Ad Hoc Teamwork

    cs.MA 2025-06 conditional novelty 5.0 of 10

    Shapley Machine reshapes rewards and TD targets so per-agent value functions approximately satisfy the Shapley axioms, and it beats the prior POAM baseline in several NAHT test environments.

  38. Ego-centric Learning of Communicative World Models for Autonomous Driving

    cs.RO 2025-06 reject novelty 5.0 of 10

    Sharing compressed latent states and planned waypoints between agents, triggered by prediction errors, improves multi-agent driving performance in CARLA while cutting communication bandwidth by roughly 50x.

  39. Reward-Independent Messaging for Decentralized Multi-Agent Reinforcement Learning

    cs.MA 2025-05 conditional novelty 5.0 of 10

    MARL-CPC lets decentralized agents learn to send informative messages through a self-supervised reconstruction objective, and outperforms message-as-action baselines in non-cooperative multi-agent tasks.

  40. Dynamic Sight Range Selection in Multi-Agent Reinforcement Learning

    cs.MA 2025-05 conditional novelty 5.0 of 10

    A UCB meta-controller that selects per-episode observation ranges improves MARL performance in several benchmark settings, but the paper's claim of consistent improvement is contradicted by its own results.

  41. Cooperative Patrol Routing: Optimizing Urban Crime Surveillance through Multi-Agent Reinforcement Learning

    cs.AI 2025-01 reject novelty 5.0 of 10

    A VDPPO-based multi-agent model plans police patrol routes that cover most of the highest-crime nodes in two of three Malaga districts, with weaker results in the third.

  42. MacLight: Multi-scene Aggregation Convolutional Learning for Traffic Signal Control

    cs.MA 2024-12 conditional novelty 5.0 of 10

    A CNN-VAE global representation plus PPO yields faster training and stable control on grid networks, though the claimed superiority over baselines is inconsistent across scenarios.

  43. SWIRL: A Staged Workflow for Interleaved Reinforcement Learning in Mobile GUI Control

    cs.AI 2025-08 conditional novelty 4.0 of 10

    A multi-agent RL workflow that interleaves single-agent updates, applied to mobile GUI control, achieves SOTA zero-shot performance and a +14.8 MATH500 gain.

  44. Multi-agent Reinforcement Learning for Robotized Coral Reef Sample Collection

    cs.RO 2025-07 conditional novelty 4.0 of 10

    An RL controller for coral sample collection, trained in Unity, is transferred zero-shot to a physical BlueROV2 using real-time underwater motion capture to drive the digital twin.

  45. PILOC: A Pheromone Inverse Guidance Mechanism and Local-Communication Framework for Dynamic Target Search of Multi-Agent in Unknown Environments

    cs.RO 2025-07 conditional novelty 4.0 of 10

    A decentralized reinforcement-learning framework using virtual pheromone marks and local map sharing outperforms MARL baselines for finding dynamic targets in simulated unknown grid environments.

  46. Multi-Agent Reinforcement Learning Scheduling to Support Low Latency in Teleoperated Driving

    cs.NI 2025-05 conditional novelty 4.0 of 10

    In ns-3 simulations, MAPPO with greedy allocation lowers average end-to-end latency and raises latency-success probability versus round-robin for teleoperated driving.

  47. Contextual Knowledge Sharing in Multi-Agent Reinforcement Learning with Decentralized Communication and Coordination

    cs.MA 2025-01 reject novelty 4.0 of 10

    A Dec-MARL framework with goal-aware and time-aware knowledge sharing is proposed, but its reported gains are contradicted by the paper's own tables.

  48. Defending Against Network Attacks for Secure AI Agent Migration in Vehicular Metaverses

    cs.NI 2024-12 conditional novelty 4.0 of 10

    A MAPPO-based pre-migration strategy with a trust-score filter is claimed to defend AI agent migration in vehicular metaverses against DDoS and malicious RSU attacks and to cut latency by roughly 43% in simulation.

  49. Agent-Temporal Credit Assignment for Optimal Policy Preservation in Sparse Multi-Agent Reinforcement Learning

    cs.MA 2024-12 reject novelty 4.0 of 10

    TAR2 redistributes sparse multi-agent rewards both across time and across agents, but its optimal-policy-preservation proof depends on a trajectory-dependent 'potential' and is not valid.

  50. GenAI-based Multi-Agent Reinforcement Learning towards Distributed Agent Intelligence: A Generative-RL Agent Perspective

    cs.AI 2025-07 unverdicted novelty 3.0 of 10

    A position paper claiming that generative-AI agents that model and predict multi-agent dynamics will replace today's reactive MARL approaches.

  51. Multi-Agent Reinforcement Learning in Cybersecurity: From Fundamentals to Applications

    cs.MA 2025-05 conditional novelty 3.0 of 10

    A narrative survey of multi-agent reinforcement learning for cyber defense, reviewing game-theoretic models, cyber gyms, and applications, concluding MARL is promising but faces scalability and simulation-to-real tran...

  52. Multi-Agent Reinforcement Learning in Wireless Distributed Networks for 6G

    cs.IT 2025-02 conditional novelty 1.0 of 10

    A comprehensive survey of multi-agent reinforcement learning for wireless distributed networks in 6G, covering structures, algorithms, enhanced techniques, and applications.

Pith tools