Pith. sign in

REVIEW 3 major objections 7 minor 2 cited by

Understanding Individual Agent Importance in Multi-Agent System via Counterfactual Reasoning

T0 review · 3 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper's central claim is that agent importance in a black-box multi-agent system can be learned as a masking probability, with less masking meaning more importance.

desk verdict A real gap and a plausible MARL-based method, but the evaluation is partly self-confirming and the key hyperparameters aren't reported. read the letter →

arxiv 2412.15619 v2 pith:VC6KCI4W submitted 2024-12-20 cs.AI cs.MA

Jianming Chen , Yawen Wang , Junjie Wang , Xiaofei Xie , jun Hu , Qing Wang , Fanjiang Xu This is my paper · ORCID
classification cs.AIcs.MA
keywords agent-levelexplanationcounterfactualreasoningmulti-agentreinforcementlearningmaskingpolicyexplainableagentimportancefidelityevaluationcentralizedtrainingwithdecentralizedexecution
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that in a trained multi-agent team, each agent's importance at a given state can be read out as a learned masking probability: the less likely a learned masking policy is to randomize an agent's action, the more that agent matters to the team's reward. It frames this as a multi-agent reinforcement-learning problem, training masking agents whose only job is to preserve the original expected reward while randomizing as many target agents as possible. Across seven tasks from well-known benchmarks, the resulting explanations beat three baselines on fidelity, with relative improvements from 11% to 118%, and outperform them when used to guide policy understanding, adversarial attacks, and policy patching. If the claim holds, black-box multi-agent policies become inspectable at the level of individual agents and time-steps, which would make targeted interventions such as finding lazy agents, attacking critical units, or patching weak links more efficient.

What carries the argument

The central object is the masking policy $\pi_\theta$: a set of $n$ agents whose action space is $\{\mathrm{keep}, \mathrm{randomize}\}$, one for each target agent. The load-bearing inversion is the readout of importance from masking probability, because the policy is trained to make randomization impact reward as little as possible, it randomizes only agents whose random actions are cheap, so the agents left with low masking probability are the important ones. The policy is trained as a DEC-POMDP with the reward-preservation objective $|J(\pi)-J(\pi_\theta)|$, a sparsity bonus $\beta$ times the number of masked agents, and a one-step temporal-difference loss in which a central critic $Q^{tot}(o,a^m)$, built with non-negative weights to satisfy the Individual-Global-Max principle, estimates the joint value of masking actions.

What would settle it

Take a small benchmark, fix a set of states, and for each state randomize each agent's action alone many times to measure the true average reward drop, then compare that ranking with EMAI's masking probabilities. If the ranking changes when the sparsity weight $\beta$ is varied or when agents are masked in groups rather than singly, the masking probability is not a stable measure of individual counterfactual importance.

Watch

Extended reading notes

Core claim

The paper claims that the counterfactual quantity "expected reward lost when an agent's action is randomized" can be characterized by a learned policy instead of by enumerating random perturbations. EMAI trains $n$ masking agents, one per target agent, that output a binary action at each time-step: keep the target agent's action or replace it with a random action. The training objective is to minimize the absolute difference between the original expected reward $J(\pi)$ and the expected reward under masking $J(\pi_\theta)$, plus a sparsity reward $\beta \sum_i a^m_{t,i}$ that encourages masking more agents; the masking probability assigned to each agent is then reported as the inverse importance. Because joint masking decisions interact across agents and across time, the masking policies are learned with centralized training and decentralized execution, using a central critic $Q^{tot}(o,a^m)$ whose value is decomposed into per-agent values. In experiments on seven tasks spanning StarCraft battles, football, and particle navigation, the method records the highest relative reward difference (RRD) among all compared explanations, with improvements of 11% to 118%; in downstream tests, attacks guided by its importance rankings reduce team rewards more, and patches guided by them improve team rewards more, than baseline-guided interventions.

Load-bearing premise

The load-bearing premise is that the probability a trained masking policy assigns to randomizing an agent's action faithfully ranks how much that agent's random action actually lowers the team's expected reward, even though the ranking is trained with a hand-set sparsity weight $\beta$ that encourages masking many agents.

Editorial extensions

If this is right

  • Users can now ask which agent is critical at any state of a black-box multi-agent system, not just which observation region or time-step matters.
  • Attackers can concentrate perturbations on a few high-importance agents, achieving larger reward reductions than attacks guided by existing explanations.
  • Defenders can patch policies by replacing the actions of critical agents with recorded high-reward actions, yielding consistent reward gains across all seven tested tasks.
  • The same importance ranking exposes low-contributing agents, pointing to targeted retraining or credit-assignment changes.
  • The approach is black-box compatible: it needs only observations and actions of the target agents, not their parameters or value functions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the masking-probability readout tracks true single-agent counterfactual impact beyond the tested tasks, an untested corollary is that importance rankings could be assembled into a surrogate for Shapley-style attribution without enumerating exponentially many coalitions.
  • The same learned counterfactual signal could be used during training, not only after it, as a curriculum signal that randomizes or overprotects high-importance agents to stress-test coordination.
  • The reported attack improvements cut both ways: the rankings that help defenders patch policies could also be mined by adversaries, so deployment of such explanations should be paired with robustness analysis of the underlying policy.
  • A direct testable extension would be to compare EMAI's rankings with brute-force single-agent randomization on a small environment, and if the match is close the method could serve as a scalable estimator of causal importance in larger systems.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes EMAI, a post-hoc explanation method for multi-agent systems that estimates the importance of each individual agent at each time step. The method defines importance via counterfactual reasoning: randomizing an important agent's action should cause a large reward drop. To avoid enumerating all agents, EMAI learns a masking policy that decides, at each step, which target agents should have their actions replaced by random actions. The masking probabilities are then interpreted as inverse importance scores. The learning problem is formulated as a decentralized POMDP and solved with a CTDE framework using a QMIX-style value decomposition. The authors evaluate EMAI on seven tasks from SMAC, GRF, and MPE, comparing against StateMask, VB, and GBA. They report that EMAI achieves higher fidelity (measured by relative reward difference, RRD) and better guidance for understanding policies, launching attacks, and patching policies.

Significance. Agent-level importance explanation for black-box multi-agent systems is a relevant and underexplored problem with practical implications for diagnosing policy failures and targeting interventions. Modeling the explanation search as a MARL problem is a novel and potentially scalable way to avoid brute-force counterfactual enumeration. However, the current experimental evidence is undermined by a near-tautological relationship between the training objective and the fidelity metric, an unreported sparsity hyperparameter, and a loss function that appears misspecified. If these issues are corrected and the method is validated against independent ground-truth counterfactual importance, the approach could be a solid contribution to explainable multi-agent reinforcement learning.

major comments (3)
  1. [Fidelity Evaluation, Eq. (2) and Eq. (5)] The fidelity metric RRD measures the reward change when the identified most critical agent is randomized. This is essentially the same quantity that EMAI's training objective (Eq. 2) minimizes: the expected reward difference before and after randomizing selected agents' actions. Because the masking probabilities directly determine which agent is selected in the fidelity test, the reported high RRD is at least partly constructed by the training objective rather than independently verified. To break this circularity, the authors should compare EMAI's importance scores against a ground-truth single-agent counterfactual estimate obtained by Monte Carlo randomization of each agent individually, and report the correlation or rank agreement.
  2. [Approach, Eq. (5), Experiments] The sparsity weight β in Eq. (5) is never reported, and no sensitivity or ablation analysis is provided. β controls the trade-off between preserving the original reward and masking as many agents as possible, and it directly shapes the learned masking probabilities. Without a reported value or a sweep over β, the fidelity and practicability results are conditional on an unspecified hyperparameter, and the mapping from masking probability to a meaningful single-agent importance score is not validated.
  3. [Approach, Eq. (9)] The loss function Ld in Eq. (9) appears misspecified. It computes the squared error between the constant J(π) and the sum Σ_t γ^t (Qtot(o, a^m_t) − Rm(a^m_t)). Since Qtot already estimates the discounted future return from time t onward, summing the discounted Qtot values over all time steps double-counts rewards and is dimensionally inconsistent. If this loss is actually used in training, it is unclear that minimizing Ld corresponds to minimizing |J(π) − J(πθ)|. The authors should derive Eq. (9) carefully or replace it with a correct surrogate that provably bounds or approximates the counterfactual reward difference.
minor comments (7)
  1. [Abstract] There are typos: 'provideed' should be 'provided' and 'demonstratee' should be 'demonstrate'.
  2. [Experiments, Figure 2] The figure reports only means and standard deviations; for several tasks the error bars overlap (e.g., SMAC-1c3s5z). Please provide statistical significance tests, such as paired bootstrap confidence intervals, to support the claim of consistent improvements.
  3. [Experiments, Experimental Setup] The hyperparameters λ (in Eq. 10) and β (in Eq. 5) are not listed. Please include their values and the training details (e.g., learning rate, network architecture) for reproducibility.
  4. [Understanding Policies] The user study reports that 75% of participants preferred EMAI and 58% found it helpful, but no confidence intervals, inter-rater agreement, or details of the questionnaire are provided. Please elaborate on the study design.
  5. [Approach, Eq. (7)] The notation ω(Q_i(o_i, a_i^m; θ)) is vague; clarify whether ω is a sum, a monotonic neural network, or another aggregation function.
  6. [Patching Policies] The similarity threshold d_th is not specified in the text or tables. Please include its value or describe how it is selected.
  7. [Related Work] Since Shapley value is discussed in related work, consider adding a Shapley-based agent attribution baseline or explicitly justify why it is not applicable in this black-box, per-time-step setting.

Circularity Check

1 steps flagged · score 6.0 of 10

Fidelity evaluation re-measures EMAI's training objective: the RRD metric is the same counterfactual reward-difference signal used to fit the masking policy, so the central fidelity claim is partially forced by construction.

  1. fitted input called prediction [Approach — Problem Modeling (Eq. 2) and Experiments — Fidelity Evaluation (RRD metric)]
    ""The optimization goal of the policy is to minimize the reward difference before and after randomization. Thus, a lower probability of an agent being selected for randomization indicates its higher importance. ... obj(πθ) = arg min θ |J(π) − J(πθ)|, (2) ... Therefore, at each time-step, we select the most critical agent based on the explanation approach and randomize its action ... Then the fidelity can be expressed as: RRD = |Re − Ro|/|Rr − Ro|.""

    EMAI's explanation is the masking probability πθ(a^m_i=1|o_i), with the paper's own mapping that lower probability means higher importance. That probability is fitted to the objective of minimizing the reward difference caused by randomizing masked agents (Eq. 2, with sparsity Eq. 5). The fidelity metric then randomizes the agent with the lowest masking probability and measures the resulting reward change, i.e., it evaluates the learned policy against exactly the counterfactual reward-difference objective used as its training signal. A high RRD therefore restates that the policy learned Eq. 2 rather than providing independent evidence that the probability equals ground-truth single-agent counterfactual importance.

full rationale

The main circularity is evaluative: EMAI's readout and its fidelity metric share the same counterfactual reward-difference objective. Because the policy is trained to keep reward nearly unchanged when it randomizes agents, the lowest-probability agent is, by the training signal, the one whose randomization the policy has learned to avoid; randomizing that agent and measuring the reward drop (RRD) verifies whether training worked, not whether the explanation matches an independent notion of importance. This is a partial circularity rather than a complete one: the baselines are external, the attack and patch studies provide some downstream evidence, and no load-bearing argument rests on self-citation. The apparent misspecification of Eq. 9 (subtracting only the instantaneous Rm from Qtot before accumulating over γ^t) is a correctness concern and would weaken the claimed link to Eq. 2, but it is not itself a circular step. Overall, the central fidelity comparison is substantially forced by the alignment between the fitted objective and the evaluation metric, so the circularity score is 6.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central method rests on a small set of design choices: the counterfactual definition of importance, the sparsity-weighted objective that shapes the masking probabilities, and the QMIX-style value decomposition. The sparsity weight β is the most consequential free parameter because it directly determines the threshold of what counts as important.

free parameters (3)
  • β (sparsity weight) = not reported
    Weight of sparsity reward in Eq. 5; controls the trade-off between masking many agents and preserving reward. Importance scores are conditional on this value, but the paper does not report it or analyze sensitivity.
  • λ (loss weight) = not reported
    Weight balancing the TD loss and the reward-difference loss in Eq. 10.
  • d_th (patching threshold) = not reported
    Threshold for observation similarity in the patching application; affects patch guidance results, though it is secondary to the central claim.
assumptions (4)
  • domain assumption Randomized-action reward drop is a valid measure of individual agent importance.
    This is the definition underlying the approach (Problem Modeling section). It is intuitively natural but not proven; important multi-agent effects such as role redundancy or delayed credit could weaken it.
  • domain assumption The masking decision process is a DEC-POMDP with the target agents fixed as part of the environment.
    Standard formulation; assumes local observations are sufficient for importance decisions.
  • standard math IGM principle and non-negative weight factorization (QMIX) correctly reconstruct the joint value function.
    Borrowed from prior MARL work (Rashid et al. 2018); the monotonicity constraint limits expressiveness but supports credit assignment.
  • ad hoc to paper Sparsity reward encourages exploration of more action randomization without distorting importance.
    The sparsity term is a design choice that directly shapes the masking probability; its interaction with importance is not theoretically characterized.
invented entities (1)
  • Masking agents
    purpose: Auxiliary policies that decide at each time-step whether to replace each target agent's action with a random one; their probabilities encode importance.
    The masking agents are a methodological construct. Their explanatory value is tested within the paper, but there is no separate falsifiable handle outside the reported experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Understanding Individual Agent Importance in Multi-Agent System via Counterfactual Reasoning." pith.science (2026). https://pith.science/paper/VC6KCI4W

@misc{pith2026241215619,
  author       = {Pith},
  title        = {Pith review of: Understanding Individual Agent Importance in Multi-Agent System via Counterfactual Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VC6KCI4W}},
  note         = {Machine review of arXiv:2412.15619}
}
read the original abstract

Explaining multi-agent systems (MAS) is urgent as these systems become increasingly prevalent in various applications. Previous work has proveided explanations for the actions or states of agents, yet falls short in understanding the black-boxed agent's importance within a MAS and the overall team strategy. To bridge this gap, we propose EMAI, a novel agent-level explanation approach that evaluates the individual agent's importance. Inspired by counterfactual reasoning, a larger change in reward caused by the randomized action of agent indicates its higher importance. We model it as a MARL problem to capture interactions across agents. Utilizing counterfactual reasoning, EMAI learns the masking agents to identify important agents. Specifically, we define the optimization function to minimize the reward difference before and after action randomization and introduce sparsity constraints to encourage the exploration of more action randomization of agents during training. The experimental results in seven multi-agent tasks demonstratee that EMAI achieves higher fidelity in explanations than baselines and provides more effective guidance in practical applications concerning understanding policies, launching attacks, and patching policies.

Figures

Figures reproduced from arXiv: 2412.15619 by the authors.

Figure 1
Figure 1. The overview of our proposed EMAI. (a) At each time-step, EMAI outputs the masking probability for the action [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. The results of the fidelity evaluation. The bar represents the mean value, and the black line on the bar represents the [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. The illustrations of EMAI identified critical agents, which is marked by the red box. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Society of Mind Meets Real-Time Strategy: A Hierarchical Multi-Agent Framework for Strategic Reasoning

    cs.AI 2025-08 conditional novelty 6.0 of 10

    A hierarchical framework of specialized imitation agents plus a strategic planner improves win rates and cuts LLM calls in text-based StarCraft II across all race matchups.

  2. InstantEdit: Text-Guided Few-Step Image Editing with Piecewise Rectified Flow

    cs.CV 2025-08 unverdicted novelty 5.0 of 10

    InstantEdit combines RectifiedFlow inversion, latent injection, disentangled prompt guidance, and Canny ControlNet to do fast few-step text-guided image editing with content preservation.

Reference graph

Works this paper leans on

55 extracted references · 51 canonical work pages · cited by 2 Pith papers

  1. [1]

    Amir, D.; and Amir, O. 2018. HIGHLIGHTS: Summarizing Agent Behavior to People. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems, 1168–1176

    work page 2018

  2. [2]

    Atrey, A.; Clary, K.; and Jensen, D. 2020. Exploratory Not Explanatory: Counterfactual Analysis of Saliency Maps for Deep Reinforcement Learning. In International Conference on Learning Representations

    work page 2020

  3. [3]

    Bewley, T.; and Lawry, J. 2021. TripleTree: A Versatile Interpretable Representation of Black Box Agents and their Environments. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 11415--11422

    work page 2021

  4. [4]

    Chang, C.-H.; Creager, E.; Goldenberg, A.; and Duvenaud, D. 2019. Explaining Image Classifiers by Counterfactual Generation. In International Conference on Learning Representations

    work page 2019

  5. [5]

    Chen, B.; Xu, M.; Liu, Z.; Li, L.; and Zhao, D. 2020. Delay-Aware Multi-Agent Reinforcement Learning for Cooperative and Competitive Environments. arXiv

    work page 2020

  6. [6]

    Cheng, Z.; Wu, X.; Yu, J.; Sun, W.; Guo, W.; and Xing, X. 2023. StateMask: Explaining Deep Reinforcement Learning through State Mask. In Oh, A.; Neumann, T.; Globerson, A.; Saenko, K.; Hardt, M.; and Levine, S., eds., Advances in Neural Information Processing Systems, volume 36, 62457--62487. Curran Associates, Inc

    work page 2023

  7. [7]

    Ci, H.; Liu, M.; Pan, X.; Zhong, F.; and Wang, Y. 2023. Proactive Multi-Camera Collaboration for 3D Human Pose Estimation. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023

    work page 2023

  8. [8]

    Coppens, Y.; Efthymiadis, K.; Lenaerts, T.; Now \'e , A.; Miller, T.; Weber, R.; and Magazzeni, D. 2019. Distilling Deep Reinforcement Learning Policies in Soft Decision Trees. In Proceedings of the IJCAI 2019 Workshop on Explainable Artificial Intelligence, 1--6

    work page 2019

Show all 55 references
  1. [9]

    Cui, Q.; Zhang, K.; and Du, S. 2023. Breaking the Curse of Multiagents in a Large State Space: RL in Markov Games with Independent Linear Function Approximation. In Proceedings of Thirty Sixth Conference on Learning Theory, volume 195 of Proceedings of Machine Learning Researc...

    2023

  2. [10]

    K.; and Collins, A

    Eckstein, M. K.; and Collins, A. G. E. 2020. Computational evidence for hierarchically structured reinforcement learning in humans. Proceedings of the National Academy of Sciences, 117: 29381--29389

    2020

  3. [11]

    Q.; and Yuen, C

    Feng, Z.; Huang, M.; Wu, D.; Wu, E. Q.; and Yuen, C. 2023. Multi-Agent Reinforcement Learning With Policy Clipping and Average Evaluation for UAV-Assisted Communication Markov Game. IEEE Transactions on Intelligent Transportation Systems, 24: 14281--14293

    2023

  4. [12]

    A.; de Freitas, N.; and Whiteson, S

    Foerster, J.; Assael, I. A.; de Freitas, N.; and Whiteson, S. 2016. Learning to Communicate with Deep Multi-Agent Reinforcement Learning. In Advances in Neural Information Processing Systems, volume 29

    2016

  5. [13]

    N.; Farquhar, G.; Afouras, T.; Nardelli, N.; and Whiteson, S

    Foerster, J. N.; Farquhar, G.; Afouras, T.; Nardelli, N.; and Whiteson, S. 2018. Counterfactual Multi-Agent Policy Gradients. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18) , 2974--2982

    2018

  6. [14]

    C.; and Vedaldi, A

    Fong, R. C.; and Vedaldi, A. 2017. Interpretable Explanations of Black Boxes by Meaningful Perturbation. In Proceedings of the IEEE International Conference on Computer Vision (ICCV)

    2017

  7. [15]

    Goyal, Y.; Wu, Z.; Ernst, J.; Batra, D.; Parikh, D.; and Lee, S. 2019. Counterfactual Visual Explanations. In Proceedings of the 36th International Conference on Machine Learning, volume 97, 2376--2384

    2019

  8. [16]

    Greydanus, S.; Koul, A.; Dodge, J.; and Fern, A. 2018. Visualizing and Understanding Atari Agents. In ICML, 1787--1796

    2018

  9. [17]

    Guo, W.; Wu, X.; Khan, U.; and Xing, X. 2021. EDGE : Explaining Deep Reinforcement Learning Policies. In Advances in Neural Information Processing Systems

    2021

  10. [18]

    J.; and Stone, P

    Hausknecht, M. J.; and Stone, P. 2015. Deep Recurrent Q-Learning for Partially Observable MDPs. In Proceedings of the AAAI Conference on Artificial Intelligence, 29--37

    2015

  11. [19]

    Hong, Y.; Jin, Y.; and Tang, Y. 2022. Rethinking Individual Global Max in Cooperative Multi-Agent Reinforcement Learning. In Advances in Neural Information Processing Systems, volume 35, 32438--32449

    2022

  12. [20]

    H.; Bhatia, K.; Abbeel, P.; and Dragan, A

    Huang, S. H.; Bhatia, K.; Abbeel, P.; and Dragan, A. D. 2018. Establishing Appropriate Trust via Critical States. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 3929--3936

    2018

  13. [21]

    Jacq, A.; Ferret, J.; Pietquin, O.; and Geist, M. 2022. Lazy-mdps: Towards interpretable rl by learning when to act. In Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems, 669--677

    2022

  14. [22]

    Jamroga, W.; and Kim, Y. 2023. Practical Model Reductions for Verification of Multi-Agent Systems. arXiv

    2023

  15. [23]

    E.; Venkatasubramanian, S.; Scheidegger, C.; and Friedler, S

    Kumar, I. E.; Venkatasubramanian, S.; Scheidegger, C.; and Friedler, S. 2020. Problems with Shapley-value-based explanations as feature importance measures. In Proceedings of the 37th International Conference on Machine Learning, volume 119, 5491--5500. PMLR

    2020

  16. [24]

    Kurach, K.; Raichuk, A.; Stańczyk, P.; Zajac, M.; Bachem, O.; Espeholt, L.; Riquelme, C.; Vincent, D.; Michalski, M.; Bousquet, O.; and Gelly, S. 2020. Google Research Football: A Novel Reinforcement Learning Environment. In Proceedings of the AAAI Conference on Artificial Int...

    2020

  17. [25]

    Li, J.; Kuang, K.; Wang, B.; Liu, F.; Chen, L.; Wu, F.; and Xiao, J. 2021. Shapley Counterfactual Credits for Multi-Agent Reinforcement Learning. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, KDD '21, 934–942

    2021

  18. [26]

    Liu, B.; Pu, Z.; Pan, Y.; Yi, J.; Liang, Y.; and Zhang, D. 2023 a . Lazy Agents: A New Perspective on Solving Sparse Reward Problem in Multi-agent Reinforcement Learning. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Mach...

    2023

  19. [27]

    Liu, D.; Dou, L.; Zhang, R.; Zhang, X.; and Zong, Q. 2023 b . Multi-Agent Reinforcement Learning-Based Coordinated Dynamic Task Allocation for Heterogenous UAVs. IEEE Transactions on Vehicular Technology, 72: 4372--4383

    2023

  20. [28]

    Liu, Z.; Zhu, Y.; and Chen, C. 2023. N A ^ 2 Q : Neural Attention Additive Model for Interpretable Multi-Agent Q-Learning. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, 22539--22558. PMLR

    2023

  21. [29]

    Louhichi, M.; Nesmaoui, R.; Mbarek, M.; and Lazaar, M. 2023. Shapley Values for Explaining the Black Box Nature of Machine Learning Model Clustering. Procedia Computer Science, 220: 806--811

    2023

  22. [30]

    Lowe, R.; Wu, Y.; Tamar, A.; Harb, J.; Abbeel, P.; and Mordatch, I. 2017. Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments. Neural Information Processing Systems (NIPS)

    2017

  23. [31]

    Luo, R.; Ni, W.; Tian, H.; Cheng, J.; and Chen, K.-C. 2023. Joint Trajectory and Radio Resource Optimization for Autonomous Mobile Robots Exploiting Multi-Agent Reinforcement Learning. IEEE Transactions on Communications, 71: 5244--5258

    2023

  24. [32]

    Lv, Z.; Xiao, L.; Du, Y.; Zhu, Y.; Han, S.; and Liu, Y.-J. 2024. Efficient Communications in Multi-Agent Reinforcement Learning for Mobile Applications. IEEE Transactions on Wireless Communications, 1--1

    2024

  25. [33]

    McCalmon, J.; Le, T.; Alqahtani, S.; and Lee, D. 2022. CAPS: Comprehensible Abstract Policy Summaries for Explaining Reinforcement Learning Agents. In Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems, AAMAS '22, 889–897

    2022

  26. [34]

    A.; Veness, J.; Bellemare, M

    Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A. A.; Veness, J.; Bellemare, M. G.; Graves, A.; Riedmiller, M. A.; Fidjeland, A.; Ostrovski, G.; Petersen, S.; Beattie, C.; Sadik, A.; Antonoglou, I.; King, H.; Kumaran, D.; Wierstra, D.; Legg, S.; and Hassabis, D. 2015. Human-leve...

    2015

  27. [35]

    A.; Amato, C.; et al

    Oliehoek, F. A.; Amato, C.; et al. 2016. A concise introduction to decentralized POMDPs, volume 1. Springer

    2016

  28. [36]

    Pan, X.; Liu, M.; Zhong, F.; Yang, Y.; Zhu, S.-C.; and Wang, Y. 2022. MATE: Benchmarking Multi-Agent Reinforcement Learning in Distributed Target Coverage Control. In Koyejo, S.; Mohamed, S.; Agarwal, A.; Belgrave, D.; Cho, K.; and Oh, A., eds., Advances in Neural Information ...

    2022

  29. [37]

    Petrillo, A.; Salvi, A.; Santini, S.; and Valente, A. S. 2018. Adaptive multi-agents synchronization for collaborative driving of autonomous vehicles with multiple communication delays. Transportation Research Part C: Emerging Technologies, 86: 372–392

    2018

  30. [38]

    Puri, N.; Verma, S.; Gupta, P.; Kayastha, D.; Deshmukh, S.; Krishnamurthy, B.; and Singh, S. 2020. Explain Your Move: Understanding Agent Actions Using Specific and Relevant Feature Attribution. In International Conference on Learning Representations

    2020

  31. [39]

    Qiu, C.; Wu, Z.; Wang, J.; Tan, M.; and Yu, J. 2023. Multiagent-Reinforcement-Learning-Based Stable Path Tracking Control for a Bionic Robotic Fish With Reaction Wheel. IEEE Transactions on Industrial Electronics, 70: 12670--12679

    2023

  32. [40]

    Rashid, T.; Samvelyan, M.; Schroeder, C.; Farquhar, G.; Foerster, J.; and Whiteson, S. 2018. QMIX : Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceed...

    2018

  33. [41]

    Samvelyan, M.; Rashid, T.; Schroeder de Witt, C.; Farquhar, G.; Nardelli, N.; Rudner, T. G. J.; Hung, C.-M.; Torr, P. H. S.; Foerster, J.; and Whiteson, S. 2019. The StarCraft Multi-Agent Challenge. In Proceedings of the 18th International Conference on Autonomous Agents and M...

    2019

  34. [42]

    B.; Trivedi, A.; and Srinivasan, D

    Saner, C. B.; Trivedi, A.; and Srinivasan, D. 2022. A Cooperative Hierarchical Multi-Agent System for EV Charging Scheduling in Presence of Multiple Charging Stations. IEEE Transactions on Smart Grid, 13: 2218--2233

    2022

  35. [43]

    J.; Hostallero, D

    Son, K.; Kim, D.; Kang, W. J.; Hostallero, D. E.; and Yi, Y. 2019. QTRAN : Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine...

    2019

  36. [44]

    Srinivas, S.; and Fleuret, F. 2021. Rethinking the Role of Gradient-based Attribution Methods for Model Interpretability. In International Conference on Learning Representations

    2021

  37. [45]

    M.; Zambaldi, V.; Jaderberg, M.; Lanctot, M.; Sonnerat, N.; Leibo, J

    Sunehag, P.; Lever, G.; Gruslys, A.; Czarnecki, W. M.; Zambaldi, V.; Jaderberg, M.; Lanctot, M.; Sonnerat, N.; Leibo, J. Z.; Tuyls, K.; and Graepel, T. 2017. Value-Decomposition Networks For Cooperative Multi-Agent Learning. arXiv preprint

    2017

  38. [46]

    Wang, J.; Ren, Z.; Liu, T.; Yu, Y.; and Zhang, C. 2021. QPLEX: Duplex Dueling Multi-Agent Q-Learning. In International Conference on Learning Representations

    2021

  39. [47]

    Wang, J.; Zhang, Y.; Kim, T.-K.; and Gu, Y. 2020. Shapley Q-Value: A Local Reward Approach to Solve Global Reward Games. In Proceedings of the AAAI Conference on Artificial Intelligence, 7285--7292

    2020

  40. [48]

    Wu, R.; Zhong, J.; Wallace, B.; Gao, X.; Huang, H.; and Si, J. 2022. Human-Robotic Prosthesis as Collaborating Agents for Symmetrical Walking. In Advances in Neural Information Processing Systems, volume 35, 27306--27320

    2022

  41. [49]

    Xu, Z.; Zhang, B.; li, d.; Zhou, G.; Zhang, Z.; and Fan, G. 2023. Dual Self-Awareness Value Decomposition Framework without Individual Global Max for Cooperative MARL. In Advances in Neural Information Processing Systems, volume 36, 73898--73918

    2023

  42. [50]

    Yu, J.; Guo, W.; Qin, Q.; Wang, G.; Wang, T.; and Xing, X. 2023. AIRS : Explanation for Deep Reinforcement Learning based Security Applications. In 32nd USENIX Security Symposium (USENIX Security 23), 7375--7392

    2023

  43. [51]

    Yu, Z.; Wang, K.; Xie, S.; Zhong, Y.; and Lv, Z. 2022. Prototypical network based on Manhattan distance. Cmes-Comput. Model. Eng. Sci, 131: 655--675

    2022

  44. [52]

    S.; and Hsieh, C.-J

    Zhang, H.; Chen, H.; Boning, D. S.; and Hsieh, C.-J. 2021. Robust Reinforcement Learning on State Observations with Learned Optimal Adversary. In International Conference on Learning Representations

    2021

  45. [53]

    Zhang, T.; Guo, S.; Tan, T.; Hu, X.; and Chen, F. 2020. Generating Adjacency-Constrained Subgoals in Hierarchical Reinforcement Learning. In Advances in Neural Information Processing Systems, volume 33, 21579--21590

    2020

  46. [54]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...

  47. [55]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.