Pith. sign in

REVIEW 5 major objections 4 minor 1 cited by

An Outlook on the Opportunities and Challenges of Multi-Agent AI Systems

T0 review · 5 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper's central claim is that a multi-agent AI system is only as good as the diversity of its agents and the alignment of its interaction topology, and that these conditions, not the number of agents, decide whether it beats a…

desk verdict A clean, honest position paper with a mostly standard formal framework; the main quantitative claim rests on a proxy the authors themselves disavow. read the letter →

arxiv 2505.18397 v3 pith:DB5JVGFE submitted 2025-05-23 cs.MA cs.AIcs.ETcs.LG

Fangqiao Tian , An Luo , Jin Du , Xun Xian , Robert Specht , Ganghua Wang , Xuan Bi , Jiawei Zhou
show 7 more authors
Ashish Kundu Jayanth Srinivasa Charles Fleming Rui Zhang Zirui Liu Mingyi Hong Jie Ding
This is my paper · ORCID
classification cs.MAcs.AIcs.ETcs.LG
keywords multi-agentAIsystemsLLMagentsensemblerobustnesstrainingdataoverlapbackdoorattacksvulnerabilitypropagationtaskallocationfeedbackintegration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the value of a multi-agent AI system (MAS) is not automatic: it beats a single-agent system only when tasks can be flexibly divided and reallocated, when agents are trained on sufficiently diverse data, and when feedback is integrated effectively. It supplies formal definitions for an individual agent, a time-varying communication topology, and feedback updates so that these conditions can be stated precisely. On the safety side, it argues that errors made by upstream agents can silently bias downstream agents even without adversarial intent, and that star and cascade topologies can amplify or suppress backdoor attacks depending on how agents' vulnerabilities align. The central message is that redundancy helps only when it brings genuine diversity, and that multi-agent design is a trade-off, not a guarantee.

What carries the argument

The framework's central objects are the formal definition of an AI agent as a tuple $(S_i, X_i, Y_i, p_i)$ with a transition kernel $p_i$, a time-varying multi-agent topology $G^{(t)}$ governed by a graph update function $\phi$, and the vulnerability functional $V(F; P, \delta)$ that measures a system's performance drop after a perturbation. The load-bearing identities are the Hoeffding bound on majority-vote error, which supplies the exponential robustness claim under independent errors, and the directional alignment score $\langle \Delta_A, \Delta_B \rangle$ between the direction in which an upstream agent transforms data and the direction to which a downstream agent is sensitive. These objects translate the intuitive claims about redundancy and vulnerability into conditions that can be checked empirically.

What would settle it

Train several LLM-based agents on identical data ($\rho = 1$) but with different random seeds or decoding temperatures and compare majority-vote accuracy against a single agent; if the ensemble consistently beats the single agent despite full data overlap, the paper's claim that redundancy collapses to a single-agent baseline is false. Alternatively, measure the actual prediction correlation between agents at each overlap ratio and check whether accuracy tracks correlation rather than $\rho$.

Watch

Extended reading notes

Core claim

The paper's central claim is that the effectiveness and safety of a multi-agent AI system hinge on measurable structural conditions rather than on the number of agents. It shows formally that majority voting among agents improves robustness exponentially only when agent errors are independent; when agents share training data, their errors become correlated and the robustness gain disappears, collapsing to a single-agent baseline at full overlap. It also shows through a two-agent feature-selection simulation that upstream errors can degrade downstream performance without any adversarial intent, and that star versus cascade topologies either amplify or attenuate backdoor vulnerabilities depending on directional alignment between agents. The conclusion is that multi-agent systems are not inherently more reliable or more effective; their benefits must be earned through diversity, feedback, and careful topology design.

Load-bearing premise

The robustness conclusion rests on treating the fraction of overlapping training data, $\rho$, as a proxy for how correlated the agents' predictions are; the paper concedes that overlap is not a direct measure of prediction correlation, so the mechanism may weaken when real LLM-based agents share pretraining corpora yet produce diverse outputs.

Editorial extensions

If this is right

  • MAS can beat single-agent systems when tasks are decomposed and reallocated in real time and communication overhead stays small; otherwise extra agents add cost without benefit.
  • Redundancy improves robustness only when agent training data are diverse; agents trained on overlapping corpora should be expected to make correlated mistakes that voting cannot correct.
  • Multi-agent pipelines can silently propagate upstream errors, so a well-designed single-agent system can outperform a poorly aligned multi-agent pipeline.
  • Backdoor and adversarial risks depend on system topology: cascade and star designs can amplify a trigger when agents' vulnerable directions align, or filter it when they do not.
  • Feedback acts as a Bayesian update on agent behavior and can rewire the agent graph, making feedback design a first-class component of MAS rather than an optional add-on.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct testable extension is to replace the overlap-ratio proxy with estimates of actual prediction correlation, which would reveal whether the robustness collapse is driven by data sharing itself or by the correlated outputs it causes.
  • The alignment-score view of vulnerability suggests a design principle the paper leaves implicit: deliberately diversifying agents' failure directions, not just their training data, could turn malicious signals into destructive interference.
  • If the Internet-of-MAS layer becomes real, the Agent Directory and Agent Gateway become new attack surfaces, since poisoning an agent's metadata or blocking messages could act like a topological backdoor.
  • The cost model for a proposal-and-refinement pair implies an engineering rule that can be tested directly: use multi-agent decomposition when the search-space reduction outweighs measured communication overhead.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes a formal framework for multi-agent AI systems, defining agents, dynamic topologies, feedback updates, and an Internet-of-MAS protocol stack. It then examines two questions: when MAS outperform single-agent systems, and how safety vulnerabilities propagate across agents. The effectiveness analysis covers task allocation, robustness via majority voting, and feedback integration, with a toy ensemble experiment on the Covtype dataset that varies training-data overlap and ensemble size. The safety analysis introduces a vulnerability functional, a directional-alignment notion, and a two-agent feature-transfer simulation on synthetic single-cell data, together with a qualitative discussion of star and cascade topologies. The paper concludes that redundancy helps only with sufficiently diverse training data and that upstream errors can covertly bias downstream agents.

Significance. The paper is useful as a position paper: it provides a compact formal vocabulary for MAS and correctly identifies the independence/diversity condition for ensemble benefits and the topology-dependence of vulnerability propagation. The probabilistic derivations (Hoeffding bound and correlated-Bernoulli calculation) are correct and appropriately conditional. The two experiments are small but illustrative; however, their supporting role is weakened by missing experimental details and, in the robustness study, by reliance on an admittedly indirect proxy. The framework's safety claims rest on an informally defined alignment notion. With the requested revisions, the paper could serve as a substantive research roadmap for the signal processing community.

major comments (5)
  1. [3.2.1] The conclusion that redundancy is beneficial 'only when their training data is sufficiently diverse' (Section 5) is not directly supported, because the experiment varies training-data overlap ρ but never measures prediction correlation. The paper itself concedes that 'overlap ratio is not a direct measure of prediction correlation.' For LLM-based agents, shared pretraining corpora can coexist with diverse outputs, and disjoint corpora do not guarantee independent errors. Please report a direct correlation measure (e.g., average pairwise prediction agreement or error correlation) across ρ, or demonstrate the proxy's validity, before drawing the headline robustness condition.
  2. [3.2.1] The empirical study is under-specified: no model class (only 'independently trained agents'), no hyperparameters, no random seeds, no per-point standard deviations or confidence intervals, and no code release are provided. With only 100 training samples per agent and random tie-breaking on a 7-class dataset, the reported accuracy differences may be within noise. Please add full reproducibility details and error bars for each (ρ, k) condition.
  3. [4.1] The text states that a two-agent pipeline can yield 'worse performance than a single-agent baseline,' but Table 1 has no explicit single-agent baseline column. The 'All Features' condition is the natural baseline, but it is not labeled as such, and the comparison logic is unclear. Please define the baseline explicitly and report standard deviations across the 10 simulations, plus seed and data-generation details.
  4. [4.1] The directional alignment score ⟨Δ_A, Δ_B⟩ is central to the vulnerability-propagation analysis, but it is never formally defined: no feature space, norm, or inner product is specified, so statements such as 'Positive alignment indicates amplification' cannot be tested or computed. Please provide a formal definition (or explicitly present the concept as an informal heuristic) and state how it would be estimated in the simulation.
  5. [2.3] The Internet-of-MAS formalization (Definitions 4–7) is presented as inspired by the AGNTCY architecture, but the sole source [19] is a non-peer-reviewed technical report co-authored by members of this team. Please clarify which elements are new to this paper and which are adopted from [19], and consider citing independent protocol efforts to avoid over-reliance on a single self-authored source.
minor comments (4)
  1. [Definitions 2-3] 'An multi-agent topology' and 'An multi-agent AI system' should be 'A multi-agent topology' and 'A multi-agent AI system'.
  2. [4.2] In the star-topology discussion, the condition M(F_star; P) < max_i M(g_i; P) is described as 'worse than its weakest agent,' but max over i is the best agent's performance; the wording should be 'worse than its best individual agent' or the inequality should use min.
  3. [2.2] The phrase 'exactly updates with feedback' overstates the Bayesian update: the update is exact only under the stated conditional-independence assumption that p(F | y, x \ F) does not depend on internal state, so this should be flagged as a modeling assumption rather than a general result.
  4. [References] Reference [27] duplicates [12] and contains a typo in the venue name ('Advances in Neural Information Processing Systemszhe'); please consolidate and correct.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found: the paper's effectiveness and safety claims are supported by self-contained experiments and stipulated definitions, with self-citations used only as background support.

full rationale

The paper's load-bearing derivations are not circular. The effectiveness claim in Section 3.2 is supported by a controlled experiment that directly manipulates the overlap ratio ρ and ensemble size k on the Covtype dataset; ρ is a manipulated input, not a fitted parameter, and the paper explicitly concedes that 'overlap ratio is not a direct measure of prediction correlation' (Section 3.2.1), which is an acknowledged proxy limitation rather than a disguised use of the target conclusion. The safety claims in Section 4.1 rest on a self-contained two-agent feature-selection simulation; the 'directional alignment' discussion cites the authors' prior work [58] as supporting evidence, but that citation is background intuition and the simulation's outcome does not depend on it. The Internet-of-MAS formalization (Section 2.3) and the cost comparison (Section 3.3) are stipulative definitions and algebraic illustrations, respectively, not predictions derived from fitted inputs. No equation is equivalent to its conclusion by construction, and no fitted value is renamed as a prediction. Although the citation to [58] is a self-citation, it is real, externally falsifiable prior work and is not load-bearing, so it does not raise the circularity score.

Assumptions & free parameters 2 free parameters · 4 assumptions · 2 invented entities

The paper introduces few fitted numbers; its claims rest on modeling choices and a proxy assumption, not on parameter estimation. The main free parameters are experimental controls, and the key conceptual construct (directional alignment) lacks independent evidence.

free parameters (2)
  • Overlap ratio ρ = 0, 0.1, 0.8, 1 (varied in experiment)
    Controlled variable in the Covtype ensemble experiment; used to demonstrate that high data overlap erodes voting diversity. Not fitted to any target, but the experiment's conclusion depends on this proxy.
  • Correlation coefficient λ = conceptual, tends to 1 in the failure-mode analysis
    Used in Section 3.2 to show that as λ→1 agent decisions become identical and majority voting stops helping; chosen by hand, not estimated from data.
assumptions (4)
  • domain assumption Agents are probabilistic transition kernels (Definition 1) with well-defined state/output spaces.
    Section 2.1: this abstraction is assumed to cover both symbolic and neural agents; not proven for arbitrary LLM-based agents.
  • ad hoc to paper Feedback is integrated via Bayes rule with known likelihood p(F|y,x), assumed independent of internal state.
    Section 2.2: the likelihood and conditional independence are assumed to hold; no empirical validation in real MAS.
  • domain assumption The Hoeffding bound analysis assumes independent agent errors in the majority vote setup.
    Section 3.2: the paper explicitly states this assumption and then shows it fails under data overlap.
  • ad hoc to paper Vulnerability is defined by a performance gap V(F;P,δ) and directional alignment in feature space determines propagation.
    Section 4.1: the alignment heuristic is supported only by the authors' prior backdoor paper [58], not by the experiments in this paper.
invented entities (2)
  • Directional alignment score ⟨Δ_A, Δ_B⟩
    purpose: Conceptual tool to predict whether a transform pushes data toward another agent's vulnerability zone.
    Introduced in Section 4.1; no measurement or falsifiable prediction is provided outside the paper.
  • Internet of MAS
    purpose: Framing for open networked MAS with Directory/ACP/MCP/Gateway; formalized in Definitions 4-7.
    The components are based on the AGNTCY whitepaper [19] by co-authors, not independently validated here.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An Outlook on the Opportunities and Challenges of Multi-Agent AI Systems." pith.science (2026). https://pith.science/paper/DB5JVGFE

@misc{pith2026250518397,
  author       = {Pith},
  title        = {Pith review of: An Outlook on the Opportunities and Challenges of Multi-Agent AI Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DB5JVGFE}},
  note         = {Machine review of arXiv:2505.18397}
}
read the original abstract

A multi-agent AI system (MAS) is composed of multiple autonomous agents that interact, exchange information, and make decisions based on internal generative models. Recent advances in large language models and tool-using agents have made MAS increasingly practical in areas like scientific discovery and collaborative automation. However, key questions remain: When are MAS more effective than single-agent systems? What new safety risks arise from agent interactions? And how should we evaluate their reliability and structure? This paper outlines a formal framework for analyzing MAS, focusing on two core aspects: effectiveness and safety. We explore whether MAS truly improve robustness, adaptability, and performance, or merely repackage known techniques like ensemble learning. We also study how inter-agent dynamics may amplify or suppress system vulnerabilities. While MAS are relatively new to the signal processing community, we envision them as a powerful abstraction that extends classical tools like distributed estimation and sensor fusion to higher-level, policy-driven inference. Through experiments on data science automation, we highlight the potential of MAS to reshape how signal processing systems are designed and trusted.

Figures

Figures reproduced from arXiv: 2505.18397 by the authors.

Figure 1
Figure 1. Architecture of a multi-agent AI system with three interconnected layers: (1) Application Layer, interacting dynamically with human users and physical environments through feedback and actions; (2) Modeling Layer, featuring specialized agents (Planner, Executor, and Monitor) coordinated by an orchestrator and a knowledge base; and (3) Computation Layer, comprising hardware infrastructures (GPU/CPU/XPU) for executing… view at source ↗
Figure 2
Figure 2. Core update and topology dynamics in an MAS. Left: Illustration of multi-agent topologies at time t−1 and t, showing how agent k receives messages y (t−1) i and y (t−1) j from connected agents and how the topology evolves via a graph update function ϕ. Right: Update mechanism for agent k at time t−1, where it updates its internal state based on received inputs and produces output y (t−1) k . Definition 3 (Multi-Agen… view at source ↗
Figure 3
Figure 3. Extension of MAS to Internet of MAS. A networked environment where agents can be registered, discovered, and composed much like modular web services with standardized interfaces and protocols. Specifically, agents register with an Agent Directory, analogous to the Domain Name System (DNS), and connect using the Agent Connect Protocol (ACP) [19], similar to service-level routing. Calls to external tools are structure… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Overview of our position on effectiveness of MAS. Nodes represent agents. Left: MAS can dynamically allocate tasks across agents to adapt to varying requirements. Middle: MAS may or may not exhibit robustness. In the upper example, even if some agents fail, the system …
Figure 5
Figure 5. Figure 5: Ensemble accuracy under varying overlap and ensemble size. Left: Accuracy decreases with overlap ratio ρ for k > 1, showing that high overlap reduces diversity and harms ensemble performance. When ρ = 1, all agents behave similarly, matching the k = 1 baseline. Right: …
Figure 6
Figure 6. Figure 6: Illustration of our experiment on propagation of attribution errors in a two-agent system for [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Nonuniformity Principle in Human-AI Coworking

    cs.AI 2026-07 conditional novelty 6.0 of 10

    Optimal oversight schedules in multi-step AI workflows place human checkpoints early and with non-decreasing gaps between them.

Reference graph

Works this paper leans on

63 extracted references · 47 canonical work pages · cited by 1 Pith paper

  1. [58]

    Demystifying poisoning backdoor attacks from a statistical perspective

    Ganghua Wang, Xun Xian, Ashish Kundu, Jayanth Srinivasa, Xuan Bi, Mingyi Hong, and Jie Ding. Demystifying poisoning backdoor attacks from a statistical perspective. In The Twelfth International Conference on Learning Representations, 2024

    work page 2024

  2. [19]

    The Internet of Agents: an open, interoperable internet for agent- agent and agent-human quantum-safe communication

    Vijay Pandey. The Internet of Agents: an open, interoperable internet for agent- agent and agent-human quantum-safe communication. Technical report, Cisco Outshift,

    work page

  3. [1]

    Market.US. Global AI agents market size, share, upcoming investments report agent type, by agent system, region and companies - industry segment outlook, market assessment, competition scenario, trends and forecast 2024-2033. Online; accessed 8 February 2025, 2024. https: //market.us/report/ai-agents-market/

    work page 2024

  4. [2]

    AutoGen: Enabling next-gen LLM applications via multi-agent conversations

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, Ahmed Hassan Awadallah, Ryen W White, Doug Burger, and Chi Wang. AutoGen: Enabling next-gen LLM applications via multi-agent conversations. In First Conference on Language Modeling, 2024. 18

    work page 2024

  5. [3]

    AutoGPT: Build, Deploy, and Run AI Agents

    Significant Gravitas. AutoGPT: Build, Deploy, and Run AI Agents. https://github.com/ Significant-Gravitas/AutoGPT, 2023. Accessed: 2025-04-14

    work page 2023

  6. [4]

    https://agntcy.org/, 2025

    AGNTCY – Building Infrastructure for the Internet of Agents. https://agntcy.org/, 2025. Accessed: 2025-08-21

    work page 2025

  7. [5]

    Theory of Games and Economic Behavior

    John von Neumann and Oskar Morgenstern. Theory of Games and Economic Behavior. Prince- ton University Press, Princeton, NJ, USA, 1944

    work page 1944

  8. [6]

    Claude E. Shannon. A mathematical theory of communication. The Bell System Technical Journal, 27:379–423, 623–656, 1948

    work page 1948

Show all 63 references
  1. [7]

    A. N. Kolmogorov. Three approaches to the quantitative definition of information. Problems of Information Transmission, 1(1):1–7, 1965

    1965

  2. [8]

    Cybernetics: or Control and Communication in the Animal and the Machine

    Norbert Wiener. Cybernetics: or Control and Communication in the Animal and the Machine. MIT Press, Cambridge, MA, USA, 1948

    1948

  3. [9]

    W. G. Walter. An imitation of life. Scientific American, 182(5):42–45, 1950

    1950

  4. [10]

    Michael L. Littman. Markov games as a framework for multi-agent reinforcement learning. In Proceedings of the Eleventh International Conference on Machine Learning, pages 157–163, 1994

    1994

  5. [11]

    Human-level control through deep reinforcement learning

    V olodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei Rusu, Joel Veness, Marc Bellemare, Alex Graves, Martin Riedmiller, Andreas Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Le...

    2015 doi

  6. [12]

    Assael, Nando de Freitas, and Shimon Whiteson

    Jakob Foerster, Yannis M. Assael, Nando de Freitas, and Shimon Whiteson. Learning to communicate with deep multi-agent reinforcement learning. In Advances in Neural Information Processing Systems, 2016

    2016

  7. [13]

    Multi- agent actor-critic for mixed cooperative-competitive environments

    Ryan Lowe, YI WU, Aviv Tamar, Jean Harb, OpenAI Pieter Abbeel, and Igor Mordatch. Multi- agent actor-critic for mixed cooperative-competitive environments. In Advances in Neural Information Processing Systems, 2017

    2017

  8. [14]

    Cambridge University Press, Cambridge, UK, 2008

    Yoav Shoham and Kevin Leyton-Brown.Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations. Cambridge University Press, Cambridge, UK, 2008

    2008

  9. [15]

    Multiagent systems: A survey from a machine learning perspective

    Peter Stone and Manuela Veloso. Multiagent systems: A survey from a machine learning perspective. In Autonomous Robots, pages 345–383, 2000

    2000

  10. [16]

    An Introduction to MultiAgent Systems

    Michael Wooldridge. An Introduction to MultiAgent Systems. John Wiley & Sons, Chichester, UK, 2nd edition, 2009

    2009

  11. [17]

    Chawla, Olaf Wiest, and Xiangliang Zhang

    Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V . Chawla, Olaf Wiest, and Xiangliang Zhang. Large Language Model Based Multi-agents: A Survey of Progress and Challenges. In Proceedings of the Thirty-Third International Joint Conference on Artificial I...

    2024

  12. [18]

    LLM Multi-Agent Systems: Challenges and Open Problems

    Shanshan Han, Qifan Zhang, Yuhang Yao, Weizhao Jin, Zhaozhuo Xu, and Chaoyang He. LLM Multi-Agent Systems: Challenges and Open Problems. ArXiv, abs/2402.03578, 2024

    2024 arXiv

  13. [20]

    Model Context Protocol (MCP)

    Anthropic. Model Context Protocol (MCP). https://modelcontextprotocol.io/, 2024. accessed 2025-08-22. 19

    2024

  14. [21]

    Beyond self-talk: A communication-centric survey of llm-based multi-agent systems

    Bingyu Yan, Xiaoming Zhang, Litian Zhang, Lian Zhang, Ziyi Zhou, Dezhuang Miao, and Chaozhuo Li. Beyond self-talk: A communication-centric survey of llm-based multi-agent systems. arXiv preprint arXiv:2502.14321, 2025

    2025 arXiv

  15. [22]

    Large language models empowered agent-based modeling and simulation: a survey and perspectives

    Chen Gao, Xiaochong Lan, Nian Li, Yuan Yuan, Jingtao Ding, Zhilun Zhou, Fengli Xu, and Yong Li. Large language models empowered agent-based modeling and simulation: a survey and perspectives. Humanities and Social Sciences Communications, 11(1):1259, 2024

    2024

  16. [23]

    Langchain: A framework for developing applications with llms

    Harrison Chase. Langchain: A framework for developing applications with llms. https: //github.com/langchain-ai/langchain, 2022

    2022

  17. [24]

    Cooperative multi-agent learning: The state of the art

    Liviu Panait and Sean Luke. Cooperative multi-agent learning: The state of the art. Autonomous Agents and Multi-Agent Systems, 11(3):387–434, 2005

    2005

  18. [25]

    Multi-agent reinforcement learning: Independent vs

    Ming Tan. Multi-agent reinforcement learning: Independent vs. cooperative agents. In Proceed- ings of the Tenth International Conference on Machine Learning, pages 330–337, 1993

    1993

  19. [26]

    Oriol Vinyals, Igor Babuschkin, Wojciech M. Czarnecki, Michaël Mathieu, Andrew Dudzik, Junyoung Chung, David Choi, Richard Powell, Timo Ewalds, Petko Georgiev, Junhyuk Oh, Dan Horgan, Manuel Kroiss, Ivo Danihelka, Aja Huang, L. Sifre, Trevor Cai, John P. Agapiou, Max Jaderberg...

    2019

  20. [27]

    Foerster, Yannis M

    Jakob N. Foerster, Yannis M. Assael, Nando de Freitas, and Shimon Whiteson. Learning to communicate with deep multi-agent reinforcement learning. In Advances in Neural Information Processing Systemszhe, pages 2137–2145, 2016

    2016

  21. [28]

    Pablo Hernandez-Leal, Bilal Kartal, and Matthew E. Taylor. A survey and critique of multiagent deep reinforcement learning. Autonomous Agents and Multiagent Systems , 33(6):750–797, 2019

    2019

  22. [29]

    Leibo, Vinicius Zambaldi, Marc Lanctot, Janusz Marecki, and Thore Graepel

    Joel Z. Leibo, Vinicius Zambaldi, Marc Lanctot, Janusz Marecki, and Thore Graepel. Multi- agent reinforcement learning in sequential social dilemmas. Proceedings of the 16th Interna- tional Conference on Autonomous Agents and Multiagent Systems, pages 464–473, 2017

    2017

  23. [30]

    Cambridge University Press, New York, NY , 2009

    Yoav Shoham and Kevin Leyton-Brown.Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations. Cambridge University Press, New York, NY , 2009

    2009

  24. [31]

    Aho, John E

    Alfred V . Aho, John E. Hopcroft, and Jeffrey D. Ullman. The design and analysis of computer algorithms. 1974

    1974

  25. [32]

    Dietterich

    Thomas G. Dietterich. Ensemble methods in machine learning. In Proceedings of the First International Workshop on Multiple Classifier Systems (MCS 2000), volume 1857 of Lecture Notes in Computer Science, pages 1–15. Springer-Verlag, 2000

    2000

  26. [33]

    Ensemble based systems in decision making.IEEE Circuits and Systems Magazine, 6(3):21–45, 2006

    Robi Polikar. Ensemble based systems in decision making.IEEE Circuits and Systems Magazine, 6(3):21–45, 2006

    2006

  27. [34]

    Bagging predictors

    Leo Breiman. Bagging predictors. Machine Learning, 24(2):123–140, 1996

    1996

  28. [35]

    Schapire

    Yoav Freund and Robert E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences , 55(1):119–139, 1997

    1997

  29. [36]

    Random forests

    Leo Breiman. Random forests. Machine Learning, 45(1):5–32, 2001

    2001

  30. [37]

    Probability inequalities for sums of bounded random variables

    Wassily Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58(301):13–30, 1963. 20

    1963

  31. [38]

    The pile: An 800gb dataset of diverse text for language modeling

    Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Skyler Presser, and Connor Leahy. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027, 2020

    2020 arXiv

  32. [39]

    Redpajama: An open dataset for training large language models

    Together Computer. Redpajama: An open dataset for training large language models. https: //github.com/togethercomputer/RedPajama-Data, 2023

    2023

  33. [40]

    The pile, common crawl, and starcoder dataset descriptions, 2024

    Hugging Face. The pile, common crawl, and starcoder dataset descriptions, 2024. Available at https://huggingface.co/datasets

    2024

  34. [41]

    Policy shaping: Integrating human feedback with reinforcement learning

    Shane Griffith, Kaushik Subramanian, Jonathan Scholz, Charles L Isbell, and Andrea L Thomaz. Policy shaping: Integrating human feedback with reinforcement learning. Advances in neural information processing systems, 26, 2013

    2013

  35. [42]

    Deep reinforcement learning from policy-dependent human feedback

    Dilip Arumugam, Jun Ki Lee, Sophie Saskin, and Michael L Littman. Deep reinforcement learning from policy-dependent human feedback. arXiv preprint arXiv:1902.04257, 2019

    1902 arXiv

  36. [43]

    Training a helpful and harmless assistant with reinforcement learning from human feedback

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, and Tom Henighan. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022

    2022 arXiv

  37. [44]

    Safe RLHF: Safe reinforcement learning from human feedback

    Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang. Safe RLHF: Safe reinforcement learning from human feedback. In The Twelfth International Conference on Learning Representations, 2024

    2024

  38. [45]

    Multi-agent reinforcement learning: A selective overview of theories and algorithms

    Kaiqing Zhang, Zhuoran Yang, and Tamer Ba¸ sar. Multi-agent reinforcement learning: A selective overview of theories and algorithms. arXiv preprint arXiv:1911.10635, 2019. URL https://arxiv.org/abs/1911.10635. Revised version (v2) last updated 28 April 2021

    1911 arXiv

  39. [46]

    MAP: Multi-human-value alignment palette

    Xinran Wang, Qi Le, Ammar Ahmed, Enmao Diao, Yi Zhou, Nathalie Baracaldo, Jie Ding, and Ali Anwar. MAP: Multi-human-value alignment palette. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum? id=NN6QHwgRrQ

    2025

  40. [47]

    Intriguing properties of neural networks

    Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfel- low, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013

    2013 arXiv

  41. [48]

    Explaining and harnessing adversar- ial examples

    Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversar- ial examples. In International Conference on Learning Representations, 2015

    2015

  42. [49]

    Adversarial examples: Attacks and defenses for deep learning

    Xiaoyong Yuan, Pan He, Qile Zhu, and Xiaolin Li. Adversarial examples: Attacks and defenses for deep learning. IEEE Transactions on Neural Networks and Learning Systems , 30(9): 2805–2824, 2019

    2019

  43. [50]

    Jailbroken: How Does LLM Safety Training Fail? In Thirty-seventh Conference on Neural Information Processing Systems, 2023

    Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How Does LLM Safety Training Fail? In Thirty-seventh Conference on Neural Information Processing Systems, 2023

    2023

  44. [51]

    Prompt injection attack against LLM- integrated applications

    Yi Liu, Gelei Deng, Yuekang Li, Kailong Wang, Zihao Wang, Xiaofeng Wang, Tianwei Zhang, Yepang Liu, Haoyu Wang, Yan Zheng, and Yang Liu. Prompt injection attack against LLM- integrated applications. arXiv preprint arXiv:2306.05499v2, 2024. URL https://arxiv. org/abs/2306.05499...

    2024 arXiv

  45. [52]

    Limiting privacy breaches in privacy-preserving data mining

    Alex Evfimievski, Johannes Gehrke, and Ramakrishnan Srikant. Limiting privacy breaches in privacy-preserving data mining. In Proceedings of the twenty-second ACM SIGMOD-SIGACT- SIGART symposium on Principles of database systems, pages 211–222. ACM, 2003

    2003

  46. [53]

    Calibrating noise to sensitivity in private data analysis

    Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Theory of cryptography conference , pages 265–284. Springer, 2006. 21

    2006

  47. [54]

    Interval privacy: A framework for privacy-preserving data collec- tion

    Jie Ding and Bangjun Ding. Interval privacy: A framework for privacy-preserving data collec- tion. IEEE Transactions on Signal Processing, 2021

    2021

  48. [55]

    Model privacy: A unified framework to understand model stealing attacks and defenses

    Ganghua Wang, Yuhong Yang, and Jie Ding. Model privacy: A unified framework to understand model stealing attacks and defenses. arXiv preprint arXiv:2402.01852, 2024

    2024 arXiv

  49. [56]

    A Unified Detection Framework for Inference-Stage Backdoor Defenses

    Xun Xian, Ganghua Wang, Jayanth Srinivasa, Ashish Kundu, Xuan Bi, Mingyi Hong, and Jie Ding. A Unified Detection Framework for Inference-Stage Backdoor Defenses. In Thirty- seventh Conference on Neural Information Processing Systems, 2023

    2023

  50. [57]

    Understanding Backdoor Attacks through the Adaptability Hypothesis

    Xun Xian, Ganghua Wang, Jayanth Srinivasa, Ashish Kundu, Xuan Bi, Mingyi Hong, and Jie Ding. Understanding Backdoor Attacks through the Adaptability Hypothesis. In Proceedings of the 40th International Conference on Machine Learning, 2023

    2023

  51. [59]

    McKee, Joel Z

    Allan Dafoe, Edward Hughes, Yoram Bachrach, Tantum Collins, Kevin R. McKee, Joel Z. Leibo, K. Larson, and Thore Graepel. Open problems in cooperative ai. ArXiv, abs/2012.08630, 2020

    2012 arXiv

  52. [60]

    Graybuck, et al

    Bosiljka Tasic, Zizhen Yao, Lucas T. Graybuck, et al. Shared and distinct transcriptomic cell types across neocortical areas. Nature, 563:72–78, 2018

    2018

  53. [61]

    Invisible backdoor attack with sample-specific triggers

    Yuezun Li, Yiming Li, Baoyuan Wu, Longkang Li, Ran He, and Siwei Lyu. Invisible backdoor attack with sample-specific triggers. In Proceedings of the IEEE International Conference on Computer Vision, pages 16443–16452, 2021. doi: 10.1109/ICCV48922.2021.01615

    2021

  54. [62]

    Targeted backdoor attacks on deep learning systems using data poisoning

    Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526, 2017. 22

    2017 arXiv

  55. [2025]

    Technical report (white paper), accessed 2025- 08-22

    URL https://outshift-headless-cms-s3.s3.us-east-2.amazonaws.com/ Internet_of_Agents_Whitepaper.pdf. Technical report (white paper), accessed 2025- 08-22

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.