An Adversary-Resistant Multi-Agent LLM System via Credibility Scoring

Abolfazl Asudeh; Mohsen Dehghankar; Sana Ebrahimi

REVIEW 5 major objections 6 minor 40 references

An Adversary-Resistant Multi-Agent LLM System via Credibility Scoring

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proposes a credibility-scoring layer that learns each agent's reliability on the fly, weights outputs by it, and reports 6-30 point accuracy gains, with stable performance even when adversarial agents are the majority.

desk verdict The framework is sensible and the paper shows real gains in several settings, but the headline claim about adversary-majority robustness is not established because the judge that supplies the learning signal is also the strongest model in the room. read the letter →

arxiv 2505.24239 v1 pith:HYAXQPJF submitted 2025-05-30 cs.MA cs.AIcs.CLcs.LG

Sana Ebrahimi , Mohsen Dehghankar , Abolfazl Asudeh This is my paper

classification cs.MAcs.AIcs.CLcs.LG

keywords multi-agentLLMsystemscredibilityscoringadversarialrobustnesscontributionscoresShapleyvalueLLM-as-judgeadversary-majoritysettingsweightedaggregation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a multi-agent LLM system can be made resistant to adversarial and low-performing agents by learning a credibility score for each agent on the fly and using those scores to weight the agents' answers when aggregating a final response. The authors model each query as an iterative cooperative game: agents communicate according to a topology, produce individual outputs, and the system combines them with a credibility-score-aware coordinator. After each round, the team receives a reward for answer quality, and the reward is distributed among agents in proportion to their contribution, estimated either by Shapley values or by an LLM judge; credibility scores are then updated multiplicatively. Across GSM8K, MMLU-MS, MATH, and ResearchQA, the authors report that introducing credibility scoring improves accuracy by 6-30 percentage points over naive coordination and keeps accuracy stable even when three of five agents are adversarial. A sympathetic reader would care because this is a general, topology-agnostic defense that can be layered onto existing multi-agent frameworks without removing or retraining agents.

What carries the argument

The load-bearing object is the credibility score $\mathrm{CrS}$ and its multiplicative update rule: $\mathrm{CrS}^{(i)}_t = \mathrm{CrS}^{(i)}_{t-1}(1 + \eta \cdot \mathrm{CSc}^{(i)} \cdot r_t)$. An agent's weight rises when it contributes to rewarded answers and falls when it contributes to penalized ones. The contribution score $\mathrm{CSc}(i)$ is the mechanism that makes reward credit assignment fair: in no-communication settings it is the Shapley value of agent $i$ over all subsets of the team's outputs, and in communication-heavy or LLM-assisted settings it is produced by an external LLM judge analyzing the dialogue log, the agent outputs, and the final answer. The credibility scores are then used to replace unweighted aggregation, either by computing a $\mathrm{CrS}$-weighted centroid in embedding space and selecting the closest answer, or by passing the scores to a trusted coordinator LLM alongside the agents' outputs.

What would settle it

Run the same five-agent GSM8K setup with a judge that assigns rewards randomly, or with a judge known to mis-score answers as the paper reports for HumanEval, and compare credibility-score-weighted coordination to naive coordination and majority voting. If CrS-weighted accuracy does not beat both baselines over 100 queries, the claim that learned credibility scores drive the gains is falsified. A sharper test is to give the judge a systematic bias in favor of the adversarial agents' answer style and check whether the faithful agents' scores still converge upward.

Watch

Extended reading notes

Core claim

The central claim is that weighting agent outputs by a learned credibility score, rather than treating all agents equally, makes multi-agent LLM coordination robust to adversarial influence even when adversaries are the majority. The credibility score $\mathrm{CrS}(i) \in [0,1]$ reflects the system's estimate of agent $i$'s reliability over previous queries; it starts at a default value and is updated after each round by distributing the team reward $r_t$ in proportion to each agent's contribution score $\mathrm{CSc}(i)$. Contribution scores are computed either exactly via Shapley values in settings without inter-agent communication, or by an LLM-as-Judge that reads the query, final answer, dialogue log, and agent outputs. The paper reports that this mechanism raises accuracy by 6-30 percentage points across all tested backbones and benchmarks, and that in a five-agent team with three adversaries the credibility-score coordinator holds accuracy near 31% on MMLU-MS whereas unweighted methods stay below 24% and fluctuate. The authors also show that the learned scores converge to separate faithful from adversarial agents over roughly fifty to one hundred queries.

Load-bearing premise

The whole credibility-learning loop depends on an external judge supplying accurate reward and contribution estimates; if the judge's scores are noisy or wrong, the credibility updates are distorted and weighted coordination can become worse than doing no weighting at all.

Editorial extensions

If this is right

Credibility-score weighting can be added to existing coordination mechanisms such as majority voting, centroid aggregation, and LLM coordinators without changing agent prompts or retraining backbones.
A five-agent team with three adversarial agents retains stable accuracy around 31% on MMLU-MS with credibility scoring, while unweighted methods fluctuate below 24%.
The quality of the external judge is part of the mechanism: replacing GPT-4o mini with LLaMA3.2 as judge lowered GSM8K accuracy by 54%.
The learned credibility scores converge over repeated queries and separate faithful from adversarial agents, so longer system lifetimes should improve weighting.
Increasing communication links beyond six edges yields diminishing returns and risks judge token-compression errors, so the benefit of credibility scoring is tied to keeping interaction logs compact.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If judge noise is the main failure mode, a natural extension the paper does not test is an ensemble of judges or a judge that calibrates its own confidence before issuing rewards; the 54% degradation with a weaker judge suggests such calibration could be as valuable as the weighting itself.
An adversary that learns to echo faithful agents' answers early in the game could keep its contribution score high and delay detection; the paper's synthetic adversaries are instructed to be subtly wrong, so adaptive adversaries are an untested boundary.
The same credibility values could be reused beyond aggregation, for example to select which agents form future teams or to route queries to the most credible agents, which would follow from the claim that the scores track true reliability.
Because the update rule is multiplicative and the reward is bounded in $[-1,1]$, a single judge error on a high-contribution agent can move its score sharply; testing the framework with occasional injected misrewards would reveal how quickly scores recover.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The core idea — learn per-agent credibility online from judge-provided reward and contribution estimates, then weight the aggregation by it — is sound, and the paper shows real gains in several settings. The framework is modular, tested across topologies, tasks, and backbones, and the authors honestly report limitations including judge dependence and synthetic adversaries. That is worth something.

But the central claim that the system can tolerate adversary-majority teams is not established on the evidence here. In the SIA and chain settings, both the reward and the contribution scores come from the same external GPT-4o-mini judge. The paper's own Section 6.3.5 shows that replacing that judge with LLaMA3.2(3B) drops GSM8K accuracy by 54%, and the authors concede that a stronger evaluator might make the multi-agent configuration unnecessary. No control condition shows the CrS-weighted team outperforming the judge alone. So the headline result may be mostly an artifact of delegating evaluation to a model that already knows the answers, rather than a demonstration that credibility learning itself detects and mitigates adversaries.

There are also statistical gaps: no seeds, no error bars, only 100 questions, and a few results cut against the claim — HumanEval underperforms a single agent, and GSM8K trails majority voting in some conditions. Learning rate and normalization are unspecified. The adversaries are synthetic and, as the authors note, often yield after a few rounds; that is a weaker threat model than a strategic, adaptive adversary.

None of this is fatal to the idea. The combination of online credibility learning with Shapley-value and LLM-judge contribution scores is not something I remember seeing verbatim in the prior work, and the positive results, though noisy, suggest the mechanism can help. The paper is a reasonable engineering contribution, but it needs a major revision to support the strong claim in the abstract.

I would send it to peer review, but with a clear expectation that the authors add a judge-only baseline, report variance, specify hyperparameters, and ideally test with a stronger judge or a ground-truth reward signal in the standalone setting to isolate the contribution of the credibility loop. Without those, the claim of adversary-majority robustness remains speculative.

This paper is for researchers working on multi-agent LLM robustness and trust aggregation. It deserves a serious referee, but the referee should push for the missing controls.

Referee Report

5 major / 6 minor

Summary. The paper proposes a multi-agent LLM coordination framework that maintains per-agent credibility scores (CrS) and updates them on the fly from contribution scores (computed via Shapley values or an LLM judge) and a reward signal. The final answer is obtained through CrS-aware aggregation, either centroid-based or via an LLM coordinator. Experiments on GSM8K, MMLU-MS, MATH, HumanEval, and Research Questions, with five-agent teams (two faithful, three adversarial) and several topologies, are used to claim 6-30 percentage point accuracy gains and tolerance of adversary-majority teams.

Significance. If the empirical claims held, the framework would be a useful general mechanism for robust multi-agent LLM systems, and the paper connects classical credit assignment (Shapley values) with LLM-based evaluation in a clean way. The release of source code and prompts is a strength. However, the current evidence is insufficient: key results are reported without variance, the flagship claim excludes a benchmark where the method loses to a single agent, simple majority voting beats CrS on GSM8K, and the entire learning loop is shown to depend critically on the external judge. The central claim therefore needs substantial additional experiments and appropriately hedged claims.

major comments (5)

[§6.3.1, Table 3] The abstract and §6.3.1 state that introducing CrS raises accuracy by 6-30 percentage points, and §6.3.1 says this holds 'across all four benchmarks'; however §6.1 lists five benchmarks including HumanEval. Table 3 shows that on HumanEval the CrS coordinator (0.16) is below the single-agent baseline (0.32 pre-communication), and §6.3.5 explicitly attributes this to judge-induced CrS distortion. This is a direct counterexample to the general claim; the claim must be restricted to benchmarks where it is supported, or the HumanEval result must be reconciled with the headline.
[§6.3.2, Figure 3d-f] The text reports that on GSM8K the CrS coordinator is 'second-best, trailing behind Majority Voting.' Since the central contribution is robustness in adversary-majority settings, a simple unweighted baseline outperforming the method on a core mathematical benchmark is a serious challenge. The explanation (noisy CSc for complex reasoning) is not quantified; please report the comparison with variance and significance, and discuss why CrS should be preferred despite this result.
[§6.3.5 and §8] The learning loop is fed by the external judge for both the reward rt and the contribution scores CSc. The paper's own experiments show that replacing GPT-4o mini with LLaMA3.2 reduces GSM8K accuracy by 54% and that GPT-4o mini miscalculates rewards on HumanEval. No control condition tests the judge alone on the same tasks, and no ablation tests a judge without access to ground-truth answers. Therefore the claimed adversary-majority tolerance may be an artifact of the judge's competence rather than of the CrS mechanism; the concession in §6.3.5 that 'directly assigning the task to a stronger evaluator might be more effective' underlines this gap. These controls are necessary to support the headline claim.
[§6.1 and §6.3] All reported accuracies are single numbers over 100 questions (Figure 3) or 50 questions (Table 3), with no confidence intervals, standard deviations, or multiple seeds. The SIA topology is stochastic (links are sampled randomly per query), so the results are subject to sampling noise; the claimed 6-30 percentage point gains may not be statistically significant. Please provide error bars or significance tests for the main tables and figures.
[§5.2, Eq. (2)] The update rule CrS_t = CrS_{t-1}(1 + η·CSc·r_t) is underspecified: η is never given a value or schedule, CSc is not defined as bounded or normalized, and no clipping or renormalization is described. Without these details the claimed invariant CrS ∈ [0,1] is not guaranteed (e.g., with η=0.5, CSc=1, r_t=1, CrS grows monotonically beyond 1), and the experiments are not reproducible. Please specify the exact hyperparameters and any normalization used.

minor comments (6)

[§6.1, §6.3.1, §8] The number of benchmarks is inconsistent: §6.1 lists five benchmarks, while §6.3.1 and §8 refer to four, omitting HumanEval; please make the count and the summary claims consistent.
[§5.1] The Shapley-value equation contains an undefined symbol 'xn' and an undefined reward function R(ot); the notation should be repaired for readability.
[Table 3] The caption and column headers of Table 3 are garbled; reformat so that each condition (pre-communication, post-communication, chain random) is clearly labeled.
[References] The references list duplicate entries for Liang et al. 2023a and 2023b with identical titles; one of the entries should be removed or corrected.
[§4, Eq. (1)] Equation (1) uses 1/N times the sum of CrS-weighted embeddings; if the intended centroid is a weighted average, the weights should be normalized by their sum rather than by N, although the cosine-distance selection may be insensitive to this scaling.
[Appendix D and §6.3.5] The notation for credibility score appears as 'Src' in Appendix D and 'CrS' elsewhere; please standardize, and also standardize the capitalization of 'LLaMA3.2' throughout.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the credibility loop is evaluated against independent ground truth, and no equation reduces the claimed result to its inputs.

full rationale

The central claim is that CrS-weighted aggregation improves accuracy in adversarial settings. The CrS update (Eq. 2) is a heuristic learning rule driven by an external judge's reward and contribution scores, and the reported accuracy is measured against benchmark ground truth rather than derived from the judge's estimates. The paper's own ablations show the loop is not tautologically beneficial: a weaker judge degrades GSM8K accuracy by 54% and GPT-4o mini mislabels incorrect HumanEval code, distorting CrS updates. Those are robustness and external-validity findings, not evidence that the output equals the input by construction. The centroid-based aggregator cited from Ebrahimi et al. (2024) is one optional integration mechanism, used alongside LLM-assisted aggregation and compared with majority voting, similarity ensembles, single-agent, and naive coordination baselines; none of these comparisons is forced by the CrS equation. No fitted parameter is relabeled as a prediction: the online-learned CrS weights are used to aggregate outputs whose correctness is assessed independently, and the paper reports cases where CrS underperforms majority voting. Therefore no step in the derivation chain reduces to its own inputs.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

CrS and CSc are algorithmic constructs, not postulated physical entities; the framework introduces no new ontology. The central claim depends on the judge's reliability, the honest faithful agents, and the embedding-space assumption, plus several hand-chosen constants that are not reported.

free parameters (4)

learning_rate_eta = not specified
Equation 2 uses η to scale CrS updates; no value or tuning procedure is given in the paper.
initial_credibility_0.5 = 0.5
Section 5 sets all credibility scores to a default of 0.5; this choice is not justified.
communication_links_m = 6
In SIA, m=6 links are sampled (Section 6.2); the authors state accuracy saturates at six links, indicating tuning.
contribution_score_normalization = not specified
CSc values in Tables 4-5 sum to 1.0, but the normalization procedure is not described in the paper.

assumptions (5)

domain assumption The external judge provides accurate reward and contribution estimates.
The entire CrS learning loop assumes GPT-4o mini produces reliable rt and CSc values (Sections 5.1, 5.2, Appendix D).
domain assumption Faithful agents genuinely attempt to answer correctly and are not malicious.
Section 6.2 assumes the non-adversarial agents pursue correct solutions without adaptive strategies.
domain assumption Adversarial agents are static and do not adapt to the CrS mechanism.
Adversaries are prompted to inject subtle errors; the Limitations admit real-world adversaries may be more sophisticated and evasive.
domain assumption Embedding-space cosine distance reflects answer quality.
Centroid-based aggregation (Section 4, Appendix C) inherits this assumption from Ebrahimi et al. 2024.
ad hoc to paper CrS remains within [0,1] under the update rule.
Equation 2 can push CrS outside [0,1] if η·CSc·rt is large; the paper does not clamp or discuss this, despite defining CrS ∈ [0,1].

how reviews work

0 comments

Cite this review

Pith. "Pith review of An Adversary-Resistant Multi-Agent LLM System via Credibility Scoring." pith.science (2026). https://pith.science/paper/HYAXQPJF

@misc{pith2026250524239,
  author       = {Pith},
  title        = {Pith review of: An Adversary-Resistant Multi-Agent LLM System via Credibility Scoring},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HYAXQPJF}},
  note         = {Machine review of arXiv:2505.24239}
}

read the original abstract

While multi-agent LLM systems show strong capabilities in various domains, they are highly vulnerable to adversarial and low-performing agents. To resolve this issue, in this paper, we introduce a general and adversary-resistant multi-agent LLM framework based on credibility scoring. We model the collaborative query-answering process as an iterative game, where the agents communicate and contribute to a final system output. Our system associates a credibility score that is used when aggregating the team outputs. The credibility scores are learned gradually based on the past contributions of each agent in query answering. Our experiments across multiple tasks and settings demonstrate our system's effectiveness in mitigating adversarial influence and enhancing the resilience of multi-agent cooperation, even in the adversary-majority settings.

Figures

Figures reproduced from arXiv: 2505.24239 by the authors.

**Figure 1.** System architecture. being outvoted or manipulated. Our approach is applicable across different team structures and integration mechanisms for existing methods. It empowers users to minimize the impact of low-performing and malicious agents within the teams with various formations and communications topologies. By leveraging this adaptability, our method enhances the resilience of multi-agent systems, ensuring mor… view at source ↗

**Figure 2.** CrS convergence for an adversary-dominated team with 3 adversarial and 2 faithful agents. subtle errors, are prompted using similar prompt template across tasks. We evaluate our method across three communication topologies1 : Stochastic Interaction Architecture (SIA). For each question, six undirected links are sampled at random from the (︁ 5 2 )︁ possible pairs, yielding diverse topologies such as trees, rings, etc… view at source ↗

**Figure 3.** Performance comparison of baseline methods versus CrS-based coordinators. 0 10 20 30 40 50 Experiment Round 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 Accuracy CrS Coordinator Naive Coordination Majority Voting Similarity Ensemble [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Baseline accuracy for a five-agent chain (one faithful, four adversarial). The “CrS Coordinator” (green) curve reflects a CrS-ordered chain, whereas all other methods use an unordered chain topology. that coordination cannot fully compensate for insufficient backbone …

**Figure 6.** Figure 6: Impact of adversarial agent count on accuracy across baseline methods compared to the CrS coordination mechanism on MMLU-MS [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

**Figure 7.** Figure 7: CrS evolution with a LLaMA-3.2(3B) judge supervising five Qwen2.5(7B) agents on GSM8K—directly comparable to Figure2a [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]

**Figure 8.** Figure 8: CrS evolution for two independent LLaMA-3.2 (3B) [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

40 extracted references · 9 canonical work pages

[1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Mistral AI. 2023. https://mistral.ai/news/announcing-mistral-7b Announcing mistral 7b . Accessed: 2025-04-17

work page 2023
[4]

Alfonso Amayuelas, Xianjun Yang, Antonis Antoniades, Wenyue Hua, Liangming Pan, and William Wang. 2024. Multiagent collaboration attack: Investigating adversarial attacks in large language model collaborations via debate. arXiv preprint arXiv:2406.14711

arXiv 2024
[5]

Meghana Moorthy Bhat, Rui Meng, Ye Liu, Yingbo Zhou, and Semih Yavuz. 2023. Investigating answerability of llms for long-form question answering. arXiv preprint arXiv:2309.08210

arXiv 2023
[6]

Justin Chih-Yao Chen, Swarnadeep Saha, and Mohit Bansal. 2023. Reconcile: Round-table conference improves reasoning via consensus among diverse llms. arXiv preprint arXiv:2309.13007

arXiv 2023
[7]

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374

arXiv 2021
[8]

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

arXiv 2021

Show all 40 references

[9]

Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. 2023. Improving factuality and reasoning in language models through multiagent debate. arXiv preprint arXiv:2305.14325

2023 arXiv
[10]

Sana Ebrahimi, Nima Shahbazi, and Abolfazl Asudeh. 2024. Requal-lm: Reliability and equity through aggregation in large language models. In NAACL-HLT (Findings)

2024
[11]

Neel Guha, Mayee Chen, Trevor Chow, Ishan Khare, and Christopher Re. 2024. Smoothie: Label free language model routing. Advances in Neural Information Processing Systems, 37:127645--127672

2024
[12]

Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang. 2024. Large language model based multi-agents: A survey of progress and challenges. arXiv preprint arXiv:2402.01680

2024 arXiv
[13]

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Xiaodong Song, and Jacob Steinhardt. 2020. https://api.semanticscholar.org/CorpusID:221516475 Measuring massive multitask language understanding . ArXiv, abs/2009.03300

2020 arXiv
[14]

Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. NeurIPS

2021
[15]

Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, et al. 2023. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308.00352

2023 arXiv
[16]

Jen-tse Huang, Jiaxu Zhou, Tailin Jin, Xuhui Zhou, Zixi Chen, Wenxuan Wang, Youliang Yuan, Maarten Sap, and Michael R Lyu. 2024. On the resilience of multi-agent systems with malicious agents. arXiv preprint arXiv:2408.00989

2024 arXiv
[17]

Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023. Camel: Communicative agents for" mind" exploration of large language model society. Advances in Neural Information Processing Systems, 36:51991--52008

2023
[18]

Junyou Li, Qin Zhang, Yangbin Yu, Qiang Fu, and Deheng Ye. 2024 a . More agents is all you need. arXiv preprint arXiv:2402.05120

2024 arXiv
[19]

Xinyi Li, Sai Wang, Siqi Zeng, Yu Wu, and Yi Yang. 2024 b . A survey on llm-based multi-agent systems: workflow, infrastructure, and challenges. Vicinagearth, 1(1):9

2024
[21]

Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. 2023 b . Encouraging divergent thinking in large language models through multi-agent debate. arXiv preprint arXiv:2305.19118

2023 arXiv
[22]

Zijun Liu, Yanzhe Zhang, Peng Li, Yang Liu, and Diyi Yang. 2023. Dynamic llm-agent network: An llm-agent collaboration framework with agent team optimization. arXiv preprint arXiv:2310.02170

2023 arXiv
[23]

Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30

2017
[24]

Ollama. 2024 a . https://ollama.com/chevalblanc/gpt-4o-mini Gpt-4o mini - cheval blanc . Accessed: 2025-04-24

2024
[25]

Ollama. 2024 b . https://ollama.com/library/llama3.2 Llama 3.2 . Accessed: 2025-04-17

2024
[26]

Silviu Pitis, Michael R Zhang, Andrew Wang, and Jimmy Ba. 2023. Boosted prompt ensembles for large language models. arXiv preprint arXiv:2304.05970

2023 arXiv
[27]

Chen Qian, Zihao Xie, Yifei Wang, Wei Liu, Yufan Dang, Zhuoyun Du, Weize Chen, Cheng Yang, Zhiyuan Liu, and Maosong Sun. 2024. Scaling large-language-model-based multi-agent collaboration. arXiv preprint arXiv:2406.07155

2024 arXiv
[28]

Corby Rosset, Ho-Lam Chung, Guanghui Qin, Ethan C Chau, Zhuo Feng, Ahmed Awadallah, Jennifer Neville, and Nikhil Rao. 2024. Researchy questions: A dataset of multi-perspective, decompositional questions for llm web agents. arXiv preprint arXiv:2402.17896

2024 arXiv
[29]

Lloyd S Shapley. 1951. Notes on the n-person game—ii: The value of an n-person game. RAND Corporation, RM-670

1951
[30]

Yoav Shoham and Kevin Leyton-Brown. 2008. Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations. Cambridge University Press, USA

2008
[31]

Junlin Wang, Jue Wang, Ben Athiwaratkun, Ce Zhang, and James Zou. 2024. Mixture-of-agents enhances large language model capabilities. arXiv preprint arXiv:2406.04692

2024 arXiv
[32]

Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171

2022 arXiv
[33]

Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. 2023. Autogen: Enabling next-gen llm applications via multi-agent conversation framework. arXiv preprint arXiv:2308.08155

2023 arXiv
[34]

Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. 2025. The rise and potential of large language model based agents: A survey. Science China Information Sciences, 68(2):121101

2025
[35]

Changrong Xiao, Sean Xin Xu, Kunpeng Zhang, Yufang Wang, and Lei Xia. 2023. Evaluating reading comprehension exercises generated by llms: A showcase of chatgpt in education applications. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Appl...

2023
[36]

An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

2024 arXiv
[37]

Yi Yang, Yitong Ma, Hao Feng, Yiming Cheng, and Zhu Han. 2025. https://doi.org/10.3390/app15073676 Minimizing hallucinations and communication costs: Adversarial debate and voting mechanisms in llm-based multi-agents . Applied Sciences, 15:3676

2025 doi
[38]

Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2023. https://openreview.net/forum?id=WE_vluYUL-X React: Synergizing reasoning and acting in language models . In The Eleventh International Conference on Learning Representations

2023
[39]

Jintian Zhang, Xin Xu, Ningyu Zhang, Ruibo Liu, Bryan Hooi, and Shumin Deng. 2023. Exploring collaboration mechanisms for llm agents: A social psychology view. arXiv preprint arXiv:2310.02124

2023 arXiv
[40]

Yusen Zhang, Ruoxi Sun, Yanfei Chen, Tomas Pfister, Rui Zhang, and Sercan \"O Arik. 2024 a . Chain of agents: Large language models collaborating on long-context tasks. arXiv preprint arXiv:2406.02818

2024 arXiv
[41]

Zaibin Zhang, Yongting Zhang, Lijun Li, Hongzhi Gao, Lijun Wang, Huchuan Lu, Feng Zhao, Yu Qiao, and Jing Shao. 2024 b . Psysafe: A comprehensive framework for psychological-based attack, defense, and evaluation of multi-agent system safety. arXiv preprint arXiv:2401.11880

2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Mistral AI. 2023. https://mistral.ai/news/announcing-mistral-7b Announcing mistral 7b . Accessed: 2025-04-17

work page 2023

[4] [4]

Alfonso Amayuelas, Xianjun Yang, Antonis Antoniades, Wenyue Hua, Liangming Pan, and William Wang. 2024. Multiagent collaboration attack: Investigating adversarial attacks in large language model collaborations via debate. arXiv preprint arXiv:2406.14711

arXiv 2024

[5] [5]

Meghana Moorthy Bhat, Rui Meng, Ye Liu, Yingbo Zhou, and Semih Yavuz. 2023. Investigating answerability of llms for long-form question answering. arXiv preprint arXiv:2309.08210

arXiv 2023

[6] [6]

Justin Chih-Yao Chen, Swarnadeep Saha, and Mohit Bansal. 2023. Reconcile: Round-table conference improves reasoning via consensus among diverse llms. arXiv preprint arXiv:2309.13007

arXiv 2023

[7] [7]

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374

arXiv 2021

[8] [8]

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

arXiv 2021