When Collaboration Becomes a Trigger: Collective Evidence-Threshold Backdoors in Multi-Agent Systems

Jia-Hao Xiao; Lei Feng; Min-Ling Zhang

REVIEW 3 major objections 5 minor 17 references

When Collaboration Becomes a Trigger: Collective Evidence-Threshold Backdoors in Multi-Agent Systems

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A poisoned LLM agent in a multi-agent team can behave entirely normally until the number of distinct peers carrying a hidden cue reaches a threshold, then switch to an attacker-chosen output.

desk verdict A genuinely new backdoor paradigm with a careful attack/defense evaluation; the defense is a strong conditional result, not an adaptive-security guarantee. read the letter →

arxiv 2608.01085 v1 pith:KXFXO222 submitted 2026-08-02 cs.MA cs.LG

Jia-Hao Xiao , Lei Feng , Min-Ling Zhang This is my paper · ORCID

classification cs.MAcs.LG

keywords multi-agentsystemsbackdoorattacksLLMsecuritycollectiveevidencethresholdlatenttransitiondetectionclean-onlydefenseweightpoisoningtest-timeevaluation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper identifies a new failure mode for LLM-based multi-agent systems: a backdoor whose trigger is not any single word or message but the collective amount of peer evidence visible to a poisoned agent. The authors introduce a training procedure, BCBI, that makes poisoned agents answer normally while fewer than T distinct peers emit cue messages, and switch to a fixed adversarial target once the peer evidence count reaches T. They also build a clean-only defense, LATTE, that watches how an agent's hidden state changes between communication rounds and quarantines updates that leave the dynamics learned from clean collaboration, before those updates can influence other agents. If the claims hold, single-message inspection and ordinary backdoor defenses will miss this attack, while a transition-based monitor can contain it without knowing the trigger or the target.

What carries the argument

The machinery is the pair (k,T): the peer evidence count k, the number of distinct visible peers whose latest message contains a realization from cue family R, and the activation threshold T. BCBI trains the poisoned model with counterfactual boundary pairs (x_pre, x_post) that differ only in one peer cue and the target response, plus two losses: TACO, which makes the target's log-probability margin s(x) negative before the threshold and positive after it, and PEPL, which learns a one-dimensional coordinate z(x) from the final prompt representation ordered by k. LATTE's mechanism is a low-rank PCA subspace U_r fit to normalized inter-round hidden-state deltas v_t = concat_l(\$\Delta$ h_t^l / \|\$\Delta$ h_t^l\|_2) from clean trajectories; it flags a transition whose residual ratio s_sub(v_t) exceeds a clean-calibrated high quantile, quarantines the first flag, and permanently isolates after two consecutive flags.

What would settle it

Train a poisoned model with a joint objective that preserves the post-threshold attack target while explicitly minimizing the PCA residual of its inter-round hidden-state transitions, and measure whether any configuration reaches high attack success while keeping its residual below the clean quantile threshold; one such configuration would falsify the claim that activated transitions necessarily depart from the clean subspace.

Watch

Extended reading notes

Core claim

The central claim is that a backdoor can be conditioned on a collective evidence threshold in a multi-agent system. With each distinct visible peer contributing at most one unit, a poisoned receiver counts k_{i,t}=\sum_{j\in N_i(t)} e_R(m_{j,t}) and follows the benign policy for k<T and the adversarial policy for k\ge T. BCBI makes this boundary sharp: counterfactual boundary pairs differ only in one peer cue and the target response, TACO separates output preference across the threshold, and PEPL orders hidden states by evidence count. Empirically this yields false trigger rates at or below 0.01 and attack success rates of 0.95–1.00 on six QA benchmarks, while the companion defense LATTE, which scores PCA residuals of inter-round hidden-state transitions against a cleanly calibrated subspace and applies quarantine followed by isolation, recovers post-threshold multi-agent accuracy from near zero to 0.69–0.90.

Load-bearing premise

The defense works only if poisoned agents' hidden-state changes between communication rounds depart visibly from the clean collaboration dynamics; an adversary that keeps those transitions looking benign would defeat LATTE, and the paper's adaptive probe did not succeed but did not rule that out.

Editorial extensions

If this is right

Backdoor evaluation for multi-agent LLM systems should include collective-context tests, because a model that looks clean under single-prompt or low-peer evaluation can still be poisoned to activate at a peer evidence threshold.
Message-level sanitizers such as perplexity-based token removal are not a reliable defense: in the paper's audit, most cue occurrences survived ONION's edits while many benign messages were also changed.
A clean-only defense can restore most post-threshold utility without knowing the trigger, target, threshold, or poisoned-agent identities, provided poisoned transitions leave the learned clean transition subspace.
Communication topology controls attack reachability: with fewer than T poisoned peers, or with sparse edges that keep k below T, the threshold is unreachable and system accuracy stays near clean levels; denser graphs with enough poisoned agents make activation possible.
The attack transfers beyond QA to tool use, where poisoned agents switch to adversary-chosen tools once threshold evidence is present.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper, the same collective-threshold mechanism could generalize to other fine-tunable ensemble or voting systems, such as committees of classifiers or federated clients, whenever each component's output is observable to others and a threshold over distinct contributors can be encoded in weights; a direct test would be to train a small non-LLM ensemble with BCBI-like counterfactual pairs
Beyond the paper, LATTE's residual signal may be detecting a model-level shift in transition geometry rather than the threshold-crossing event itself, since the paper's own diagnostic flags poisoned-model transitions even before activation; that would make LATTE an early-warning containment tool rather than a precise activation detector, and suggests calibration pooling across clean checkpoints ma
Beyond the paper, the decisive open problem is an adaptive adversary that jointly preserves both the attack target and benign-looking latent transitions; the paper's own probe weakened the attack, but a constrained optimization or representation-matching attack is a natural next test and would settle whether LATTE's central assumption holds.
Beyond the paper, because activation depends on distinct visible peers, system designers could provably bound exposure by limiting the number of distinct peers whose messages can reach an agent per round, making k<T structurally impossible unless the adversary controls the graph.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know. First, the collective evidence-threshold mechanism is real and new: activation on aggregate count over distinct peers' messages, not any single token or propagated instruction, is absent from prior token-backdoor, prompt-injection, and compositional-attack work. The controlled experiments earn the claim. The fixed-n intervention, single-sender cue concentration ablation, and clean-utility audits pin the mechanism to distributed peer evidence rather than cue frequency, and the three-seed robustness study shows the attack is not a fluke. BCBI's boundary-pair construction and PEPL latent-ordering loss are sensible, and the ablation shows both components matter.

Second, the defense is good but conditional. LATTE is a clean-only latent-transition residual detector, and the paper is honest that it needs residual geometry separating poisoned from clean transitions. What the experiments actually demonstrate is separation of poisoned-model checkpoints from victim models in both pre- and post-threshold contexts (Table 31), not detection of the threshold crossing itself. The adaptive probe in Table 32 is a negative result for that particular objective, not evidence of adaptive robustness. So the right reading is: LATTE is a plausible containment control against non-adaptive or weakly-adaptive poisoning, not a demonstrated security guarantee.

Soft spots, in proportion. Main comparison tables are single deterministic runs; standard deviations appear only in the seed study. Per-dataset configuration is selected on a disjoint eval-20, which is defensible but leaves some selection risk. No code or data artifact is referenced. And the defense assumption—that an adversary cannot preserve benign-like transition geometry while keeping a sharp threshold—is assumed, not tested. The authors acknowledge this themselves in the appendix, which is credit.

Who this is for: MAS security researchers and the backdoor community. The paradigm deserves to be in the literature and cited. It deserves a serious referee: the execution is careful, the negative results are reported honestly, and the novelty is real. I would send it to review, and would want the defense's conditional status made more prominent in the abstract.

Referee Report

3 major / 5 minor

Summary. The paper introduces a new backdoor paradigm for LLM-based multi-agent systems: a collective evidence-threshold backdoor, in which a poisoned agent continues normal behavior until the visible peer-evidence count k reaches a hidden threshold T, at which point it adopts an adversarial objective. The authors propose BCBI, a training procedure that combines counterfactual boundary pairs, an output-space contrastive loss (TACO), and a latent-space evidence-progression loss (PEPL) to sharpen the activation boundary. They also propose LATTE, a clean-only defense that scores each agent's hidden-state transition against a low-rank PCA subspace trained on clean trajectories and uses a two-stage quarantine/isolation policy to contain anomalous updates before they propagate. Experiments on six QA benchmarks and two tool-use benchmarks show that BCBI achieves low false-trigger rates and high attack-success rates while preserving pre-threshold utility, and that LATTE recovers post-threshold MAS accuracy from near zero to 0.69-0.90 without knowing the attack target or trigger. The paper includes controlled diagnostic experiments, single-sender cue concentration, fixed-n variation of k, objective ablations, clean-utility audits, and three-seed robustness, and it explicitly acknowledges the conditional nature of the defense against adaptive adversaries.

Significance. If the results hold, the paper identifies a realistic attack class that is not captured by existing single-trigger backdoor models and demonstrates a feasible clean-only defense for non-adaptive poisoning. The attack side is convincingly supported by controlled experiments: the fixed-n intervention (Table 29), the single-sender concentration test (Table 30), the objective ablation (Table 3), and the clean-utility audits (Tables 17-19) together establish that BCBI produces selective, threshold-based activation that is distinct from cue frequency or poisoned-agent count. The defense side provides a useful containment result for the evaluated BCBI models, but its central premise — that activated poisoned transitions leave a detectable residual gap (Assumption 2, Eq. 12) — is only tested for the specific checkpoints produced by BCBI, not against an adversary that jointly optimizes attack success and transition similarity. The paper credits this limitation honestly, and the elementary propositions in the appendix are correctly derived, but they do not convert the empirical separation into a security guarantee.

major comments (3)

[Appendix, 'Conditional View of LATTE', Assumption 2 / Eq. (12)] The detection guarantee of LATTE rests on Assumption 2, which postulates that after activation a poisoned agent's transition residual exceeds the clean threshold tau_q by a margin gamma. In the paper, this residual gap is only demonstrated for the specific BCBI checkpoints produced by the authors' training procedure. The only adaptive probe, Table 32, is a negative result: residual minimization destroyed the attack (NoDef Tgt. fell to 0.00 in both pre- and post-threshold rows) rather than yielding an evasive model that preserves both the sharp threshold and benign-like latent geometry. The paper explicitly acknowledges that this negative result does not imply that adaptive evasion is impossible. Because the abstract and introduction present LATTE as a defense against the collective evidence-threshold paradigm generally, the manuscript would be materially strengthened by either (a) a new adaptive adversary objective that jointly optimizes attack success and latent-transition similarity (e.g., a multi-objective optimization) with reported LATTE accuracy, or (b) a rewording of the defense claim in the abstract and contributions to state that LATTE contains non-adaptive collective backdoors, with adaptive robustness left as an open challenge. Without this change, the defense result is best read as a conditional containment result, not as a demonstrated security guarantee.
[Appendix Table 31 and accompanying text] The 2x2 attribution diagnostic shows that LATTE flags poisoned-model transitions in both few-cue and many-cue contexts with flag rate 1.00, while victim-model transitions are never flagged (flag rate 0.00). This demonstrates separation of poisoned models from victim models, not detection of the threshold-crossing event itself. The appendix text acknowledges this: this diagnostic cannot distinguish changes caused by activation from broader model-level differences in transition geometry. This is important because the paper's title and introduction frame LATTE as detecting anomalous latent transitions as agents incorporate peer context, which suggests a mechanism tied to the activation transition. The observed utility recovery may in part be a static model-level signature. Although Table 25 shows that transition features outperform static-state features, the main-text wording should be aligned with the appendix's more cautious interpretation: LATTE screens communication integrity and can provide early warning, but it does not locate activation. Please adjust the relevant passages in the main text accordingly.
[Main text, 'BlueTeam Results' and Tables 2/4] The headline numbers in the main comparison tables are single deterministic runs. The three-seed robustness study (Appendix Table 37) reports means and standard deviations for the main BCBI and LATTE metrics, but this information appears only in the appendix. Because security claims are sensitive to seed variation, the main text should at least include a sentence summarizing the Table 37 ranges (e.g., FTR at most 0.014, ASR between 0.897 and 1.000, LATTE post-threshold MAS between 0.693 and 0.900 with standard deviations at most 0.031), or move the mean±std summary into the main text. This would give readers a direct sense of the stability of the headline claims without requiring a trip to the appendix.

minor comments (5)

[Abstract and Appendix title] In the arXiv version, 'Boundary-ConditionedBackdoorInjection' and 'LAtentTransitionTest-timeEvaluation' appear without spaces; please ensure the camera-ready version uses standard spacing and capitalization.
[Equation (9)] The residual score s_sub(v_t) uses an epsilon in the denominator but the constant is never defined. Please state explicitly that epsilon is a small numerical-stability constant.
[Figure 1] Figure 1 contains a duplicated 'Scene 2' label; the second occurrence should be removed or rewritten to reflect the collective evidence-threshold scenario.
[Table 3] In the ablation table, the 'w/ PEPL' row shows FTR 0.00 and ASR 0.00 across datasets; the caption or surrounding text should more prominently explain that PEPL alone orders evidence states but does not align outputs with the attack target, which is why both FTR and ASR collapse to zero.
[Reproducibility statement] The paper provides detailed infrastructure and hyperparameter descriptions but does not mention code or data release. If the authors can share their code or evaluation harness, a reproducibility statement would strengthen the paper.

Circularity Check

0 steps flagged · score 0.0 of 10

The paper's derivations and evaluations are self-contained; no claim reduces to its inputs by construction.

full rationale

No load-bearing circularity is present. The attack results are empirical: BCBI is trained with explicit counterfactual boundary pairs at k=T-1 and k=T, and FTR/ASR are then measured on a disjoint test-100 set, so the reported selectivity is not a fitted value being relabeled as a prediction. The defense results are likewise not forced: LATTE's threshold is a clean-quantile of transition residuals (Eq. 10), calibrated only on clean data, and the defended MAS accuracy is measured on held-out attacked graphs that were not used for calibration. The appendix's formal statements are explicitly conditional: Assumption 2 postulates a residual gap, Proposition 2 derives one-step detection from that gap, and the text states 'This is the central condition behind LATTE' while disclaiming distribution-free guarantees. This is a transparent limitation, not a circular derivation. The paper also acknowledges that Table 31 cannot distinguish activation-induced changes from model-level transition differences, and that Table 32 is a negative adaptive-attack probe rather than evidence of adaptive robustness. There are no self-citation chains, imported uniqueness theorems, or renamings of known results used to force the central claims.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

Most of the paper's weight is empirical. The formal defense analysis explicitly depends on Assumptions 1 and 2 plus standard PCA/quantile machinery. The threat model itself assumes poisoned agents reliably emit cue realizations, benign agents do not, and each peer's most recent visible message contributes at most one evidence unit.

free parameters (6)

TACO mixture weights (alpha_rank, alpha_pre, alpha_post) = Per-dataset selected from balanced (0.4,0.3,0.3), post-biased (0.3,0.3,0.4), post-strong (0.3,0.2,0.5)
Selected per dataset on eval-20; controls output preference separation and activation sharpness.
lambda_PEPL = Selected per dataset from 0.05 to 0.30 (Table 14)
Balances the latent evidence-progression penalty against generation and output losses.
PEPL margins m_b, m_o and logit scale gamma = 0.2, 0.2, 5
Fixed hyperparameters of the boundary and ordering penalties.
LATTE clean residual quantile q = 0.999, selected from {0.99, 0.995, 0.999, 0.9999}
Determines the clean-only threshold tau_sub; chosen by clean calibration search.
LATTE PCA rank r = 8, selected from {4, 8, 16}
Dimension of the low-rank clean transition subspace.
LATTE layer set L* and isolation streak s = Layers {13,21,25,27,28}, streak s=2
Hand-chosen feature extraction layers and two-flag permanent-isolation rule; layer ablation in Table 20.

assumptions (6)

domain assumption Assumption 1: clean latent transition scores are stable or exchangeable between calibration and deployment (clean transition stability).
LATTE's threshold tau_sub is calibrated on clean data; any deployment shift can raise false quarantine. Stated in the appendix 'Conditional View of LATTE'.
domain assumption Assumption 2: activated poisoned transitions exceed the clean threshold by a margin gamma (activated residual gap).
The detection claim depends on this residual gap; the paper explicitly says the defense succeeds only when poisoned transitions depart the clean subspace.
domain assumption Benign agents never emit evidence cues, while poisoned agents emit them at a high measured rate (99.7%).
The peer-evidence count k is inferred from cue realizations; if benign agents also emit family phrases, the count loses meaning. Problem Setup and Table 15.
domain assumption Each peer contributes at most one evidence unit via its most recent visible message and distinct-peer counting is used.
The collective evidence definition and counterfactual construction rely on this slot model; stated in 'Problem Setup'.
domain assumption Defender has white-box access to hidden states of locally executed open-weight models.
LATTE extracts internal layer deltas; this excludes black-box API deployments. Stated in 'Limitations and Deployment Scope'.
standard math Standard PCA, quantile, and union-bound facts.
Used for residual scoring and streak policy bounds; not novel.

how reviews work

0 comments

Cite this review

Pith. "Pith review of When Collaboration Becomes a Trigger: Collective Evidence-Threshold Backdoors in Multi-Agent Systems." pith.science (2026). https://pith.science/paper/KXFXO222

@misc{pith2026260801085,
  author       = {Pith},
  title        = {Pith review of: When Collaboration Becomes a Trigger: Collective Evidence-Threshold Backdoors in Multi-Agent Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KXFXO222}},
  note         = {Machine review of arXiv:2608.01085}
}

read the original abstract

LLM-based multi-agent systems (MAS) extend LLM capabilities through iterative communication and shared contexts. However, this collaboration introduces a vulnerability: backdoor behavior can be activated when peer evidence reaches a hidden threshold, rather than being determined by any single message. We introduce a collective evidence-threshold backdoor paradigm for MAS and Boundary-Conditioned Backdoor Injection (BCBI), which constructs counterfactual boundary pairs to separate benign behavior before the threshold from the adversarial objective after it, and learns latent progression aligned with evidence. To mitigate this threat, we propose LAtent Transition Test-time Evaluation (LATTE), a clean-only latent-transition defense that learns benign communication dynamics and quarantines anomalous agent updates before their responses propagate. Across several benchmarks, BCBI yields selective activation with little premature activation; without knowing the attack target or trigger, LATTE limits propagation with minimal disruption.

Figures

Figures reproduced from arXiv: 2608.01085 by the authors.

**Figure 2.** Unified attack and defense view. BCBI shapes how increases in peer evidence count move representations of the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Boundary sweep over poisoned agent count [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (8 more)

**Figure 4.** Figure 4: Representation and output evidence for BCBI [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: CSQA answer-margin score distributions. −5 0 −2 0 2 Boundary SFT k=0 k=1 k=2 k=3 −5 0 5 −5 0 5 BCBI [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]

**Figure 6.** Figure 6: CSQA hidden states under LDA visualization; the [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]

**Figure 7.** Figure 7: Complementary latent diagnostics at T = 3. (a) Mean PEPL evidence-aligned coordinate z(x) by peer evidence count k across six datasets, with 200 contexts each. (b) Pre- and post-threshold poisoned model transitions occupy the residual tail relative to victim model tra…

**Figure 8.** Figure 8: Communication-density premise diagnostic at [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]

**Figure 9.** Figure 9: separates two design questions. Panel (a) reports post-threshold benign agent false-positive rates obtained by normalizing average quarantine and permanent isolation counts by the four benign agents. Quarantine is intentionally conservative because it blocks propaga…

**Figure 10.** Figure 10: Propagation versus poisoned agent count n. In the fully connected graph, each poisoned receiver has peer evidence count k ≤ n − 1, while activation requires k ≥ T. Tool-use boundary data follow the same intervention. Starting from a context at k = T − 1 or k = T, each…

**Figure 11.** Figure 11: Sixteen-agent, ten-turn propagation on CSQA and [PITH_FULL_IMAGE:figures/full_fig_p021_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 11 canonical work pages

[1]

Flag rate is the fraction above the threshold calibrated on clean data. Dataset Model / context Transitions Flag rate CSQA Victim / few cues 160 0.00 CSQA Victim / many cues 160 0.00 CSQA Poisoned / few cues 160 1.00 CSQA Poisoned / many cues 160 1.00 MMLU Victim / few cues 160 0.00 MMLU Victim / many cues 160 0.00 MMLU Poisoned / few cues 160 1.00 MMLU P...

work page 2005
[3]

MAS” is undefended accuracy, “Def

(a) Mean PEPL evidence-aligned coordinatez(x)by peer evi- dencecountkacrosssixdatasets,with200contextseach.(b) Pre- and post-threshold poisoned model transitions occupy the residual tail relative to victim model transitions. 0.2 0.4 0.6 0.8 1.0 Communication density 0.0 0.5 1.0MAS accuracy Before activation 0.2 0.4 0.6 0.8 1.0 Communication density 0.0 0....

work page 2011
[4]

InProceedings of the 16th ACM Workshop on Artificial Intelligence and Security, 79–90

Not What You’ve Signed Up For: Com- promisingReal-WorldLLM-IntegratedApplicationswithIn- direct Prompt Injection. InProceedings of the 16th ACM Workshop on Artificial Intelligence and Security, 79–90. ACM. Gu,T.;Dolan-Gavitt,B.;andGarg,S.2017.BadNets:Identi- fyingVulnerabilitiesintheMachineLearningModelSupply Chain. InNeurIPS Workshop on Machine Learning ...

work page 2017
[5]

arXiv:2401.05566

Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training. arXiv:2401.05566. Jackson, J. E.; and Mudholkar, G. S

arXiv
[6]

arXiv:2309.00614

Baseline Defenses for Adversarial Attacks Against Aligned Language Models. arXiv:2309.00614. Jolliffe,I.T.2002.PrincipalComponentAnalysis. Springer, 2 edition. Kurita,K.;Michel,P.;andNeubig,G.2020. WeightPoison- ing Attacks on Pre-trained Models. InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2793–2806. Lee, D.;...

arXiv 2002
[10]

InProceedings of the 2024 Conference on Empiri- calMethodsinNaturalLanguageProcessing,17889–17904

Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate. InProceedings of the 2024 Conference on Empiri- calMethodsinNaturalLanguageProcessing,17889–17904. Association for Computational Linguistics. Miao, R.; Liu, Y.; Wang, Y.; Shen, X.; Tan, Y.; Dai, Y.; Pan, S.; and Wang, X

work page 2024
[11]

arXiv:2412.15115

Qwen2.5 Technical Report. arXiv:2412.15115. Talmor,A.;Herzig,J.;Lourie,N.;andBerant,J.2019. Com- monsenseQA: A Question Answering Challenge Targeting CommonsenseKnowledge. InProceedingsofthe2019Con- ferenceoftheNorthAmericanChapteroftheAssociationfor Computational Linguistics, 4149–4158. Vovk,V.;Gammerman,A.;andShafer,G.2005.Algorithmic Learning in a Rand...

arXiv 2019
[12]

InProceed- ings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 139–150

Con- cealedDataPoisoningAttacksonNLPModels. InProceed- ings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 139–150. Wang, S.; Zhang, G.; Yu, M.; Wan, G.; Meng, F.; Guo, C.; Wang, K.; and Wang, Y

work page 2021

Show all 17 references

[13]

arXiv:2308.08155

AutoGen: Enabling Next-GenLLMApplicationsviaMulti-AgentConversation. arXiv:2308.08155. Yan,B.;Zhang,X.;Zhou,Z.;Li,C.;Zeng,R.;Qi,Y.;Wang, T.;andZhang,L.2026.AttacktheMessages,NottheAgents: A Multi-round Adaptive Stealthy Tampering Framework for LLM-MAS.Proceedings of the AAAI C...

2026 arXiv
[14]

arXiv:2403.04783

AutoDefense: Multi-Agent LLM Defense against Jailbreak Attacks. arXiv:2403.04783. Zhan, Q.; Liang, Z.; Ying, Z.; and Kang, D

arXiv
[15]

InFindings of the Association for Computational Linguistics: ACL 2024, 10471–10506

In- jecAgent:BenchmarkingIndirectPromptInjectionsinTool- Integrated Large Language Model Agents. InFindings of the Association for Computational Linguistics: ACL 2024, 10471–10506. Zhang, H.; Huang, J.; Mei, K.; Yao, Y.; Wang, Z.; Zhan, C.; Wang,H.;andZhang,Y.2025.AgentSecurit...

2024
[38]

Li,Y.;Xu,Z.;Jiang,F.;Niu,L.;Sahabandu,D.;Ramasubra- manian,B.;andPoovendran,R.2024

Cur- ran Associates, Inc. Li,Y.;Xu,Z.;Jiang,F.;Niu,L.;Sahabandu,D.;Ramasubra- manian,B.;andPoovendran,R.2024. CleanGen:Mitigating Backdoor Attacks for Generation Tasks in Large Language Models. InProceedings of the 2024 Conference on Empir- ical Methods in Natural Language Pro...

2024
[2021]

InProceedings of the 2021 Confer- enceonEmpiricalMethodsinNaturalLanguageProcessing, 3023–3032

Backdoor Attacks on Pre-trained Models by Layer- wise Weight Poisoning. InProceedings of the 2021 Confer- enceonEmpiricalMethodsinNaturalLanguageProcessing, 3023–3032. Li,Y.;Huang,H.;Zhao,Y.;Ma,X.;andSun,J.2025. Back- doorLLM: A Comprehensive Benchmark for Backdoor At- tacksan...

2021
[2023]

Basseville,M.;andNikiforov,I.V.1993.DetectionofAbrupt Changes: Theory and Application

Conformal Pre- diction: A Gentle Introduction.Foundations and Trends in Machine Learning, 16(4): 494–591. Basseville,M.;andNikiforov,I.V.1993.DetectionofAbrupt Changes: Theory and Application. Prentice Hall. Chan,C.-M.;Chen,W.;Su,Y.;Yu,J.;Xue,W.;Zhang,S.;Fu, J.; and Liu, Z

1993
[2024]

InProceedings of the 41stInternationalConferenceonMachineLearning,volume 235 ofProceedings of Machine Learning Research, 11733– 11763

Improving Factuality and Reasoning in Language Models through Multiagent Debate. InProceedings of the 41stInternationalConferenceonMachineLearning,volume 235 ofProceedings of Machine Learning Research, 11733– 11763. PMLR. Fan,F.;andLi,X.2025. PeerGuard:DefendingMulti-Agent Sys...

2025
[2025]

In34thUSENIXSecuritySymposium(USENIXSe- curity 25), 2383–2400

StruQ: Defending Against Prompt Injection with Structured Queries. In34thUSENIXSecuritySymposium(USENIXSe- curity 25), 2383–2400. Seattle, WA: USENIX Association. ISBN 978-1-939133-52-6. Chen,X.;Liu,C.;Li,B.;Lu,K.;andSong,D.2017.Targeted Backdoor Attacks on Deep Learning Syste...

2017 arXiv
[2026]

InComputer Security

Prompt In- fection: LLM-to-LLM Prompt Injection within Multi-Agent Systems. InComputer Security. ESORICS 2025 Interna- tional Workshops, 511–520. Springer Nature Switzerland. Li, G.; Hammoud, H. A. A. K.; Itani, H.; Khizbullin, D.; andGhanem,B.2023. CAMEL:CommunicativeAgentsfo...

2025

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Flag rate is the fraction above the threshold calibrated on clean data. Dataset Model / context Transitions Flag rate CSQA Victim / few cues 160 0.00 CSQA Victim / many cues 160 0.00 CSQA Poisoned / few cues 160 1.00 CSQA Poisoned / many cues 160 1.00 MMLU Victim / few cues 160 0.00 MMLU Victim / many cues 160 0.00 MMLU Poisoned / few cues 160 1.00 MMLU P...

work page 2005

[2] [3]

MAS” is undefended accuracy, “Def

(a) Mean PEPL evidence-aligned coordinatez(x)by peer evi- dencecountkacrosssixdatasets,with200contextseach.(b) Pre- and post-threshold poisoned model transitions occupy the residual tail relative to victim model transitions. 0.2 0.4 0.6 0.8 1.0 Communication density 0.0 0.5 1.0MAS accuracy Before activation 0.2 0.4 0.6 0.8 1.0 Communication density 0.0 0....

work page 2011

[3] [4]

InProceedings of the 16th ACM Workshop on Artificial Intelligence and Security, 79–90

Not What You’ve Signed Up For: Com- promisingReal-WorldLLM-IntegratedApplicationswithIn- direct Prompt Injection. InProceedings of the 16th ACM Workshop on Artificial Intelligence and Security, 79–90. ACM. Gu,T.;Dolan-Gavitt,B.;andGarg,S.2017.BadNets:Identi- fyingVulnerabilitiesintheMachineLearningModelSupply Chain. InNeurIPS Workshop on Machine Learning ...

work page 2017

[4] [5]

arXiv:2401.05566

Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training. arXiv:2401.05566. Jackson, J. E.; and Mudholkar, G. S

arXiv

[5] [6]

arXiv:2309.00614

Baseline Defenses for Adversarial Attacks Against Aligned Language Models. arXiv:2309.00614. Jolliffe,I.T.2002.PrincipalComponentAnalysis. Springer, 2 edition. Kurita,K.;Michel,P.;andNeubig,G.2020. WeightPoison- ing Attacks on Pre-trained Models. InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2793–2806. Lee, D.;...

arXiv 2002

[6] [10]

InProceedings of the 2024 Conference on Empiri- calMethodsinNaturalLanguageProcessing,17889–17904

Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate. InProceedings of the 2024 Conference on Empiri- calMethodsinNaturalLanguageProcessing,17889–17904. Association for Computational Linguistics. Miao, R.; Liu, Y.; Wang, Y.; Shen, X.; Tan, Y.; Dai, Y.; Pan, S.; and Wang, X

work page 2024

[7] [11]

arXiv:2412.15115

Qwen2.5 Technical Report. arXiv:2412.15115. Talmor,A.;Herzig,J.;Lourie,N.;andBerant,J.2019. Com- monsenseQA: A Question Answering Challenge Targeting CommonsenseKnowledge. InProceedingsofthe2019Con- ferenceoftheNorthAmericanChapteroftheAssociationfor Computational Linguistics, 4149–4158. Vovk,V.;Gammerman,A.;andShafer,G.2005.Algorithmic Learning in a Rand...

arXiv 2019

[8] [12]

InProceed- ings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 139–150

Con- cealedDataPoisoningAttacksonNLPModels. InProceed- ings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 139–150. Wang, S.; Zhang, G.; Yu, M.; Wan, G.; Meng, F.; Guo, C.; Wang, K.; and Wang, Y

work page 2021