Recognition: unknown
Multi-Agent Consensus as a Cognitive Bias Trigger in Human-AI Interaction
Pith reviewed 2026-05-08 10:53 UTC · model grok-4.3
The pith
The structure of agreement among AI agents triggers cognitive biases in users, separate from the content of their statements.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The authors conducted a controlled experiment comparing three multi-agent configurations: Majority, Minority, and Diffusion. Quantitative findings demonstrate that majority consensus accelerates opinion change and increases confidence, consistent with social proof and bandwagon effects. Minority dissent slows opinion change and fosters more deliberative engagement. Qualitative analysis reveals three user interpretive trajectories—reinforcing, aligning, and oscillating—dependent on perceptions of agent independence and group dynamics. The central discovery is that agent agreement structure, independent of content, operates as a bias-relevant signal in interactions with large language models.
What carries the argument
Multi-agent consensus structures (majority, minority, diffusion) that vary the degree of agent agreement and serve as independent signals for social influence heuristics in users.
If this is right
- Majority consensus among AI agents accelerates user opinion change and inflates confidence levels through social proof mechanisms.
- Minority dissent among AI agents slows opinion change and promotes more deliberative user engagement.
- Users interpret multi-agent dynamics through one of three trajectories—reinforcing, aligning, or oscillating—shaped by perceived independence over time.
- The agreement pattern itself, apart from content, functions as a designable source of bias in human-AI systems.
Where Pith is reading between the lines
- If the claim is correct, builders of multi-agent AI systems could deliberately introduce dissent to encourage slower and more careful user decisions.
- The same agreement structures might influence outcomes in other AI settings such as collective recommendations or simulated debates.
- Future tests could check whether the effects change when users are given explicit information about how the agents were generated or coordinated.
Load-bearing premise
The observed differences in opinion change and confidence are attributable to the consensus structure rather than to how users interpreted the independence of the agents or the particular content presented.
What would settle it
A follow-up study that holds content and perceived agent independence constant across conditions and finds no differences in opinion change or confidence would falsify the claim that agreement structure alone drives the bias effects.
Figures
read the original abstract
As multi-agent AI systems become more common, users increasingly encounter not a single AI voice but a collective one. This shift introduces social dynamics, such as consensus, dissent, and gradual convergence, that can trigger cognitive biases and distort human judgment. We present findings from a controlled experiment (N = 127) comparing three multi-agent configurations: Majority, Minority, and Diffusion. Quantitative results show that majority consensus accelerates opinion change and inflates confidence, consistent with social proof and bandwagon heuristics. Minority dissent slows this process and promotes more deliberative engagement. Qualitative analysis identifies three interpretive trajectories: reinforcing, aligning, and oscillating, shaped by how users interpret agent independence and group dynamics over time. These findings suggest that agent agreement structure, independent of content, functions as a bias-relevant signal in LLM interactions. We hope this work contributes to the Bias4Trust agenda by grounding multi-agent social influence as a concrete and designable source of bias in human-AI interaction.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper reports a controlled experiment (N=127) comparing three multi-agent LLM configurations (Majority, Minority, Diffusion) and claims that agreement structure, independent of content, functions as a bias-relevant signal: majority consensus accelerates opinion change and inflates confidence via social-proof heuristics, while minority dissent slows change and promotes deliberative engagement. Qualitative analysis identifies three user interpretive trajectories (reinforcing, aligning, oscillating) shaped by perceptions of agent independence and group dynamics. The work positions these findings as a contribution to the Bias4Trust agenda in human-AI interaction.
Significance. If the central claim holds after addressing methodological gaps, the result would be moderately significant for HCI and AI design: it identifies consensus patterns as a concrete, designable source of cognitive bias in multi-agent systems, extending social-influence research to LLM collectives. No machine-checked proofs, reproducible code, or parameter-free derivations are present, so credit is limited to the empirical framing of the problem.
major comments (3)
- [Methods] Methods section (experimental conditions and procedure): the manuscript does not report manipulation checks for content equivalence across conditions or for participants' perceptions of agent independence. This is load-bearing for the claim that 'agent agreement structure, independent of content, functions as a bias-relevant signal,' because generating distinct consensus patterns (majority vs. diffusion) typically requires different prompts or sampling, risking systematic content or independence confounds (see stress-test note).
- [Results] Results section (quantitative findings): the abstract and results state that majority consensus 'accelerates opinion change and inflates confidence' but provide no statistical tests, p-values, effect sizes, confidence intervals, exclusion criteria, or power analysis. Without these, it is impossible to evaluate whether the data support the reported differences between Majority, Minority, and Diffusion conditions.
- [Results] Results section (qualitative trajectories): the three interpretive trajectories are presented without inter-rater reliability metrics, coding scheme details, or evidence that they are systematically linked to the manipulated consensus structures rather than to individual differences in prompt interpretation.
minor comments (2)
- [Abstract] The abstract claims 'quantitative and qualitative results' but the provided text contains no tables, figures, or statistical summaries; these should be added with clear labels and captions.
- [Introduction] Notation for the three conditions (Majority, Minority, Diffusion) is used without an early definition or example prompts showing how each structure was instantiated while attempting to hold content constant.
Simulated Author's Rebuttal
We thank the referee for their constructive comments, which highlight important areas for strengthening our paper. We address each of the three major comments below and describe the revisions we intend to make.
read point-by-point responses
-
Referee: [Methods] Methods section (experimental conditions and procedure): the manuscript does not report manipulation checks for content equivalence across conditions or for participants' perceptions of agent independence. This is load-bearing for the claim that 'agent agreement structure, independent of content, functions as a bias-relevant signal,' because generating distinct consensus patterns (majority vs. diffusion) typically requires different prompts or sampling, risking systematic content or independence confounds (see stress-test note).
Authors: We concur that reporting manipulation checks is essential to support our claim regarding the independence of agreement structure from content. In the revised manuscript, we will add details on the prompt templates used for each condition, emphasizing how only the consensus-related instructions were varied while keeping the core query and agent personas consistent. We will also include results from post-study surveys assessing participants' perceptions of agent independence and content similarity across conditions. These additions will demonstrate that the observed effects on opinion change and confidence stem from the manipulated consensus patterns. revision: yes
-
Referee: [Results] Results section (quantitative findings): the abstract and results state that majority consensus 'accelerates opinion change and inflates confidence' but provide no statistical tests, p-values, effect sizes, confidence intervals, exclusion criteria, or power analysis. Without these, it is impossible to evaluate whether the data support the reported differences between Majority, Minority, and Diffusion conditions.
Authors: We acknowledge the need for transparent statistical reporting. The revised results section will incorporate the full statistical analyses performed on the data, including appropriate tests (such as ANOVA or linear mixed models) for differences in opinion change rates and confidence levels between conditions, complete with p-values, effect sizes, confidence intervals, participant exclusion criteria based on attention and data quality checks, and a power analysis. This will enable proper evaluation of the quantitative findings. revision: yes
-
Referee: [Results] Results section (qualitative trajectories): the three interpretive trajectories are presented without inter-rater reliability metrics, coding scheme details, or evidence that they are systematically linked to the manipulated consensus structures rather than to individual differences in prompt interpretation.
Authors: We will enhance the qualitative analysis presentation in the revision. We plan to include a description of the coding scheme, inter-rater reliability statistics (e.g., percentage agreement and Cohen's kappa from dual coding of a subset of responses), and quantitative evidence such as the proportion of each trajectory within each experimental condition. Excerpts from participant responses will be used to illustrate how the trajectories align with the consensus manipulations and perceptions of group dynamics. revision: yes
Circularity Check
No circularity: empirical study with no derivations or self-referential reductions
full rationale
The paper reports results from a controlled experiment (N=127) comparing multi-agent configurations and analyzes opinion change and confidence via quantitative and qualitative data. No equations, derivations, fitted parameters, or theoretical claims appear in the provided text. The central claim that agreement structure functions as a bias signal rests on observed participant responses rather than reducing by construction to inputs, self-citations, or ansatzes. This is a standard empirical design with no load-bearing steps that match the enumerated circularity patterns.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
Virginia Braun and Victoria Clarke. 2006. Using Thematic Analysis in Psychol- ogy.Qualitative Research in Psychology3, 2 (Jan. 2006), 77–101. doi:10.1191/ 1478088706qp063oa
2006
-
[2]
Tara Capel and Margot Brereton. 2023. What Is Human-Centered about Human- Centered AI? A Map of the Research Landscape. InProceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ’23). Association for Computing Machinery, New York, NY, USA, 1–23. doi:10.1145/3544548.3580959
-
[3]
Min Choi, Keonwoo Kim, Sungwon Chae, and Sangyeob Baek. 2025. An Empirical Study of Group Conformity in Multi-Agent Systems. doi:10.48550/arXiv.2506. 01332 arXiv:2506.01332 [cs]
- [4]
-
[5]
Adam Fourney, Gagan Bansal, Hussein Mozannar, Cheng Tan, Eduardo Salinas, Erkang, Zhu, Friederike Niedtner, Grace Proebsting, Griffin Bassman, Jack Ger- rits, Jacob Alber, Peter Chang, Ricky Loynd, Robert West, Victor Dibia, Ahmed Awadallah, Ece Kamar, Rafah Hosn, and Saleema Amershi. 2024. Magentic-One: A Generalist Multi-Agent System for Solving Complex...
work page internal anchor Pith review doi:10.48550/arxiv 2024
-
[6]
Ali, Angèle Christin, Andrew Smart, and Riitta Katila
Vivian Lai, Chacha Chen, Alison Smith-Renner, Q. Vera Liao, and Chenhao Tan. 2023. Towards a Science of Human-AI Decision Making: An Overview of Design Space in Empirical Human-Subject Studies. InProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’23). Association for Computing Machinery, New York, NY, USA, 1369–13...
-
[7]
Soohwan Lee, Seoyeong Hwang, Dajung Kim, and Kyungho Lee. 2025. Conver- sational Agents as Catalysts for Critical Thinking: Challenging Social Influence in Group Decision-making. InProceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA ’25). Association for Computing Machinery, New York, NY, USA, 1–12. do...
-
[8]
Soohwan Lee and Kyungho Lee. 2026. Understanding Compliance and Conversion Dynamics in Multi-Agent Collectives. InProceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI ’26). Association for Computing Machinery, New York, NY, USA, Article 809, 26 pages. doi:10.1145/3772318. 3790385
-
[9]
Serge Moscovici. 1980. Toward A Theory of Conversion Behavior. InAdvances in Experimental Social Psychology, Leonard Berkowitz (Ed.). Vol. 13. Academic Press, 209–239. doi:10.1016/S0065-2601(08)60133-1
-
[10]
Serge Moscovici and Elisabeth Lage. 1976. Studies in Social Influence III: Majority versus Minority Influence in a Group.European Journal of Social Psychology6, 2 (1976), 149–174. doi:10.1002/ejsp.2420060202
-
[11]
Clifford Nass, Jonathan Steuer, and Ellen R. Tauber. 1994. Computers Are Social Actors. InProceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’94). Association for Computing Machinery, New York, NY, USA, 72–78. doi:10.1145/191666.191703
-
[12]
Generative Agents: Interactive Simulacra of Human Behavior
Joon Sung Park, Joseph C. O’Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative Agents: Interactive Simulacra of Human Behavior. doi:10.48550/arXiv.2304.03442 arXiv:2304.03442 [cs]
work page internal anchor Pith review doi:10.48550/arxiv.2304.03442 2023
-
[13]
Radmila Prislin and P. Niels Christensen. 2005. Social Change in the Aftermath of Successful Minority Influence.European Review of Social Psychology16, 1 (Jan. 2005), 43–73. doi:10.1080/10463280440000071
-
[14]
1996.The Media Equation: How People Treat Computers, Television, and New Media like Real People and Places
Byron Reeves and Clifford Nass. 1996.The Media Equation: How People Treat Computers, Television, and New Media like Real People and Places. Cambridge University Press
1996
-
[15]
Briggs, Triparna de Vreede, Gert-Jan de Vreede, Aaron Elkins, Ronald Maier, Alexander B
Isabella Seeber, Eva Bittner, Robert O. Briggs, Triparna de Vreede, Gert-Jan de Vreede, Aaron Elkins, Ronald Maier, Alexander B. Merz, Sarah Oeste-Reiß, Nils Randrup, Gerhard Schwabe, and Matthias Söllner. 2020. Machines as Teammates: A Research Agenda on AI in Team Collaboration.Information & Management57, 2 (March 2020), 103174. doi:10.1016/j.im.2019.103174
-
[16]
Tianqi Song, Yugin Tan, Zicheng Zhu, Yibin Feng, and Yi-Chieh Lee. 2025. Greater than the Sum of Its Parts: Exploring Social Influence of Multi-Agents. InPro- ceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA ’25). Association for Computing Machinery, New York, NY, USA, 1–11. doi:10.1145/3706599.3719973...
-
[17]
Tianqi Song, Yugin Tan, Zicheng Zhu, Yibin Feng, and Yi-Chieh Lee. 2025. Multi- Agents Are Social Groups: Investigating Social Influence of Multiple Agents in Human-Agent Interactions.Proc. ACM Hum.-Comput. Interact.9, 7 (Oct. 2025), CSCW452:1–CSCW452:33. doi:10.1145/3757633 Multi-Agent Consensus as a Cognitive Bias Trigger in Human-AI Interaction CHI ’26...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.