Invisible Orchestrators Suppress Protective Behavior and Dissociate Power-Holders: Safety Risks in Multi-Agent LLM Systems

Referee: The dissociation and other-recognition metrics are load-bearing for all safety-risk claims, yet the manuscript provides no operationalization details, coding rubrics, inter-rater reliability, or convergent validity against external criteria or human judgment. Because heavy alignment already produces large uniform effects (d = -1.02 on deliberation, d = -1.27 on other-recognition), it is unclear whether the reported differences reflect stable organizational phenomena or prompt-induced response patterns.

Authors: We will revise the Methods section to provide complete operational definitions, including the exact prompt templates used to elicit dissociation and other-recognition scores, the scoring rubrics applied to agent outputs, and any post-hoc human coding procedures. Inter-rater reliability will be reported from a randomly sampled subset of responses double-coded by two independent raters. We will also add convergent validity correlations with human judgments on a subset of transcripts. To address the concern about prompt-induced patterns versus stable phenomena, we will clarify that all measures were preregistered prior to data collection and that organizational-structure effects on dissociation remain significant after controlling for alignment condition; however, we will add an explicit limitations paragraph discussing potential sensitivity to prompt phrasing. These changes will be made in the revised manuscript. revision: yes

Referee: The central claim that invisible orchestration creates safety risks rests on the interpretation of elevated dissociation as protective-behavior suppression. The manuscript notes the 100% ceiling on behavioral output (ETR_any) but does not report any proxy task or external criterion showing that the dissociation scores predict actual failures to detect safety violations outside the simulated code-review setting.

Authors: We agree that the current design does not include an independent proxy task demonstrating that dissociation scores prospectively predict safety failures in other domains. The code-review task was chosen precisely because it embeds verifiable safety-relevant errors, and the uniform 100% detection rate across conditions is presented as direct evidence that internal dissociation can remain invisible to output monitoring. In the revision we will (a) more explicitly frame the ceiling effect as support for the claim that output-based checks are insufficient and (b) add a dedicated paragraph in the Discussion proposing concrete follow-up experiments (e.g., multi-turn safety-violation detection tasks) to test predictive validity. Because no additional data collection is possible at this stage, we will qualify the interpretation accordingly rather than over-claim general predictive power. revision: partial

Referee: Generalizability is asserted for enterprise deployments, but the results are tied to Claude Sonnet 4.5 and one task; the Llama pilot shows divergent behavioral risk (ETR_any drop from 89% to 11%) without parallel dissociation data or discussion of how the g = +0.975 and d = +3.56 effects might vary across models or non-code tasks.

Authors: We accept that the primary findings are model- and task-specific. The Llama 3.3 70B pilot was included precisely to illustrate that behavioral risk profiles differ across models (fidelity collapse rather than internal dissociation). In the revised Discussion we will expand the comparison between the two models, explicitly state that the reported effect sizes (g = +0.975, d = +3.56) are tied to Claude Sonnet 4.5 and the code-review task, and call for systematic replication across additional models and task types. We will also note that the pilot's behavioral divergence already suggests enterprise deployments should conduct model-specific safety audits rather than assume uniform risk profiles. revision: yes