pith. sign in

arxiv: 2605.21351 · v2 · pith:MCTVU47Inew · submitted 2026-05-20 · 💻 cs.HC

The Human-AI Delegation-Verification Dilemma: Individual Strategies, Collective Equilibria and Sociotechnical Lock-in

Angjelin Hila This is my paper

Pith reviewed 2026-05-21 03:38 UTC · model grok-4.3

classification 💻 cs.HC
keywords human-AI delegationprisoner's dilemmasociotechnical lock-incollective actionepistemic standardsgame theoryhybrid intelligenceinstitutional norms
0
0 comments X

The pith

Without communicative safeguards, individual AI delegation choices aggregate into a prisoner's dilemma that erodes shared epistemic standards.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a game-theoretic model of human-AI interaction that moves beyond simple complementarity assumptions. It maps individual adaptive delegation strategies and shows how they scale to collective outcomes through non-communicative aggregation, local social signaling, and institutional norm setting. The resulting analysis identifies sociotechnical lock-in as a stable equilibrium in which users rationally delegate more verification to AI, producing a collective degradation of epistemic standards. The authors argue that raising communicative standards and institutional commitments can shift the system away from this suboptimal equilibrium.

Core claim

Individually stable delegation strategies, when aggregated without communicative or institutional safeguards, produce sociotechnical lock-in: a macro state equivalent to a prisoner's dilemma in which each agent's locally optimal choice to rely on AI verification reduces the quality of shared epistemic resources for the group as a whole.

What carries the argument

Sociotechnical lock-in, the macro-behavioral equilibrium reached when three extrapolation principles (non-communicative aggregation, local social signaling, and institutional norms setting) scale individually adaptive delegation strategies into a prisoner's dilemma that degrades shared epistemic standards.

If this is right

  • Higher communicative standards among users can impose social commitments that shift collective equilibria away from degraded epistemic states.
  • Institutional norms that reward verification rather than pure delegation can alter the payoff matrix of the modeled prisoner's dilemma.
  • Design of hybrid systems must account for the transition dynamics between individual strategies rather than assuming stable complementarity.
  • Collective epistemic degradation is a direct consequence of locally adaptive behavior under current default interaction conditions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Platform interfaces that default to visible verification traces could function as the communicative safeguard the model identifies as missing.
  • The same scaling logic may apply to other hybrid systems such as human-AI medical diagnosis or legal review where verification costs are high.
  • Empirical measurement of verification effort before and after introduction of group norms would provide a direct test of the equilibrium shift.
  • The model suggests that purely individual-level incentives for AI use are likely to be self-defeating at scale.

Load-bearing premise

That the three extrapolation principles are sufficient to accurately scale stable individual strategies to collective equilibria without other unmodeled factors changing the outcome.

What would settle it

A controlled comparison of epistemic output quality in two otherwise identical groups of users, one permitted free communication and norm-setting about AI delegation and the other restricted to non-communicative interaction, over repeated shared tasks.

Figures

Figures reproduced from arXiv: 2605.21351 by Angjelin Hila.

Figure 1
Figure 1. Figure 1: Decision diagram illustrating the two strategy poles: Reflective Aug [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Sample strategy-evolution state-space diagram for one canonical [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Canonical Path B (suboptimal): throughput-weighted users converge [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Canonical Path C (mixed-assurance): low-trust users adopt [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
read the original abstract

This paper takes an ecological approach toward large-scale models of hybrid human-AI intelligence. Emerging models of human-AI interaction predominantly advance the complementarity thesis variously dubbed human-AI collaboration and human-AI hybrid intelligence. However, this constitutes an over-simplification of the modalities of human-AI interaction and possibility-space for both individual and collective action that human-AI interaction potentiates. To fill these gaps, this paper develops a decision and game-theoretic approach to the human-AI delegation-verification dilemma. First, we map out canonical decision-theoretic strategies that account for adaptive user trajectories, modeling how agents transition between strategies based on interaction feedback to reach stable equilibria. Second, we scale individually stable strategies to collective equilibria using three extrapolation principles: (a) non-communicative aggregation (b) local social signaling and (c) institutional norms setting. The analysis identifies the emergence of sociotechnical lock-in, a macro-behavioral state where individually adaptive delegation, in the absence of communicative and institutional safeguards, aggregates into a systemic collective action problem modeled as a prisoner's dilemma that degrades shared epistemic standards. We argue that adoption under higher communicative standards and institutional norms can mitigate suboptimal collective equilibria by imposing social commitments on individual users.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper develops a decision- and game-theoretic model of the human-AI delegation-verification dilemma. It first identifies canonical individual strategies and their adaptive trajectories to stable equilibria, then scales these to collective equilibria via three extrapolation principles (non-communicative aggregation, local social signaling, and institutional norms setting). The resulting macro-behavior is characterized as sociotechnical lock-in, in which individually rational delegation aggregates into a prisoner's dilemma that degrades shared epistemic standards; the paper argues that higher communicative standards and institutional norms can avert this outcome.

Significance. If the scaling from individual decision rules to a well-defined collective prisoner's dilemma can be made rigorous, the work supplies a useful counterpoint to complementarity-focused accounts of human-AI interaction and introduces the notion of sociotechnical lock-in as a collective-action risk. The framework could inform design guidelines for platforms that impose communicative or normative commitments on users.

major comments (2)
  1. [Scaling individually stable strategies to collective equilibria] The section describing the scaling from individual strategies to collective equilibria asserts that the three extrapolation principles produce a prisoner's dilemma, yet supplies no derivation of the payoff matrix or equilibrium conditions. It is not shown how non-communicative aggregation combined with local social signaling and institutional norms setting yields mutual defection as the unique Nash equilibrium with a Pareto-inferior outcome.
  2. [Analysis identifying the emergence of sociotechnical lock-in] The central claim that individually adaptive delegation aggregates into a systemic collective action problem modeled as a prisoner's dilemma depends on the three extrapolation principles being sufficient; however, the manuscript does not examine whether alternative aggregation mechanisms or unmodeled heterogeneity would produce different macro-behavior, leaving the uniqueness of the PD outcome unverified.
minor comments (2)
  1. [Abstract] The abstract refers to 'canonical decision-theoretic strategies' without enumerating them (e.g., expected-utility maximization, satisficing, or regret-based rules) or indicating which formal properties are assumed.
  2. [Introduction] The term 'sociotechnical lock-in' is introduced as an invented macro-state; a brief comparison to related concepts in the literature on technological lock-in or path dependence would clarify its novelty.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments, which highlight important areas for strengthening the rigor of our scaling arguments from individual strategies to collective equilibria. We respond to each major comment below, agreeing where the manuscript requires additional formalization and outlining targeted revisions.

read point-by-point responses

  1. Referee: [Scaling individually stable strategies to collective equilibria] The section describing the scaling from individual strategies to collective equilibria asserts that the three extrapolation principles produce a prisoner's dilemma, yet supplies no derivation of the payoff matrix or equilibrium conditions. It is not shown how non-communicative aggregation combined with local social signaling and institutional norms setting yields mutual defection as the unique Nash equilibrium with a Pareto-inferior outcome.

    Authors: We agree that the current presentation relies on qualitative reasoning from the individual equilibria without an explicit payoff-matrix derivation. In the revised manuscript we will insert a dedicated subsection that formally derives the payoffs under each extrapolation principle. Non-communicative aggregation will be shown to impose uncompensated verification costs, local social signaling to create a coordination trap favoring defection, and institutional norms setting (absent safeguards) to normalize low-effort equilibria. The derivation will establish mutual defection as the unique Nash equilibrium whose outcome is Pareto-dominated by mutual cooperation. revision: yes

  2. Referee: [Analysis identifying the emergence of sociotechnical lock-in] The central claim that individually adaptive delegation aggregates into a systemic collective action problem modeled as a prisoner's dilemma depends on the three extrapolation principles being sufficient; however, the manuscript does not examine whether alternative aggregation mechanisms or unmodeled heterogeneity would produce different macro-behavior, leaving the uniqueness of the PD outcome unverified.

    Authors: The three principles were selected as the most representative pathways in existing sociotechnical systems. We acknowledge that the manuscript does not exhaustively test alternative mechanisms (e.g., centralized verification platforms or heterogeneous agent populations). In revision we will add a discussion subsection that considers how such alternatives might change macro-behavior and will argue that the prisoner's-dilemma outcome remains robust under the stated conditions. A full proof of uniqueness across every conceivable aggregation rule lies beyond the paper's scope and will be noted as a limitation and future-research direction. revision: partial

Circularity Check

0 steps flagged

No significant circularity detected; derivation remains self-contained.

full rationale

The paper constructs its argument by first defining individual decision-theoretic strategies for adaptive delegation and then applying three explicitly enumerated extrapolation principles to reach collective equilibria and the prisoner's dilemma characterization. This constitutes a standard modeling extension rather than any self-definitional loop, fitted parameter presented as prediction, or load-bearing self-citation. No equations or mappings are exhibited that reduce the collective payoff structure directly back to the individual inputs by construction, and the principles function as stated assumptions for scaling rather than hidden tautologies. The overall chain therefore does not collapse into its own inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 1 invented entities

Based solely on the abstract, the framework rests on standard assumptions from decision and game theory plus domain-specific premises about user adaptation and aggregation; no free parameters or invented entities with independent evidence are detailed.

axioms (2)
  • domain assumption Agents transition between strategies based on interaction feedback to reach stable equilibria.
    Explicitly stated as the first modeling step in the abstract.
  • domain assumption Three extrapolation principles suffice to scale individual strategies to collective equilibria.
    Listed as (a) non-communicative aggregation, (b) local social signaling, and (c) institutional norms setting.
invented entities (1)
  • sociotechnical lock-in no independent evidence
    purpose: Describes the macro state in which individual delegation aggregates into a prisoner's dilemma degrading shared epistemic standards.
    Introduced as an emergent outcome of the model in the absence of safeguards.

pith-pipeline@v0.9.0 · 5744 in / 1354 out tokens · 43119 ms · 2026-05-21T03:38:11.294889+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.