Pith. sign in

REVIEW 3 major objections 4 minor 34 references

Agency in the Age of AI

T0 review · 3 major / 4 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that the harms of generative AI are best understood as attacks on human agency, and that a quantitative, multi-agent theory of agency is needed to evaluate them.

desk verdict A clear, honest position paper that frames generative-AI harms as attacks on agency; the taxonomy is useful, but the promised quantitative payoff is a bet, not a result. read the letter →

arxiv 2502.00648 v1 pith:Z3D7PTAR submitted 2025-02-02 cs.AI cs.MA

Samarth Swarup This is my paper · ORCID
classification cs.AIcs.MA
keywords agencygenerativeAIagent-basedmodelingBDImodelPlanningTheoryofharmsmisinformationempowerment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This position paper argues that the many harms of generative AI—misinformation, manipulation, erosion of trust, automation bias, and more—are best understood as attacks on human agency, the ability to choose goals and act on them. The author proposes a six-part classification of attacks on agency and shows how each known harm fits. To make this lens usable, the paper calls for extending the standard planning-based model of agency (BDI) in four directions: making it multi-agent, cognitive, self-monitoring, and quantitative. It then proposes agent-based simulations as the testbed for measuring how attacks diminish or augment agency. The payoff, if the program is carried out, is a unifying framework for evaluating both harms and benefits of AI rather than treating each problem piecemeal.

What carries the argument

The load-bearing framework is the Belief-Desire-Intention (BDI) model, a computational operationalization of Bratman's Planning Theory of Agency in which an agent holds beliefs about the world, desires for preferred states, and intentions (plans) to achieve goals. The paper's argument works by considering an adversary who seeks to reduce the agent's agency and enumerating six attack types (A1–A6) that map onto the components of the BDI model: blocking actions, disrupting planning, influencing desires and goal selection, making desires seem unachievable, corrupting belief formation, and degrading the environment. To extend the model, the paper draws on the sociological Relational Theory of Agency, the enactive view of agency (with self-individuation), and information-theoretic measures—empowerment and Markov/causal blankets—intended to give a quantitative handle on how agency changes through interactions among agents, adversaries, and the environment.

What would settle it

A concrete test would be to implement the extended BDI model with an empowerment-based agency measure in a two-agent adversarial scenario and check whether the measure decreases when the adversary corrupts beliefs (A5) but not when it blocks actions (A1); failing to distinguish these attack types, or failing to show any decrease under a known successful misinformation attack, would falsify the claim that the quantitative measure captures attacks on agency.

Watch

Extended reading notes

Core claim

The paper's central claim is stated directly: agency is the most appropriate theoretical lens to view the problems of generative AI harms. Concretely, the author argues that nearly every category of harm—threats to democratic representation and accountability, the liar's dividend, integrity attacks and institutionalized misinformation, automation bias, and loss of control to AI—can be classified as one of six attacks on an agent's beliefs, desires, plans, goal selection, belief formation, or environment (A1–A6). The author further claims that this classification shows the limitations of current agency theory: agency is fundamentally multi-agent, cognition and self-monitoring must be part of the model, and the theory must become quantitative so changes in agency can be measured. The paper proposes that these extensions be operationalized in agent-based models that include baseline humans, humans augmented with AI tools, and autonomous AI agents, with information-theoretic measures such as empowerment used to quantify agency.

Load-bearing premise

The proposal depends on the assumption that agency can be made quantitative in a way that is computable for realistic, large-scale agent-based models, which the paper itself admits is currently hard to apply beyond simple cases.

Editorial extensions

If this is right

  • The six attack types provide a common vocabulary for classifying existing and future harms of generative AI, including both malicious and unintentional harms.
  • If agency becomes measurable, simulations can compare scenarios (e.g., elections, epidemics) in terms of how much agency is lost or gained, enabling evaluation of interventions before deployment.
  • The extended BDI model would require including self-monitoring in agents so they can detect attacks on their own agency, which current models lack.
  • The program implies that studying AI harms requires multi-agent, cognitive, and quantitative theories of agency, not just single-agent planning models.
  • Agent-based models that include LLM-based agents, augmented humans, and baseline humans would become a new class of generative models whose outputs are scenario evaluations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the quantitative agency program succeeds, the same machinery could measure not only loss but augmentation of agency, turning the framework into a design tool for 'better futures' where AI tools are evaluated by whether they increase an agent's effective options—an extension the paper only gestures at.
  • The paper's attack taxonomy (A1–A6) could be operationalized as a benchmark: each harm category becomes a test scenario with a known adversary, and the proposed measure should show agency decreasing monotonically with attack intensity, a directly testable reading the author does not spell out.
  • The framework may naturally extend to collective agency, such as climate action at global scale, as the paper notes in passing; one could test whether group-level agency measures emerge from agent-level interactions in an agent-based model, connecting this work to global-coordination problems.
  • A concrete extension: in an agent-based model with LLM-based agents, inject misinformation of varying volumes and measure the empowerment or free energy of the agents; the paper's picture predicts a decrease, and if none is observed, the quantitative core would be falsified.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This paper is a blue-sky position paper arguing that the diverse harms of generative AI—misuse by malicious actors, changes to the information landscape, corruption of the tools themselves, and unintended consequences—are best understood through the lens of agency. It introduces a taxonomy of six types of attacks on agency (A1–A6), ranging from blocking an agent's actions to manipulating its beliefs, goals, or environment. It then derives four observations (O1–O4) about what a more complete theory of agency would need: multi-agent character, cognitive mechanisms, self-monitoring, and quantitative assessment. The proposed research program combines an extended BDI model with relational and enactive theories of agency and information-theoretic measures, to be evaluated in agent-based simulations. The paper is explicitly a research agenda rather than a report of completed results.

Significance. If the proposed program succeeds, it would provide a common theoretical vocabulary for harms that are currently discussed piecemeal, and the A1–A6 taxonomy is already a useful organizing device for comparing different failure modes. The paper is notably candid about the gap between existing formal measures of agency and the scale of realistic agent-based models, and it identifies genuine limitations of the standard BDI framework, such as the absence of self-monitoring. However, as it stands, the central claim is programmatic: there is no formal derivation, no empirical validation, and no concrete algorithm or measure supplied. The credibility of the proposal rests on future advances, especially on a computable quantitative notion of agency (O4), which the paper itself concedes is currently limited to the simplest settings.

major comments (3)
  1. [§3, Observation O4] The proposed evaluation pipeline depends on O4 being realized as a measure that can be computed in realistic multi-agent simulations. The paper cites empowerment and Markov/causal blankets as promising tools but immediately concedes that 'it is hard to apply this formalism practically to any but the simplest of agents and environments.' Since A1–A6 attacks require at least an agent, an adversary, and an environment interacting at scale, this is not a peripheral technicality; it is the load-bearing point on which the promise of evaluating harms in agent-based models rests. The manuscript should either develop a concrete candidate measure, specify the formal properties such a measure must satisfy, or provide a minimal worked example showing how one of the A1–A6 attacks would be quantified. Without one of these, the claim in §4 that we can 'evaluate particular scenarios' does not follow.
  2. [§2, taxonomy A1–A6] The mapping of harms to attack categories is asserted through brief examples, but no criteria are given for when a harm belongs to one category rather than another, and the categories are not shown to be mutually exclusive or exhaustive. For instance, the 'loss of control to AI decision-makers' is classified as A3, but it could plausibly be read as A4 or A5 (belief corruption) or as A6 (structural elimination of good options). Because the central claim is that the agency lens unifies the harms, the taxonomy needs a more systematic justification, including a discussion of boundary cases and overlaps. This could be addressed by adding explicit decision rules for classification or by acknowledging and analyzing ambiguous cases.
  3. [§2, Observation O3] Observation O3 states that if an agent cannot detect attacks on its agency, it 'effectively doesn't have agency,' with the example of an agent that keeps re-planning despite every plan failing. This is a strong conceptual claim that conflates agency with the epistemic capacity to detect interference. An agent whose plans are consistently foiled may still be an agent; what is diminished is its efficacy or success, not necessarily its status as an agent. If O3 is intended only as a design requirement for defensive or self-protective systems, the text should say so explicitly, because as written it conflicts with the Planning Theory's own notion of autonomous agency and it underpins the later call for self-monitoring.
minor comments (4)
  1. [Throughout] There are several typographical and formatting issues, such as 'Charlottesville, V A' on the title page and inconsistent spacing in the references; the manuscript would benefit from a careful proofreading pass.
  2. [§2] The statement that the harms 'by no means comprehensive' is honest, but it raises the question of how the taxonomy would handle harms not listed; a sentence on criteria for adding new attack types would strengthen the framework.
  3. [§3.1] The discussion of scaling mentions de Mooij et al. and Chopra et al., but Chopra et al.'s 'On the limits of agency in agent-based models' appears highly relevant to the O4 concern and is cited only in passing; the paper should engage with its findings on the tractability of agency measures in ABMs.
  4. [§4] The bullet list of potential benefits of generative AI is somewhat disconnected from the preceding argument; adding one or two sentences linking each benefit to the notion of augmented agency would make the section cohere with the rest of the paper.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is a blue-sky argument mapping generative-AI harms onto an external agency framework, with no fitted derivation or load-bearing self-citation chain.

full rationale

The paper's central claim is that agency, understood through the Planning Theory and the BDI model, is an appropriate lens for studying harms from generative AI. The argument proceeds by proposing an adversarial thought experiment (attacks A1-A6) and then classifying the harms from Section 1 under those attack types. This is an interpretive mapping onto an externally established framework (Bratman; Rao et al.), not a derivation whose conclusion is equivalent to its input by construction. The observations O1-O4 are stated as requirements for an extended theory, not as results derived from fitted parameters. The quantitative ambition (O4) is supported by reference to external information-theoretic and Markov/causal blanket work by other authors, and the paper explicitly concedes that 'it is hard to apply this formalism practically to any but the simplest of agents and environments.' The author's own prior work is cited only as examples of agent-based modeling practice, simulation analytics, and network agency; none of these self-citations is load-bearing for the central claim, and no uniqueness theorem is imported from the author's previous publications to force a choice. No equation in the paper reduces to itself, no fitted parameter is renamed as a prediction, and no known result is merely relabeled as an organizing principle. The paper is a research agenda with an admittedly unfulfilled quantitative requirement, but that is a limitation and a correctness risk, not a circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper does not introduce new mathematical free parameters or invented entities. Its central claim rests on several domain assumptions about the adequacy of BDI-style agency, the completeness of the attack taxonomy, and the practical feasibility of quantitative agency measures in large simulations; these are explicitly framed as open challenges.

assumptions (4)
  • domain assumption The Planning Theory of Agency and the BDI model are a reasonable starting point for analyzing AI harms.
    Section 2 adopts Bratman's planning theory and BDI as the baseline framework without defending it against alternative theories of agency, though multiple definitions are cited.
  • domain assumption The harms of generative AI can be exhaustively categorized as attacks on agency (A1-A6).
    Section 2 introduces the thought experiment and maps prior harm categories to A1-A6, but this mapping is an interpretive claim, not proven.
  • domain assumption A quantitative measure of agency (e.g., via information theory, empowerment, or Markov blankets) can be defined and computed for agents in complex social simulations.
    Section 3 proposes using information-theoretic ideas for measurement, but acknowledges practical limitations.
  • domain assumption Agent-based simulations can represent baseline humans, AI-augmented humans, and autonomous AI agents interacting in realistic scenarios.
    Section 3.1 assumes feasibility of such simulations and lists scaling and uncertainty as challenges.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Agency in the Age of AI." pith.science (2026). https://pith.science/paper/Z3D7PTAR

@misc{pith2026250200648,
  author       = {Pith},
  title        = {Pith review of: Agency in the Age of AI},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z3D7PTAR}},
  note         = {Machine review of arXiv:2502.00648}
}
read the original abstract

There is significant concern about the impact of generative AI on society. Modern AI tools are capable of generating ever more realistic text, images, and videos, and functional code, from minimal prompts. Accompanying this rise in ability and usability, there is increasing alarm about the misuses to which these tools can be put, and the intentional and unintentional harms to individuals and society that may result. In this paper, we argue that \emph{agency} is the appropriate lens to study these harms and benefits, but that doing so will require advancement in the theory of agency, and advancement in how this theory is applied in (agent-based) models.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 25 canonical work pages

  1. [1]

    automation bias

    Saar Alon-Barkat and Madalina Busuioc. Human–ai interactions in public sector decision making: “automation bias” and “selective adherence” to algorithmic advice. Journal of Public Administration Research and Theory, 33 0 (1): 0 153--169, February 2022. ISSN 1477-9803. doi:10.1093/jopart/muac007

    doi:10.1093/jopart/muac007 2022

  2. [2]

    Societal adaptation to advanced ai

    Jamie Bernardi, Gabriel Mukobi, Hilary Greaves, Lennart Heim, and Markus Anderljung. Societal adaptation to advanced ai. May 2024. doi:10.48550/ARXIV.2405.10295

    arXiv doi:10.48550/arxiv.2405.10295 2024

  3. [3]

    Artificial intelligence crime: An overview of malicious use and abuse of ai

    Tais Fernanda Blauth, Oskar Josef Gstrein, and Andrej Zwitter. Artificial intelligence crime: An overview of malicious use and abuse of ai. IEEE Access, 10: 0 77110--77122, 2022. ISSN 2169-3536. doi:10.1109/access.2022.3191790

    work page arXiv 2022

  4. [4]

    Intention, Plans, and Practical Reason

    Michael Bratman. Intention, Plans, and Practical Reason. Cambridge, MA: Harvard University Press, Cambridge, 1987

    work page 1987

  5. [5]

    Structures of agency: Essays

    Michael E Bratman. Structures of agency: Essays. Oxford University Press, 2007

    work page 2007

  6. [6]

    Michael E. Bratman. Shared Agency: A Planning Theory of Acting Together. Oxford University Press, Oxford, 2013

    work page 2013

  7. [7]

    Relational agency: Relational sociology, agency and interaction

    Ian Burkitt. Relational agency: Relational sociology, agency and interaction. European Journal of Social Theory, 19 0 (3): 0 322--339, 2016. doi:10.1177/1368431015591426

    work page doi:10.1177/1368431015591426 2016

  8. [8]

    On the limits of agency in agent-based models

    Ayush Chopra, Shashank Kumar, Nurullah Giray-Kuru, Ramesh Raskar, and Arnau Quera-Bofarull. On the limits of agency in agent-based models. September 2024. doi:10.48550/ARXIV.2409.10568

    arXiv doi:10.48550/arxiv.2409.10568 2024

Show all 34 references
  1. [9]

    On the role of social interaction in individual agency

    Hanne De Jaegher and Tom Froese. On the role of social interaction in individual agency. Adaptive Behavior, 17 0 (5): 0 444--460, 2009

    2009

  2. [10]

    A framework for modeling human behavior in large-scale agent-based epidemic simulations

    Jan de Mooij, Parantapa Bhattacharya, Davide Dell’Anna, Mehdi Dastani, Brian Logan, and Samarth Swarup. A framework for modeling human behavior in large-scale agent-based epidemic simulations. Simulation, 99 0 (12): 0 1183--1211, August 2023. ISSN 1741-3133. doi:10.1177/003754...

    2023 doi

  3. [11]

    Sensorimotor Life

    Ezequiel Di Paolo , Thomas Buhrmann, and Xabier Barandiaran. Sensorimotor Life. Oxford University Press, 2017. doi:10.1093/acprof:oso/9780198786849.001.0001

    2017

  4. [12]

    What is agency? American Journal of Sociology, 103 0 (4): 0 962--1023, January 1998

    Mustafa Emirbayer and Ann Mische. What is agency? American Journal of Sociology, 103 0 (4): 0 962--1023, January 1998

    1998

  5. [13]

    Algorithmic bias: Senses, sources, solutions

    Sina Fazelpour and David Danks. Algorithmic bias: Senses, sources, solutions. Philosophy Compass, 16 0 (8), June 2021. ISSN 1747-9991. doi:10.1111/phc3.12760

    2021 doi

  6. [14]

    Magentic-one: A generalist multi-agent system for solving complex tasks

    Adam Fourney, Gagan Bansal, Hussein Mozannar, Cheng Tan, Eduardo Salinas, Erkang, Zhu , Friederike Niedtner, Grace Proebsting, Griffin Bassman, Jack Gerrits, Jacob Alber, Peter Chang, Ricky Loynd, Robert West, Victor Dibia, Ahmed Awadallah, Ece Kamar, Rafah Hosn, and Saleema A...

    2024 doi arXiv

  7. [15]

    Is it an agent, or just a program?: A taxonomy for autonomous agents

    Stan Franklin and Art Graesser. Is it an agent, or just a program?: A taxonomy for autonomous agents. In Intelligent Agents III, pages 21--35. Springer-Verlag, 1997

    1997

  8. [16]

    Large language models empowered agent-based modeling and simulation: a survey and perspectives

    Chen Gao, Xiaochong Lan, Nian Li, Yuan Yuan, Jingtao Ding, Zhilun Zhou, Fengli Xu, and Yong Li. Large language models empowered agent-based modeling and simulation: a survey and perspectives. Humanities and Social Sciences Communications, 11 0 (1), September 2024. ISSN 2662-99...

    2024 doi

  9. [17]

    Maryanne Garry, Way Ming Chan, Jeffrey Foster, and Linda A. Henkel. Large language models (llms) and the institutionalization of misinformation. Trends in Cognitive Sciences, 28 0 (12): 0 1078--1088, December 2024. ISSN 1364-6613. doi:10.1016/j.tics.2024.08.007

    2024 doi

  10. [18]

    Social conceptions of knowledge and action: DAI foundations and open systems semantics

    Les Gasser. Social conceptions of knowledge and action: DAI foundations and open systems semantics. Artificial Intelligence, 47: 0 107--138, 1991

    1991

  11. [19]

    Values define agency: Ecological and enactive perspectives reconsidered

    Bert H Hodges. Values define agency: Ecological and enactive perspectives reconsidered. Adaptive Behavior, 31 0 (6): 0 559--576, March 2022. ISSN 1741-2633. doi:10.1177/10597123221076876

    2022 doi

  12. [20]

    Empowerment for continuous agent-environment systems

    Tobias Jung, Daniel Polani, and Peter Stone. Empowerment for continuous agent-environment systems. Adaptive Behavior, 19 0 (1): 0 16--39, January 2011

    2011

  13. [21]

    Should you still learn to code in an A.I

    Sarah Kessler. Should you still learn to code in an A.I. world? The New York Times, 2024. URL https://www.nytimes.com/2024/11/24/business/computer-coding-boot-camps.html. Nov 24

    2024

  14. [22]

    Comparing varieties of agency theory in economics, political science, and sociology: An illustration from state policy implementation

    Edgar Kiser. Comparing varieties of agency theory in economics, political science, and sociology: An illustration from state policy implementation. Sociological Theory, 17 0 (2): 0 146--170, 1999

    1999

  15. [23]

    How ai threatens democracy

    Sarah Kreps and Doug Kriner. How ai threatens democracy. Journal of Democracy, 34 0 (4): 0 122--131, October 2023. ISSN 1086-3214. doi:10.1353/jod.2023.a907693

    2023 doi

  16. [24]

    Decomposing causality into its synergistic, unique, and redundant components

    Álvaro Martínez-Sánchez, Gonzalo Arranz, and Adrián Lozano-Durán. Decomposing causality into its synergistic, unique, and redundant components. Nature Communications, 15 0 (1), November 2024. ISSN 2041-1723. doi:10.1038/s41467-024-53373-4

    2024 doi

  17. [25]

    Mortveit, Christopher L

    Zakaria Mehrab, Logan Stundal, Samarth Swarup, Srinivasan Venaktramanan, Bryan Lewis, Henning S. Mortveit, Christopher L. Barrett, Abhishek Pandey, Chad R. Wells, Alison P. Galvani, Burton H. Singer, David A. Leblang, Rita R. Colwell, and Madhav Marathe. Network agency: A n ag...

    2024

  18. [26]

    Raghunathan

    Trivellore E. Raghunathan. Synthetic data. Annual Review of Statistics and Its Application, 8 0 (1): 0 129--140, March 2021. ISSN 2326-831X. doi:10.1146/annurev-statistics-040720-031848

    2021 doi

  19. [27]

    Maxwell J. D. Ramstead, Michael D. Kirchhoff, Axel Constant, and Karl J. Friston. Multiscale integration: B eyond internalism and externalism. Synthese, 198 0 (1): 0 41--70, Jan 2021. ISSN 1573-0964. doi:10.1007/s11229-019-02115-x. URL https://doi.org/10.1007/s11229-019-02115-x

    2021 doi

  20. [28]

    BDI agents: from theory to practice

    Anand S Rao, Michael P Georgeff, et al. BDI agents: from theory to practice. In Proceedings of the First International Conference on Multi-Agent Systems (ICMAS), volume 95, pages 312--319, 1995

    1995

  21. [29]

    Rosas, Pedro A

    Fernando E. Rosas, Pedro A. M. Mediano, Martin Biehl, Shamil Chandaria, and Daniel Polani. Causal blankets: Theory and algorithmic framework. arXiv:2008.12568v2 [nlin.AO], 2020

    2008 arXiv

  22. [30]

    Schiff, and Nat\' a lia S

    Kaylyn Jackson Schiff, Daniel S. Schiff, and Nat\' a lia S. Bueno. The liar's dividend: C an politicians claim misinformation to evade accountability? American Political Science Review, pages 1--20, February 2024. ISSN 1537-5943. doi:10.1017/s0003055423001454

    2024 doi

  23. [31]

    Susan P. Shapiro. Agency theory. Annu. Rev. Sociol., 31: 0 263--284, 2005

    2005

  24. [32]

    Marathe, and Christopher L

    Samarth Swarup, Achla Marathe, Madhav V. Marathe, and Christopher L. Barrett. Simulation analytics for social and behavioral modeling. In Paul K. Davis, Angela O'Mahony, and Jonathan Pfautz, editors, Social-Behavioral Modeling for Complex Systems, pages 617--632. Wiley, 2019

    2019

  25. [33]

    Network agency

    Stefano Tasselli and Martin Kilduff. Network agency. Academy of Management Annals, 15 0 (1): 0 68--110, jan 2021. doi:10.5465/annals.2019.0037

    2021

  26. [34]

    Williams and Randall Beer

    Paul L. Williams and Randall Beer. Information dynamics of evolved agents. In S. Doncieux, B. Girard, A. Guillot, J. Hallam, J.-A. Meyer, and J-B. Mouret, editors, From Animals to Animats 11: Proceedings of the 11th International Conference on Simulation of Adaptive Behavior, ...

    2010

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.