Pith. sign in

REVIEW 4 major objections 4 minor 35 references

Model-Based Soft Maximization of Suitable Metrics of Long-Term Human Power

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Softly maximizing aggregate human power—how many goals people can reach—could give AI agents a safe, goal-agnostic objective.

desk verdict A serious, transparent proposal for a goal-agnostic power objective that deserves review, but the referee should push hard on the unresolved fixed-point non-uniqueness. read the letter →

arxiv 2508.00159 v2 pith:3ATTLBWS submitted 2025-07-31 cs.AI cs.CYcs.LGecon.THmath.OC

Jobst Heitzig , Ram Potham This is my paper
classification cs.AIcs.CYcs.LGecon.THmath.OC
keywords humanpowerAIalignmentempowermentboundedrationalityaggregatewelfaremulti-agentreinforcementlearningcorrigibilitypower-seeking
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes that an AI agent's objective should be, not a task reward, but a soft long-term aggregate of human power—specifically, of how many different goals each human could effectively reach. It constructs a metric, ICCEA power, from a world model alone, without trying to infer any particular human's current goals, and derives algorithms to compute or learn a policy that softly maximizes the aggregate over people and time. The central claim, stated cautiously in the paper, is that such a power-managing agent would act as a helpful instruction follower, ask for confirmation, respect social norms, allocate resources fairly, and by construction not disempower humans, making it safer than direct utility-based objectives. If true, this would move AI objective design from task-specific reward engineering to a structural, goal-agnostic metric of human capability.

What carries the argument

The load-bearing object is the ICCEA power metric and its aggregate. ICCEA stands for informationally and cognitively constrained effective autonomous power, and it counts how many possible goals a human could choose to reach with more or less certainty, given bounded rationality and expectations about others. The design is carried by three transformations: $f_G(v)=v^\zeta$ with $\zeta>1$ makes the robot prefer reliable over uncertain empowerment; $f_H(w)=-2^{-\xi w}$ with $\xi=1$ makes the aggregate strongly inequality-averse and protects a person's last bit of power; and $F_H(y)=-(-y)^\eta$ with $\eta>1$ adds intertemporal inequality aversion. The soft policy $\pi_r(a)\propto(-Q_r(s,a))^{-\beta_r}$ with finite $\beta_r$ keeps the robot from over-optimizing. These pieces are assembled into equations (1)–(9), which give a recursive way to compute human behavior priors, effective goal-attainment probabilities, and the robot's long-term value.

What would settle it

Compute the fixed points of equations (1)–(3) in the two-state example in the Supplement: for rationality $\beta$ between the two bifurcation points, three distinct solutions exist, so the equations alone do not determine the human behavior prior; if a continuation method selects one branch and real human choices match a different branch, the model's assessment is wrong. Alternatively, give the robot an overconfident human-rationality prior in the key-and-door environment and check whether it stops asking for confirmation and chooses actions that reduce a fallible human's goal-reaching probability; if it does, the safety claim fails.

Watch

Extended reading notes

Core claim

The paper's discovery is a concrete, decomposable objective: individual power is $W_h(s)=\log_2 \sum_{g_h} V_h^e(s,g_h)^\zeta$, the log of the effective number of goals human $h$ can attain, where $V_h^e$ is the goal-reaching probability under the robot's actual soft policy and a modeled boundedly rational human policy; aggregate power is $U_r(s)=-\left(\sum_h X_h(s)^{-\xi}\right)^\eta$, with $X_h=2^{W_h}$, chosen to be inequality- and risk-averse and to protect a human's last bit of power. The robot then maximizes $V_r(s)=\mathbb{E}\sum_{t}\gamma_r^t U_r(s_t)$ using a power-law soft policy $\pi_r(a)\propto(-Q_r(s,a))^{-\beta_r}$. In a small gridworld, an agent trained on this objective alone learned to fetch a key, unlock a door, and move out of the human's way for every possible human goal, without ever observing that goal. The paper's cautious conclusion is that softly maximizing this aggregate human-power objective is a promising, safer alternative to maximizing any extrinsic utility.

Load-bearing premise

The load-bearing premise is that the robot has a decent stochastic world model with accurate priors about humans' rationality, habits, social norms, and mutual expectations; if those priors are wrong, the agent optimizes a distorted power metric and its empowering actions rest on false beliefs about what humans can achieve.

Editorial extensions

If this is right

  • An agent using this objective will make conditional commitments so that humans can steer it, because its model of human behavior assumes humans cautiously respond to the actions the robot has committed to.
  • It will ask for confirmation before irreversible or error-prone actions, with the number of confirmation rounds growing with human and robot patience and shrinking with human rationality.
  • It will follow relevant social norms, split resources equally unless power translates very non-concavely, and offer a finite menu of options sized to human bounded rationality rather than the largest possible menu.
  • Because the objective is defined over human power at all future times, the agent is incentivized to preserve options, avoid irreversible change, and protect its own ability to keep empowering humans.
  • The same objective can be evaluated by backward induction in acyclic games and approximated by two-phase temporal-difference learning in larger environments, so it is algorithmically accessible, not just a formal construction.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One implication the paper leaves implicit is that the approach shifts the hard alignment problem from guessing human preferences to building accurate structural world models and human-behavior priors; if those priors are systematically wrong, the agent will optimize a distorted power metric and may still disempower humans through miscalibration.
  • A testable extension would compare power-maximizing and utility-maximizing agents in a suite of small environments with hidden, changing human goals, measuring how often humans lose access to at least one reachable goal; the power objective should show near-zero disempowerment in all runs.
  • The softness parameter $\beta_r$ functions as a caution dial: lowering it makes the agent more sensitive to rare destructive outcomes during learning, while raising it sharpens optimization; deployment could anneal $\beta_r$ downward in safety-critical states.
  • The metric could be combined with a regularizer that penalizes divergence between the modeled beliefs $\mu_{-h}$ and the actual policies $\pi_{-h}$, which would directly address the paper's own scenario where the robot manipulates humans' mutual expectations.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a model-based objective for agentic AI in which the agent softly maximizes a long-term, inequality- and risk-averse aggregate of a structural metric of human 'ICCEA power'. The metric is defined through Bellman-like equations involving possible human goals, boundedly rational human policies shaped by norms and expectations, and a human model that assumes the robot keeps its commitments. The authors derive the metric from desiderata (separability, Pigou-Dalton inequality aversion, risk aversion), give backward-induction and two-phase reinforcement-learning algorithms, and illustrate the resulting behavior with analytic examples and a small gridworld experiment. The paper's cautious central claim is that softly maximizing such a power aggregate might be a beneficial objective for agentic AI systems, and safer than direct utility-based objectives.

Significance. If the framework were fully well-posed, this would be a significant conceptual contribution: it offers a goal-agnostic, structural alternative to reward inference, explicitly incorporates bounded rationality and social norms, and derives the objective from transparent ethical and formal desiderata. The paper is unusually candid about its limitations, and it ships reproducible code for the gridworld experiment. These are real strengths. However, the formal status of the central objective is not fully established, and the safety claims are conditional on a particular fixed-point selection and on accurate human-behavior priors. The paper is best read as a proposal with a strong axiomatic skeleton rather than as a completed safety guarantee.

major comments (4)
  1. [Section 2 ('Existence and (non-)uniqueness') and Supplement B] Equations (1)–(9) do not define a unique U_r or π_r in cyclic environments. The main text admits multiple fixed points and conjectures that a β-homotopy selects a unique 'principal' solution, but Supplement B's two-state example shows that continuation in β from β=0 traces the largest branch only until β₂ ≈ 0.78, where the branch folds back; the authors themselves write that this 'casts some doubts whether a continuation approach using β is successful in all cases.' Because the Section 5 safety claim ('not disempower humans (by definition)') is stated for the objective, not for one selected fixed point, the paper needs either a proven selection rule (e.g., via γ-homotopy or a regularization limit) or an explicit restriction of the claims to environments where uniqueness can be guaranteed. This is a load-bearing mathematical gap, not a presentation issue.
  2. [Section 3.1 ('Anticipated convergence')] The two-phase reinforcement-learning algorithm is not proven to converge. The text states that Phase 2 convergence 'might still fail' because the update operator is not a contraction, and no convergence theorem is supplied even for the tabular case. Since the paper presents these algorithms as a way to 'compute that metric' in complex environments, the missing guarantee leaves the computational route heuristic. Please provide a formal convergence result for the tabular setting under stated conditions, or clearly label the algorithm as an approximate heuristic without guarantees.
  3. [Section 4.2 and Section 5] The gridworld experiment is a proof-of-concept but does not support the comparative safety claim. The robot is trained to maximize V_r, so observing that it learns a policy that increases X_h is an instance of the objective being optimized, not independent evidence that this objective is safer than direct utility-based objectives; no baseline or ablation against a utility-maximizing agent is reported. The phrase 'not disempower humans (by definition)' should be understood as conditional on the selected fixed point and on correct model priors, and the 'safer than direct utility-based objectives' assessment remains a conjecture. The conclusion should state these conditions explicitly.
  4. [Section 2 'Framework' and Section 5] The objective is defined relative to the robot's priors on human behavior (ν_h, π0_h, β_h, μ_−h) and on the goal set G_h. The paper correctly flags 'wishful thinking' as a fault scenario, but the issue is more central than a caveat: a robot with a misspecified μ_−h or β_h can compute a high V_e_h and therefore a high U_r while its actions in fact reduce humans' real attainable goals. For the safety claim to be meaningful, the paper should formalize a robustness condition—for example, conservatively maximizing over an uncertainty set of priors, or proving that U_r is monotone in the accuracy of the priors—rather than treating accurate priors as an unmodeled assumption.
minor comments (4)
  1. [Equation (5)] The power-law softmax π_r(s)(a) ∝ (−Q_r(s,a_r))^{−β_r} is undefined when Q_r(s,a_r)=0; the later discussion of adding ε to Q_r and X_h should be moved into the definition in Table 1 or stated as a standing assumption.
  2. [Table 2, row 'Range of W_h is symmetric around 0'] The desideratum says 'Make each trajectory fulfill exactly one g_h', whereas Section 2.1 says 'each possible state trajectory fulfills at least one possible goal'; clarify whether G_h is required to be a partition of terminal states or merely a cover, since this affects X_h and the symmetry claim.
  3. [Section 3.1, Phase 1] The phrase 'use expected SARSA targets on a time-scale faster than π_h' is underspecified; state the learning-rate conditions or cite a specific two-timescale convergence result.
  4. [Supplement E] Potential-based reward shaping is used to train the human model; the standard invariance result is for optimal policies in single-agent MDPs, but the learned quantity here is a boundedly rational policy π_h under a fixed μ_−h, so please state why PBRS does not bias the derived prior in the bounded-rationality case.

Circularity Check

1 steps flagged · score 4.0 of 10

Only one advertised safety property is definitional; the rest of the axiomatic derivation and gridworld demonstration are transparent and not fitted.

  1. self definitional [Section 5, Conclusion and Outlook, final bullet list]
    "• not disempower humans (by definition)."

    Eq. (8) defines U_r(s) = -(Σ_h X_h(s)^{-ξ})^{-η}, and eq. (7) defines X_h(s) = Σ_{g_h} V_e_h(s,g_h)^ζ. Hence U_r is constructed to be a strictly monotone increasing transform of the human power metric X_h. An agent that maximizes U_r is therefore, by definition, an agent whose objective is to avoid reducing X_h. The safety property 'not disempower humans' is thus not a derived or independently validated consequence; it is a restatement of the chosen objective. The paper openly says 'by definition', so this is transparent, but it is still a claim that reduces to its own input construction rather than an emergent result.

full rationale

Most of the paper is an explicit mechanism-design derivation: behavioral consequences such as commitment-making, norm-following, confirmation-asking, and resource-sharing are computed from the chosen equations (Table 1) and desiderata (Table 2). Parameters ζ, ξ, η, β_r are selected by axioms, not fitted to external data, so there is no fitted-input-called-prediction problem. The gridworld experiment is a consistency check that an agent trained to maximize U_r unlocks the door, which is expected because opening new reachable goals directly increases X_h; it is not an external benchmark but is not circular in a hidden way. The non-uniqueness problem for cyclic environments, acknowledged by the authors and demonstrated in Supplement B, is a correctness gap rather than a circularity: the 'principal' fixed point is not proven to exist uniquely, so the safety claims are not currently well-defined for cyclic worlds. Self-citations to Potham and Harms 2025 and Lowe et al. 2025 are not load-bearing. The only genuine circular element is the Conclusion bullet 'not disempower humans (by definition)', which restates the objective's definition. Score 4 reflects that this definitional safety claim is part of the advertised strengths, while the central derivation retains substantial independent content.

Assumptions & free parameters 7 free parameters · 7 assumptions · 0 invented entities

The central object is a new metric, not a new physical entity. The free parameters are mostly normative design choices, not fitted constants; the axioms are explicit modeling and ethical assumptions, plus the crucial world-model assumption that the paper itself flags as fragile.

free parameters (7)
  • zeta (reliability exponent) = 2.0
    Risk-aversion power in eq. (7); chosen >1 and set to 2 partly to make W_h symmetric.
  • xi (inequality aversion exponent) = 1.0
    Power in eq. (8); set to 1 to protect the last bit of any human's power.
  • eta (intertemporal inequality exponent) = 1.1
    Power in eq. (8); set just above 1 to limit power trading across time.
  • beta_r (robot soft-optimization exponent) = 5.0 final, annealed from 1.0
    Soft-max power in eq. (5); finite value chosen to allow exploration and avoid over-optimization.
  • gamma_h and gamma_r (discount factors) = 0.99 for both
    Discount factors in the gridworld; chosen to promote farsighted behavior.
  • Human behavior priors (beta_h, nu_h, pi0_h, mu_-h) = Not estimated; assumed known from the world model
    These parameters define the boundedly rational human policy in eqs. (1)-(2) and are central to the metric.
  • Goal set G_h = All open cells in gridworld; partition-like sets in theory
    The choice of which events count as possible goals determines what 'power' means.
assumptions (7)
  • domain assumption The robot operates from a decent stochastic world model with known transition kernel P(s'|s,a) and observation functions.
    Invoked in Section 2 'Framework'; all downstream computation is based on this model.
  • domain assumption Separability axioms for goal aggregation and for aggregation across humans (Fleming 1952).
    These normative axioms justify the separable forms of W_h and U_r in Section 2.
  • domain assumption Uniform prior over possible human goals g_h.
    Section 2.2 'Aggregation across uncertainty' takes expectations over independent uniform draws.
  • domain assumption Human decisions follow a softmax mixture of habitual and boundedly rational behavior (eq. (2)).
    This is the paper's model of bounded rationality and social norms.
  • ad hoc to paper Goal events are subsets of states with mutual unreachability, and G_h covers all trajectories so that X_h > 0.
    Section 2.1; otherwise X_h^{-xi} can be infinite in eq. (8).
  • domain assumption Pigou-Dalton inequality aversion and protection of the last bit of power are adopted as normative constraints.
    These social-choice principles determine the concave form of f_H and the choice xi = 1.
  • domain assumption Common rescaling of all goal-attainment probabilities should not change the policy.
    This invariance requirement fixes the power-law forms of f_G, F_H, and pi_r in Section 2.2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Model-Based Soft Maximization of Suitable Metrics of Long-Term Human Power." pith.science (2026). https://pith.science/paper/3ATTLBWS

@misc{pith2026250800159,
  author       = {Pith},
  title        = {Pith review of: Model-Based Soft Maximization of Suitable Metrics of Long-Term Human Power},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3ATTLBWS}},
  note         = {Machine review of arXiv:2508.00159}
}
read the original abstract

Power is a key concept in AI safety: power-seeking as an instrumental goal, sudden or gradual disempowerment of humans, power balance in human-AI interaction and international AI governance. At the same time, power as the ability to pursue diverse goals is essential for wellbeing. This paper explores the idea of promoting both safety and wellbeing by forcing AI agents explicitly to empower humans and to manage the power balance between humans and AI agents in a desirable way. Using a principled, partially axiomatic approach, we design a parametrizable and decomposable objective function that represents an inequality- and risk-averse long-term aggregate of human power. It takes into account humans' bounded rationality and social norms, and, crucially, considers a wide variety of possible human goals. We derive algorithms for computing that metric by backward induction or approximating it via a form of multi-agent reinforcement learning from a given world model. We exemplify the consequences of (softly) maximizing this metric in a variety of paradigmatic situations and describe what instrumental sub-goals it will likely imply. Our cautious assessment is that softly maximizing suitable aggregate metrics of human power might constitute a beneficial objective for agentic AI systems that is safer than direct utility-based objectives.

Figures

Figures reproduced from arXiv: 2508.00159 by the authors.

Figure 1
Figure 1. Overview of proposed approach for deriving [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The gridworld environment. The robot learns to [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

35 extracted references · 29 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Amiel, Y.; Creedy, J.; and Hurn, S. 1999. Measuring attitudes towards inequality. Scandinavian Journal of Economics, 101(1): 83--96

    work page 1999

  4. [4]

    Baker, C.; Saxe, R.; and Tenenbaum, J. 2011. Bayesian theory of mind: Modeling joint belief-desire attribution. In Proceedings of the annual meeting of the cognitive science society, volume 33

    work page 2011

  5. [5]

    V.; and Duflo, E

    Banerjee, A. V.; and Duflo, E. 2011. Poor economics: A radical rethinking of the way to fight global poverty. Public Affairs

    work page 2011

  6. [6]

    Baum, W. M. 1974. On two types of deviation from the matching law: bias and undermatching 1. Journal of the experimental analysis of behavior, 22(1): 231--242

    work page 1974

  7. [7]

    Bengio, Y.; Cohen, M.; Fornasiere, D.; Ghosn, J.; Greiner, P.; MacDermott, M.; Mindermann, S.; Oberman, A.; Richardson, J.; Richardson, O.; et al. 2025. Superintelligent agents pose catastrophic risks: Can scientist ai offer a safer path? arXiv preprint arXiv:2502.15657

    arXiv 2025

  8. [8]

    Cao, H.; Cohen, S.; and Szpruch, L. 2021. Identifiability in inverse reinforcement learning. Advances in Neural Information Processing Systems, 34: 12362--12373

    work page 2021

Show all 35 references
  1. [9]

    Du, Y.; Tiomkin, S.; Kiciman, E.; Polani, D.; Abbeel, P.; and Dragan, A. 2020. Ave: Assistance via empowerment. Advances in Neural Information Processing Systems, 33: 4560--4571

    2020

  2. [10]

    Fleming, M. 1952. A cardinal concept of welfare. The Quarterly Journal of Economics, 66(3): 366--384

    1952

  3. [11]

    Gao, L.; Schulman, J.; and Hilton, J. 2023. Scaling laws for reward model overoptimization. In International Conference on Machine Learning, 10835--10866. PMLR

    2023

  4. [12]

    R.; Zurek, M.; Brown, D

    Ghosal, G. R.; Zurek, M.; Brown, D. S.; and Dragan, A. D. 2023. The effect of modeling human rationality level on learning rewards from multiple feedback types. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, 5983--5992

    2023

  5. [13]

    K.; Holt, C

    Goeree, J. K.; Holt, C. A.; and Palfrey, T. R. 2016. Quantal response equilibrium: A stochastic theory of games. In Quantal response equilibrium. Princeton University Press

    2016

  6. [14]

    Hill Jr, T. E. 2002. Human welfare and moral worth: Kantian perspectives. Clarendon Press

    2002

  7. [15]

    S.; Polani, D.; and Nehaniv, C

    Klyubin, A. S.; Polani, D.; and Nehaniv, C. L. 2005. Empowerment: A universal agent-centric measure of control. In 2005 ieee congress on evolutionary computation, volume 1, 128--135. IEEE

    2005

  8. [16]

    Krakovna, V.; Orseau, L.; Martic, M.; and Legg, S. 2018. Measuring and avoiding side effects using relative reachability. arXiv preprint arXiv:1806.01186

    2018 arXiv

  9. [17]

    Z.; Vezhnevets, A

    Leibo, J. Z.; Vezhnevets, A. S.; Diaz, M.; Agapiou, J. P.; Cunningham, W. A.; Sunehag, P.; Haas, J.; Koster, R.; Du \'e \ n ez-Guzm \'a n, E. A.; Isaac, W. S.; et al. 2024. A theory of appropriateness with applications to generative artificial intelligence. arXiv preprint arXi...

    2024 arXiv

  10. [18]

    J.; and Heidari, H

    London, A. J.; and Heidari, H. 2024. Beneficent intelligence: a capability approach to modeling benefit, assistance, and associated moral failures through AI systems. Minds and Machines, 34(4): 41

    2024

  11. [19]

    A.; Barez, F.; Franklin, M.; Haupt, A.; Heitzig, J.; Holliday, W

    Lowe, R.; Edelman, J.; Zhi-Xuan, T.; Klingefjord, O.; Hain, E.; Wang, V.; Sarkar, A.; Bakker, M. A.; Barez, F.; Franklin, M.; Haupt, A.; Heitzig, J.; Holliday, W. H.; Jara-Ettinger, J.; Kasirzadeh, A.; Kearns, R. O.; Kirkpatrick, J. R.; Koh, A.; Lehman, J.; Levine, S.; Revel, ...

    2025

  12. [20]

    Myers, V.; Ellis, E.; Levine, S.; Eysenbach, B.; and Dragan, A. 2024. Learning to assist humans without inferring rewards. arXiv preprint arXiv:2411.02623

    2024 arXiv

  13. [21]

    Nussbaum, M. 2019. Aristotelian social democracy. In Liberalism and the Good, 203--252. Routledge

    2019

  14. [22]

    K.; and Suzumura, K

    Pattanaik, P. K.; and Suzumura, K. 1996. Individual rights and social evaluation: a conceptual framework. Oxford Economic Papers, 48(2): 194--212

    1996

  15. [23]

    Pigou, A. C. 1912. Wealth and welfare. Macmillan and Company, limited

    1912

  16. [24]

    Potham, R.; and Harms, M. 2025. Corrigibility as a Singular Target: A Vision for Inherently Reliable Foundation Models. arXiv preprint arXiv:2506.03056

    2025 arXiv

  17. [25]

    Quiggin, J. 1982. A theory of anticipated utility. Journal of economic behavior & organization, 3(4): 323--343

    1982

  18. [26]

    Rapoport, A.; and Felsenthal, D. S. 1990. Efficacy in small electorates under plurality and approval voting. Public Choice, 64(1): 57--71

    1990

  19. [27]

    Reddy, S.; Levine, S.; and Dragan, A. 2022. First contact: Unsupervised human-machine co-adaptation via mutual information maximization. Advances in Neural Information Processing Systems, 35: 31542--31556

    2022

  20. [28]

    Robeyns, I. 2006. The capability approach in practice. Journal of political philosophy, 14(3)

    2006

  21. [29]

    Safari, M.; Shalbaf, R.; Bagherzadeh, S.; and Shalbaf, A. 2024. Classification of mental workload using brain connectivity and machine learning on electroencephalogram data. Scientific Reports, 14(1): 9153

    2024

  22. [30]

    Salge, C.; and Polani, D. 2017. Empowerment as replacement for the three laws of robotics. Frontiers in Robotics and AI, 4: 260425

    2017

  23. [31]

    Sen, A. 2014. Development as freedom (1999). The globalization and development reader: Perspectives on development and global change, 525

    1999

  24. [32]

    P.; Du \'e \ n ez-Guzm \'a n, E

    Smith, C.; Trivedi, R.; Clifton, J.; Hammond, L.; Khan, A.; Vezhnevets, S.; Agapiou, J. P.; Du \'e \ n ez-Guzm \'a n, E. A.; Matyas, J.; Karmon, D.; et al. 2024. The Concordia Contest: Advancing the Cooperative Intelligence of Language Agents. In NeurIPS 2024 Competition Track

    2024

  25. [33]

    M.; Hadfield-Menell, D.; and Tadepalli, P

    Turner, A. M.; Hadfield-Menell, D.; and Tadepalli, P. 2020. Conservative agency via attainable utility preservation. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 385--391

    2020

  26. [34]

    M.; Smith, L.; Shah, R.; Critch, A.; and Tadepalli, P

    Turner, A. M.; Smith, L.; Shah, R.; Critch, A.; and Tadepalli, P. 2019. Optimal policies tend to seek power. arXiv preprint arXiv:1912.01683

    2019 arXiv

  27. [35]

    Zhuang, S.; and Hadfield-Menell, D. 2020. Consequences of misaligned AI. Advances in Neural Information Processing Systems, 33: 15763--15773

    2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.