Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

Observation Interference in Partially Observable Assistance Games

T0 review · 2 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Partially observable assistance games can make observation interference necessary at the level of individual actions, but never at the level of whole policies.

desk verdict The action/policy interference distinction is a genuine contribution, but the main theorems as stated are false on a simple infinite-state counterexample; a finiteness restriction fixes them. read the letter →

arxiv 2412.17797 v2 pith:7C3XCHEA submitted 2024-12-23 cs.AI cs.GTcs.LGcs.MA

Scott Emmons , Caspar Oesterheld , Vincent Conitzer , Stuart Russell This is my paper · ORCID
classification cs.AIcs.GTcs.LGcs.MA
keywords partiallyobservableassistancegamesobservationinterferencevalueofinformationhuman-AIalignmentdecentralizedPOMDPBoltzmannrationalityAIdeception
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper studies partially observable assistance games (POAGs), a two-player model in which a human and an AI assistant share the same reward function, but only the human knows the reward parameters and both sides see the world only partially. It asks whether a perfectly aligned assistant would ever benefit from interfering with the human's observations. The paper's central finding is split: in some POAGs every optimal policy pair requires the assistant to take an action that gives the human a strictly less informative signal, even though a state- and reward-equivalent non-interfering action exists (Proposition 4.4); yet in every POAG there is some optimal policy pair in which the assistant does not interfere when interference is judged at the level of whole policies rather than isolated actions (Theorem 4.7). This reconciles action-level interference with the classical result that the value of information is nonnegative, and extends that result to cooperative multi-agent settings. The paper also identifies two further incentives for interference, querying the preferences of a naive human and simplifying decisions for a Boltzmann-rational human, and supports the theory with a product-selection experiment.

What carries the argument

The paper's central object is the Blackwell informativeness order: an observation family $P$ is more informative than $\hat{P}$ when $\hat{P}$ can be produced from $P$ by a fixed stochastic function independent of the state. Definition 3.2 calls an action observation-interfering if a state- and reward-equivalent action exists whose induced human observation distribution is strictly more informative; Definition 4.6 applies the same comparison to whole policies, allowing the human to condition on the assistant's known strategy. The proof of Theorem 4.7 carries the argument by choosing, among optimal continuation policies, one that minimizes the Shannon entropy $H(P(S_{t+1} \mid O^H_{t+1}))$ of the human's posterior over states; Lemma B.4 shows a strictly more informative policy strictly lowers this entropy, so no entropy-minimizing optimal policy can be observation-interfering at the policy level.

What would settle it

Run an exhaustive search over optimal policy pairs in the formal finite POAG of Example 4.3 (Appendix B.3); the paper's Theorem 4.7 predicts at least one optimal pair is non-interfering at the policy level, so finding that every optimal pair interferes at the policy level under Definition 4.6 would disprove the theorem.

Watch

Extended reading notes

Core claim

At the level of individual actions, the assistant may have to suppress information in order to convey its own private information. In the paper's CUDA version-list example, the assistant knows which software versions are compatible while the human does not; the assistant's optimal policy suppresses the incompatible versions from the human's view, even though suppressing nothing has identical state-transition and reward effects. That suppression counts as observation interference under the paper's action-level definition. But when interference is defined relative to the human's inference given the assistant's whole policy, the same suppression becomes informative rather than destructive, and the paper proves that an optimal policy pair always exists in which the assistant does not interfere at the policy level. Thus the paper establishes that observation interference is sometimes necessary action-by-action, never necessary as a strategy, and that the classic nonnegative value of information carries over to common-payoff multi-agent settings.

Load-bearing premise

The proof of the no-policy-level-interference theorem assumes discrete state and observation spaces so that a Shannon entropy minimizer among optimal policies exists, while the theorem is stated without that restriction.

Editorial extensions

If this is right

  • Action-level observation interference can appear in optimal solutions even with a perfectly aligned assistant and a human who observes and acts naively.
  • There is always an optimal solution without policy-level interference, so the assistant's hidden actions function as a communication device rather than a necessary manipulation.
  • Adding an unbounded assistant-to-human communication channel removes the first incentive (Theorem 4.5), and a two-way channel restores non-interference under naive human behavior (Theorem 5.3).
  • If the human is Boltzmann-rational, suppressing information can improve decisions, and for every positive rationality parameter there exists a POAG where all optimal policy pairs interfere with observations.
  • Experimental results in a product-selection game show that more assistant private information strengthens the incentive to interfere, while more human rationality weakens it.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • These results suggest normative judgments about AI information-hiding should be made at the policy or communication level rather than at the level of individual actions; the same visible suppression can be benign or harmful depending on whether the human can infer the assistant's policy.
  • The entropy-minimization proof points to a concrete design rule: among optimal assistant policies, choose the one minimizing the expected posterior entropy of the human's state belief, which yields non-interference at the policy level.
  • If future assistants use such suppression, transparency about the assistant's policy could convert seemingly harmful interference into legitimate communication, for example by announcing that information is being hidden because it is irrelevant or incorrect.
  • The gap between finite and continuous POAGs means Theorem 4.7 may not hold for continuous-state settings, and checking that extension would refine the theory beyond what the paper proves.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper introduces partially observable assistance games (POAGs) and defines two notions of observation interference: action-level interference, which compares individual assistant actions by Blackwell informativeness while holding state-transition and reward effects fixed, and policy-level interference, which makes the same comparison at the level of entire policies. The main theoretical claims are: if the assistant has no private information, there exists an optimal policy pair without action-level interference (Theorem 4.2); if the assistant has private information, action-level interference can be unavoidable in every optimal pair, even with a naive human (Proposition 4.4); an unbounded A-to-H communication channel removes this necessity (Theorem 4.5); in every POAG there exists an optimal policy pair without policy-level interference (Theorem 4.7); with a naive human, interference can be needed to elicit the human's preferences (Proposition 5.2); and under Boltzmann-rational human behavior, interference can be optimal even without private information (Proposition 6.3). The paper closes with a Monte Carlo study of a product-selection game that quantifies the tradeoff between communicating the assistant's private information and destroying the human's private information.

Significance. If the results are read with the intended finite-state qualifications, the paper makes a clean and useful contribution: it separates action-level from policy-level observation tampering, identifies communication of private information as a driver of the former, and extends the nonnegative-value-of-information principle to cooperative multiagent settings. The appendix contains detailed proofs and fully formalized counterexample games, which is a clear strength. The experimental section is a reasonable illustration of the theory. The significance is moderate: the qualitative conclusions are likely to matter for assistance-game and AI-safety audiences, but the universal quantifier over 'any POAG' is not justified as stated, and the theorems need to be restricted to finite, or otherwise compact/discrete, POAGs before the central claims are sound.

major comments (2)
  1. [§2.1, §4.2, §4.3 (Theorems 4.2, 4.5, 4.7; Appendices B.2, B.5)] The universal theorems are stated for 'any POAG' under Definition 2.1, where the state, action, and observation sets are arbitrary, but the proofs require finiteness or compactness. Theorem 4.7's proof in Appendix B.5 minimizes Shannon entropy of the human's posterior, uses a uniform-random human policy over AH, and invokes the extreme value theorem on a closed and bounded policy set; none of these is guaranteed by Definition 2.1. This is not merely a missing hypothesis: consider the one-shot POAG with S=[0,1] (uniform initial state), AH=[0,1], AA=ℕ, A having no private information, and reward R(s,aH,aA)=1 if |aH−s|<1/8 else 0, where under action n the human observes (n, bin_n(s)) with bin_n the index of the length-2^{-n} dyadic interval containing s. For every n≥3 the human can play the bin midpoint and obtain reward 1, so all such n are optimal; but action n+1 is strictly more informative than n and has identical state-transition and reward effects, so every optimal action is action-level interfering under Definition 3.2. Similarly, shifting any distribution over AA up by one yields a more informative policy with the same effects, so every policy is policy-level interfering under Definition 4.6. Thus Theorems 4.2 and 4.7 are false as stated, and Theorems 4.5 and 5.3 inherit the same quantifier problem. The fix is to restrict Definition 2.1 and all quantified theorems to finite, or otherwise compact/discrete, POAGs, which covers the paper's examples and the standard DecPOMDP setting.
  2. [Appendix B.5 (Theorem 4.7)] Even within finite POAGs, the proof of Theorem 4.7 should justify that the one-step entropy-minimizing choices assemble into an optimal pair for the infinite-horizon objective. The argument shows that every finite prefix is compatible with some optimal policy pair, but without a compactness or limiting argument the resulting limit policy pair need not be optimal. This is the same structural issue as the finiteness problem above and should be addressed when the theorem is restated with the appropriate hypotheses.
minor comments (5)
  1. [Definition 4.6] The displayed definition repeats the human policy in the conditioning event, writing PoH_{t+1}(· | πH, st+1, ˆπA_{0:t}, πH); it should presumably condition on πH only once.
  2. [Appendix B.3] In the formalization of Example 4.3, the human observation kernel is written as OH(oH | s, aA, aH) = 1[oA_i = s_{i+1} aA_i], which appears to contain typos; it should presumably be 1[oH_i = s_{i+10} aA_i].
  3. [Example 6.2 formalization] The reward definition in Appendix D has a stray line '1[aH = 1]1[s ∈ {sa, sb}] + 1[aH = 2]1[s ∈ {sc, sd}]' after the four numbered equations; either remove it or explain its intended role.
  4. [Figure 1] The Monte Carlo results are reported without error bars or confidence intervals, so small differences between curves, especially at low β, are hard to distinguish from sampling noise; reporting standard errors or multiple seeds would strengthen the experimental claim.
  5. [Definition 6.1] The Boltzmann-rational response definition has πH appearing inside the expectation on both sides of the defining equation; please clarify whether this is intended as a fixed-point condition and, if so, how existence and uniqueness are guaranteed in the examples.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's theorems are derived from POAG axioms, Blackwell informativeness, and optimality definitions, with no fitted parameters or self-citation chains carrying the central claims.

full rationale

The paper's central claims, Proposition 4.4 and Theorem 4.7, are derived from the POAG definition, the Blackwell-informativeness ordering, and standard optimality/Best-response definitions. No parameter is fitted to data and then renamed as a prediction. The experiments in Section 7 rest on the analytic Proposition 7.2, which is proved from the model's Boltzmann selection rule rather than estimated from the simulation; the Monte Carlo results illustrate the analytic tradeoff rather than constitute an independent confirmation that could be circular. The statement in Section 7.3 that the experiment 'confirms our hypothesis based on Theorem 4.2' is a qualitative interpretation of a simulation of a different (Boltzmann) human model, not a reduction of Theorem 4.2 to its own assumptions. The only self-citations with overlapping authorship, e.g., Lang et al. (2024) and Shah et al. (2020), are used as context or as a prior RLHF example; they are not load-bearing for the paper's new theorems. The proof of Theorem 4.7 uses entropy minimization and the extreme value theorem, which are substantive proof tools rather than definitions of the target conclusion. A possible technical gap exists for non-finite POAGs (the proof assumes discrete entropy and compact policy sets), but this is a correctness or generality concern, not circularity; the same applies to the skeptical construction with continuous state and action spaces. Overall, the derivation chain is self-contained against the stated model, and no circular step can be exhibited by quoting an equation or construction that reduces to its own input.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The central theorems are proved from the POAG/DecPOMDP formalism, Blackwell informativeness, and standard dynamic programming; they do not fit parameters to data. The analysis carries three structural premises: a finiteness or discreteness premise needed by the entropy argument in Theorem 4.7 (unstated in Definition 2.1), the paper's own naive-human plausibility restriction behind the second incentive, and the Boltzmann model behind the third. Example-specific utilities are hand-chosen to make existence proofs concrete.

free parameters (4)
  • Boltzmann rationality coefficient beta = swept over {0.01, 0.03, 0.1, 0.3, 1, 3, 10, 30, 100} in experiments; set to 1 WLOG in Proposition 6.3
    Beta parameterizes the human error model in Definition 6.1 and the product selection game; it is a modeling input, not fitted to data, but the strength and even sign of the interference incentive depends on it (Appendix E, Figure 1a).
  • Example 6.2 utility values (0, 1, 7 and tldr expected value 4) = utilities 1 and 7 at beta=1; interference preferred (3.928 vs 3.862)
    Hand-picked so that at beta=1 the interfering tldr page yields higher expected utility than the more informative man page; the difference is small and reverses for small beta, so the example is an existence construction, not a robust quantitative prediction.
  • Product game size and priors = d=5, H_i and R_i ~ Unif(0,1), k in {0,...,4}, 30,000 trials, beta=infinity for Figure 1b
    Experimental design choices that set the scale of the simulated tradeoff; no search or fitting over these values is reported, but the qualitative ordering in Figure 1b depends on them.
  • Example 5.1 preference utilities = 1 for weak preference, 10 for strong preference, 0 otherwise
    Chosen so that signaling the strong (GPU vs CPU) preference is worth distorting the human's choice; the example requires a separation between strong and weak preference magnitudes.
assumptions (6)
  • domain assumption Optimal policy pairs maximize expected discounted shared reward with perfect coordination; both players know each other's policy
    Adopted in Definition 2.1 and used throughout; it is the assistance-game solution concept from Shah et al. (2020).
  • standard math Blackwell's informativeness order: a signal is more informative if the other can be stochastically generated from it for all states (Definition 3.1)
    Underpins Definitions 3.2 and 4.6; taken from Blackwell (1951, 1953).
  • domain assumption Finite or discrete state, observation, and action spaces and compact policy sets for the entropy-minimization proof of Theorem 4.7
    Unstated in Definition 2.1. Shannon entropy H(P(S|O)), the closedness and boundedness of the policy set, and the extreme value theorem in Appendix B.5 only work for finite or discrete and compact spaces, so the theorem's stated scope 'any POAG' is not established in the continuous case.
  • standard math Deterministic optimal policies exist in common-payoff games with perfect recall and no absentmindedness
    Invoked in the proof sketch of Theorem 4.2 (Appendix B.2), following Kuhn (1953).
  • ad hoc to paper Naive human behavior (Definitions 3.6 and 3.7): the human takes observations at face value and myopically maximizes immediate reward when choices do not affect state transitions
    The paper's own plausibility criterion; Proposition 5.2's necessity of interference holds only under this restriction, which the paper concedes could otherwise be handled by an unnatural communication convention.
  • domain assumption Boltzmann rationality as a model of human decision making (Definition 6.1)
    Used for the third incentive; the existence claim in Proposition 6.3 is relative to this model, and the constructed POAG scales rewards with 1/beta.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Observation Interference in Partially Observable Assistance Games." pith.science (2026). https://pith.science/paper/7C3XCHEA

@misc{pith2026241217797,
  author       = {Pith},
  title        = {Pith review of: Observation Interference in Partially Observable Assistance Games},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7C3XCHEA}},
  note         = {Machine review of arXiv:2412.17797}
}
read the original abstract

We study partially observable assistance games (POAGs), a model of the human-AI value alignment problem which allows the human and the AI assistant to have partial observations. Motivated by concerns of AI deception, we study a qualitatively new phenomenon made possible by partial observability: would an AI assistant ever have an incentive to interfere with the human's observations? First, we prove that sometimes an optimal assistant must take observation-interfering actions, even when the human is playing optimally, and even when there are otherwise-equivalent actions available that do not interfere with observations. Though this result seems to contradict the classic theorem from single-agent decision making that the value of information is nonnegative, we resolve this seeming contradiction by developing a notion of interference defined on entire policies. This can be viewed as an extension of the classic result that the value of information is nonnegative into the cooperative multiagent setting. Second, we prove that if the human is simply making decisions based on their immediate outcomes, the assistant might need to interfere with observations as a way to query the human's preferences. We show that this incentive for interference goes away if the human is playing optimally, or if we introduce a communication channel for the human to communicate their preferences to the assistant. Third, we show that if the human acts according to the Boltzmann model of irrationality, this can create an incentive for the assistant to interfere with observations. Finally, we use an experimental model to analyze tradeoffs faced by the AI assistant in practice when considering whether or not to take observation-interfering actions.

Figures

Figures reproduced from arXiv: 2412.17797 by the authors.

Figure 1
Figure 1. Incentives to interfere with observations in the product selection game. (Left) When [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. The effect of varying β on the assistant’s incentive for observation interference in Example 6.2. Specifically, the y axis indicates the difference between the expected utility under non-interference minus the expected utility under interference. Similarly, if she observes sb or sc, her expected utility is exp(1) exp(1) + exp(0). (6) Thus, overall her expected utility is 1 2 7 exp(7) exp(7) + exp(0) + 1 2 exp(1) exp… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Align AI to Dynamic Human-AI Workflows

    cs.AI 2026-07 conditional novelty 5.0 of 10

    Alignment should be redefined as optimizing joint human-AI interaction trajectories in which preferences co-evolve, rather than emulating static preferences over isolated outputs.

Reference graph

Works this paper leans on

47 extracted references · 33 canonical work pages · cited by 1 Pith paper

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    and Ng, A

    Abbeel, P. and Ng, A. Apprenticeship learning via inverse reinforcement learning. In ICML, 2004

    work page 2004

  3. [3]

    Claude can now search the web, 2025

    Anthropic . Claude can now search the web, 2025. URL https://www.anthropic.com/news/web-search

    work page 2025

  4. [4]

    Information Theory, volume 19 of Interscience Tracts in Pure and Applied Mathematics

    Ash, R. Information Theory, volume 19 of Interscience Tracts in Pure and Applied Mathematics. John Wiley & Sons, 1965

    work page 1965

  5. [5]

    and Zeugmann, T

    Balbach, F. and Zeugmann, T. Recent developments in algorithmic teaching. In Language and Automata Theory and Applications. Springer, 2009

    work page 2009

  6. [6]

    Positive value of information in games

    Bassan, B., Gossner, O., Scarsini, M., and Zamir, S. Positive value of information in games. International Journal of Game Theory, 32 0 (1): 0 17--31, 2003. ISSN 1432-1270. doi:10.1007/s001820300142

    work page doi:10.1007/s001820300142 2003

  7. [7]

    S., Givan, R., Immerman, N., and Zilberstein, S

    Bernstein, D. S., Givan, R., Immerman, N., and Zilberstein, S. The complexity of decentralized control of Markov decision processes. Mathematics of Operations Research, 27 0 (4): 0 819--840, 2002. doi:10.1287/moor.27.4.819.297. URL https://dl.acm.org/doi/10.1287/moor.27.4.819.297

    doi:10.1287/moor.27.4.819.297 2002

  8. [8]

    Equivalent comparisons of experiments

    Blackwell, D. Equivalent comparisons of experiments. The annals of mathematical statistics, pp.\ 265--272, 1953

    work page 1953

Show all 47 references
  1. [9]

    Blackwell, D. et al. Comparison of experiments. In Proceedings of the second Berkeley symposium on mathematical statistics and probability, volume 2, pp.\ 93--102, 1951

    1951

  2. [10]

    and Lopes, M

    Cakmak, M. and Lopes, M. Algorithmic and human teaching of sequential decision tasks. In AAAI, 2012

    2012

  3. [11]

    and Thomas, J

    Cover, T. and Thomas, J. Elements of Information Theory. John Wiley & Sons, 1991

    1991

  4. [12]

    Blackwell's informativeness theorem using diagrams

    de Oliveira , H. Blackwell's informativeness theorem using diagrams. Games and Economic Behavior, 109: 0 126--131, 2018. ISSN 0899-8256. doi:https://doi.org/10.1016/j.geb.2017.12.008. URL https://www.sciencedirect.com/science/article/pii/S0899825617302270

    2018 doi

  5. [13]

    Uncertain reward-transition MDP s for negotiable reinforcement learning

    Desai, N. Uncertain reward-transition MDP s for negotiable reinforcement learning. Technical report, Technical report, UC Berkeley, Berkeley, California, USA, 2017

    2017

  6. [14]

    A decision-theoretic model of assistance

    Fern, A., Natarajan, S., Judah, K., and Tadepalli, P. A decision-theoretic model of assistance. JAIR, 50 0 (1): 0 71--104, 2014

    2014

  7. [15]

    An approach to communication equilibria

    Forges, F. An approach to communication equilibria. Econometrica: Journal of the Econometric Society, pp.\ 1375--1385, 1986

    1986

  8. [16]

    and Kearns, M

    Goldman, S. and Kearns, M. On the complexity of teaching. Journal of Computer and System Sciences, 50 0 (1): 0 20--31, 1995

    1995

  9. [17]

    Try Deep Research and our new experimental model in Gemini, your AI assistant , 2024

    Google . Try Deep Research and our new experimental model in Gemini, your AI assistant , 2024. URL https://blog.google/products/gemini/google-gemini-deep-research/

    2024

  10. [18]

    Long context, 2025

    Google . Long context, 2025. URL https://ai.google.dev/gemini-api/docs/long-context

    2025

  11. [19]

    J., Abbeel, P., and Dragan, A

    Hadfield-Menell, D., Russell, S. J., Abbeel, P., and Dragan, A. Cooperative inverse reinforcement learning. Advances in neural information processing systems, 29, 2016

    2016

  12. [20]

    The off-switch game

    Hadfield-Menell, D., Dragan, A., Abbeel, P., and Russell, S. The off-switch game. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, IJCAI'17, pp.\ 220–227. AAAI Press, 2017. ISBN 9780999241103

    2017

  13. [21]

    R., Millman, K

    Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N. J., Kern, R., Picus, M., Hoyer, S., van Kerkwijk, M. H., Brett, M., Haldane, A., del R \' i o, J. F., Wiebe, M., Peterson, P., G \' e rard...

    2020 doi

  14. [22]

    other-play

    Hu, H., Lerer, A., Peysakhovich, A., and Foerster, J. “other-play” for zero-shot coordination. In International Conference on Machine Learning, pp.\ 4399--4410. PMLR, 2020

    2020

  15. [23]

    Hunter, J. D. Matplotlib: A 2d graphics environment. Computing in Science & Engineering, 9 0 (3): 0 90--95, 2007. doi:10.1109/MCSE.2007.55

    2007 doi

  16. [24]

    I., Tauman, Y., and Zamir, S

    Kamien, M. I., Tauman, Y., and Zamir, S. On the value of information in a strategic conflict. Games and Economic Behavior, 2 0 (2): 0 129--153, 1990. ISSN 0899-8256. doi:https://doi.org/10.1016/0899-8256(90)90026-Q. URL https://www.sciencedirect.com/science/article/pii/0899825...

    1990

  17. [25]

    and Friedman, N

    Koller, D. and Friedman, N. Probabilistic Graphical Models: Principles and Techniques. MIT Press, 2009

    2009

  18. [26]

    Kuhn, H. W. Extensive games and the problem of information. In Kuhn, H. W. and Tucker, A. W. (eds.), Contributions to the Theory of Games (AM-28), Volume II, chapter 11, pp.\ 193--216. Princeton University Press, Princeton, 1953. ISBN 9781400881970. doi:doi:10.1515/97814008819...

    1953 doi

  19. [27]

    and Dragan, A

    Laidlaw, C. and Dragan, A. The Boltzmann policy distribution: Accounting for systematic suboptimality in human models. In International Conference on Learning Representations, 2021

    2021

  20. [28]

    When your AI deceives you: Challenges with partial observability of human evaluators in reward learning

    Lang, L., Foote, D., Russell, S., Dragan, A., Jenner, E., and Emmons, S. When your AI deceives you: Challenges with partial observability of human evaluators in reward learning. arXiv preprint arXiv:2402.17747, 2024

    2024 arXiv

  21. [29]

    Signaling and mediation in games with common interests

    Lehrer, E., Rosenberg, D., and Shmaya, E. Signaling and mediation in games with common interests. Games and Economic Behavior, 68 0 (2): 0 670--682, 2010. ISSN 0899-8256. doi:https://doi.org/10.1016/j.geb.2009.08.007. URL https://www.sciencedirect.com/science/article/pii/S0899...

    2010 doi

  22. [30]

    Luce, R. D. Individual Choice Behavior: A Theoretical Analysis. Dover, Mineola, NY, 1959

    1959

  23. [31]

    Conditional logit analysis of qualitative choice behavior

    McFadden, D. Conditional logit analysis of qualitative choice behavior. In Zarembka, P. (ed.), Frontiers of Econometrics. Academic Press, New York, 1973

    1973

  24. [32]

    and Russell, S

    Ng, A. and Russell, S. Algorithms for inverse reinforcement learning. In ICML, 2000

    2000

  25. [33]

    Introducing ChatGPT search , 2024

    OpenAI . Introducing ChatGPT search , 2024. URL https://openai.com/index/introducing-chatgpt-search/

    2024

  26. [34]

    pandas-dev/pandas: Pandas, February 2020

    pandas development team, T. pandas-dev/pandas: Pandas, February 2020. URL https://doi.org/10.5281/zenodo.3509134

    2020 doi

  27. [35]

    S., Goldstein, S., O’Gara, A., Chen, M., and Hendrycks, D

    Park, P. S., Goldstein, S., O’Gara, A., Chen, M., and Hendrycks, D. AI deception: A survey of examples, risks, and potential solutions. Patterns, 5 0 (5), 2024

    2024

  28. [36]

    and Amir, E

    Ramachandran, D. and Amir, E. Bayesian inverse reinforcement learning. In IJCAI, 2007

    2007

  29. [37]

    Reif, J. H. The complexity of two-player games of incomplete information. Journal of computer and system sciences, 29 0 (2): 0 274--301, 1984

    1984

  30. [38]

    and Norvig, P

    Russell, S. and Norvig, P. Artificial Intelligence: A modern approach. Pearson, 3 edition, 2010

    2010

  31. [39]

    D., Abbeel, P., Dragan, A., and Russell, S

    Shah, R., Freire, P., Alex, N., Freedman, R., Krasheninnikov, D., Chan, L., Dennis, M. D., Abbeel, P., Dragan, A., and Russell, S. Benefits of assistance over reward learning, 2020

    2020

  32. [40]

    A new formalism, method and open issues for zero-shot coordination

    Treutlein, J., Dennis, M., Oesterheld, C., and Foerster, J. A new formalism, method and open issues for zero-shot coordination. In International Conference on Machine Learning, pp.\ 10413--10423. PMLR, 2021

    2021

  33. [41]

    R., Toni, F., Belardinelli, F., and Everitt, T

    Ward, F. R., Toni, F., Belardinelli, F., and Everitt, T. Honesty is the best policy: Defining and mitigating AI deception. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=EmxpDiPgRu

    2023

  34. [42]

    Waskom, M. L. seaborn: statistical data visualization. Journal of Open Source Software, 6 0 (60): 0 3021, 2021. doi:10.21105/joss.03021. URL https://doi.org/10.21105/joss.03021

    2021 doi

  35. [43]

    R., He, H., and Feng, S

    Wen, J., Zhong, R., Khan, A., Perez, E., Steinhardt, J., Huang, M., Bowman, S. R., He, H., and Feng, S. Language models learn to mislead humans via RLHF . arXiv preprint arXiv:2409.12822, 2024

    2024 arXiv

  36. [44]

    D ata S tructures for S tatistical C omputing in P ython

    W es M c K inney. D ata S tructures for S tatistical C omputing in P ython. In S t\'efan van der W alt and J arrod M illman (eds.), P roceedings of the 9th P ython in S cience C onference , pp.\ 56 -- 61, 2010. doi:10.25080/Majora-92bf1922-00a

    2010 doi

  37. [45]

    Targeted manipulation and deception emerge when optimizing LLM s for user feedback

    Williams, M., Carroll, M., Narang, A., Weisser, C., Murphy, B., and Dragan, A. Targeted manipulation and deception emerge when optimizing LLM s for user feedback. arXiv preprint arXiv:2411.02306, 2024

    2024 arXiv

  38. [46]

    Learning to interactively learn and assist

    Woodward, M., Finn, C., and Hausman, K. Learning to interactively learn and assist. In Proceedings of the AAAI conference on artificial intelligence, pp.\ 2535--2543, 2020

    2020

  39. [47]

    Maximum entropy inverse reinforcement learning

    Ziebart, B., Maas, A., Bagnell, J., and Dey, A. Maximum entropy inverse reinforcement learning. In AAAI, 2008

    2008

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.