REVIEW 3 cited by
Embedded Agency
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Traditional models of rational action treat the agent as though it is cleanly separated from its environment, and can act on that environment from the outside. Such agents have a known functional relationship with their environment, can model their environment in every detail, and do not need to reason about themselves or their internal parts. We provide an informal survey of obstacles to formalizing good reasoning for agents embedded in their environment. Such agents must optimize an environment that is not of type "function"; they must rely on models that fit within the modeled environment; and they must reason about themselves as just another physical system, made of parts that can be modified and that can work at cross purposes.
Forward citations
Cited by 3 Pith papers
-
Safety from Honesty in a Disinterested AI Predictor
Under consequence-invariant posterior training and sparsity of coordinated harm patterns, the training mass on dangerous guarded Predictors is bounded by C_bad times R_shell.
-
A game theory for foundation models shows new paths to rational cooperation through similarity inference
Foundation-model agents that plan by predicting both the world and themselves can rationally cooperate in one-shot social dilemmas by inferring behavioral similarity from interaction history.
-
Towards chemistries in dynamical systems
A finite dynamical system can be described as a Petri-net chemistry when a token map and transition map are compatible; an optional uniqueness criterion restricts when this description is least ambiguous.
Discussion (0). Continue with ORCID to comment.