Defines canonical credal set oracles as Knaster-Tarski least fixed points of isotone operators on closed credal sets, proving self-consistency and reduction to point estimates when non-performative.
Good and safe uses of AI Oracles
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
abstract
It is possible that powerful and potentially dangerous artificial intelligence (AI) might be developed in the future. An Oracle is a design which aims to restrain the impact of a potentially dangerous AI by restricting the agent to no actions besides answering questions. Unfortunately, most Oracles will be motivated to gain more control over the world by manipulating users through the content of their answers, and Oracles of potentially high intelligence might be very successful at this \citep{DBLP:journals/corr/AlfonsecaCACAR16}. In this paper we present two designs for Oracles which, even under pessimistic assumptions, will not manipulate their users into releasing them and yet will still be incentivised to provide their users with helpful answers. The first design is the counterfactual Oracle -- which choses its answer as if it expected nobody to ever read it. The second design is the low-bandwidth Oracle -- which is limited by the quantity of information it can transmit.
fields
cs.AI 3representative citing papers
Models AGI safety frameworks with causal influence diagrams to compare optimization objectives and causal assumptions.
The paper characterizes ASI and examines scaling, paradigm shifts, recursive self-improvement, and multi-agent collectives as routes from AGI to ASI, together with frictions and open questions about acceleration.
citing papers explorer
-
Unbiased Canonical Set-Valued Oracles Via Lattice Theory
Defines canonical credal set oracles as Knaster-Tarski least fixed points of isotone operators on closed credal sets, proving self-consistency and reduction to point estimates when non-performative.
-
Modeling AGI Safety Frameworks with Causal Influence Diagrams
Models AGI safety frameworks with causal influence diagrams to compare optimization objectives and causal assumptions.
-
From AGI to ASI
The paper characterizes ASI and examines scaling, paradigm shifts, recursive self-improvement, and multi-agent collectives as routes from AGI to ASI, together with frictions and open questions about acceleration.