REVIEW 8 cited by
Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Ensuring that AI systems reliably and robustly avoid harmful or dangerous behaviours is a crucial challenge, especially for AI systems with a high degree of autonomy and general intelligence, or systems used in safety-critical contexts. In this paper, we will introduce and define a family of approaches to AI safety, which we will refer to as guaranteed safe (GS) AI. The core feature of these approaches is that they aim to produce AI systems which are equipped with high-assurance quantitative safety guarantees. This is achieved by the interplay of three core components: a world model (which provides a mathematical description of how the AI system affects the outside world), a safety specification (which is a mathematical description of what effects are acceptable), and a verifier (which provides an auditable proof certificate that the AI satisfies the safety specification relative to the world model). We outline a number of approaches for creating each of these three core components, describe the main technical challenges, and suggest a number of potential solutions to them. We also argue for the necessity of this approach to AI safety, and for the inadequacy of the main alternative approaches.
Forward citations
Cited by 8 Pith papers
-
Mining Verdict Boundaries for Neural Network Verification
BMiner speeds up Branch-and-Bound neural network verification by using exponential and gradient-guided search to skip subproblems on the way to each path's verdict boundary, cutting average verification time by 17–30%.
-
Towards chemistries in dynamical systems
A finite dynamical system can be described as a Petri-net chemistry when a token map and transition map are compatible; an optional uniqueness criterion restricts when this description is least ambiguous.
-
Re:Form -- Reducing Human Annotations in Scalable Formal Software Verification with RL in LLMs: A Preliminary Study on Dafny
A minimal-prior pipeline with automated data curation and verifier-driven RL lets small LLMs generate verifiable Dafny specifications and beat larger proprietary models on a synthetic compositional benchmark.
-
The Other Mind: How Language Models Exhibit Human Temporal Cognition
Larger LLMs develop a subjective 'present' around the current date, and their year similarity judgments follow a logarithmic Weber-Fechner compression, with supporting neural and representational evidence.
-
The Limits of Predicting Agents from Behaviour
Observed behavior only weakly constrains an intentional agent's choices under distribution shift, and its perceived fairness and harm cannot be identified from behavior alone.
-
Beyond the Proxy: Trajectory-Distilled Guidance for Offline GFlowNet Training
TD-GFN uses IRL-derived edge rewards to prune the environment DAG and sample backward trajectories, training offline GFlowNets directly from ground-truth terminal rewards without a proxy reward model.
-
Out of Control -- Why Alignment Needs Formal Control Theory (and an Alignment Control Stack)
A position paper proposing that AI alignment adopt formal optimal control and a ten-layer Alignment Control Stack for organizing and interoperating control interventions.
-
Report on NSF Workshop on Science of Safe AI
An NSF workshop report articulating a cross-disciplinary research agenda for designing and verifying safe, trustworthy AI systems.
Discussion (0). Sign in to comment.