REVIEW 4 cited by
Tools for Verifying Neural Models' Training Data
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
It is important that consumers and regulators can verify the provenance of large neural models to evaluate their capabilities and risks. We introduce the concept of a "Proof-of-Training-Data": any protocol that allows a model trainer to convince a Verifier of the training data that produced a set of model weights. Such protocols could verify the amount and kind of data and compute used to train the model, including whether it was trained on specific harmful or beneficial data sources. We explore efficient verification strategies for Proof-of-Training-Data that are compatible with most current large-model training procedures. These include a method for the model-trainer to verifiably pre-commit to a random seed used in training, and a method that exploits models' tendency to temporarily overfit to training data in order to detect whether a given data-point was included in training. We show experimentally that our verification procedures can catch a wide variety of attacks, including all known attacks from the Proof-of-Learning literature.
Forward citations
Cited by 4 Pith papers
-
In Which Areas of Technical AI Safety Could Geopolitical Rivals Cooperate?
Based on a four-risk typology, the paper concludes that verification mechanisms and codified protocols are the least risky areas for cooperation between geopolitical rivals on technical AI safety.
-
Towards Data Governance of Frontier AI Models
Training data can serve as a governance lever for frontier AI through five proposed mechanisms: canary tokens, mandatory filtering, dataset reporting, data security, and know-your-customer rules.
-
International Security Applications of Flexible Hardware-Enabled Guarantees
A policy report argues that hardware-level guarantees on AI chips could make international AI governance agreements stable if states value winning only modestly and fear catastrophe sufficiently.
-
The Societal Impact of Foundation Models: Advancing Evidence-based AI Policy
A dissertation that synthesizes prior work on foundation models into a three-part framework: conceptual framing, empirical measurement (HELM, FMTI), and evidence-based AI policy.
Discussion (0). Continue with ORCID to comment.