REVIEW 3 major objections 4 minor 1 cited by
Technical Requirements for Halting Dangerous AI Activities
T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Halting dangerous AI development requires deep control over AI compute, this paper argues.
desk verdict A useful compute-governance synthesis with a good intervention-plan matrix, but the headline necessity claim is stated more strongly than the paper's own caveats support. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is AI compute treated as a physical chokepoint: chips, fabs, and datacenters that can be located, monitored, and shut off. Around this chokepoint the paper assembles a catalog of interventions, with the most load-bearing items being hardware-enabled mechanisms embedded in chips to verify location and enforce usage policies, plus datacenter- and fab-level monitoring. The analytical table maps every intervention to three capacities—restrict training, restrict inference, and restrict post-training—and to five governance plans, making the dependence structure explicit. The comparison shows that compute-control functions appear in every plan, while software-only monitoring is consistently rated easier to subvert and often low in readiness.
What would settle it
A concrete disconfirmation would be a demonstration that frontier-dangerous AI capabilities can be obtained from ordinary consumer GPUs outside declared datacenters and untracked chips, without relying on a few advanced fabs—or, less dramatically, an empirical estimate that the compute required for dangerous capabilities falls below any monitorable threshold within a few years.
Extended reading notes
Core claim
The central claim is that a halt is primarily a hardware-governance problem. Based on current AI development, the main intervention point is AI compute, because advanced chip production is concentrated in a small number of fabs and large training runs are physically located in datacenters. The paper's table grades each intervention for technological readiness and for whether each plan requires it, using the categories required, maybe required, and helpful. In every plan considered, at least one of three compute-control functions—chip tracking, datacenter monitoring, or manufacturing restriction—is required; for most plans, preventing model weights and capabilities from proliferating is also essential. A number of these required technologies, including hardware-enabled governance mechanisms, currently lack functional prototypes.
Load-bearing premise
The load-bearing premise is that dangerous AI requires large, concentrated, trackable compute, so chips and datacenters are the right chokepoints; if algorithmic advances or consumer hardware make dangerous AI possible outside those physical chokepoints, the halt mechanisms lose their grip.
Editorial extensions
If this is right
- If the central claim is right, governments should start international tracking of AI chips now, because delayed tracking is harder and most halt plans require historical shipment data.
- Hardware-enabled governance mechanisms need to be developed and standardized before an emergency, since several plans depend on them and they are currently at low technological readiness.
- Restricting model release and securing model weights are not optional extras for most plans; without them, leaked capabilities could make compute-based control impossible.
- A global chip production moratorium is feasible partly because advanced chip fabrication is concentrated in a small number of fabs, and it could even be enforced unilaterally as a last resort.
- Compute monitoring alone may need to be complemented by restrictions on algorithmic research, since algorithmic progress could lower the compute needed for dangerous AI.
Reading between the lines
- A testable corollary is that the effectiveness of compute-centric halts can be quantified by measuring how much dangerous capability can be produced below proposed compute thresholds; if that elasticity is large, compute control alone will be underpowered regardless of readiness.
- The paper's emergency-shutdown appendix suggests an undeveloped weak spot: if a rogue or malicious AI spreads to consumer hardware and botnets rather than staying in datacenters, the shutdown problem shifts from governance to infrastructure-level response, such as grid or network controls.
- Extending the analysis, monitoring algorithmic progress and researcher activity could become as important as monitoring compute if algorithmic efficiency improvements continue at recent rates; the paper lists this as helpful for most plans rather than required.
- One policy consequence the paper leaves implicit is that compute-control infrastructure is dual-use: the same systems that enable a halt could also enable continuous surveillance of AI developers, so safeguards on the governance mechanisms themselves would need to be designed in.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper catalogs technical interventions for halting or restricting dangerous AI development and deployment, groups them into six categories (chip location, chip manufacturing, compute monitoring, non-compute monitoring, limiting proliferation, and research), and maps them to five governance plans: two novel plans (Last-minute Wake-up and Chip Production Moratorium) and three existing proposals (A Narrow Path, Keep the Future Human, and Superintelligence Strategy). The paper grades each intervention's technological readiness and each plan's dependency on it in Table 1, and concludes that all plans require substantial control over AI compute through chip tracking, datacenter monitoring, or manufacturing restrictions.
Significance. If its central claim is accepted with appropriate qualifications, this paper provides a valuable organizing framework for AI governance research and policy. It makes explicit which technical capabilities a coordinated halt would demand, identifies low-readiness interventions that need development, and usefully distinguishes capacities for restricting training, inference, and post-training. The paper is transparent about the preliminary nature of its assessments and includes an appendix that acknowledges concrete failure scenarios, such as rogue AI running on consumer hardware or a botnet. Its main weaknesses are an over-generalized conclusion and at least one unsupported quantitative claim in the Chip Production Moratorium discussion.
major comments (3)
- [Section 6 (Conclusion)] The sentence 'All of these plans requires substantial control over AI compute through chip tracking, datacenter monitoring, or manufacturing restrictions' is stated unconditionally, but the paper's own analysis in Section 2 and Appendix A identifies scenarios in which dangerous AI activities could be enabled by algorithmic progress or run on diffuse consumer hardware, making compute control neither necessary nor sufficient. As written, the conclusion overstates what the preceding analysis supports. The conclusion should be reformulated as a conditional claim: compute control is necessary for plans of this type provided dangerous AI activities require large, concentrated, trackable compute, and the paper should explicitly carry over the caveats from Section 2 and Appendix A.
- [Section 4 (Chip Production Moratorium)] The text claims that restricting algorithmic research and governing existing compute 'may not be necessary to delay dangerous AI by decades,' but no evidence or model is provided for the 'decades' quantification. This is a load-bearing empirical claim about the plan's effectiveness, and it is not supported by the cited work in this section. The authors should either remove the specific 'decades' claim or provide a citation or analysis that justifies it.
- [Section 5 and Table 1] The technological readiness grades (High/Medium/Low) and dependency classifications (Required/Maybe required/Helpful) in Table 1 are presented as key results and are used in the conclusion ('Most plans also rely on interventions currently at low technological readiness'). However, the paper does not provide a documented rubric, an inter-rater reliability check, a sensitivity analysis, or any validation of these expert judgments. The authors call them 'preliminary best estimates,' but given that the table is a central contribution, more methodological transparency is needed: how were grades assigned, by whom, and how robust are the conclusions to alternative grades?
minor comments (4)
- [Section 6 (Conclusion)] The first sentence contains a grammatical error: 'All of these plans requires' should be 'All of these plans require.'
- [Section 4 heading and body] The sentence 'The plans in Section 4were highlighted because they, to some extent, involve halting dangerous AI activities' has a missing space before 'were'.
- [Table 1] The legend for the table uses symbols to denote Required, Maybe required, and Helpful, but these symbols do not appear in the plain-text version. Please ensure the symbols are visible in the formatted publication.
- [Section 5] The text preceding Table 1 explains the meaning of the grades but does not state whether these grades reflect only the authors' judgment or a broader elicitation; adding this detail would help readers calibrate their interpretation of the matrix.
Circularity Check
No significant circularity: the paper is a qualitative catalog and plan assessment, and its central conclusion transparently summarizes the authors' judgments rather than being derived from fitted inputs or a self-citation chain.
full rationale
The paper contains no equations, fitted parameters, or statistical predictions; its 'derivation chain' is a catalog of technical interventions plus a qualitative dependency assessment for five governance plans. The central claim—that all listed plans require substantial control over AI compute—is a synthesis of the plan descriptions themselves: Last-minute Wake-up is defined as 'implementing a global compute monitoring regime,' Chip Production Moratorium as seeking to 'globally pause the production of new AI compute,' A Narrow Path uses a 'licensing regime based on compute thresholds,' Keep the Future Human uses 'hardware-enabled compute governance,' and Superintelligence Strategy relies on 'hardware export control.' The conclusion is therefore a summary of the authors' own characterizations, not an empirical prediction equal to its input by construction. The paper also does not present the compute-centric premise as an unfalsifiable definition: it cites external work (Sastry et al., 2024; Ho et al., 2024) for the importance of compute and algorithmic progress, and it explicitly flags in Appendix A that rogue AI could run on consumer hardware or a distributed botnet, making shutdown 'very difficult or effectively impossible.' This shows the authors acknowledge conditions under which compute control would be insufficient, rather than defining the conclusion into existence. Self-citations to Scher & Thiergart (2024) and Wasil et al. (2024) appear for specific verification mechanisms and are not load-bearing uniqueness theorems; removing them would not collapse the central argument. No fitted input is relabeled as a prediction, no known result is merely renamed, and no ansatz is smuggled in via citation. The only potential concerns—self-assessment of one's own plans and dependence on the unproven premise that dangerous AI requires large, concentrated compute—are correctness and robustness risks, not circularity.
Assumptions & free parameters
assumptions (4)
- domain assumption The main intervention point for these mechanisms is AI compute.
- domain assumption Production of advanced AI chips is concentrated in dozens of highly advanced fabs.
- domain assumption Governments may coordinate to halt AI development.
- ad hoc to paper Technological readiness can be graded as High/Medium/Low and dependencies as Required/Maybe/Helpful.
Cite this review
Pith. "Pith review of Technical Requirements for Halting Dangerous AI Activities." pith.science (2026). https://pith.science/paper/M527JAH4
@misc{pith2026250709801,
author = {Pith},
title = {Pith review of: Technical Requirements for Halting Dangerous AI Activities},
year = {2026},
howpublished = {\url{https://pith.science/paper/M527JAH4}},
note = {Machine review of arXiv:2507.09801}
}
read the original abstract
The rapid development of AI systems poses unprecedented risks, including loss of control, misuse, geopolitical instability, and concentration of power. To navigate these risks and avoid worst-case outcomes, governments may proactively establish the capability for a coordinated halt on dangerous AI development and deployment. In this paper, we outline key technical interventions that could allow for a coordinated halt on dangerous AI activities. We discuss how these interventions may contribute to restricting various dangerous AI activities, and show how these interventions can form the technical foundation for potential AI governance plans.
Forward citations
Cited by 1 Pith paper
-
How to Catch a GPU: A Taxonomy of Verification and Enforcement Mechanisms for International AI Agreements
Verification of international AI agreements will fail first at detecting hidden compute facilities, around the 10,000-H100-equivalent scale, before other enforcement mechanisms break.
Reference graph
Works this paper leans on
-
[1]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
-
[2]
Aarne, O., Fist, T., and Withers, C. Secure, governable chips. Center for a New American Security. https://www. cnas. org/publications/reports/secure-governable-chips, 2024
work page 2024
-
[3]
Aguirre, A. Keep the Future Human: Why and How We Should Close the Gates to AGI and Superintelligence, and What We Should Build Instead . arXiv preprint arXiv:2311.09452, 2025. URL https://arxiv.org/abs/2311.09452v4
arXiv 2025
-
[4]
What Information Should Be Shared with Whom "Before and During Training"?
Belfield, H. What Information Should Be Shared with Whom " Before and During Training "?, December 2024. URL http://arxiv.org/abs/2501.10379. arXiv:2501.10379 [cs]
work page Pith review arXiv 2024
-
[5]
N., Zhang, Y.-Q., Xue, L., Shalev-Shwartz, S., et al
Bengio, Y., Hinton, G., Yao, A., Song, D., Abbeel, P., Darrell, T., Harari, Y. N., Zhang, Y.-Q., Xue, L., Shalev-Shwartz, S., et al. Managing extreme ai risks amid rapid progress. Science, 384 0 (6698): 0 842--845, 2024
work page 2024
-
[6]
Brass, A. and Aarne, O. Location verification for ai chips. Technical report, Institute for AI Policy and Strategy, April 2024. URL https://www.iaps.ai/research/location-verification-for-ai-chips. Available at: https://www.iaps.ai/research/location-verification-for-ai-chips
work page 2024
-
[7]
L., Bucknall, B., Haupt, A., Wei, K., Scheurer, J., Hobbhahn, M., Sharkey, L., Krishna, S., Hagen, M
Casper, S., Ezell, C., Siegmann, C., Kolt, N., Curtis, T. L., Bucknall, B., Haupt, A., Wei, K., Scheurer, J., Hobbhahn, M., Sharkey, L., Krishna, S., Hagen, M. V., Alberti, S., Chan, A., Sun, Q., Gerovitch, M., Bau, D., Tegmark, M., Krueger, D., and Hadfield-Menell, D. Black- Box Access is Insufficient for Rigorous AI Audits . In The 2024 ACM Conference o...
arXiv 2024
-
[8]
Statement on AI Risk CAIS , March 2023
Center for AI Safety . Statement on AI Risk CAIS , March 2023. URL https://www.safe.ai/work/statement-on-ai-risk
work page 2023
Show all 41 references
-
[9]
K., and Anderljung, M
Chan, A., Wei, K., Huang, S., Rajkumar, N., Perrier, E., Lazar, S., Hadfield, G. K., and Anderljung, M. Infrastructure for ai agents. arXiv preprint arXiv:2501.10114, 2025
2025 arXiv
-
[10]
Locking machine learning models into hardware
Clifford, E., Saravanan, A., Langford, H., Zhang, C., Zhao, Y., Mullins, R., Shumailov, I., and Hayes, J. Locking machine learning models into hardware. arXiv preprint arXiv:2405.20990, 2024
2024 arXiv
-
[11]
Here Comes The AI Worm : Unleashing Zero -click Worms that Target GenAI - Powered Applications , January 2025
Cohen, S., Bitton, R., and Nassi, B. Here Comes The AI Worm : Unleashing Zero -click Worms that Target GenAI - Powered Applications , January 2025. URL http://arxiv.org/abs/2403.02817. arXiv:2403.02817 [cs] version: 2
2025 arXiv
-
[12]
W., Webster, K., Epstein, G
Crawford, F. W., Webster, K., Epstein, G. L., Roberts, D., Fair, J., and Nevo, S. Securing Commercial Nucleic Acid Synthesis . RAND Corporation, Santa Monica, CA , 2024. doi:10.7249/RRA3329-1
2024 doi
-
[13]
AI-Enabled Coups: How a Small Group Could Use AI to Seize Power
Davidson, T., Finnveden, L., and Hadshar, R. AI-Enabled Coups: How a Small Group Could Use AI to Seize Power . 2025. URL https://www.forethought.org/research/ai-enabled-coups-how-a-small-group-could-use-ai-to-seize-power. Accessed: 2025-04-17
2025
-
[14]
Compute forecast
Dean, R. Compute forecast. AI 2027, April 2025. URL https://ai-2027.com/research/compute-forecast. Available at: https://ai-2027.com/research/compute-forecast
2027
-
[15]
Sophon: Non-fine-tunable learning to restrain task transferability for pre-trained models
Deng, J., Pang, S., Chen, Y., Xia, L., Bai, Y., Weng, H., and Xu, W. Sophon: Non-fine-tunable learning to restrain task transferability for pre-trained models. In 2024 IEEE Symposium on Security and Privacy (SP), pp.\ 2553--2571. IEEE, 2024
2024
-
[16]
and Koessler, L
Heim, L. and Koessler, L. Training Compute Thresholds : Features and Functions in AI Regulation , August 2024. URL http://arxiv.org/abs/2405.10799. arXiv:2405.10799 [cs]
2024 arXiv
-
[17]
Superintelligence Strategy : Expert Version , March 2025
Hendrycks, D., Schmidt, E., and Wang, A. Superintelligence Strategy : Expert Version , March 2025. URL http://arxiv.org/abs/2503.05628. arXiv:2503.05628 [cs]
2025 arXiv
-
[18]
C., Atkinson, D., Thompson, N., and Sevilla, J
Ho, A., Besiroglu, T., Erdil, E., Owen, D., Rahman, R., Guo, Z. C., Atkinson, D., Thompson, N., and Sevilla, J. Algorithmic progress in language models, March 2024. URL http://arxiv.org/abs/2403.05812. arXiv:2403.05812 [cs]
2024 arXiv
-
[19]
A Sketch of Potential Tripwire Capabilities for AI
Karnofsky, H. A Sketch of Potential Tripwire Capabilities for AI . dec 2024
2024
-
[20]
Risk thresholds for frontier ai
Koessler, L., Schuett, J., and Anderljung, M. Risk thresholds for frontier ai. arXiv preprint arXiv:2406.14713, 2024
2024 arXiv
-
[21]
Kulp, G., Gonzales, D., Smith, E., Heim, L., Puri, P., Vermeer, M. J. D., and Winkelman, Z. Hardware-Enabled Governance Mechanisms: Developing Technical Solutions to Exempt Items Otherwise Classified Under Export Control Classification Numbers 3A090 and 4A090. RAND Corporation...
2024 doi
-
[22]
The Rogue Replication Threat Model
METR. The Rogue Replication Threat Model . https://metr.org/blog/2024-11-12-rogue-replication-threat-model/, 11 2024
2024
-
[23]
A Narrow Path , December 2024
Miotti, A., Bilge, T., Kasten, D., and Newport, J. A Narrow Path , December 2024. URL https://www.narrowpath.co/
2024
-
[24]
A., and Alstott, J
Nevo, S., Lahav, D., Karpur, A., Bar-On, Y., Bradley, H. A., and Alstott, J. Securing AI Model Weights : Preventing Theft and Misuse of Frontier Models . Technical report, RAND Corporation, May 2024. URL https://www.rand.org/pubs/research_reports/RRA2849-1.html
2024
-
[25]
OpenAI Model Spec , February 2025
OpenAI . OpenAI Model Spec , February 2025. URL https://model-spec.openai.com/2025-02-12.html
2025
-
[26]
Interim report: Mechanisms for flexible hardware-enabled guarantees
Petrie, J., Aarne, O., Ammann, N., and Dalrymple, D. Interim report: Mechanisms for flexible hardware-enabled guarantees. Technical report, 8 2024
2024
-
[27]
F., Sanders, J., Rahman, R., and Heim, L
Pilz, K. F., Sanders, J., Rahman, R., and Heim, L. Trends in ai supercomputers. arXiv preprint arXiv:2504.16026, 2025
2025 arXiv
-
[28]
R., Jackson, K., and Newman, J
Raman, D., Madkour, N., Murphy, E. R., Jackson, K., and Newman, J. Intolerable risk threshold recommendations for artificial intelligence. arXiv preprint arXiv:2503.05812, February 2025
2025 arXiv
-
[29]
K., Ngo, R., Pilz, K., Gor, G., Bluemke, E., Shoker, S., Egan, J., Trager, R
Sastry, G., Heim, L., Belfield, H., Anderljung, M., Brundage, M., Hazell, J., O'Keefe, C., Hadfield, G. K., Ngo, R., Pilz, K., Gor, G., Bluemke, E., Shoker, S., Egan, J., Trager, R. F., Avin, S., Weller, A., Bengio, Y., and Coyle, D. Computing Power and the Governance of Artif...
2024 arXiv
-
[30]
and Thiergart, L
Scher, A. and Thiergart, L. Mechanisms to Verify International Agreements About AI Development , November 2024. URL https://techgov.intelligence.org/research/mechanisms-to-verify-international-agreements-about-ai-development
2024
-
[31]
International agreements on ai safety: Review and recommendations for a conditional ai safety treaty
Scholefield, R., Martin, S., and Barten, O. International agreements on ai safety: Review and recommendations for a conditional ai safety treaty. arXiv preprint arXiv:2503.18956, 2025
2025 arXiv
-
[32]
\'O ., Korinek, A., et al
Seger, E., Dreksler, N., Moulange, R., Dardaman, E., Schuett, J., Wei, K., Winter, C., Arnold, M., h \'E igeartaigh, S. \'O ., Korinek, A., et al. Open-sourcing highly capable foundation models: An evaluation of risks, benefits, and alternative methods for pursuing open-source...
2023 arXiv
-
[33]
Can AI Scaling Continue Through 2030? , 2024
Sevilla, J., Besiroglu, T., Cottier, B., You, J., Roldán, E., Villalobos, P., and Erdil, E. Can AI Scaling Continue Through 2030? , 2024. URL https://epoch.ai/blog/can-ai-scaling-continue-through-2030. Accessed: 2025-03-29
2024
-
[34]
Structured access: an emerging paradigm for safe ai deployment
Shevlane, T. Structured access: an emerging paradigm for safe ai deployment. arXiv preprint arXiv:2201.05159, 2022
2022 arXiv
-
[35]
Model evaluation for extreme risks, September 2023
Shevlane, T., Farquhar, S., Garfinkel, B., Phuong, M., Whittlestone, J., Leung, J., Kokotajlo, D., Marchal, N., Anderljung, M., Kolt, N., Ho, L., Siddarth, D., Avin, S., Hawkins, W., Kim, B., Gabriel, I., Bolina, V., Clark, J., Bengio, Y., Christiano, P., and Dafoe, A. Model e...
2023 arXiv
-
[36]
AI catastrophes and rogue deployments
Shlegeris, B. AI catastrophes and rogue deployments . June 2024. URL https://www.alignmentforum.org/posts/ceBpLHJDdCt3xfEok/ai-catastrophes-and-rogue-deployments
2024
-
[37]
Ai behind closed doors: a primer on the governance of internal deployment
Stix, C., Pistillo, M., Sastry, G., Hobbhahn, M., Ortega, A., Balesni, M., Hallensleben, A., Goldowsky-Dill, N., and Sharkey, L. Ai behind closed doors: a primer on the governance of internal deployment. arXiv preprint arXiv:2504.12170, 2025
2025 arXiv
-
[38]
Tamper-resistant safeguards for open-weight llms
Tamirisa, R., Bharathi, B., Phan, L., Zhou, A., Gatti, A., Suresh, T., Lin, M., Wang, J., Wang, R., Arel, R., et al. Tamper-resistant safeguards for open-weight llms. arXiv preprint arXiv:2408.00761, 2024
2024 arXiv
-
[39]
and Allen, G
Thadani, A. and Allen, G. C. Mapping the semiconductor supply chain. Center for Strategic and International Studies, 11, May 2023
2023
-
[40]
R., Reed, T., Miller, J
Wasil, A. R., Reed, T., Miller, J. W., and Barnett, P. Verification methods for international ai agreements. arXiv preprint arXiv:2408.16074, 2024
2024 arXiv
-
[41]
Defense against the AI dark arts: Threat assessment and coalition defense
Zelikow, P., Cu \'e llar, M.-F., Schmidt, E., and Matheny, J. Defense against the AI dark arts: Threat assessment and coalition defense. Report, Hoover Institution, Stanford, CA, 12 2024. A Publication of the Hoover Institution
2024
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.