Pith. sign in

REVIEW 3 major objections 4 minor 2 cited by

The Reality of AI and Biorisk

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A review of the publicly available evidence concludes that current large language models and AI-enabled biological tools do not pose an immediate biorisk, while the studies supporting that conclusion are too nascent and methodologically…

desk verdict A careful narrative review that correctly flags the immature evidence base, but its 'no immediate risk' conclusion leans too hard on null red-team results without a sensitivity check. read the letter →

arxiv 2412.01946 v3 pith:2YWJOFD3 submitted 2024-12-02 cs.AI

Aidan Peppin , Anka Reuel , Stephen Casper , Elliot Jones , Andrew Strait , Usman Anwar , Anurag Agrawal , Sayash Kapoor
show 5 more authors
Sanmi Koyejo Marie Pellat Rishi Bommasani Nick Frosst Sara Hooker
This is my paper · ORCID
classification cs.AI
keywords bioriskAIsafetylargelanguagemodelsbiologicaltoolsredteamingthreatbiosecuritywhole-chainriskanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether current AI systems can meaningfully increase biorisk and whether existing tests can detect such an increase. Reviewing two dominant threat models—LLMs as sources of biological information and planning, and AI-enabled biological tools that might help synthesize harmful artifacts—it finds the evidence base nascent, often speculative, and methodologically limited. The available studies, including the largest red-team exercises, find no statistically significant uplift in biological attack capability from LLM access compared with internet access alone. For biological tools, there are no known real-world misuse cases, and capability critiques suggest such tools currently underperform at core tasks. The paper concludes that current models do not pose an immediate biorisk, while emphasizing that biorisk remains a future risk requiring more rigorous, whole-chain evaluation.

What carries the argument

The organizing device is the biorisk chain—the sequence from malicious intent through biological idea, conversion to data such as a pathogen genome, synthesis into a live artifact, culturing and testing, and release into the environment. The paper evaluates each threat model against this chain and against a marginal-risk standard: does the AI system add capability beyond what the internet and existing tools already provide? It also applies methodological criteria—baseline controls, sample size, expertise matching, and transparency—to weigh the red-team studies.

What would settle it

A randomized controlled red-team study with a validated biological-tasking rubric, a sufficiently large and expertise-matched participant pool, and an internet-only control arm that finds a statistically significant uplift in end-to-end attack capability would overturn the paper's central conclusion.

Watch

Extended reading notes

Core claim

The paper's central claim is that current general-purpose LLMs and current AI biological tools do not meaningfully increase biorisk, and that the two dominant threat models—information-and-planning uplift and synthesis of harmful biological artifacts—are not yet supported by sound theory or robust methods. The authors do not assert that AI biorisk is impossible; they argue that the empirical record is too immature to justify treating it as an immediate threat. They ground this in the pattern across red-team studies: the only studies comparing LLM-plus-internet access against internet-only access found no statistically significant uplift, or in one case reported an unclear significance level. For biological tools, they emphasize capability gaps, data-access bottlenecks, and formidable laboratory, skill, and material barriers along the biorisk chain, alongside the absence of any documented real-world misuse.

Load-bearing premise

The load-bearing premise is that the red-team studies with null results were sensitive enough to detect a meaningful biorisk uplift if one existed, and that the absence of documented misuse of biological tools is evidence of low risk rather than of inattention.

Editorial extensions

If this is right

  • If the central claim is correct, regulators should not treat current general-purpose LLM biorisk evaluations as evidence of immediate danger; null red-team results should be reported with full methodological transparency and explicit internet-access baselines.
  • Biorisk assessment should shift from isolated capability tests to whole-chain analyses that include access to materials, specialized skills, and laboratory facilities.
  • Policy and research attention should concentrate on AI models developed specifically for biological purposes, such as biological tools and biology-tuned LLMs, rather than all general-purpose models.
  • Compute-based thresholds tied to biological sequence data are unreliable because greater compute does not reliably yield greater capability, and the definition of a biological model can be manipulated.
  • The absence of significant uplift in current studies does not rule out future risk; continued evaluation of new model generations, especially those trained on biological data, remains necessary.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference (not in paper): If the null red-team results are accepted, the policy implication extends beyond the paper's explicit recommendations—compute-based thresholds and mandatory biorisk evaluations for general-purpose models are likely to divert resources from more tractable risks unless tied to demonstrated capability pathways.
  • Editorial inference (not in paper): The paper's whole-chain framing suggests a concrete evaluation design: measure an AI model's contribution at each biorisk-chain stage (materials acquisition, synthesis, culturing, dispersal) with internet-only control arms, rather than relying on a single composite score.
  • Editorial inference (not in paper): The marginal-risk baseline could be tested directly by comparing LLM responses against top search-engine results on dual-use protocols; the paper implies this comparison but does not run it.
  • Editorial inference (not in paper): For biological tools, a stronger falsification target would be a demonstration that a model trained on biological sequences proposes a novel pathogen design that expert reviewers cannot distinguish from experimentally validated designs; absent that, the paper's limited-risk conclusion stands.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This paper reviews the publicly available evidence for two AI-biorisk threat models: (1) that LLMs uplift access to biological information and planning, and (2) that AI-enabled biological tools (BTs) uplift the ability to synthesize harmful biological artifacts. It compares the major red-team studies, discusses capability and data limitations of BTs, and concludes that existing studies are nascent, often speculative, and methodologically limited, and that the available literature suggests current LLMs and BTs do not pose an immediate risk. The paper closes with recommendations for whole-chain risk analysis, a focus on specialized biological models, and more rigorous, higher-validity empirical evaluations.

Significance. If the conclusion holds, the paper provides a useful corrective to policy and media narratives that treat current LLMs and BTs as imminent biorisk amplifiers, and its recommendations for whole-chain, baseline-controlled evaluation are sensible. The review is valuable for assembling and comparing the red-team studies in Table 1 and for emphasizing the distinction between information access and the full biorisk chain. The paper is carefully hedged in several places and explicitly notes that null findings do not rule out future, more capable models. Its main weakness is that the central inference from non-significant red-team results to 'no meaningful uplift' depends on the sensitivity of instruments that are not shown to be sensitive, and the BT conclusion leans on absence of known misuse rather than on a direct test.

major comments (3)
  1. [§3.1.2, Table 1] The paper converts non-significant red-team results into the positive claim in §3.1.3 that information access via current, publicly available LLMs does not meaningfully increase risk. This inference requires the red-team instruments to be sensitive enough to detect a meaningful uplift if one existed, but no power analysis, effect-size justification, or validation of the scoring rubrics is provided for Mouton et al. (2024) or OpenAI (2024a). With sample sizes of 45 and 100 and non-public rubrics, non-significance alone is weak evidence of absence; the manuscript should either supply a sensitivity or equivalence analysis or reframe the conclusion as 'no statistically significant uplift detected in exploratory, likely underpowered studies.'
  2. [§3.1.2, Table 1] The table caption states that 'All studies which compare uplift to internet access find a non-significant increase in risk,' yet the Anthropic row reports 'Minor uplift (unclear of statistical significance).' A 'minor uplift' with unreported significance is not established as a non-significant finding, so the caption and surrounding text overstate the consistency of the null results and should be revised.
  3. [§3.2.3] The assessment that BTs 'present limited risk' rests in part on 'no known examples of current AI biological tools being misused to cause real-world harm' as well as on capability limitations. The absence of documented misuse is weak evidence, particularly because the paper itself notes that no direct empirical misuse studies and no whole-chain analyses exist. The conclusion should be explicitly framed as an evidence-limited statement—for example, 'there is currently no direct evidence of BT misuse and capability limitations suggest limited uplift'—rather than a firm 'limited risk' claim.
minor comments (4)
  1. [§3.1.2] The sentence beginning 'important to note that the studies reviewed here highlight their own methodological limitations' is a sentence fragment and should be attached to the preceding discussion.
  2. [§4] The OpenAI o1 system card is cited as evidence for the conclusion that the model poses 'limited risk'; because this is a developer self-assessment, the manuscript should flag it as such and ideally compare it with independent evaluations.
  3. [References] Some reference entries contain OCR or transcription errors; for example, the Lentzos et al. entry begins 'he urgent need for an overhaul of global biorisk management' and should read 'The urgent need...'.
  4. [§3.2.3] The phrase 'no known examples of current AI biological tools being misused' should be dated explicitly (e.g., 'as of January 2025') so that the claim is not misread as a permanent or absolute absence.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the review's conclusion is a qualitative synthesis of external red-team studies and capability assessments, with only peripheral self-citations.

full rationale

This is a qualitative literature review, not a formal derivation. The central claims—that current LLMs and BTs do not meaningfully uplift biorisk and that existing studies are nascent—are inferred from externally produced red-team studies (Mouton et al. 2024; OpenAI 2024a; Anthropic 2024), capability critiques of AlphaFold and other biological tools, and barrier analyses. None of these inputs is defined in terms of the conclusion, and no parameter is fitted to a subset of data and then renamed as a prediction. The paper explicitly flags the limitations of the evidence: 'the studies reviewed here highlight their own methodological limitations, for example the relatively limited sample sizes' and notes that Soice et al. and Mouton et al. 'position findings as exploratory, rather than conclusive.' The most plausible concern—that non-significant red-team results are read as evidence of no uplift without a formal sensitivity or power analysis—is a statistical-validity objection, not circularity; it attacks the evidential weight of external studies, not a construction in which the output is equivalent to the input. Self-citations (Hooker 2024; Kapoor et al. 2024; Reuel et al. 2024) appear only in supporting roles—defining marginal risk, discussing compute-threshold limitations, or recommending more technical work—and none is load-bearing for the biorisk conclusion. Therefore no circular step is identifiable, and the score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters or invented entities appear because this is a qualitative review. The axioms listed are framing assumptions inherited from the cited biorisk and marginal-risk literature; they are not derived within the paper, and the paper's recommendations depend on them.

assumptions (4)
  • domain assumption The biorisk chain model (intent, idea, data, artifact, cultivation, dispersal) is a valid decomposition of biological attack pathways.
    Used throughout Sections 2 and 3 to argue that information access alone is insufficient and that whole-chain analysis is needed. If this decomposition is wrong, the recommendations lose much of their force.
  • domain assumption Marginal risk, defined as uplift relative to internet access, is the correct baseline for assessing AI biorisk.
    Invoked in Section 3.1.2 via NTIA and Kapoor et al. This baseline drives the interpretation of red-team results as non-significant and is central to the paper's conclusion about current LLMs.
  • domain assumption Publicly available evidence surveyed is representative of all relevant AI-biorisk research.
    The paper explicitly limits itself to publicly available evidence and draws conclusions about the 'available literature.' If important non-public or gray-literature studies exist, the conclusions could change.
  • domain assumption Absence of known misuse of biological tools is informative about current risk.
    Section 3.2.3 uses 'no known examples' plus capability limitations to conclude limited risk. This treats absence of evidence as some evidence of low risk, which is a load-bearing interpretive step.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Reality of AI and Biorisk." pith.science (2026). https://pith.science/paper/2YWJOFD3

@misc{pith2026241201946,
  author       = {Pith},
  title        = {Pith review of: The Reality of AI and Biorisk},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2YWJOFD3}},
  note         = {Machine review of arXiv:2412.01946}
}
read the original abstract

To accurately and confidently answer the question 'could an AI model or system increase biorisk', it is necessary to have both a sound theoretical threat model for how AI models or systems could increase biorisk and a robust method for testing that threat model. This paper provides an analysis of existing available research surrounding two AI and biorisk threat models: 1) access to information and planning via large language models (LLMs), and 2) the use of AI-enabled biological tools (BTs) in synthesizing novel biological artifacts. We find that existing studies around AI-related biorisk are nascent, often speculative in nature, or limited in terms of their methodological maturity and transparency. The available literature suggests that current LLMs and BTs do not pose an immediate risk, and more work is needed to develop rigorous approaches to understanding how future models could increase biorisks. We end with recommendations about how empirical work can be expanded to more precisely target biorisk and ensure rigor and validity of findings.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Preliminary suggestions for rigorous GPAI model evaluations

    cs.CY 2025-07 conditional novelty 4.0 of 10

    A RAND team turned a 64-paper literature review into a preliminary best-practice checklist for rigorously evaluating general-purpose AI models.

  2. Forbidden Science: Dual-Use AI Challenge Benchmark and Scientific Refusal Tests

    cs.CL 2025-02 reject novelty 4.0 of 10

    A new 512-prompt benchmark claims to measure LLM over-refusal on scientific dual-use questions, but its design and labeling flaws undermine the claim.

Reference graph

Works this paper leans on

76 extracted references · 55 canonical work pages · cited by 2 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Ballard, Joshua Bambrick, Sebastian W

    Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew J. Ballard, Joshua Bambrick, Sebastian W. Bodenstein, David A. Evans, Chia-Chun Hung, Michael O’Neill, David Reiman, Kathryn Tunyasuvunakool, Zachary Wu, Akvilė Žemgulytė, Eirini Arvaniti, Charles Beattie, Ottavia Bertolli, Alex...

    2024

  3. [3]

    Advanced technology: Examining threats to national security, 2023

    Gregory C Allen. Advanced technology: Examining threats to national security, 2023. URL https://www.hsgac.senate.gov/wp-content/uploads/Allen-Testimony.pdf

    work page 2023

  4. [4]

    Preparing the federal response to advanced technologies, 2023

    Jeff Alstott. Preparing the federal response to advanced technologies, 2023. URL https://www.hsgac.senate.gov/wp-content/uploads/Alstott-Testimony.pdf

    work page 2023

  5. [5]

    Frontier threats red teaming for AI safety, 2023

    Anthropic . Frontier threats red teaming for AI safety, 2023. URL https://www.anthropic.com/news/frontier-threats-red-teaming-for-ai-safety

    work page 2023

  6. [6]

    Claude 3 model card, 2024

    Anthropic . Claude 3 model card, 2024. URL https://docs.anthropic.com/en/docs/resources/model-card

    work page 2024

  7. [7]

    Humphreys, Qian Cong, David Baker, and Frank DiMaio

    Minkyung Baek, Ivan Anishchenko, Ian R. Humphreys, Qian Cong, David Baker, and Frank DiMaio . Efficient and accurate prediction of protein structure using RoseTTAFold 2, 2023. URL http://biorxiv.org/lookup/doi/10.1101/2023.05.24.542179

    doi:10.1101/2023.05.24.542179 2023

  8. [8]

    AI and BioRisk : An explainer, 2023

    Steph Batalis. AI and BioRisk : An explainer, 2023. URL https://cset.georgetown.edu/publication/ai-and-biorisk-an-explainer/

    work page 2023

Show all 76 references
  1. [9]

    Could AI -designed proteins be weaponized? 2024

    Ewen Callaway. Could AI -designed proteins be weaponized? 2024. URL https://www.nature.com/articles/d41586-024-00699-0

    2024

  2. [10]

    Campbell, Jonathan Herington, and Andrew D

    Quintina L. Campbell, Jonathan Herington, and Andrew D. White. Censoring chemical data to mitigate dual use risk, 2023. URL http://arxiv.org/abs/2304.10510

    2023

  3. [11]

    risk, alignment, and guarding against doomsday scenarios

    Rocco Casagrande. Statement to the US senate AI insight forum on “risk, alignment, and guarding against doomsday scenarios", 2023. URL https://www.schumer.senate.gov/imo/media/doc/Rocco

    2023

  4. [12]

    Schneider, Andrew W

    Jun Cheng, Guido Novati, Joshua Pan, Clare Bycroft, Akvilė Žemgulytė, Taylor Applebaum, Alexander Pritzel, Lai Hong Wong, Michal Zielinski, Tobias Sargeant, Rosalia G. Schneider, Andrew W. Senior, John Jumper, Demis Hassabis, Pushmeet Kohli, and Žiga Avsec. Accurate proteome-w...

    2023 doi

  5. [13]

    Cummings, Kaitlin M

    Christopher L. Cummings, Kaitlin M. Volk, Anna A. Ulanova, Do Thuy Uyen Ha Lam, and Pei Rou Ng. Emerging biosecurity threats and responses: A review of published and gray literature. In Benjamin D. Trump, Marie-Valentine Florin, Edward Perkins, and Igor Linkov (eds.), Emerging...

    2021 doi

  6. [14]

    Aya expanse: Combining research breakthroughs for a new multilingual frontier, 2024

    John Dang, Shivalika Singh, Daniel D'souza, Arash Ahmadian, Alejandro Salamanca, Madeline Smith, Aidan Peppin, Sungjin Hong, Manoj Govindassamy, Terrence Zhao, Sandra Kublik, Meor Amer, Viraat Aryabumi, Jon Ander Campos, Yi-Chern Tan, Tom Kocmi, Florian Strub, Nathan Grinsztaj...

    2024 arXiv

  7. [15]

    Dauparas, I

    J. Dauparas, I. Anishchenko, N. Bennett, H. Bai, R. J. Ragotte, L. F. Milles, B. I. M. Wicky, A. Courbet, R. J. de Haas, N. Bethel, P. J. Y. Leung, T. F. Huddy, S. Pellock, D. Tischer, F. Chan, B. Koepnick, H. Nguyen, A. Kang, B. Sankaran, A. K. Bera, N. P. King, and D. Baker....

    2022 doi

  8. [16]

    Kevin M. Esvelt. Credible pandemic virus identification willtrigger the immediate proliferation of agents as lethal as nuclear devices, 2022. URL https://www.hsgac.senate.gov/wp-content/uploads/imo/media/doc/Esvelt

    2022

  9. [17]

    Synthetic biology: Applying engineering to biology, 2005

    European Commission . Synthetic biology: Applying engineering to biology, 2005

    2005

  10. [18]

    Regulation ( EU ) 2024/1689 of the European Parliament : Laying down harmonised rules on artificial intelligence ( Artificial Intelligence Act) , 2024

    European Parliament . Regulation ( EU ) 2024/1689 of the European Parliament : Laying down harmonised rules on artificial intelligence ( Artificial Intelligence Act) , 2024. URL http://data.europa.eu/eli/reg/2024/1689/oj/eng

    2024

  11. [19]

    AI in infectious diseases: The role of datasets

    Cesar de la Fuente-Nunez. AI in infectious diseases: The role of datasets. 2024. URL https://pmc.ncbi.nlm.nih.gov/articles/PMC11537278/

    2024

  12. [20]

    A national security insider does the math on the dangers of AI

    Lauren Goode. A national security insider does the math on the dangers of AI . 2024. ISSN 1059-1028. URL https://www.wired.com/story/jason-matheny-national-security-insider-dangers-of-ai/

    2024

  13. [21]

    AlphaFold 3 predicts the structure and interactions of all of life’s molecules, 2024

    Google DeepMind . AlphaFold 3 predicts the structure and interactions of all of life’s molecules, 2024. URL https://blog.google/technology/ai/google-deepmind-isomorphic-alphafold-3-ai-model/

    2024

  14. [22]

    Machine learning and deep learning in synthetic biology: Key architectures, applications, and challenges

    Manoj Kumar Goshisht. Machine learning and deep learning in synthetic biology: Key architectures, applications, and challenges. 2024. URL https://doi.org/10.1021/acsomega.3c05913

    2024 doi

  15. [23]

    Seven myths & realities about do-it-yourself biology, 2013

    Daniel Grushkin, Todd Kuiken, and Piers Millet. Seven myths & realities about do-it-yourself biology, 2013. URL https://www.wilsoncenter.org/sites/default/files/media/documents/publication/7_myths_final.pdf

    2013

  16. [24]

    Managing risks from AI -enabled biological tools, 2024

    John Halstead. Managing risks from AI -enabled biological tools, 2024. URL https://www.governance.ai/post/managing-risks-from-ai-enabled-biological-tools

    2024

  17. [25]

    Sofroniew, Deniz Oktay, Zeming Lin, Robert Verkuil, Vincent Q

    Thomas Hayes, Roshan Rao, Halil Akin, Nicholas J. Sofroniew, Deniz Oktay, Zeming Lin, Robert Verkuil, Vincent Q. Tran, Jonathan Deaton, Marius Wiggert, Rohil Badkundri, Irhum Shafkat, Jun Gong, Alexander Derry, Raul S. Molina, Neil Thomas, Yousuf Khan, Chetan Mishra, Carolyn K...

    2024 doi

  18. [26]

    On the limitations of compute thresholds as a governance strategy, 2024

    Sara Hooker. On the limitations of compute thresholds as a governance strategy, 2024. URL http://arxiv.org/abs/2407.05694

    2024 arXiv

  19. [27]

    Takashi Inagaki, Akari Kato, Koichi Takahashi, Haruka Ozaki, and Genki N. Kanda. LLMs can generate robotic scripts from goal-oriented instructions in biological laboratory automation, 2023. URL http://arxiv.org/abs/2304.10267

    2023 arXiv

  20. [28]

    Synthetic biology and biosecurity: Challenging the “myths”

    Catherine Jefferson, Filippa Lentzos, and Claire Marris. Synthetic biology and biosecurity: Challenging the “myths”. 2014. URL https://pmc.ncbi.nlm.nih.gov/articles/PMC4139924/

    2014

  21. [29]

    Carter, Nazish Alexanian, Oliver Crook, Samuel Curtis, Richard Moulange, Shrestha Rath, Sophie Rose, and Jennifer Clark

    Nazish Jeffery, Sarah R. Carter, Nazish Alexanian, Oliver Crook, Samuel Curtis, Richard Moulange, Shrestha Rath, Sophie Rose, and Jennifer Clark. Bio x AI : Policy recommendations for a new frontier, 2023. URL https://fas.org/publication/bio-x-ai-policy-recommendations/

    2023

  22. [30]

    Jeong, Saurabh Garg, Zachary C

    Daniel P. Jeong, Saurabh Garg, Zachary C. Lipton, and Michael Oberst. Medical adaptation of large language and vision-language models: Are we making progress?, 2024. URL https://arxiv.org/abs/2411.04118

    2024 arXiv

  23. [31]

    John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, Alex Bridgland, Clemens Meyer, Simon A. A. Kohl, Andrew J. Ballard, Andrew Cowie, Bernardino Romera-Paredes, Stani...

    2021

  24. [32]

    David E. Kaplan. The cult at the end of the world. 1996. URL https://www.wired.com/1996/07/aum/

    1996

  25. [33]

    Ho, Percy Liang, and Arvind Narayanan

    Sayash Kapoor, Rishi Bommasani, Kevin Klyman, Shayne Longpre, Ashwin Ramaswami, Peter Cihon, Aspen Hopkins, Kevin Bankston, Stella Biderman, Miranda Bogen, Rumman Chowdhury, Alex Engler, Peter Henderson, Yacine Jernite, Seth Lazar, Stefano Maffulli, Alondra Nelson, Joelle Pine...

    2024 arXiv

  26. [34]

    Masha Karelina, Joseph Noh, and Ron O. Dror. How accurately can one predict drug binding modes using AlphaFold models?, 2023. URL https://www.biorxiv.org/content/10.1101/2023.05.18.541346v1

    2023 doi

  27. [35]

    Zilinskas, and Jens H

    Milton Leitenberg, Raymond A. Zilinskas, and Jens H. Kuhn. The Soviet Biological Weapons Program: A History. Harvard University Press, 2012. URL https://www.jstor.org/stable/j.ctt2jbscf

    2012

  28. [36]

    Koblentz, and J Rodgers

    F Lentzos, G.D. Koblentz, and J Rodgers. he urgent need for an overhaul of global biorisk management. CTC Sentinel., 15 0 (4), 2022. URL https://ctc.westpoint.edu/wp-content/uploads/2022/04/CTC-SENTINEL-042022.pdf

    2022

  29. [37]

    The data provenance initiative: A large scale audit of dataset licensing and attribution in ai, 2023

    Shayne Longpre, Robert Mahari, Anthony Chen, Naana Obeng-Marnu, Damien Sileo, William Brannon, Niklas Muennighoff, Nathan Khazam, Jad Kabbara, Kartik Perisetla, Xinyi Wu, Enrico Shippole, Kurt Bollacker, Tongshuang Wu, Luis Villa, Sandy Pentland, and Sara Hooker. The data prov...

    2023 arXiv

  30. [38]

    Lee, Campbell S

    Shayne Longpre, Robert Mahari, Ariel N. Lee, Campbell S. Lund, Hamidah Oderinwale, William Brannon, Nayan Saxena, Naana Obeng-Marnu, Tobin South, Cole J Hunter, Kevin Klyman, Christopher Klamm, Hailey Schoelkopf, Nikhil Singh, Manuel Cherep, Ahmad Mustafa Anis, An Dinh, Caroli...

    2024

  31. [39]

    AI -powered biological warfare is ‘biggest issue,’ former Google exec warns , 2022

    Ryan Lovelace. AI -powered biological warfare is ‘biggest issue,’ former Google exec warns , 2022. URL https://www.washingtontimes.com/news/2022/sep/12/ai-powered-biological-warfare-biggest-issue-former/

    2022

  32. [40]

    Biological sequence models in the context of the AI directives, 2024

    Nicole Maug. Biological sequence models in the context of the AI directives, 2024. URL https://epochai.org/blog/biological-sequence-models-in-the-context-of-the-ai-directives

    2024

  33. [41]

    Dozens of Top Scientists Sign Effort to Prevent A.I

    Cade Metz. Dozens of Top Scientists Sign Effort to Prevent A.I. Bioweapons . 2024. URL https://www.nytimes.com/2024/03/08/technology/biologists-ai-agreement-bioweapons.html

    2024

  34. [42]

    Towards responsible governance of biological design tools, 2023

    Richard Moulange, Max Langenkamp, Tessa Alexanian, Samuel Curtis, and Morgan Livingston. Towards responsible governance of biological design tools, 2023. URL http://arxiv.org/abs/2311.15936

    2023 arXiv

  35. [43]

    Mouton, Caleb Lucas, and Ella Guest

    Christopher A. Mouton, Caleb Lucas, and Ella Guest. The operational risks of AI in large-scale biological attacks: Results of a red-team study, 2024. URL https://www.rand.org/pubs/research_reports/RRA2977-2.html

    2024

  36. [44]

    Written testimony for Senate Homeland Security and Governmental Affairs Subcommittee on Emerging Threats and Spending Oversight hearing on Advanced Technology , 2023

    Dewey Murdoch. Written testimony for Senate Homeland Security and Governmental Affairs Subcommittee on Emerging Threats and Spending Oversight hearing on Advanced Technology , 2023. URL https://www.hsgac.senate.gov/wp-content/uploads/Murdick-Testimony.pdf

    2023

  37. [45]

    Biodefense in the Age of Synthetic Biology

    National Academies of Sciences, Engineering, and Medicine . Biodefense in the Age of Synthetic Biology. The National Academies Press, Washington, DC, 2018. ISBN 978-0-309-46518-2. doi:10.17226/24890. URL https://nap.nationalacademies.org/catalog/24890/biodefense-in-the-age-of-...

    2018 doi

  38. [46]

    Biotechnology Research in an Age of Terrorism

    National Academies Press . Biotechnology Research in an Age of Terrorism. National Academies Press, 2004. URL http://www.nap.edu/catalog/10827

    2004

  39. [47]

    Evidence-based laboatory biorisk management - science and technology roadmap

    National Science and Technology Council . Evidence-based laboatory biorisk management - science and technology roadmap. Health Security Threats Subcommittee, 2022. URL https://www.whitehouse.gov/wp-content/uploads/2022/04/04-2022-NSTC-ST-Biorisk-Research-Roadmap_FINAL.pdf

    2022

  40. [48]

    Dual-use foundation models with widely available model weights, 2024

    National Telecommunications and Information Administration . Dual-use foundation models with widely available model weights, 2024. URL https://www.ntia.gov/sites/default/files/publications/ntia-ai-open-model-report.pdf

    2024

  41. [49]

    Baccus, and Chris Ré

    Eric Nguyen, Michael Poli, Marjan Faizi, Armin Thomas, Callum Birch-Sykes, Michael Wornow, Aman Patel, Clayton Rabideau, Stefano Massaroli, Yoshua Bengio, Stefano Ermon, Stephen A. Baccus, and Chris Ré. HyenaDNA : Long-range genomic sequence modeling at single nucleotide resol...

    2023 arXiv

  42. [50]

    Managing misuse risk for dual-use foundation models, 2024

    Gaithersburg Md NIST. Managing misuse risk for dual-use foundation models, 2024. URL https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.800-1.ipd.pdf

    2024

  43. [51]

    White paper 3: Risks of AIxBio , 2024

    NSCEB . White paper 3: Risks of AIxBio , 2024. URL https://www.biotech.senate.gov/wp-content/uploads/2024/01/NSCEB_AIxBio_WP3_Risks.pdf

    2024

  44. [52]

    Building an early warning system for LLM -aided biological threat creation, 2024 a

    OpenAI . Building an early warning system for LLM -aided biological threat creation, 2024 a . URL https://openai.com/index/building-an-early-warning-system-for-llm-aided-biological-threat-creation/

    2024

  45. [53]

    OpenAI and Los Alamos National Laboratory announce bioscience research partnership , 2024 b

    OpenAI . OpenAI and Los Alamos National Laboratory announce bioscience research partnership , 2024 b . URL https://openai.com/index/openai-and-los-alamos-national-laboratory-work-together/

    2024

  46. [54]

    o1 system card, 2024 c

    OpenAI . o1 system card, 2024 c . URL https://openai.com/index/openai-o1-system-card/

    2024

  47. [55]

    B arriers to B ioweapons

    Sonia Ben Ouagrham-Gormley. B arriers to B ioweapons . Cornell Press, 2014. URL https://www.cornellpress.cornell.edu/book/9780801452888/barriers-to-bioweapons/#bookTabs=4 . [Accessed 29-11-2024]

    2014

  48. [56]

    Read, Edward N

    Randy J. Read, Edward N. Baker, Charles S. Bond, Elspeth F. Garman, and Mark J. van Raaij. AlphaFold and the future of structural biology. 2023. URL https://pmc.ncbi.nlm.nih.gov/articles/PMC10324484/

    2023

  49. [57]

    Kochenderfer, and Robert Trager

    Anka Reuel, Ben Bucknall, Stephen Casper, Tim Fist, Lisa Soder, Onni Aarne, Lewis Hammond, Lujain Ibrahim, Alan Chan, Peter Wills, Markus Anderljung, Ben Garfinkel, Lennart Heim, Andrew Trask, Gabriel Mukobi, Rylan Schaeffer, Mauricio Baker, Sara Hooker, Irene Solaiman, Alexan...

    2024 arXiv

  50. [58]

    US senators express bipartisan alarm about AI , focusing on biological attack, 2023

    Reuters . US senators express bipartisan alarm about AI , focusing on biological attack, 2023. URL https://www.reuters.com/technology/us-senators-express-bipartisan-alarm-about-ai-focusing-biological-attack-2023-07-25/

    2023

  51. [59]

    Towards quantifying the risk of LLMs raising lone-wolf bioterrorism (forthcoming)

    Luca Righetti. Towards quantifying the risk of LLMs raising lone-wolf bioterrorism (forthcoming). 2024

    2024

  52. [60]

    The near-term impact of AI on biological misuse, 2024

    Sophie Rose, Richard Moulange, James Smith, and Cassidy Nelson. The near-term impact of AI on biological misuse, 2024. URL https://www.longtermresilience.org/wp-content/uploads/2024/07/CLTR-Report-The-near-term-impact-of-AI-on-biological-misuse-July-2024-1.pdf

    2024

  53. [61]

    Who should we fear more: Biohackers, disgruntled postdocs, or bad governments? a simple risk chain model of biorisk

    Anders Sandberg and Cassidy Nelson. Who should we fear more: Biohackers, disgruntled postdocs, or bad governments? a simple risk chain model of biorisk. 2020. URL https://pmc.ncbi.nlm.nih.gov/articles/PMC7310205/

    2020

  54. [62]

    Shivalika Singh, Freddie Vargus, Daniel Dsouza, Börje F. Karlsson, Abinaya Mahendiran, Wei-Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura OMahony, Mike Zhang, Ramith Hettiarachchi, Joseph Wilson, Marina Machado, Luisa Souza Moura, Dominik Krzemiński, Hakimeh F...

    2024 arXiv

  55. [63]

    Soice, Rafael Rocha, Kimberlee Cordova, Michael Specter, and Kevin M

    Emily H. Soice, Rafael Rocha, Kimberlee Cordova, Michael Specter, and Kevin M. Esvelt. Can large language models democratize access to dual-use biotechnology?, 2023. URL http://arxiv.org/abs/2306.03809

    2023 arXiv

  56. [64]

    Terwilliger, Dorothee Liebschner, Tristan I

    Thomas C. Terwilliger, Dorothee Liebschner, Tristan I. Croll, Christopher J. Williams, Airlie J. McCoy , Billy K. Poon, Pavel V. Afonine, Robert D. Oeffner, Jane S. Richardson, Randy J. Read, and Paul D. Adams. AlphaFold predictions are valuable hypotheses, and accelerate but ...

    2023 doi

  57. [65]

    Science in the age of AI , 2024

    The Royal Society . Science in the age of AI , 2024. URL https://royalsociety.org/-/media/policy/projects/science-in-the-age-of-ai/science-in-the-age-of-ai-report.pdf

    2024

  58. [66]

    Executive order on the safe, secure, and trustworthy development and use of artificial intelligence, 2023

    The White House . Executive order on the safe, secure, and trustworthy development and use of artificial intelligence, 2023. URL https://www.whitehouse.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-a...

    2023

  59. [67]

    de Brevern

    Sébastien Tourlet, Ragousandirane Radjasandirane, Julien Diharce, and Alexandre G. de Brevern. AlphaFold 2 update and perspectives. 2023. URL https://www.mdpi.com/2673-7426/3/2/25

    2023

  60. [68]

    AI safety institute approach to evaluations, 2024

    UK AI Safety Institute . AI safety institute approach to evaluations, 2024. URL https://www.gov.uk/government/publications/ai-safety-institute-approach-to-evaluations/ai-safety-institute-approach-to-evaluations

    2024

  61. [69]

    The Bletchley Declaration by Countries Attending the AI Safety Summit, 1-2 November 2023 , 2023

    UK Government . The Bletchley Declaration by Countries Attending the AI Safety Summit, 1-2 November 2023 , 2023. URL https://www.gov.uk/government/publications/ai-safety-summit-2023-the-bletchley-declaration/the-bletchley-declaration-by-countries-attending-the-ai-safety-summit...

    2023

  62. [70]

    New commitment to deepen work on severe AI risks concludes AI seoul summit, 2024

    UK Government . New commitment to deepen work on severe AI risks concludes AI seoul summit, 2024. URL https://www.gov.uk/government/news/new-commitmentto-deepen-work-on-severe-ai-risks-concludes-ai-seoul-summit

    2024

  63. [71]

    History of the biological weapons convention

    United Nations . History of the biological weapons convention. URL https://disarmament.unoda.org/biological-weapons/about/history/

  64. [72]

    Amerithrax investigative summary

    The U nited S tates D epartment of J ustice. Amerithrax investigative summary. 2010. URL https://www.justice.gov/archive/amerithrax/docs/amx-investigative-summary.pdf

    2010

  65. [73]

    Biosafety levels, 2015

    US Department of Health and Human Sciences . Biosafety levels, 2015. URL https://www.phe.gov/s3/BioriskManagement/biosafety/Pages/Biosafety-Levels.aspx

    2015

  66. [74]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2023. URL http://arxiv.org/abs/1706.03762

    2023 arXiv

  67. [75]

    What is dual-use research of concern?, 2020

    World Health Organisation . What is dual-use research of concern?, 2020. URL https://www.who.int/news-room/questions-and-answers/item/what-is-dual-use-research-of-concern

    2020

  68. [76]

    Specialized foundation models struggle to beat supervised baselines, 2024

    Zongzhe Xu, Ritvik Gupta, Wenduo Cheng, Alexander Shen, Junhong Shen, Ameet Talwalkar, and Mikhail Khodak. Specialized foundation models struggle to beat supervised baselines, 2024. URL https://arxiv.org/abs/2411.02796

    2024 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.