Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectives
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:IALG2JYVrecord.jsonopen to challenge →
read the original abstract
Recent decisions by leading AI labs to either open-source their models or to restrict access to their models has sparked debate about whether, and how, increasingly capable AI models should be shared. Open-sourcing in AI typically refers to making model architecture and weights freely and publicly accessible for anyone to modify, study, build on, and use. This offers advantages such as enabling external oversight, accelerating progress, and decentralizing control over AI development and use. However, it also presents a growing potential for misuse and unintended consequences. This paper offers an examination of the risks and benefits of open-sourcing highly capable foundation models. While open-sourcing has historically provided substantial net benefits for most software and AI development processes, we argue that for some highly capable foundation models likely to be developed in the near future, open-sourcing may pose sufficiently extreme risks to outweigh the benefits. In such a case, highly capable foundation models should not be open-sourced, at least not initially. Alternative strategies, including non-open-source model sharing options, are explored. The paper concludes with recommendations for developers, standard-setting bodies, and governments for establishing safe and responsible model sharing practices and preserving open-source benefits where safe.
This paper has not been read by Pith yet.
Forward citations
Cited by 7 Pith papers
-
Toward Open Weight Models Without Risks: Separating Public and Private Capabilities in LLMs
Tiered Language Models use a secret key to induce an alternative computation graph over shared weights, enabling private capabilities in the keyed mode while the public mode shows none.
-
Next-Billion AI Index: The compass for AI utility and adoption in the global majority
Introduces nexbax, a diagnostic framework with three themes and 10 dimensions for evaluating AI economic viability, operational practicality, and societal integrity in next-billion-user contexts.
-
Open AI in the Wild: Adoption and Adaptation of Open Models on r/LocalLLaMA
Thematic analysis of r/LocalLLaMA discussions finds users define openness via reliability, local control, privacy, and adaptation under compute, licensing, and usability constraints.
-
A governance horizon for ethical-use constraints in open-weight AI models
Ethical constraint evidence on open-weight AI models decays with a half-life of 1.31 derivation steps on Hugging Face, creating a governance horizon at seven generations where 80% of models lack traceable information.
-
Why Open Source? A Game-Theoretic Analysis of the AI Race
A game-theoretic R&D race model shows that pure Nash equilibria for open-sourcing decisions exist and are computationally tractable in both discrete and continuous settings.
-
Strategic commitments shape collective cybersecurity under AI inequality
Subsidized commitment by a small group of defenders in an evolutionary game model significantly increases strong defense adoption, suppresses attacks, and improves system resilience under AI access inequality.
-
Strategic commitments shape collective cybersecurity under AI inequality
Targeted subsidies for committed defenders in an evolutionary game model of AI-unequal cybersecurity significantly increase strong defense adoption, suppress attacks, and enhance overall resilience.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.