REVIEW 2 major objections 4 minor 3 cited by
Stop treating `AGI' as the north-star goal of AI research
T0 review · 2 major / 4 minor · reviewed 2026-08-09 · deepseek-v4-flash
Pith's one-line read This paper argues that the contested term 'AGI' should be abandoned as the guiding north-star of AI research, because it undermines goal-setting and amplifies six research traps.
desk verdict Useful trap taxonomy and honest treatment of counterarguments, but the 'drop AGI' conclusion is stronger than the evidence—the paper's own §4.1 concedes the middle ground. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central machinery is a diagnostic framework of six 'traps' — obstacles to productive goal-setting — drawn from prior documented problems in AI research (underspecification, SOTA-chasing, exploratory-confirmatory confusion, value-ladenness, exclusion, technical debt). For each trap, the paper first establishes the problem through existing work and then argues that AGI discourse amplifies it. The 'north-star' metaphor (an astronomical guide used for navigation) supplies the target of the critique: instead of one guiding star, the authors argue for a pluralistic constellation of specific goals. The traps do the argumentative work, and the three recommendations directly answer them: specificity counteracts consensus and bad science, pluralism counteracts the goal lottery and exclusion, and inclusion counteracts normalized exclusion.
What would settle it
A study comparing subfields that routinely use 'AGI' framing with matched subfields that state specific technical or societal goals could falsify the causal claim: if the specific-goal subfields show no better goal-setting (measured by hypothesis clarity, external validity, and community diversity), the proposed remedy would fail. A simpler check is longitudinal — whether papers using 'AGI' in their framing exhibit more underspecification, such as missing hypotheses or post-hoc evaluations, than matched papers with concrete goals.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that 'AGI' functions less as a research target and more as an ambiguous symbol: a shared word that conceals deep disagreements about what the field is for. The authors argue that framing AI research as a march toward AGI aggravates six traps: Illusion of Consensus (a familiar term masks contested goals), Supercharging Bad Science (vague concepts worsen underspecification, conflate science with engineering, and blur confirmatory and exploratory work), Presuming Value-Neutrality (seemingly technical definitions quietly embed political and ethical choices), Goal Lottery (goals are adopted because of incentives and luck rather than merit), Generality Debt (appeals to generality postpone hard decisions about what to build and for whom), and Normalized Exclusion (grand AGI narratives sideline the disciplines and communities who should shape goals). The authors conclude that the AI research community should abandon AGI as a north-star goal and adopt three remedies — specificity, pluralism, and inclusion — and they concede that they cannot rule out that improved AGI accounts could avoid these traps, while arguing such a modified pursuit still conflicts with their recommendations, with the cultural hype of the term, and with a focus on benefiting human beings.
Load-bearing premise
The paper assumes that AGI discourse actively causes or aggravates the six problems it names, rather than merely reflecting deeper structural forces such as funding incentives and industry concentration that would persist even if the term 'AGI' were abandoned.
Editorial extensions
If this is right
- Research leaders and funders would orient toward concrete, measurable goals, such as specific deployment contexts or clearly scoped benchmarks, instead of 'human-level' general intelligence.
- Evaluation practice would shift toward explicit hypotheses, external validity, and a clearer separation of confirmatory and exploratory claims.
- The community would deliberately sustain multiple research agendas and spread resources across them, rather than concentrating on a single grand objective.
- Goal-setting would involve more disciplines and affected communities, changing both the questions asked and who gets to ask them.
- The term 'AGI' would lose its role as the default justification for large compute investments and hype-prone claims.
Reading between the lines
- If AGI talk is mostly a symptom of deeper structural forces — funding incentives, industry concentration, media cycles — then dropping the term may not by itself fix the six traps; a causal comparison of subfields that use AGI framing against those that do not would test this.
- The same north-star critique likely applies to other sweeping goals such as 'transformative AI' or 'superintelligence,' a direction the authors themselves gesture toward.
- One testable consequence of the paper's argument: papers that frame their contribution as progress toward AGI should show measurably more underspecification — for example, missing hypotheses or post-hoc evaluations — than matched papers with specific goals.
- Implementing the recommendations would imply observable changes in resource distribution across research topics, which could be tracked through funding and publication data over the coming years.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This position paper argues that the AI research community should stop treating 'AGI' as the north-star goal of AI research. It identifies six 'traps' that AGI discourse allegedly aggravates (Illusion of Consensus, Supercharging Bad Science, Presuming Value-Neutrality, Goal Lottery, Generality Debt, Normalized Exclusion) and proposes three recommendations: goal specificity, pluralism of goals and approaches, and greater inclusion in goal setting. The paper then rebuts the alternative view that improved definitions of AGI could avoid these traps, offering three reasons: conflict with the recommendations, the cultural baggage of AGI undermining hype-vs-reality distinctions, and the alternative goal of benefiting humans.
Significance. If the paper's central claim were fully established, it would have practical import for how AI research goals are framed by major labs, funding agencies, and policy bodies. The six-traps framework is a useful organizing device, and the paper benefits from a wide-ranging and current bibliography, careful engagement with at least one strong counterargument (Morris et al. 2024), and unusually transparent author-contribution details. However, the paper is a position piece whose normative conclusion rests on empirical and causal claims about the effects of AGI discourse that are asserted more than demonstrated. The most important strength is the honest caveat in Section 4.1, which at the same time exposes the gap between the premises and the categorical conclusion.
major comments (2)
- [§4.1, §4.2, footnote 9] The paper's own Section 4.1 concedes that 'we cannot rule out the possibility of efforts that mitigate these same problems while retaining AGI as a goal,' and footnote 9 explicitly labels the reasons against improved definitions as pro-tanto rather than all-things-considered. The central conclusion that the community must stop treating AGI as a north-star therefore requires the empirical claim in Section 4.2, Reason 2: that 'no matter how well or poorly defined, AGI has acquired a cultural significance' that irreparably exacerbates hype. That claim is asserted with examples but not with systematic evidence. The six traps are about underspecified, value-neutral, non-pluralistic, and exclusionary goal-setting, which could in principle be repaired by precise operational definitions such as Morris et al. (2024). As written, the strongest conclusion entailed by the paper's own premises is 'stop treating vague or current AGI discourse as the north-star goal,' not 'stop treating AGI as the north-star goal.' This is a load-bearing gap: either the conclusion should be narrowed, or Section 4.2 needs evidence or a structured argument for why the term's cultural associations are irreparable.
- [§2.2, §2.6] Throughout the paper, AGI discourse is described with causal language: it 'supercharges' bad science (§2.2), 'aggravates problems of exclusion' (§2.6), and 'intensifies' existing problems (§2.6). The cited evidence largely demonstrates problems in AI research and associations with AGI rhetoric, but not that AGI discourse is a significant cause rather than a correlate or symptom of deeper structural forces such as funding incentives and industry power. The policy recommendation depends on this causal direction: if AGI talk is epiphenomenal, dropping AGI as a north-star would not bring the promised benefits. The paper would be stronger if it explicitly acknowledged this limitation, for example by framing the conclusion as conditional on the causal claim, or by presenting a concrete mechanism and testable implications for how changes in discourse would alter research practices or funding decisions.
minor comments (4)
- [Appendix B] Yann LeCun's surname is misspelled as 'LeCunn' in the sentence beginning 'Speaking to TIME'; this should be corrected.
- [Throughout] The manuscript contains numerous spacing artifacts, such as 'Y et,' 'T echnology,' and 'V alue,' likely from typesetting or OCR. These should be cleaned up before final publication.
- [§2.1] The paper quotes Mueller (2024) as calling AGI 'a meaningless concept, an emperor with no clothes,' but does not engage with the nuances of that critique; a one-sentence gloss would help the reader understand why this strong dismissal is included alongside more measured accounts.
- [§3, Recommendation 1] The Whisper mixture-of-experts example illustrates goal specificity well, but it is not connected back to AGI discourse; a brief note explaining how this example contrasts with an AGI-framed goal would improve readability.
Circularity Check
No significant circularity; the six traps are independently evidenced and the paper's self-citations are not load-bearing.
full rationale
This is a position paper with no fitted parameters, equations, or data-derived predictions. The central claim that AGI discourse aggravates six obstacles to productive goal-setting is supported by a broad set of external sources (e.g., Summerfield 2023, Mueller 2024, Altmeyer et al. 2024, Morris et al. 2024, Raji et al. 2021, Herrmann et al. 2024) and by concrete examples, rather than by a formal derivation from the paper's own prior work. The cited earlier papers by the same authors (Blili-Hamelin et al. 2024; Blili-Hamelin and Hancox-Li 2023) are used to support claims that AGI definitions are contested and value-laden, but those claims are independently corroborated by the paper's Appendix A table of divergent definitions and by other cited critiques; removing the self-citations would not collapse the argument. No 'prediction' is fitted and then renamed as a discovery, no uniqueness theorem is imported from the authors' own prior work, and no ansatz is smuggled in via citation. The normative conclusion to stop treating AGI as a north-star goal is an argued recommendation rather than an empirical result derived from its inputs. The skeptical concern that Section 4.2 Reason 2 (the cultural irreparability of 'AGI') is asserted rather than proven is a matter of evidential support and underdetermination, not circularity. The derivation chain is therefore self-contained, and the circularity score is low.
Assumptions & free parameters
assumptions (5)
- domain assumption The AI research community has a shared responsibility to set goals and to distinguish hype from reality.
- domain assumption Pluralism of goals and approaches is beneficial for research progress.
- domain assumption Specificity in goal formulations reduces ambiguity and is desirable.
- domain assumption Inclusion of diverse communities and disciplines leads to better research outcomes.
- domain assumption AGI discourse causes or worsens the six identified traps, not merely correlates with them.
Cite this review
Pith. "Pith review of Stop treating `AGI' as the north-star goal of AI research." pith.science (2026). https://pith.science/paper/BGQZBHI7
@misc{pith2026250203689,
author = {Pith},
title = {Pith review of: Stop treating `AGI' as the north-star goal of AI research},
year = {2026},
howpublished = {\url{https://pith.science/paper/BGQZBHI7}},
note = {Machine review of arXiv:2502.03689}
}
read the original abstract
The AI research community plays a vital role in shaping the scientific, engineering, and societal goals of AI research. In this position paper, we argue that focusing on the highly contested topic of `artificial general intelligence' (`AGI') undermines our ability to choose effective goals. We identify six key traps -- obstacles to productive goal setting -- that are aggravated by AGI discourse: Illusion of Consensus, Supercharging Bad Science, Presuming Value-Neutrality, Goal Lottery, Generality Debt, and Normalized Exclusion. To avoid these traps, we argue that the AI research community needs to (1) prioritize specificity in engineering and societal goals, (2) center pluralism about multiple worthwhile approaches to multiple valuable goals, and (3) foster innovation through greater inclusion of disciplines and communities. Therefore, the AI research community needs to stop treating `AGI' as the north-star goal of AI research.
Forward citations
Cited by 3 Pith papers
-
Deep Hype in Artificial General Intelligence: Uncertainty, Sociotechnical Fictions and the Governance of AI Futures
AGI hype is reframed as 'deep hype': a long-term, unverifiable overpromissory dynamic sustained by sociotechnical fictions and venture capital speculation.
-
Is the `Agent' Paradigm a Limiting Framework for Next-Generation Intelligent Systems?
The paper argues that the agent paradigm in AI is conceptually ambiguous and anthropocentric, and that next-generation intelligence may be better pursued through system-level, world-model, and material-computing frameworks.
-
NLP Meets the World: Toward Improving Conversations With the Public About Natural Language Processing Research
NLP researchers should define cognitive terms, temper expectations, and candidly address ethical failures when speaking with the public.
Reference graph
Works this paper leans on
-
[1]
Hern´ andez-Orallo, J., Loe, B
URL https://www.sciencedirect.com/ science/article/pii/S1389041713000338. Hern´ andez-Orallo, J., Loe, B. S., Cheke, L., Mart´ ınez- Plumed, F., and ´O h ´Eigeartaigh, S. General intelligence disentangled via a generality metric for natural and ar- tificial intelligence. Scientific Reports , 11(1):22822, November 2021. ISSN 2045-2322. doi: 10.1038/ s41598-0...
-
[2]
doi: 10.1145/3531146.3533158. URL https://dl. acm.org/doi/10.1145/3531146.3533158. Rastogi, C., Stelmakh, I., Shen, X., Meila, M., Echenique, F., Chawla, S., and Shah, N. B. To ArXiv or not to ArXiv: A Study Quantifying Pros and Cons of Posting Preprints Online, June 2022. URL http://arxiv.org/abs/ 2203.17259. arXiv:2203.17259 [cs]. Rombach, R., Blattmann...
-
[4]
Publisher: Nature Publish- ing Group
URL https://www.nature.com/articles/ s41598-024-56648-4 . Publisher: Nature Publish- ing Group. Bertelsen, P . S., Bossen, C., Knudsen, C., and Pedersen, A. M. Data work and practices in healthcare: A scoping review. International Journal of Medical Informat- ics, 184:105348, April 2024. ISSN 1386-5056. doi: 10.1016/j.ijmedinf.2024.105348. URL https://www...
-
[5]
doi: 10.1145/3287560.3287577. URL https://dl. acm.org/doi/10.1145/3287560.3287577. Y oung, M., Ehsan, U., Singh, R., Tafesse, E., Gilman, M., Harrington, C., and Metcalf, J. Participa- tion versus scale: Tensions in the practical de- mands on participatory AI. First Monday , April
-
[6]
ISSN 0360-0300. doi: 10.1145/3502289. URL https://doi.org/10.1145/3502289. Cave, S. The Problem with Intelligence: Its V alue- Laden History and the Future of AI. In Proceed- ings of the AAAI/ACM Conference on AI, Ethics, and Society, AIES ’20, pp. 29–35, New Y ork, NY , USA, February 2020. Association for Computing Machinery. ISBN 978-1-4503-7110-0. doi:...
-
[7]
URL http://jmlr.org/papers/v23/20- 1335.html. DeepMind, G. About, 2025. URL https://deepmind. google/about/. [Online; accessed 19-May-2025]. Dehghani, M., Tay, Y ., Gritsenko, A. A., Zhao, Z., Houlsby, N., Diaz, F., Metzler, D., and Vinyals, O. The Bench- mark Lottery, July 2021. URL http://arxiv.org/ abs/2107.07002. Delgado, F., Y ang, S., Madaio, M., an...
arXiv 2025
-
[8]
URL https://doi.org/10.1007/978-94- 017-0475-5_8. Anderson, E. The Epistemology of Democracy. Epis- teme, 3(1-2):8–22, June 2006. ISSN 1750-0117, 1742-3600. doi: 10 . 3366 / epi . 2006 . 3 . 1 - 2 . 8. URL https://www.cambridge.org/ core/journals/episteme/article/ 10 Stop Treating ‘AGI’ as the North-star Goal of AI Research abs/epistemology-of-democracy/ ...
-
[9]
Dori-Hacohen, S., Montenegro, R
URL https://doi.org/10.1007/s11098- 021-01657-6. Dori-Hacohen, S., Montenegro, R. E., Murai, F., Hale, S. A., Sung, K., Blain, M., and Edwards-Johnson, J. Fair- ness via AI: Bias Reduction in Medical Information. In The 4th F AccTRec W orkshop on Responsible Recommen- dation at RecSys , 2021. Dotan, R. and Milli, S. V alue-laden disciplinary shifts in mac...
Show all 31 references
-
[11]
doi: 20240428092319000
ISSN 1396-0466. doi: 20240428092319000. URL https://firstmonday.org/ojs/index. php/fm/article/view/13636. Goertzel, B. Artificial general intelligence: concept, sta te of the art, and future prospects. Journal of Artificial General Intelligence , 5(1):1, 2014. URL https:// scien...
2014
- [13]
-
[17]
Kaack, L
URL https://www.nature.com/articles/ d41586-025-00110-6 . Kaack, L. H., Donti, P . L., Strubell, E., Kamiya, G., Creutzig, F., and Rolnick, D. Aligning artificial intel- ligence with climate change mitigation. Nature Climate Change, 12(6):518–527, 2022. Kelly, S. M. Elon Musk s...
2022 arXiv
-
[19]
org/10.1007/978-1-4615-4201-8_9
URL https://doi. org/10.1007/978-1-4615-4201-8_9 . Mikesell, L., Bromley, E., and Khodyakov, D. Eth- ical Community-Engaged Research: A Literature Review. American Journal of Public Health , 103 (12):e7–e14, December 2013. ISSN 0090-0036. doi: 10.2105/AJPH.2013.301605. URL htt...
2013
-
[20]
doi: 10.1126/ science
ISSN 0036-8075, 1095-9203. doi: 10.1126/ science . ado7069. URL https://www.science. org/doi/10.1126/science.ado7069. Morris, M. R., Sohl-Dickstein, J., Fiedel, N., Warkentin, T., Dafoe, A., Faust, A., Farabet, C., and Legg, S. Position: levels of AGI for operationalizing prog...
-
[25]
Publisher: Nature Publish- ing Group
URL https://www.nature.com/articles/ s41467-023-36741-4 . Publisher: Nature Publish- ing Group. Shilton, K. V alues and Ethics in Human-Computer Inter- action. F oundations and Trends® in Human–Computer Interaction, 12(2):107–171, 2018. ISSN 1551- 3955, 1551-3963. doi: 10.1561...
2018
-
[26]
Tenopir, C., Levine, K., Allard, S., Christian, L., V olentine, R., Boehm, R., Nichols, F., Nicholas, D., Jamali, H
ISBN 978-0-19-284388-3. Tenopir, C., Levine, K., Allard, S., Christian, L., V olentine, R., Boehm, R., Nichols, F., Nicholas, D., Jamali, H. R., Herman, E., and Watkinson, A. Trustworthiness and au- thority of scholarly information in a digital age: Results of an international...
2016 doi
-
[29]
Grossman, G
URL https://ieeexplore.ieee.org/ abstract/document/9684742. Grossman, G. AGI is coming faster than we think: We must get ready now. V entureBeat, 2023. URL https://venturebeat.com/ai/agi-is- coming-faster-than-we-think-we-must- get-ready-now/ . Accessed: Jan 17, 2025. Gruetzem...
2023
-
[30]
doi: 20240428092301000
ISSN 1396-0466. doi: 20240428092301000. URL https://firstmonday.org/ojs/index. php/fm/article/view/13642. Y u, D., Rosenfeld, H., and Gupta, A. The ‘AI divide’ between the Global North and Global South, Jan- uary 2023. URL https://www.weforum.org/ stories/2023/01/davos23-ai-di...
2023
-
[31]
to ensure that [AGI] benefits all of humanity
ISSN 1099-1743. doi: 10 . 1002 / sres . 2881. URL https://onlinelibrary.wiley. com/doi/abs/10.1002/sres.2881. eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/sres.2881. 24 Stop Treating ‘AGI’ as the North-star Goal of AI Research A. Definitions of AGI and Related Concep...
2024 doi
-
[32]
[AGI] has the po- tential to drive one of the greatest transformations in his- tory
of Google DeepMind states that “[AGI] has the po- tential to drive one of the greatest transformations in his- tory.” In a recent briefing ( Browne, 2025), Demis Hass- abis stated that, though current systems still have limita- tions, over the next 5–10 years “a lot of those ca...
1976
-
[130]
emnlp-main.130/
URL https://aclanthology.org/2022. emnlp-main.130/. Agre, P . E. Toward a critical technical practice: Lessons learned in trying to reform AI. In Social science, tech- nical systems, and cooperative work , pp. 131–157. Psy- chology Press, 2014. Ag¨ uera y Arcas, B. and Norvig,...
2022
-
[417]
doi: https://doi.org/10.1016/j.cogsys.2013.06
2013 doi
-
[1423]
DiPaolo, J
URL https://aclanthology.org/N19- 1423/. DiPaolo, J. What’s wrong with epistemic trespassing? Philosophical Studies , 179(1):223–243, January 2022. ISSN 1573-0883. doi: 10.1007/s11098- 021- 01657-
2022 doi
-
[2013]
ISSN 1747-9991. doi: 10 . 1111 / phc3 . 12000. URL https://onlinelibrary.wiley. com/doi/abs/10.1111/phc3.12000. Mulligan, D. K., Koopman, C., and Doty, N. Privacy is an essentially contested concept: a multi-dimensional ana- lytic for mapping privacy. Philosophical Transaction...
-
[2016]
URL https:// royalsocietypublishing.org/doi/10
doi: 10.1098/rsta.2016.0118. URL https:// royalsocietypublishing.org/doi/10. 1098/rsta.2016.0118. Publisher: Royal Society. 18 Stop Treating ‘AGI’ as the North-star Goal of AI Research Narayanan, A. and Kapoor, S. AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t...
2016
-
[2018]
URL https://openai.com/charter. OpenAI. About, 2025a. URL https://openai.com/ about/. [Online; accessed 19-May-2025]. OpenAI. Planning for AGI and beyond, 2025b. URL https://openai.com/index/planning- for-agi-and-beyond/ . [Online; accessed 19- May-2025]. OpenAI. Security on t...
2025 arXiv
-
[2021]
doi: 10.1145/ 3433949
ISSN 0001-0782, 1557-7317. doi: 10.1145/ 3433949. URL https://dl.acm.org/doi/10. 1145/3433949. Gabriel, I. Artificial Intelligence, V alues, and Alignment . Minds and Machines , 30(3):411–437, September 2020. ISSN 0924-6495, 1572-8641. doi: 10.1007/s11023-020- 09539- 2. URL htt...
2020 doi
-
[2022]
cc/paper_files/paper/2022/hash/ 17a234c91f746d9625a75cf8a8731ee2- Abstract-Conference.html
URL https://proceedings.neurips. cc/paper_files/paper/2022/hash/ 17a234c91f746d9625a75cf8a8731ee2- Abstract-Conference.html . Bommasani, R., Singer, S., Appel, R. E., Cen, S., Cooper, A. F., Cryst, E., Gailmard, L. A., Gonzalez, J. E., Ho, D. E., Klaus, I., Lee, M. M., Liang, ...
2022 arXiv
-
[2023]
https://www.nytimes
— nytimes.com. https://www.nytimes. com/2023/05/22/technology/ai-photo- labels-google-apple.html , 2023. [Accessed 24-01-2025]. Graziul, C., Belikov, A., Chattopadyay, I., Chen, Z., Fang, H., Girdhar, A., Jia, X., Krafft, P . M., Kleiman-Weiner, M., Lewis, C., Liang, C., Mucho...
2023
-
[2024]
doi: 20240428092345000
ISSN 1396-0466. doi: 20240428092345000. URL https://firstmonday.org/ojs/index. php/fm/article/view/13626. Alexandrova, A. and Fabian, M. Democratising Measure- ment: or Why Thick Concepts Call for Coproduction. European Journal for Philosophy of Science , 12(1):7, January 2022...
2022 doi
-
[2025]
AI supply chain
URL https://techpolicy.press/ deepseek-and-the-race-to-agi-how- global-ai-competition-puts-ethical- accountability-at-risk . United Nations. Governing AI for Humanity. Final Report, United Nations, New Y ork, NY , September 2024. URL https://www.un.org/sites/un2.un. org/files/...
2024
-
[4331]
Kleinberg, J
URL https://www.nytimes.com/2025/ 03/04/opinion/ezra-klein-podcast-ben- buchanan.html . Kleinberg, J. and Raghavan, M. Algorithmic monocul- ture and social welfare. Proceedings of the National Academy of Sciences , 118(22):e2018340118, 2021. doi: 10 . 1073 / pnas . 2018340118....
2025
Reviewed August 9, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.