REVIEW 3 major objections 4 minor 21 references
The Living Guide of Machine Learning for Particle Physics
T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read The HEP-ML Living Review is frozen and succeeded by a curated, annotated Living Guide because near-comprehensive manual bibliographies have become unsustainable and less useful as the field has grown by an order of magnitude.
desk verdict A well-documented announcement of a useful new community resource whose core 'curation is better' claim is plausible but unmeasured. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the HEP-ML Living Guide itself: a versioned field guide organized along two independent axes (HEP application and ML method) with cross-links between them. Every included paper or cluster carries a short annotation stating why it was chosen and what it establishes. Sections are written by named contributors, timestamped, and periodically released as citable snapshots with their own DOI, so a citation points at a fixed state and at the people who produced it. The selection criteria are foundational importance, methodological clarity, and practical use for newcomers, and the resource deliberately makes no claim of completeness.
What would settle it
Count citations and contributions to the archived Living Review versus the Living Guide over the next two years: if the frozen review continues to be cited at its pre-freeze rate while the Guide attracts few contributors or readers, the claim that near-complete bibliographies are no longer particularly useful would be falsified.
Extended reading notes
Core claim
The central claim is that the near-comprehensive, community-contributed Living Review model that served machine learning in particle physics from 2020 has been overtaken by the field it helped document. The literature grew from roughly two hundred papers a year to several hundred a year, the flat taxonomy could no longer answer questions like which papers established simulation-based inference or which architectures work for fast simulation, and the community built its own ecosystem of reviews, benchmarks, and software. The paper therefore freezes the Living Review as an archival snapshot up to 1 June 2026 and replaces it with the HEP-ML Living Guide, a curated field guide that does not claim completeness and instead recommends, annotates, and maps the literature.
Load-bearing premise
The Guide's design rests on the untested premise, based only on informal conversations with researchers, that newcomers chiefly want a few recommended papers, a handful of tutorials, and a map of subfield connections rather than a comprehensive list.
Editorial extensions
If this is right
- Newcomers to any subfield can start from a short list of recommended papers and tutorials instead of a multi-thousand-entry bibliography.
- Citations to community resources become tied to a fixed version and to the named authors of each section, repairing the credit and reproducibility problems of a rolling document.
- The frozen Living Review remains available as an archival snapshot, preserving near-comprehensive coverage of the literature up to 1 June 2026 for historical citation.
- The Guide deliberately narrows its scope to particle physics, linking to neighboring curated resources for cosmology, astroparticle physics, and accelerators rather than duplicating them.
- Maintenance becomes sustainable because named experts write sections as one-time contributions, sections stay until someone updates them, and releases are tagged periodically.
Reading between the lines
- The two-axis taxonomy and annotation requirement could become a template for other disciplines facing literature explosion, making this paper a reusable model rather than a one-off change.
- By leaving out nuclear, heavy-ion, and astroparticle physics, the Guide may push those communities to maintain their own linked guides, leading to a federated network of curated entry points.
- The authors' informal user interviews could be replaced by a systematic usage study; one testable prediction is that newcomers reach a working understanding of a subfield faster through recommended starting points than through comprehensive lists.
- Because section coverage follows community interest, well-funded subfields may receive frequent updates while smaller areas stagnate, a silent bias the paper acknowledges only indirectly.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper announces a directional change in a widely used community resource for machine learning in particle physics (HEP-ML). The authors argue that the HEP–ML Living Review, a near-comprehensive community-maintained bibliography, is no longer sustainable or as useful as it was in 2020, because the literature has grown by more than an order of magnitude and has diversified into many subfields. They therefore freeze the Living Review as an archival snapshot dated 1 June 2026 and replace it with a curated HEP–ML Living Guide, which offers annotated recommended starting points, curated paper lists, benchmarks, and software references organized along two independent axes (HEP application and ML method). The paper describes the history and lessons of the original Review, presents publication-growth data from INSPIRE-HEP, reports citation statistics for the Review, and details the structure, scope, and maintenance model of the new Guide.
Significance. The paper is a practical community-resource transition statement rather than a technical research contribution. Its factual premises are documented: Section 2.2 and Figure 1 provide INSPIRE-HEP query results showing the growth of HEP-ML literature, and Section 2.3 gives concrete INSPIRE citation counts (81 URL-only, 217 arXiv-only, 20 both) for the Living Review. The authors are transparent about what they did and did not measure, and they explicitly acknowledge the self-referential nature of evaluating their own project. The significance, if the new Guide succeeds, is real but modest: it could improve onboarding and orientation for newcomers and cross-subfield researchers. However, the central prescriptive claim—that a curated guide is better for users than a near-comprehensive bibliography—rests on anecdotal evidence. The paper does not currently provide enough empirical support to justify the strength of the conclusion, and the sustainability argument is asserted rather than demonstrated. Still, the direction is defensible and the proposed resource is a reasonable community experiment; the claims need tempering or additional evidence.
major comments (3)
- [Section 4, Conclusions] The claim that students and postdocs want 'three to five papers to read first, two or three tutorials or reviews, and a map of how the subfield connects to its neighbors' is supported only by 'Conversations with researchers at all career stages,' with no number of conversations, no sampling frame, no protocol, and no results. This is the load-bearing empirical premise for abandoning near-comprehensive coverage and moving to a curated guide. As written, it is a design hypothesis, not a finding. To make the argument rigorous, the authors should either (a) report a user study or survey with a described methodology and results, (b) document the anecdotal evidence in a more falsifiable way (e.g., number of researchers consulted, their career stages, the questions asked, and the range of responses), or (c) explicitly reframe the premise as a design assumption and soften the conclusion accordingly. As it stands, the paper's central move from 'comprehensive lists are no longer enough' to 'a curated Guide is more useful' is not fully supported by the presented evidence.
- [Section 3, Commitment 5] The conclusion states that the field 'has become too large and too mature for a near-complete manual bibliography to stay sustainable or particularly useful.' The 'particularly useful' clause is stronger than the evidence in Section 2.3 supports: the INSPIRE search returns 81 entries citing only the URL, 217 citing only the arXiv article, and 20 citing both, which indicates ongoing use of the comprehensive resource even if the exact usage mode (navigation vs. bibliographic reference) is not known. The new Guide's design rationale is about orientation, so the more precise claim would be that a near-complete list is less useful for early orientation and navigation, while still acknowledging the archival value for comprehensive searches, reference chasing, and citation audits. The current wording overstates the case and should be revised to distinguish these functions.
- [Section 3, Commitment 5] The commitment to 'Sustainability by design' is asserted but not demonstrated. The paper explains why the old model was unsustainable at the scale of several hundred new papers per year, but the new Guide's maintenance model depends on named community members writing sections as one-time contributions, with sections staying static until someone updates them. The paper does not address the risk that some subfields, especially smaller or less active ones, may never attract a section author, which would undermine the Guide's goal of providing a map of subfield connections. The self-selection mechanism ('contributions follow real community investment') may produce uneven coverage biased toward currently popular areas. The authors should discuss a fallback plan for orphaned sections, perhaps by listing a provisional roster of section editors or by describing how the coordinators will handle gaps.
minor comments (4)
- [Section 3.2] The text says 'It now adds that many in a few months,' referring to roughly 200 papers per year in the first year. The claim would be clearer with a specific number or a comparison of the slopes in Figure 1, for example, the number of papers added in the most recent year versus the first year.
- [Section 3.2] The two-axis taxonomy (HEP application and ML method) is described as the organizing principle, but the mechanics of cross-linking are not specified. It would be helpful to state how cross-links are represented in the version-controlled repository (e.g., tags, a lookup table, or bidirectional links in the markdown) so that contributors know how to maintain them.
- [Section 3.2] The sentence 'We do not expect this to trigger controversial discussions that would call for extra layers of moderation' presupposes an outcome. If a contested contribution does arise, the governance model does not specify a resolution process. A one-sentence contingency statement would make the maintenance model more robust.
- [References] The paper's title, 'The Living Guide of Machine Learning for Particle Physics,' is identical to the name of the new resource. A subtitle such as 'A Transition from the Living Review' would make the article's purpose immediately clear to readers searching for the old Review. Also, Reference [1] (the arXiv paper for the original Review) has no DOI while Reference [3] has a Zenodo DOI; please verify that the citation details are consistent with the description in Section 3.2 of the archived Resource.
Circularity Check
No circularity: the paper's argument is an empirical/design claim with checkable external evidence, not a derivation that reduces to its own inputs.
full rationale
The paper contains no equations or derivations, so there is no equation-level circularity. Its central claim is that the near-comprehensive Living Review should be frozen and replaced by a curated, annotated Living Guide. The evidence for the scale problem is external and checkable: Figure 1 is an INSPIRE-HEP retrieval, and Section 2.3 cites INSPIRE fulltext counts of citations to the authors' own review (81 URL-only, 217 arXiv-only, 20 both), which is a factual, independently verifiable measure of adoption rather than a circular justification. The weakest point is Section 2.3's 'What users need now' paragraph, which rests on undocumented 'Conversations with researchers at all career stages' to support the premise that newcomers want three to five recommended papers rather than comprehensive lists. That is an unverified empirical assumption, and the Guide's design then follows from it, but this is a weakness of evidence, not circularity: the premise is not defined in terms of the conclusion, and the conclusion is not forced by the premise's wording. The self-citations to the authors' own Living Review are not load-bearing in a circular way, because the cited resource's properties are publicly documented and its citation counts are externally retrievable. Overall, the transition argument is a design proposal with a soft evidential basis, but no step reduces by construction to its own inputs.
Assumptions & free parameters
assumptions (4)
- domain assumption Finding papers is no longer the hard part; the hard part is finding a way in.
- domain assumption Newcomers want three to five papers, two or three tutorials, and a map, not a comprehensive bibliography.
- domain assumption A curated guide with named, timestamped sections will be sustainable with light voluntary maintenance.
- domain assumption Machine learning for particle physics can be cleanly separated from nuclear, heavy-ion, astroparticle, and cosmology ML.
Cite this review
Pith. "Pith review of The Living Guide of Machine Learning for Particle Physics." pith.science (2026). https://pith.science/paper/HFLS2V3X
@misc{pith2026260809531,
author = {Pith},
title = {Pith review of: The Living Guide of Machine Learning for Particle Physics},
year = {2026},
howpublished = {\url{https://pith.science/paper/HFLS2V3X}},
note = {Machine review of arXiv:2608.09531}
}
read the original abstract
We started the Living Review of Machine Learning for Particle Physics (HEP-ML Living Review) in 2020 as a community-maintained, near-comprehensive bibliography of machine learning in particle physics. The field was then growing faster than any single researcher could follow, finding the relevant papers was hard, and a structured, continuously updated reference paid off immediately. Since then the literature has grown by more than an order of magnitude, the methods reach far beyond the classification and generation tasks of the early years, and the community has built its own ecosystem of topic-specific reviews, benchmark papers, and software frameworks. The original model no longer serves this field well, and we can no longer sustain it. We therefore change direction. We freeze the Living Review as an archival reference covering the literature up to 1 June 2026, where it remains a stable record of the first phase of HEP-ML. A new resource, the HEP-ML Living Guide, replaces it. It does not list everything. It curates, it annotates, and it points readers to foundational and representative work, so that researchers can find their way into a mature and rapidly diversifying field. In this article we explain why we make this change and how the new resource works.
Reference graph
Works this paper leans on
-
[1]
M. Feickert and B. Nachman,A Living Review of Machine Learning for Particle Physics, arXiv:2102.02770 [hep-ph]
-
[2]
M. Feickert, B. Nachman, C. Krause, and R. Winterhalder,A Living Review of Machine Learning for Particle Physics,https://iml-wg.github.io/HEPML-LivingReview/
-
[3]
C. Krause, R. Winterhalder, M. Feickert, B. Nachman, J. Raine, and HEP-ML Community, A living review of machine learning for particle physics, July, 2026. https://doi.org/10.5281/zenodo.21626667
-
[4]
Stein,Machine Learning in Cosmology, https://github.com/georgestein/ml-in-cosmology
G. Stein,Machine Learning in Cosmology, https://github.com/georgestein/ml-in-cosmology
-
[5]
Gully-Santiago,Awesome Astrodata,https://github.com/gully/awesome-astrodata
M. Gully-Santiago,Awesome Astrodata,https://github.com/gully/awesome-astrodata
-
[6]
IRIS-HEP,Awesome HEP,https://github.com/iris-hep/awesome-hep
-
[7]
C. Krause, J. Raine, and R. Winterhalder,“The HEP-ML Living Review” talk at ML4Jets 2023,https://indico.cern.ch/event/1253794/contributions/5588545/
arXiv 2023
-
[8]
K. Cranmer and J. Lo,Simulation-based inference, https://simulation-based-inference.org
Show all 21 references
-
[9]
Mishra-Sharma,Awesome Neural SBI, https://github.com/smsharma/awesome-neural-sbi
S. Mishra-Sharma,Awesome Neural SBI, https://github.com/smsharma/awesome-neural-sbi
-
[10]
AI/ML for Particle Accelerators Collaboration,The AI/ML for particle accelerators living review,https://ml-accel-review.org
-
[11]
Plehn, A
T. Plehn, A. Butter, B. Dillon, T. Heimel, C. Krause, and R. Winterhalder,Modern Machine Learning for LHC Physicists, arXiv:2211.01421 [hep-ph]
-
[12]
Halverson,TASI Lectures on Physics for Machine Learning, arXiv:2408.00082 [hep-th]
J. Halverson,TASI Lectures on Physics for Machine Learning, arXiv:2408.00082 [hep-th]
-
[13]
J. Y. Arazet al.,Les Houches guide to reusable ML models in LHC analyses, arXiv:2312.14575 [hep-ph]
-
[14]
Butteret al.,The Machine Learning landscape of top taggers, SciPost Phys.7(2019) 014, arXiv:1902.09914 [hep-ph]
A. Butteret al.,The Machine Learning landscape of top taggers, SciPost Phys.7(2019) 014, arXiv:1902.09914 [hep-ph]
2019 arXiv
-
[15]
Kasieczkaet al.,The LHC Olympics 2020 a community challenge for anomaly detection in high energy physics, Rept
G. Kasieczkaet al.,The LHC Olympics 2020 a community challenge for anomaly detection in high energy physics, Rept. Prog. Phys.84(2021) 12, 124201, arXiv:2101.08320 [hep-ph]
2021 arXiv
-
[16]
T. Aarrestadet al.,The Dark Machines Anomaly Score Challenge: Benchmark Data and Model Independent Event Classification for the Large Hadron Collider, SciPost Phys.12 (2022) 1, 043, arXiv:2105.14027 [hep-ph]
2022 arXiv
-
[17]
Amramet al.,CaloChallenge 2022: a community challenge for fast calorimeter simulation, Rept
O. Amramet al.,CaloChallenge 2022: a community challenge for fast calorimeter simulation, Rept. Prog. Phys.88(2025) 11, 116201, arXiv:2410.21611 [physics.ins-det]
2025
-
[18]
Shanahanet al.,Snowmass 2021 Computational Frontier CompF03 Topical Group Report: Machine Learning, arXiv:2209.07559 [physics.comp-ph]
P. Shanahanet al.,Snowmass 2021 Computational Frontier CompF03 Topical Group Report: Machine Learning, arXiv:2209.07559 [physics.comp-ph]. – 9 –
2021 arXiv
-
[19]
Badgeret al.,Machine learning and LHC event generation, SciPost Phys.14(2023) 4, 079, arXiv:2203.07460 [hep-ph]
S. Badgeret al.,Machine learning and LHC event generation, SciPost Phys.14(2023) 4, 079, arXiv:2203.07460 [hep-ph]
2023 arXiv
-
[20]
Huetschet al.,The landscape of unfolding with machine learning, SciPost Phys.18(2025) 2, 070, arXiv:2404.18807 [hep-ph]
N. Huetschet al.,The landscape of unfolding with machine learning, SciPost Phys.18(2025) 2, 070, arXiv:2404.18807 [hep-ph]
2025 arXiv
-
[21]
T. K. Aarrestadet al.,Building an AI-native Research Ecosystem for Experimental Particle Physics: A Community Vision, arXiv:2602.17582 [hep-ex]. – 10 –
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.