Recognition: unknown
Visible, Trackable, Forkable: Opening the Process of Science
Pith reviewed 2026-05-10 16:05 UTC · model grok-4.3
The pith
Making the full scientific process visible, trackable, and forkable would accelerate progress, improve accessibility, and strengthen trustworthiness of claims.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that opening the process of science itself, not merely its outputs, would produce a step change in the speed of scientific progress, the accessibility of scientific reasoning, the trustworthiness of scientific claims, and the scalability of scientific quality assurance. Three required properties are identified: the process must be visible rather than hidden behind finished papers, every change must be trackable and attributable, and anyone must be able to fork from any prior state while preserving shared history. Such a workflow is inherently verifiable by humans, automated tools, and AI agents, mirroring the practices that transformed software development.
What carries the argument
The visible, trackable, and forkable workflow that records the full evolving sequence of questions, interpretations, dead ends, and direction changes rather than only final results.
If this is right
- Errors would be corrected through ongoing amendments rather than punished by retraction of finished papers.
- Alternative research directions would share a common history instead of appearing only as competing isolated publications.
- The research flow would become verifiable by automated tools and AI agents in addition to human readers.
- Broader participation would become possible while maintaining quality at larger scales, as seen in software.
- The unit of scientific contribution would shift from polished products to the documented sequence of reasoning steps.
Where Pith is reading between the lines
- Such workflows could enable new forms of credit and incentives that reward documented corrections and intermediate insights rather than only final positive results.
- Training and education in a field could incorporate forking real research paths to let students practice on live histories instead of static examples.
- Disputes over priority or interpretation could be settled by direct reference to the shared, timestamped record instead of separate publications.
Load-bearing premise
The benefits observed in software development from visible, trackable, and forkable workflows will translate directly to scientific research without major domain-specific barriers.
What would settle it
A large-scale adoption of visible, trackable, forkable systems in one research field that produces no measurable rise in the rate of error correction by amendment, the breadth of contributions, or the speed of progress compared with traditional closed-process publishing.
read the original abstract
The way science is currently practiced shows conclusions but hides how they were reached. Researchers work privately, polish their results, publish a finished paper, and defend it. Errors are punished by retraction rather than corrected by amendment. Alternative directions are pursued through competing papers with no shared history. The reasoning, the dead ends, the trade-offs, the corrections: everything that would let others understand how a conclusion was reached is invisible. Two decades of open science reform have addressed this by opening specific artifacts: papers, data, code, notebooks, protocols. Each is valuable, but the unit remains a finished product. None opens the thinking process itself: the evolving sequence of questions, interpretations, dead ends, and direction changes that constitutes the actual scientific contribution. This paper argues that opening the process of science (not just its outputs) would produce a step change in the speed of scientific progress, the accessibility of scientific reasoning, the trustworthiness of scientific claims, and the scalability of scientific quality assurance. We identify three properties the workflow needs: visible (the process is open, not just the product), trackable (every change is recorded and attributable), and forkable (anyone can branch from any point with shared history preserved). A visible, trackable flow is inherently verifiable: by humans, by automated tools, by AI agents. Software development adopted this flow decades ago, and the results (faster correction, broader contribution, maintained quality at scale) demonstrate the opportunity for science.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper argues that current scientific practice conceals the iterative process of research (questions, dead ends, corrections) behind polished final outputs, and that two decades of open science have only opened artifacts rather than the workflow itself. It proposes that making the process visible, trackable, and forkable—modeled on software development—would produce a step change in the speed of progress, accessibility of reasoning, trustworthiness of claims, and scalability of quality assurance, with verifiability by humans, tools, and AI agents.
Significance. If the proposed workflow properties can be realized and the software analogy holds after accounting for domain differences, the framework could meaningfully advance open science by shifting focus from static outputs to dynamic, amendable processes, enabling broader participation and continuous correction at scale. The clear tripartite characterization of needed properties is a constructive contribution that could guide tool and platform development.
major comments (2)
- [Abstract] Abstract: The central claim of a 'step change' in speed, accessibility, trustworthiness, and scalability rests entirely on the unexamined assumption that software-development benefits will transfer directly. The manuscript provides no analysis of countervailing science-specific factors (non-digital physical experiments that resist forking, career incentives favoring secrecy until publication, or the gatekeeping role of peer review versus continuous amendment), which is load-bearing for whether the asserted benefits materialize.
- [The three properties] The section introducing the three properties: The definition of 'forkable' (anyone can branch from any point with shared history preserved) does not address how this would apply when scientific contributions include unique physical or wet-lab components that cannot be digitally versioned or replicated in the same way as code, undermining the claim that the workflow is inherently verifiable and scalable.
minor comments (2)
- [Abstract] The abstract contains several long sentences that could be split to improve readability of the core argument.
- The contrast with existing open-science efforts (papers, data, code, notebooks) would benefit from one or two concrete citations to recent reviews or platforms to ground the claim that none currently opens the thinking process.
Simulated Author's Rebuttal
We thank the referee for the detailed and constructive report. The comments correctly identify areas where the manuscript relies on the software analogy without sufficient qualification of domain differences. We address both major comments below and will incorporate revisions to clarify scope and add discussion of limitations.
read point-by-point responses
-
Referee: [Abstract] The central claim of a 'step change' in speed, accessibility, trustworthiness, and scalability rests entirely on the unexamined assumption that software-development benefits will transfer directly. The manuscript provides no analysis of countervailing science-specific factors (non-digital physical experiments that resist forking, career incentives favoring secrecy until publication, or the gatekeeping role of peer review versus continuous amendment).
Authors: We agree that the manuscript presents the benefits as following from the three properties without a dedicated analysis of transfer barriers. The core argument is that visible, trackable, and forkable workflows enable the same mechanisms of correction and contribution seen in software, but we will add a new subsection in the discussion that explicitly examines the listed factors. For physical experiments we will note that protocols and raw data can still be versioned and forked digitally even if the physical instance cannot; for incentives we will reference ongoing shifts toward preprinting and open workflows; and for peer review we will discuss hybrid models where continuous amendment supplements rather than replaces gatekeeping. The 'step change' phrasing will be qualified as potential rather than guaranteed. revision: yes
-
Referee: [The three properties] The definition of 'forkable' (anyone can branch from any point with shared history preserved) does not address how this would apply when scientific contributions include unique physical or wet-lab components that cannot be digitally versioned or replicated in the same way as code, undermining the claim that the workflow is inherently verifiable and scalable.
Authors: The definition is scoped to the digital record of the process. We will revise the forkable section to state explicitly that forking operates on the documented workflow (questions, decisions, data, protocols, code) while physical or wet-lab elements are addressed through detailed, versioned descriptions that enable attempted replication or variation by others. Verifiability then applies to the traceable digital artifacts and reasoning chain, which humans, tools, and AI agents can inspect, rather than to non-replicable physical instances. This mirrors how software forks preserve history even when hardware differs. The scalability claim will be limited accordingly. revision: yes
Circularity Check
No circularity: proposal relies on external software analogy without self-referential reduction or fitted inputs.
full rationale
The paper's central argument is a forward-looking proposal that visible, trackable, and forkable workflows (modeled on software development) would improve scientific progress, accessibility, trustworthiness, and quality assurance. This rests on an analogy to observed software outcomes rather than any derivation, equation, or self-citation that reduces the claimed benefits back to the paper's own definitions or inputs. No load-bearing self-citations, uniqueness theorems, ansatzes, or renamings of known results appear in the provided text. The three properties are introduced as requirements for the workflow, not as self-defining the predicted outcomes. The argument is therefore self-contained against external benchmarks and does not exhibit circularity.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Software development practices of visible, trackable, and forkable workflows will produce analogous benefits when applied to scientific research
Forward citations
Cited by 1 Pith paper
-
The Research Guide: From Informal Role to Profession
The authors argue that guiding non-PhD learners through authentic research requires a dedicated profession with its own training, career structure, and recognition because existing models and programs fall short.
Reference graph
Works this paper leans on
-
[1]
Stephen Curry et al. Ending publication bias: A values-based approach to surface null and negative results.PLoS Biology, 23(9):e3003368, 2025. doi: 10.1371/journal.pbio.3003368
-
[2]
John P. A. Ioannidis. Why most published research findings are false.PLoS Medicine, 2(8):e124, 2005. doi: 10.1371/journal.pmed.0020124
-
[3]
Zachary C. Lipton and Jacob Steinhardt. Troubling trends in machine learning scholarship.arXiv preprint arXiv:1807.03341, 2018. URLhttps://arxiv.org/abs/1807.03341
-
[4]
Liz Lyon. Transparency: The emerging third dimension of open science and open data.LIBER Quarterly, 25(4):153–171, 2016. doi: 10.18352/lq.10113
-
[5]
Marcus R. Munafò, Brian A. Nosek, Dorothy V . M. Bishop, Katherine S. Button, Christopher D. Chambers, Nathalie Percie du Sert, Uri Simonsohn, Eric-Jan Wagenmakers, Jennifer J. Ware, and John P. A. Ioannidis. A manifesto for reproducible science.Nature Human Behaviour, 1:0021, 2017. doi: 10.1038/s41562-016-0021
-
[6]
Brian A. Nosek, George Alter, George C. Banks, Denny Borsboom, Sara D. Bowman, Steven J. Breckler, Stuart Buck, Christopher D. Chambers, Gilbert Chin, Garret Christensen, et al. Promoting an open research culture.Science, 348(6242):1422–1425, 2015. doi: 10.1126/science.aab2374. Preprint: https://osf.io/vj54c/
-
[7]
Open Science Collaboration. Estimating the reproducibility of psychological science.Science, 349 (6251):aac4716, 2015. doi: 10.1126/science.aac4716. Author manuscript: https://osf.io/ 447b3/
-
[8]
Karthik Ram. Git can facilitate greater reproducibility and increased transparency in science.Source Code for Biology and Medicine, 8(7), 2013. doi: 10.1186/1751-0473-8-7
-
[9]
Robert T. Thibault et al. Open science 2.0: Towards a truly collaborative research ecosystem.PLoS Biology, 21(10):e3002362, 2023. doi: 10.1371/journal.pbio.3002362
-
[10]
Wilkinson, Michel Dumontier, IJsbrand Jan Aalbersberg, Gabrielle Appleton, Myles Axton, Arie Baak, Niklas Blomberg, Jan-Willem Boiten, Luiz Bonino da Silva Santos, Philip E
Mark D. Wilkinson, Michel Dumontier, IJsbrand Jan Aalbersberg, Gabrielle Appleton, Myles Axton, Arie Baak, Niklas Blomberg, Jan-Willem Boiten, Luiz Bonino da Silva Santos, Philip E. Bourne, et al. The FAIR guiding principles for scientific data management and stewardship.Scientific Data, 3:160018,
-
[11]
doi: 10.1038/sdata.2016.18. 11
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.