REVIEW 3 major objections 5 minor 4 references
The Research Impact of the Jodrell Bank Observatory and Other Facilities affected by the UK Science Funding Cuts in 2025
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper counts facility mentions in every 2025 astrophysics preprint to measure the research output put at risk by UK funding cuts, reporting 80 e-MERLIN papers and 1,874 Rubin papers.
desk verdict Timely, transparent mention counts for UK-threatened facilities, but the 'research impact' framing overreaches: the paper's own Rubin example shows that mentions are not data use. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is a string-matching search over the LaTeX source files of the 2025 astrophysics preprint corpus. For each facility the paper defines a list of names — for e-MERLIN, the array name plus each individual telescope in the network — and counts a paper as using the facility if any name appears, with case-insensitive matching for long forms and case-sensitive matching for short forms. One e-MERLIN variant adds European VLBI Network terms to capture joint international use. The resulting paper counts are combined with citation totals and per-paper averages, and a normalised citation distribution is compared with the JWST distribution to show that the e-MERLIN average is not driven by outliers.
What would settle it
A manual audit of the 80 counted e-MERLIN papers, checking how many actually contain e-MERLIN observations in their data section rather than merely naming the array in passing, would settle whether the mention count is a genuine measure of research use.
Extended reading notes
Core claim
On its own terms, the paper's discovery is a set of mention-based impact statistics for the six affected facilities. Searching the full LaTeX text of the 2025 astrophysics preprint corpus, it finds that e-MERLIN appears in 80 papers (192 citations, 2.40 citations per paper), a count that grows to 161 papers and 384 citations when European VLBI Network terms are added. The James Clerk Maxwell Telescope appears in 235 papers (402 citations, 1.71 per paper); the Birmingham Solar Oscillations Network in 18 papers (51 citations, 2.83); the Extremely Large Telescope in 490 papers (1,336 citations, 2.73); the Square Kilometre Array in 1,416 papers (4,520 citations, 3.19); and the Vera C. Rubin Observatory in 1,874 papers (7,296 citations, 3.89). The paper notes that e-MERLIN's top subject categories are high-energy astrophysical phenomena and galaxies, that its most-cited paper follows up a fast X-ray transient, and that many Rubin mentions appeared before that observatory's first test images because it began operating halfway through 2025. It frames these totals as the research impact of facilities that are not being prioritised under the new budget.
Load-bearing premise
The counts rest on treating a mention of a facility name in a paper's source text as evidence that the paper used that facility's data; the paper's own Rubin numbers show mentions can predate any possible data use, so this premise can fail.
Editorial extensions
If this is right
- If the mention counts are accepted as a measure of research output, e-MERLIN supported at least 80 published papers in 2025, and 161 when its joint European VLBI Network role is included, giving the funding cut a concrete annual output to weigh against the savings.
- The e-MERLIN citation distribution tracks the shape of the JWST distribution, so the lower average citation rate is a general feature of these papers, not an artifact of one or two highly cited outliers.
- The per-facility counts — 235 for the James Clerk Maxwell Telescope, 490 for the Extremely Large Telescope, 1,416 for the Square Kilometre Array, 1,874 for the Rubin Observatory — provide a common baseline that stakeholders can compare across the cut facilities and against future years.
- Because the Rubin Observatory only began operating midway through 2025 and still dominates the mention counts, the numbers also show that preprints can mention a facility for planned science, not only for data already taken.
Reading between the lines
- The paper's own Rubin observation points to a limitation it does not fully apply to e-MERLIN: a mention is not proof of data use, so the 80 e-MERLIN papers are best read as an upper bound on direct usage rather than a precise count of it.
- A natural extension would be to calibrate mention counts against the facilities' actual data archives or observing logs, converting 'papers that mention the telescope' into 'papers whose data products came from the telescope' and separating UK-led from international use; this would make the impact measure directly relevant to a UK funding decision.
- If the cuts are implemented, the paper's 2025 counts form a before/after baseline: a measurable prediction of its own logic is that e-MERLIN and JCMT mention rates in 2026 and 2027 should fall relative to comparable facilities, and the speed of that drop would test whether the impact is as large as the counts suggest.
- The extreme spread among facilities — 18 papers for BiSON versus 1,874 for Rubin — implies that raw mention counts cannot by themselves adjudicate the cuts, because a small but irreplaceable niche facility can score low while still being essential to its subfield.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper reports a text-mining analysis of all 2025 arXiv astro-ph submissions, counting mentions of e-MERLIN, JCMT, BiSON, ELT, SKA, and the Vera C. Rubin Observatory in LaTeX sources as a proxy for research impact. It finds 80 papers mentioning e-MERLIN (161 when EVN-related terms are included), reports associated citation statistics, and compares e-MERLIN's average citation rate with that of JWST. The stated purpose is to inform the public and policy-makers debating the UKRI/STFC funding cuts.
Significance. The paper addresses a timely and policy-relevant question, and it has the virtue of using publicly available data with a transparent, mechanical counting procedure; the citation data obtained through the NASA ADS API is a further reproducible element. If the mention counts were validated against actual data usage, the results could serve as a useful descriptive indicator of the visibility of affected facilities in the 2025 literature. However, the principal measure conflates textual mention with data use, and the paper itself provides a direct counterexample for Rubin; consequently, the headline 'research impact' claims are not currently supported. The paper's strength is its data provenance and simplicity; its weakness is the gap between what is measured and what is concluded.
major comments (3)
- [Section 2 and Section 3.2] The central operational definition, stated in Section 2 as 'if a particular paper mentions the names of any of the telescopes from a pre-defined list, that paper is counted as using the data from that telescope,' equates textual mentions with data use. The paper's own Section 3.2 analysis of the Rubin Observatory shows that 1874 papers mention Rubin even though the observatory only began operating halfway through 2025 and a good number of those papers were released before first test images. This internal counterexample demonstrates that mentions do not imply data use, yet the same unvalidated proxy is then applied to e-MERLIN, JCMT, and the other facilities. Because the Abstract and Section 4 present these counts as 'research impact,' this conflation is load-bearing and the conclusions overstate what the data show.
- [Section 3.1] The paper reports that 'a paper using data from e-MERLIN was cited 2.40' times on average, compared with 3.93 for JWST, and Figure 1 is claimed to show that the averages are not skewed by outliers. No uncertainties are provided for any of the citation means, no statistical test is applied to the JWST comparison, and the figure's axes are unlabelled and its content is not legible in the submitted version. Given the typical heavy-tailed distribution of citation counts and the small e-MERLIN sample size of 80, the apparent gap between 2.40 and 3.93 may not be significant; bootstrap confidence intervals or a Poisson treatment should be reported.
- [Section 2 and Section 4] The exact search-term lists and matching code are not included in the manuscript; the reader is instead referred to Lewis et al. (2026), which the authors state used a different search scope (title and abstract only) rather than the full-text search employed here. Because the methodology differs, the companion paper does not suffice for reproducibility, and the reader cannot independently verify the headline count of 80 e-MERLIN papers. The manuscript should include the complete term list for each facility and a precise description of the matching rules, or an appendix with the extraction code.
minor comments (5)
- [Title] The title contains a spacing error: 'F acilities' should be 'Facilities.'
- [Section 4] In the Conclusions, 'who's years of experience' should be 'whose years of experience,' and 'to asses if' should be 'to assess if.'
- [Section 3] The terminology for the measured quantity is inconsistent: 'mentioned using data' (Section 3.1), 'was attributed in' (Section 3.2), 'citing' and 'quoting' (Section 3.2) all refer to the same string-matching operation. A single term, such as 'mentions,' should be used consistently.
- [Footnotes] The footnote references to RAS and BBC articles are not included in the reference list; if they are to be cited, they should be moved to the bibliography.
- [Section 1] The claim that funding cuts were 'on the order of 30%' and later reduced to '2.7%' would benefit from a direct citation to the cited RAS article, since the numbers are central to the motivation.
Circularity Check
No circular derivation: the mention-count statistics are computed directly from external arXiv and ADS data, with only minor, non-load-bearing self-citations to the authors' companion methodology paper.
full rationale
The paper's headline quantity, the 80 e-MERLIN papers, is obtained by applying the Section 2 string-matching rule to arXiv LaTeX sources, an external dataset acknowledged in the Acknowledgments ('All data used in this paper comes from papers publicly available on the arXiv'). Citation counts are separately indexed from the NASA ADS API. There is no fitted parameter being renamed as a prediction and no equation in which an output is defined in terms of itself; the 'mention equals data use' rule is an explicit operationalization of 'research impact', not a hidden self-definition. The self-citations to R. F. Lewis et al. (2026) provide methodological detail and the JWST comparison distribution, and the Conclusions explicitly state that the search scope differs from that companion paper ('here we search through the entire manuscript whereas in R. F. Lewis et al. (2026) we only search through the title and abstract'). Thus the central counts do not reduce to the companion paper's outputs. The paper itself notes that Rubin was mentioned in 1874 papers even though it only began operating halfway through 2025, undercutting the proxy's validity, but this is a correctness and framing caveat rather than a circularity. Under the scoring rubric, the few self-citations are minor and not load-bearing, so the circularity score is low.
Assumptions & free parameters
assumptions (3)
- domain assumption Mentioning a facility name in a paper's LaTeX indicates use of that facility's data.
- domain assumption The set of all astrophysics arXiv preprints in 2025 represents the full research output of the affected facilities.
- ad hoc to paper The pre-defined search-term list is complete enough to capture all relevant mentions.
Cite this review
Pith. "Pith review of The Research Impact of the Jodrell Bank Observatory and Other Facilities affected by the UK Science Funding Cuts in 2025." pith.science (2026). https://pith.science/paper/OAXUJJOT
@misc{pith2026260802723,
author = {Pith},
title = {Pith review of: The Research Impact of the Jodrell Bank Observatory and Other Facilities affected by the UK Science Funding Cuts in 2025},
year = {2026},
howpublished = {\url{https://pith.science/paper/OAXUJJOT}},
note = {Machine review of arXiv:2608.02723}
}
read the original abstract
The United Kingdom Research and Innovation body and the Science and Technology Facilities Council recently announced funding cuts to facilities at the Jodrell Bank Observatory, specifically the e-MERLIN network of radio telescopes as well as a few other facilities across the world like the James Clerk Maxwell Telescope. These funding cuts have been extremely disturbing to the Astronomical community, as is evident from widespread news coverage about these cuts. Here we present a short analysis of the research impact e-MERLIN and the other affected facilities had in 2025 from a dataset consisting of every Astrophysics paper submitted to the arXiv during the year. This serves to help members of the community and all stakeholders make better sense of the research output of the facilities that are not being prioritised under the new budget.
Figures
Reference graph
Works this paper leans on
-
[1]
Anderson, G. E., Lamb, G. P., Gompertz, B. P., et al. 2025, The radio flare and multi-wavelength afterglow of the short GRB 231117A: energy injection from a violent shell collision, https://arxiv.org/abs/2508.14650
arXiv 2025
-
[2]
Lewis, R. F., Shah, H., & Alfred, A. 2026, Astrophysics Wrapped 2025: Year-in-Review of Every Astrophysics arXiv Paper from 2025, https://arxiv.org/abs/2602.12303
arXiv 2026
-
[3]
Yadav, M., Troja, E., Ricci, R., et al. 2025, Radio observations point to a moderately relativistic outflow in the fast X-ray transient EP241021a, https://arxiv.org/abs/2505.08781
arXiv 2025
-
[4]
Zhang, X., Yu, W., Yan, Z., Xing, Y., & Zhang, B. 2025, A flaring radio counterpart to a fast radio burst reveals a newborn magnetized engine, https://arxiv.org/abs/2501.14247
arXiv 2025
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.