REVIEW 3 major objections 7 minor 1 cited by
COSMIC's Large-Scale Search for Technosignatures during the VLA sky Survey: Survey Description and First Results
T0 review · 3 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read A postprocessing pipeline for a commensal interferometric SETI search returned no candidate signals toward 511 stars, setting isotropic power limits between roughly 2.3e11 and 2.1e16 watts.
desk verdict The qualitative null result is credible, but the quoted EIRP limits are built on an SNR threshold the pipeline never applied; the numbers need recomputation before the paper can be trusted. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is the ARTISTIC pipeline together with the CRICKETS RFI mask that feeds it. CRICKETS flags frequency bins whose excess kurtosis deviates from Gaussian noise, using data from a calibrator observation to produce a list of dirty channels to blank. ARTISTIC then applies an SNR threshold, removes hits seen in many beams, and keeps signals found in only one coherent beam for dynamic-spectrum checks; for hits in two or more beams it compares source proximity and the coherent-to-incoherent power ratio, which is expected to equal the number of antennas used in the beamformer. The sensitivity claim is carried by the distance-squared EIRP formula, which converts each star's minimum detectable flux into a transmitter power limit.
What would settle it
Inject a synthetic narrowband signal into a target's coherent beam and also, at lower amplitude, into an adjacent beam with a known overlap; if the ARTISTIC pipeline discards it as RFI, the single-beam assumption is falsified for realistic spillover. Separately, recompute EIRP_min with an SNR threshold of 100 instead of 8; if the limits rise by a factor roughly equal to 100/8, the published sensitivity range is too optimistic.
Extended reading notes
Core claim
The central claim is that a logical, automated postprocessing chain can separate astrophysical narrowband signals from terrestrial interference well enough to run a wide-area technosignature search without manual inspection of every hit. The authors demonstrate this on 511 stars: after applying the CRICKETS kurtosis mask and an SNR cut of 100, the remaining hits were distributed across all beams, and no signal was confined to a single coherent beam in the way the pipeline expects of an astronomical emitter. They therefore report no unidentifiable signals and set equivalent isotropic radiated power limits of 2.32e11 to 2.09e16 W for the observed stars, by computing EIRP_min = 4*pi*$d^{2}$*F_min with F_min = 13.92 Jy at an assumed 8-$\sigma$ sensitivity and a roughly 8 Hz channel width. The survey has recorded more than 950,000 unique pointings since 2023, so the pipeline is presented as the route to searching that entire database.
Load-bearing premise
The search's null result depends on treating any signal seen in more than one coherent beam as terrestrial interference, and it also assumes the 8-sigma flux limit of 13.92 Jy, not the actual signal-to-noise cut of 100, is the sensitivity floor.
Editorial extensions
If this is right
- The same filter chain can be applied to the 950,000 recorded pointings, converting a database of millions of hourly hits into a short candidate list without human review.
- For the nearest stars in the sample, the null result rules out transmitters weaker than about 10^11 W, assuming the single-beam criterion is valid, meaning even modest planetary radars would have been seen.
- Because COSMIC observes in commensal mode, expanding the search to the full survey adds no extra telescope time, so the technique can ride along on future large sky surveys.
- The pipeline's decision logic is simple enough to be reimplemented or learned by a classifier, so the method could scale to other interferometric arrays.
Reading between the lines
- Because the pipeline treats multi-beam detections as RFI, a genuine narrowband signal bright enough to appear through sidelobes or in overlapping coherent beams would be filtered out; injecting synthetic signals at known beam offsets would directly test this.
- The quoted limits use an 8-sigma flux density of 13.92 Jy, while the postprocessing applied an SNR cut of 100; if the higher cut is the true detection threshold, the EIRP limits would be roughly an order of magnitude weaker than stated.
- A natural extension is to run the same pipeline on the full survey database and publish per-star EIRP limit maps, turning the technique into a statistical constraint on the prevalence of transmitting civilizations across a large fraction of the sky.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper describes the COSMIC commensal technosignature processing on the VLA during VLASS, with emphasis on a new postprocessing pipeline: CRICKETS for kurtosis-based RFI flagging and ARTISTIC for candidate filtering. As a pilot, the authors processed 511 Gaia DR2 sources from roughly 30 minutes of VLASS observations, searched by seticore over a Doppler range of ±50 Hz/s, applied kurtosis masks and multi-beam/SNR filters, and found no surviving candidates. They quote equivalent isotropic power limits between 10^11 and 10^16 W based on a 13.92 Jy, 8-sigma sensitivity and an 8 Hz channel width.
Significance. If the quantitative limits are corrected, this is a useful open-source pipeline paper for commensal SETI searches and a cleanly described pilot null result. The software availability, the explicit flowchart of the filtering logic, and the use of a real observational field are strengths. However, the central quantitative claim currently rests on a sensitivity threshold that does not match the pipeline's SNR>100 filter and on an equation that omits the bandwidth; these issues must be fixed before the quoted EIRP limits can be taken at face value.
major comments (3)
- [Section 5, Eq. (3), and abstract] The EIRP limits are computed with Fmin = 13.92 Jy, described as the 8-sigma VLA sensitivity, but the ARTISTIC filtering in the same section retains only events with SNR > 100. If the postprocessing SNR is measured on the same noise statistics as the sensitivity-calculator sigma, all sub-100-sigma events are rejected and the minimum detectable flux is at least a factor of ~12.5 (100/8) higher, shifting the entire quoted 10^11 to 10^16 W range upward by that factor. Please recompute the limits with the actual postprocessing threshold, or justify why the 8-sigma value is still representative despite the SNR>100 cut.
- [Section 5, Eq. (3)] Equation (3) as written, EIRP_min = 4*pi*d^2*Fmin, has units of watts only if Fmin is an integrated flux (W/m^2), but the text states Fmin = 13.92 Jy, which is a spectral flux density (W/m^2/Hz), and then sets the bandwidth to ~8 Hz. The reported lower value for 4.3 pc numerically includes the 8 Hz factor, so the equation is missing the bandwidth term or Fmin is misdefined. Please correct the equation or the definition so that the units and the quoted values agree.
- [Section 4 and Figure 11] The pipeline excludes by construction events detected in all coherent and incoherent beams, and it labels most events seen in four or more beams as likely RFI. If an authentic signal were strong enough to appear in sidelobes or overlapping beams, it would be filtered out, producing a false negative. The text partially mitigates this for nearby beams, but the false-negative rate is not quantified and the unconditional all-beam cut remains a strong assumption. Please state this assumption explicitly in the conclusions, or estimate the expected sidelobe/overlap contamination for the 511 fields.
minor comments (7)
- [Sections 1, 2.5, and 5] The real-time search threshold is given as SNR 8 in Section 1, 'above ten' in Section 2.5, and the postprocessing threshold is 100 in Section 5; please state the real-time threshold unambiguously and distinguish it from the ARTISTIC filter.
- [Sections 2.2 and 5] The test field is dated 15 April 2023 in Section 2.2 but 25 April 2023 in Section 5; please use one consistent date.
- [Section 5] The sentence 'we reduced the 29,390 signals down 77% to 9,708 signals' is arithmetically inconsistent: 9,708 is about 33% of 29,390, so the reduction is about 67%.
- [Section 2.3 and Figure 11] Figure 11 uses a calibration grade threshold of >0.65 while the text defines a good calibration as a grade above 0.6; please reconcile the two values.
- [Section 4] The text contains a broken LaTeX macro ('textsc') immediately before 'CRICKETS'.
- [Section 5] The sentence defining Fmin is grammatically incomplete: 'The Fmin value is determined by dividing the minimum flux density the bandwidth of the transmitting signal' needs to be rephrased and made dimensionally consistent.
- [Abstract and Section 5] The abstract's mention of 950,000 pointings could be misread as the scope of the null result; please state explicitly that the first results are for the 511-source test field only.
Circularity Check
No circularity: the null result is a search outcome and the EIRP limits rest on an external sensitivity calculator and Gaia-based distances; the SNR-threshold mismatch is a correctness concern, not a circular reduction.
full rationale
None of the load-bearing steps reduces to its own input. The reported null result is the output of a filtering pipeline that defines what counts as a candidate, but that is a search criterion rather than a circular derivation: the paper does not define the EIRP limits in terms of the filter output, and the limiting quantities do not come from the pipeline. Equation (3) computes EIRP_min from Fmin = 13.92 Jy taken from the VLA sensitivity calculator and from distances based on Gaia DR2 parallaxes via the Czech et al. (2021) catalog; neither quantity is fitted to the fact that no single-beam candidates survived. The citations to Tremblay et al. (2024) describe the real-time COSMIC hardware, calibration scoring, and beamforming ratio; these are background or standard interferometric relations, not an imported uniqueness theorem that forces the null result. The multibeam rejection rule is an explicit assumption about how a true technosignature would appear and is therefore a potential false-negative risk, but it is not a self-definitional step. The internal inconsistency between the 8-sigma Fmin and the SNR=100 postprocessing cut used in Section 5 is a sensitivity-bookkeeping/correctness issue that could bias the quoted limits, but it is not a circularity because the limit is not derived from the discarded-hit statistics. The target catalog and distances are externally anchored to Gaia, so the Czech et al. self-citation is not load-bearing. Overall the derivation chain is self-contained for circularity purposes.
Assumptions & free parameters
free parameters (5)
- Number of frequency bins in CRICKETS =
256
- Excess kurtosis threshold =
5
- Postprocessing SNR threshold =
100
- Calibration grade threshold =
0.6 / 0.65
- Minimum flux density for EIRP calculation =
13.92 Jy
assumptions (4)
- domain assumption All signals detected in a short observation toward a calibrator are RFI and not signals of interest.
- domain assumption Astronomical noise is Gaussian in each frequency bin, so large excess kurtosis implies RFI.
- domain assumption A genuine technosignature will be detected in only one coherent beam, or otherwise pass the multi-beam logic.
- domain assumption The RFI environment during target observations matches the calibrator observation used to build the kurtosis mask.
Cite this review
Pith. "Pith review of COSMIC's Large-Scale Search for Technosignatures during the VLA sky Survey: Survey Description and First Results." pith.science (2026). https://pith.science/paper/EUOC6O5Y
@misc{pith2026250117997,
author = {Pith},
title = {Pith review of: COSMIC's Large-Scale Search for Technosignatures during the VLA sky Survey: Survey Description and First Results},
year = {2026},
howpublished = {\url{https://pith.science/paper/EUOC6O5Y}},
note = {Machine review of arXiv:2501.17997}
}
abstract
Developing algorithms to search through data efficiently is a challenging part of searching for signs of technology beyond our solar system. We have built a digital signal processing system and computer cluster on the backend of the Karl G. Jansky Very Large Array (VLA) in New Mexico in order to search for signals throughout the Galaxy consistent with our understanding of artificial radio emissions. In our first paper, we described the system design and software pipelines. In this paper, we describe a postprocessing pipeline to identify persistent sources of interference, filter out false positives, and search for signals not immediately identifiable as anthropogenic radio frequency interference during the VLA Sky Survey. As of 01 September 2024, the Commensal Open-source Multi-mode Interferometric Cluster had observed more than 950,000 unique pointings. This paper presents the strategy we employ when commensally observing during the VLA Sky Survey and a postprocessing strategy for the data collected during the survey. To test this postprocessing pipeline, we searched toward 511 stars from the $Gaia$ catalog with coherent beams. This represents about 30 minutes of observation during VLASS, where we typically observe about 2000 sources per hour in the coherent beamforming mode. We did not detect any unidentifiable signals, setting isotropic power limits ranging from 10$^{11}$ to 10$^{16}$W.
Figures
Figures from the paper (12 more)
Forward citations
Cited by 1 Pith paper
-
A search for narrowband technosignatures from LTT 3780 with the Allen Telescope Array and the Karl G. Jansky Very Large Array
No narrowband radio technosignatures were detected from LTT 3780 across ~30 hours of ATA and VLA observations spanning 1–10 GHz, setting EIRP limits of 4.7×10¹²–3.6×10¹³ W.
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter doi edition editor eprint howpublished institution journal key month number organization pages publisher school series title misctitle type volume year version url label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts ...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION format.url url empty "" new.block "" url * "" * if FUNCTION format.eprint eprint empty "" archivePrefix empty "" archivePrefix "arXiv" = new.block " " eprint * " " * new.block " " eprint * " " * if if if FUNCTION format.doi doi empty "" " " doi * " " * if FUNCTION format.pid doi empty eprint empty ur...
-
[3]
e9<3˹|2yk^ ԨN= &6o_ X;o o6ZƤNˏ/ϏC (1͗ͯp|x (G^F51&h:<)n Mdэ>N gBE ( 2 5g>Ě wAE9 wfqL\ @ n̴
thebibliography [1] 20pt to REFERENCES 6pt =0pt -12pt 10pt plus 3pt =0pt =0pt =1pt plus 1pt =0pt =0pt -12pt =13pt plus 1pt =20pt =13pt plus 1pt \@M =10000 =-1.0em =0pt =0pt 0pt =0pt =1.0em @enumiv\@empty 10000 10000 `\.\@m \@noitemerr \@latex@warning Empty `thebibliography' environment \@ifnextchar \@reference \@latexerr Missing key on reference command E...
2021
-
[4]
Astropy Collaboration , Robitaille , T. P., Tollerud , E. J., et al. 2013, , 558, A33, 10.1051/0004-6361/201322068
-
[5]
Briggs, F. H., Bell, J. F., & Kesteven, M. J. 2000, The Astronomical Journal, 120, 3351, 10.1086/316861
-
[6]
2024, , 167, 10, 10.3847/1538-3881/acf576
Choza , C., Bautista , D., Croft , S., et al. 2024, , 167, 10, 10.3847/1538-3881/acf576
-
[7]
Clark , M. A., LaPlante , P. C., & Greenhill , L. J. 2013, International Journal of High Performance Computing Applications, 27, 178, 10.1177/1094342012444794
-
[8]
Comrie , A., Wang , K.-S., Ford , P., et al. 2020, CARTA: The Cube Analysis and Rendering Tool for Astronomy , 1.3.0, Zenodo, 10.5281/zenodo.3377984
Show all 47 references
-
[9]
2021, , 133, 064502, 10.1088/1538-3873/abf329
Czech , D., Isaacson , H., Pearce , L., et al. 2021, , 133, 064502, 10.1088/1538-3873/abf329
2021 doi
-
[10]
F., Hinsen , K., & Hugunin , J
Dubois , P. F., Hinsen , K., & Hugunin , J. 1996, Comput. Phys. Commun., 10, 262
1996
-
[11]
2008, The Big Ear Wow! Signal
Ehman, J. 2008, The Big Ear Wow! Signal. http://www.bigear.org/wow20th.htm
2008
-
[12]
2019, turboSETI: Python-based SETI search algorithm , Astrophysics Source Code Library, record ascl:1906.006
Enriquez , E., & Price , D. 2019, turboSETI: Python-based SETI search algorithm , Astrophysics Source Code Library, record ascl:1906.006. 1906.006
2019
-
[13]
D., & Fuller , G
Etoka , S., Gray , M. D., & Fuller , G. A. 2012, , 423, 647, 10.1111/j.1365-2966.2012.20900.x
2012
-
[14]
I., Siemion , A
Gajjar , V., Perez , K. I., Siemion , A. P. V., et al. 2021, , 162, 33, 10.3847/1538-3881/abfd36
2021 doi
-
[15]
1999, Philosophical Transactions: Mathematical, Physical and Engineering Sciences, 357, 3277
Gray, M. 1999, Philosophical Transactions: Mathematical, Physical and Engineering Sciences, 357, 3277. http://www.jstor.org/stable/1353849
1999
-
[16]
J., Jacobs , D
Hazelton , B. J., Jacobs , D. C., Pober , J. C., & Beardsley , A. P. 2017, The Journal of Open Source Software, 2, 140, 10.21105/joss.00140
2017 doi
-
[17]
2019, in Bulletin of the American Astronomical Society, Vol
Hickish , J., Beasley , T., Bower , G., et al. 2019, in Bulletin of the American Astronomical Society, Vol. 51, 269
2019
-
[18]
Houston , K. M. 2023, Acta Astronautica, 212, 505, 10.1016/j.actaastro.2023.08.009
2023 doi
-
[19]
2023, , 166, 245, 10.3847/1538-3881/ad06b1
Huang , B.-L., Tao , Z.-Z., & Zhang , T.-J. 2023, , 166, 245, 10.3847/1538-3881/ad06b1
2023 doi
-
[20]
Hunter , J. D. 2007, Comput. Sci. Eng., 9, 90, 10.1109/MCSE.2007.55
2007 doi
-
[21]
A., Gajjar , V., Keane , E
Johnson , O. A., Gajjar , V., Keane , E. F., et al. 2023, , 166, 193, 10.3847/1538-3881/acf9f5
2023 doi
-
[23]
A., Chandler, C
Lacy, M., Baum, S. A., Chandler, C. J., et al. 2020, Publications of the Astronomical Society of the Pacific, 132, 035001, 10.1088/1538-3873/ab63eb
2020 doi
-
[24]
2022, , 938, 1, 10.3847/1538-4357/ac90bd
Li , J.-K., Zhao , H.-C., Tao , Z.-Z., Zhang , T.-J., & Xiao-Hui , S. 2022, , 938, 1, 10.3847/1538-4357/ac90bd
2022 doi
-
[25]
2024, in American Astronomical Society Meeting Abstracts, Vol
Li , M., & Margot , J.-L. 2024, in American Astronomical Society Meeting Abstracts, Vol. 243, American Astronomical Society Meeting Abstracts, 159.06
2024
-
[27]
2023 b , , 166, 182, 10.3847/1538-3881/acf83d
---. 2023 b , , 166, 182, 10.3847/1538-3881/acf83d
2023 doi
-
[29]
2023 b , Nature Astronomy, 7, 492, 10.1038/s41550-022-01872-z
---. 2023 b , Nature Astronomy, 7, 492, 10.1038/s41550-022-01872-z
2023 doi
-
[30]
MacMahon , D. H. E., Price , D. C., Lebofsky , M., et al. 2018, , 130, 044502, 10.1088/1538-3873/aa80d2
2018 doi
-
[31]
P., Waters , B., Schiebel , D., Young , W., & Golap , K
McMullin , J. P., Waters , B., Schiebel , D., Young , W., & Golap , K. 2007, in Astronomical Society of the Pacific Conference Series, Vol. 376, Astronomical Data Analysis Software and Systems XVI, ed. R. A. Shaw , F. Hill , & D. J. Bell , 127
2007
- [32]
-
[33]
M., & Gary , D
Nita , G. M., & Gary , D. E. 2010, , 406, L60, 10.1111/j.1745-3933.2010.00882.x
2010
-
[34]
M., Sadykov , V
Nita , G. M., Sadykov , V. M., Kosovichev , A. G., & Oria , V. 2019, in AGU Fall Meeting Abstracts, Vol. 2019, SH31E--3340
2019
-
[35]
Oliphant , T. E. 2007, Comput. Sci. Eng., 9, 10, 10.1109/MCSE.2007.58
2007 doi
-
[36]
2016, Jansky Very Large Array Primary Beam Characteristics , Tech
Perley , R. 2016, Jansky Very Large Array Primary Beam Characteristics , Tech. rep., National Radio Astronomy Observatory (NRAO) . https://library.nrao.edu/public/memos/evla/EVLAM_195.pdf
2016
-
[37]
C., Enriquez , J
Price , D. C., Enriquez , J. E., Brzycki , B., et al. 2020, , 159, 86, 10.3847/1538-3881/ab65f1
2020 doi
-
[39]
2019 b , , 884, 14, 10.3847/1538-4357/ab3fa8
---. 2019 b , , 884, 14, 10.3847/1538-4357/ab3fa8
2019 doi
-
[40]
Z., Smith , S., Price , D
Sheikh , S. Z., Smith , S., Price , D. C., et al. 2021, Nature Astronomy, 5, 1153, 10.1038/s41550-021-01508-8
2021 doi
-
[41]
Siemion , A. P. V., Demorest , P., Korpela , E., et al. 2013, , 767, 94, 10.1088/0004-637X/767/1/94
2013 doi
-
[42]
S., & Pisano , D
Smith , E., Lynch , R. S., & Pisano , D. J. 2022, , 164, 123, 10.3847/1538-3881/ac7e47
2022 doi
-
[43]
C., Sheikh , S
Smith , S., Price , D. C., Sheikh , S. Z., et al. 2021, Nature Astronomy, 5, 1148, 10.1038/s41550-021-01479-w
2021 doi
-
[44]
2023, , 166, 190, 10.3847/1538-3881/acfc1e
Tao , Z.-Z., Huang , B.-L., Luan , X.-H., et al. 2023, , 166, 190, 10.3847/1538-3881/acfc1e
2023 doi
-
[45]
Taylor , M. B. 2005, in ASP Conference Series, Vol. 347, Astronomical Data Analysis Software and Systems XIV, ed. P. Shopbell , M. Britton , & R. Ebert , 29
2005
-
[46]
D., Varghese , S
Tremblay , C. D., Varghese , S. S., Hickish , J., et al. 2024, , 167, 35, 10.3847/1538-3881/ad0fe0
2024 doi
-
[47]
2023, , 166, 146, 10.3847/1538-3881/acf12a
Wang , Y.-C., Tao , Z.-Z., Zhang , Z.-S., et al. 2023, , 166, 146, 10.3847/1538-3881/acf12a
2023 doi
-
[48]
1997, Astron
Weber , R., Faye , C., Biraud , F., & Dansou , J. 1997, Astron. Astrophys. Suppl. Ser., 126, 161, 10.1051/aas:1997257
1997 doi
-
[49]
H., Billingham , J., Edelson , R
Wolfe , J. H., Billingham , J., Edelson , R. E., et al. 1981, in NASA Conference Publication, Vol. 2156, NASA Conference Publication, ed. J. Billingham , 391
1981
-
[50]
T., Kanodia, S., & Lubar, E
Wright, J. T., Kanodia, S., & Lubar, E. 2018, The Astronomical Journal, 156, 260, 10.3847/1538-3881/aae099
2018 doi
-
[51]
2020, , 891, 174, 10.3847/1538-4357/ab7376
Zhang , Z.-S., Werthimer , D., Zhang , T.-J., et al. 2020, , 891, 174, 10.3847/1538-4357/ab7376
2020 doi
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.