{"id":"a5c5518d-2276-4a64-ba74-e229aec1864d","arxiv_id":"2501.17997","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A new postprocessing pipeline for the COSMIC/VLA technosignature search found no candidate signals toward 511 Gaia stars in a 30-minute VLASS test, setting isotropic power limits of 10^11 to 10^16 W.","lead":"The COSMIC project searched 511 nearby stars for artificial radio signals during a 30-minute stretch of the VLA Sky Survey and found no candidates. The paper describes the filtering pipeline that turned millions of raw detections into a clean null result, and it gives upper limits on transmitter power, between 10^11 and 10^16 watts, for the stars examined.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The quoted EIRP limits use an 8-sigma (13.92 Jy) threshold while the ARTISTIC pipeline actually retained only SNR>100 events; the limits are optimistic by at least an order of magnitude and need recomputation.","rationale":"I read the paper in good faith as a survey-methods paper with a modest, honestly presented null result. The central claim has two parts: no unidentifiable signals were detected, and the search sets isotropic power limits of 10^11–10^16 W. The first part is credible: the pipeline reduced 29,390 hits to zero surviving candidates, and the paper does not overstate this as a detection. The second part is where the argument is weakest. The EIRP calculation uses an 8-sigma sensitivity of 13.92 Jy from the VLA sensitivity calculator, but the postprocessing pipeline explicitly applies an SNR threshold of 100 before any beam-based filtering. That means signals just above the quoted 8-sigma level were never candidates, so the limits do not describe the search as executed. The factor is at least 12.5 in flux (100/8), and could be larger if the VLA sensitivity calculator does not correspond to the coherent-beam seticore SNR. The multibeam assumption is a genuine false-negative risk, but it is less decisive because the pipeline has branches that examine multi-beam detections via the coherent/incoherent ratio and source proximity; the threshold mismatch, by contrast, is a direct inconsistency between the stated sensitivity and the implemented filter. The reader's verdict of CONDITIONAL is therefore appropriate: the paper should be revised to recompute the limits with the actual threshold, or to present a measured sensitivity curve from injections. My concern does not move the verdict; it reinforces the need for the condition. I marked agreement as partial because the reader also identifies the SNR mismatch as a related fragility, but frames the primary weakest assumption as the single-beam RFI cut, whereas I view the threshold mismatch as the more load-bearing issue for the abstract's quantitative claim.","tokens_in":17010,"tokens_out":7635,"duration_ms":78723,"concrete_test":"Inject synthetic narrowband, drifting signals with known flux densities spanning 13.92 Jy to roughly 200 Jy into the raw voltage stream for a subset of the 511 targets, then run the full seticore + ARTISTIC pipeline. The recovered SNR and pass/fail status will directly measure the minimum detectable EIRP; if an 8-sigma-equivalent injection does not survive the SNR=100 cut, recompute the abstract's EIRP range using the measured threshold and update the quoted values.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The qualitative null result (no unidentifiable signals) is plausible, but the quantitative central claim—isotropic power limits from 10^11 to 10^16 W—rests on a sensitivity threshold that the pipeline did not actually apply. Section 5 states that after the CRICKETS kurtosis mask, ARTISTIC uses \"an SNR threshold of 100\" and reduces 29,390 real-time hits to 9,708. The EIRP calculation in the same section instead uses F_min = 13.92 Jy, described as the 8-sigma VLA sensitivity limit, together with an 8 Hz channel width. Because every event below SNR=100 was discarded, the search could not have detected an 8-sigma signal; a 100-sigma-equivalent flux is at least 12.5 times larger if sensitivities scale linearly, shifting the entire EIRP range upward and changing the abstract's headline numbers. This is compounded by inconsistent thresholds across the text: Section 1 says SNR=8, Section 2.5 says \"above ten,\" and Section 5 uses 100. The multibeam RFI cut (Figure 11) is a real but secondary false-negative risk; the threshold mismatch is more load-bearing because it directly undermines the quantitative limits contained in the strongest claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper describes the COSMIC commensal technosignature processing on the VLA during VLASS, with emphasis on a new postprocessing pipeline: CRICKETS for kurtosis-based RFI flagging and ARTISTIC for candidate filtering. As a pilot, the authors processed 511 Gaia DR2 sources from roughly 30 minutes of VLASS observations, searched by seticore over a Doppler range of ±50 Hz/s, applied kurtosis masks and multi-beam/SNR filters, and found no surviving candidates. They quote equivalent isotropic power limits between 10^11 and 10^16 W based on a 13.92 Jy, 8-sigma sensitivity and an 8 Hz channel width.","tokens_in":17356,"tokens_out":7475,"duration_ms":75625,"significance":"If the quantitative limits are corrected, this is a useful open-source pipeline paper for commensal SETI searches and a cleanly described pilot null result. The software availability, the explicit flowchart of the filtering logic, and the use of a real observational field are strengths. However, the central quantitative claim currently rests on a sensitivity threshold that does not match the pipeline's SNR>100 filter and on an equation that omits the bandwidth; these issues must be fixed before the quoted EIRP limits can be taken at face value.","major_comments":[{"comment":"The EIRP limits are computed with Fmin = 13.92 Jy, described as the 8-sigma VLA sensitivity, but the ARTISTIC filtering in the same section retains only events with SNR > 100. If the postprocessing SNR is measured on the same noise statistics as the sensitivity-calculator sigma, all sub-100-sigma events are rejected and the minimum detectable flux is at least a factor of ~12.5 (100/8) higher, shifting the entire quoted 10^11 to 10^16 W range upward by that factor. Please recompute the limits with the actual postprocessing threshold, or justify why the 8-sigma value is still representative despite the SNR>100 cut.","section":"Section 5, Eq. (3), and abstract"},{"comment":"Equation (3) as written, EIRP_min = 4*pi*d^2*Fmin, has units of watts only if Fmin is an integrated flux (W/m^2), but the text states Fmin = 13.92 Jy, which is a spectral flux density (W/m^2/Hz), and then sets the bandwidth to ~8 Hz. The reported lower value for 4.3 pc numerically includes the 8 Hz factor, so the equation is missing the bandwidth term or Fmin is misdefined. Please correct the equation or the definition so that the units and the quoted values agree.","section":"Section 5, Eq. (3)"},{"comment":"The pipeline excludes by construction events detected in all coherent and incoherent beams, and it labels most events seen in four or more beams as likely RFI. If an authentic signal were strong enough to appear in sidelobes or overlapping beams, it would be filtered out, producing a false negative. The text partially mitigates this for nearby beams, but the false-negative rate is not quantified and the unconditional all-beam cut remains a strong assumption. Please state this assumption explicitly in the conclusions, or estimate the expected sidelobe/overlap contamination for the 511 fields.","section":"Section 4 and Figure 11"}],"minor_comments":[{"comment":"The real-time search threshold is given as SNR 8 in Section 1, 'above ten' in Section 2.5, and the postprocessing threshold is 100 in Section 5; please state the real-time threshold unambiguously and distinguish it from the ARTISTIC filter.","section":"Sections 1, 2.5, and 5"},{"comment":"The test field is dated 15 April 2023 in Section 2.2 but 25 April 2023 in Section 5; please use one consistent date.","section":"Sections 2.2 and 5"},{"comment":"The sentence 'we reduced the 29,390 signals down 77% to 9,708 signals' is arithmetically inconsistent: 9,708 is about 33% of 29,390, so the reduction is about 67%.","section":"Section 5"},{"comment":"Figure 11 uses a calibration grade threshold of >0.65 while the text defines a good calibration as a grade above 0.6; please reconcile the two values.","section":"Section 2.3 and Figure 11"},{"comment":"The text contains a broken LaTeX macro ('textsc') immediately before 'CRICKETS'.","section":"Section 4"},{"comment":"The sentence defining Fmin is grammatically incomplete: 'The Fmin value is determined by dividing the minimum flux density the bandwidth of the transmitting signal' needs to be rephrased and made dimensionally consistent.","section":"Section 5"},{"comment":"The abstract's mention of 950,000 pointings could be misread as the scope of the null result; please state explicitly that the first results are for the 511-source test field only.","section":"Abstract and Section 5"}],"recommendation":"major_revision","confidential_remarks":"The paper is best suited as an instrument/pipeline paper, and the pilot null result is a reasonable demonstration. The central quantitative limits need correction, but the problems are fixable within the manuscript's scope: the EIRP limits must be recomputed with the actual postprocessing threshold, and Eq. (3) must be repaired. I would not recommend rejection on these grounds."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the honest headline: this is a useful, modest instrument paper. The qualitative null result is credible, but the quantitative upper limits are built on a sensitivity threshold the pipeline never actually applied. Fix that and the paper becomes a solid methods-plus-first-results contribution.\n\nWhat's genuinely new: the ARTISTIC postprocessing pipeline and the CRICKETS kurtosis-based RFI mask. The paper documents the multibeam coincidence logic and the dynamic spectra checks, and the team has made the software public. That's real, reproducible engineering. The null result toward 511 Gaia stars from 30 minutes of VLASS is new, and the paper is appropriately careful to call it a test, not a survey conclusion. The Voyager check is a nice sanity test, and the W51 maser plot gives some confidence the beamformer is pointing where they think.\n\nThe soft spots are real. The stress-test note is correct: Section 5 says they applied an SNR threshold of 100, while the EIRP calculation uses F_min = 13.92 Jy, described as the 8-sigma VLA sensitivity limit. That's roughly a factor of 12.5 in flux if sensitivity scales linearly. The quoted 10^11 to 10^16 W limits are therefore likely an order of magnitude too low. There are also inconsistent SNR thresholds elsewhere: Section 1 says 8, Section 2.5 says 'above ten,' and Section 5 uses 100. The kurtosis threshold of 5 is described as optimal but with no sensitivity analysis for how it trades off against real signals. The multibeam rejection—requiring a signal to appear in only one beam—is a genuine false-negative risk, though it is secondary because most RFI does appear across beams.\n\nNone of this kills the qualitative claim. It is plausible that no unidentifiable signals were present in this 30-minute field. But the paper's central numbers need recomputation, and the threshold choices need a sensitivity analysis before the limits can be quoted. This deserves a serious referee: a careful revision with corrected EIRP values and a robustness test would make it a trustworthy reference. I'd send it to review, conditional.","headline":"The qualitative null result is credible, but the quoted EIRP limits are built on an SNR threshold the pipeline never applied; the numbers need recomputation before the paper can be trusted.","tokens_in":17879,"tokens_out":2275,"would_cite":false,"duration_ms":21848,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A postprocessing pipeline for a commensal interferometric SETI search returned no candidate signals toward 511 stars, setting isotropic power limits between roughly 2.3e11 and 2.1e16 watts.","keywords":["technosignatures","SETI","radio frequency interference","coherent beamforming","commensal observing","isotropic radiated power limits","narrowband signal search","postprocessing pipeline"],"falsifier":"Inject a synthetic narrowband signal into a target's coherent beam and also, at lower amplitude, into an adjacent beam with a known overlap; if the ARTISTIC pipeline discards it as RFI, the single-beam assumption is falsified for realistic spillover. Separately, recompute EIRP_min with an SNR threshold of 100 instead of 8; if the limits rise by a factor roughly equal to 100/8, the published sensitivity range is too optimistic.","tokens_in":16768,"feed_emoji":"📡","tokens_out":8117,"duration_ms":79713,"temperature":0.7,"pith_summary":"The paper reports the first large-scale test of COSMIC, a digital signal processing system that piggybacks on a radio sky survey to search for narrowband artificial signals. To turn millions of raw detections into a manageable list, the authors built a postprocessing pipeline, ARTISTIC, that filters out radio frequency interference using a kurtosis-based frequency mask, a signal-to-noise cut, and the rule that a genuine source should appear in only one coherent beam. Applied to 511 stars from an astrometric catalog in roughly 30 minutes of survey data, the pipeline reduced thousands of hits to zero unidentifiable signals. If this result stands, it means that among the surveyed stars and frequencies, no transmitter with an equivalent isotropic power between about $10^{11}$ and $10^{16}$ watts was active and pointed so that we could see it. The broader aim is to show that commensal observing plus automated filtering can search nearly a million pointings without overwhelming human review.","feed_headline":"No alien radio signals found around 511 nearby stars","feed_subtitle":"A new filter pipeline cut 29,390 candidate hits to zero, probing transmitters as weak as 10^11 watts.","key_machinery":"The machinery is the ARTISTIC pipeline together with the CRICKETS RFI mask that feeds it. CRICKETS flags frequency bins whose excess kurtosis deviates from Gaussian noise, using data from a calibrator observation to produce a list of dirty channels to blank. ARTISTIC then applies an SNR threshold, removes hits seen in many beams, and keeps signals found in only one coherent beam for dynamic-spectrum checks; for hits in two or more beams it compares source proximity and the coherent-to-incoherent power ratio, which is expected to equal the number of antennas used in the beamformer. The sensitivity claim is carried by the distance-squared EIRP formula, which converts each star's minimum detectable flux into a transmitter power limit.","core_discovery":"The central claim is that a logical, automated postprocessing chain can separate astrophysical narrowband signals from terrestrial interference well enough to run a wide-area technosignature search without manual inspection of every hit. The authors demonstrate this on 511 stars: after applying the CRICKETS kurtosis mask and an SNR cut of 100, the remaining hits were distributed across all beams, and no signal was confined to a single coherent beam in the way the pipeline expects of an astronomical emitter. They therefore report no unidentifiable signals and set equivalent isotropic radiated power limits of 2.32e11 to 2.09e16 W for the observed stars, by computing EIRP_min = 4*pi*$d^{2}$*F_min with F_min = 13.92 Jy at an assumed 8-$\\sigma$ sensitivity and a roughly 8 Hz channel width. The survey has recorded more than 950,000 unique pointings since 2023, so the pipeline is presented as the route to searching that entire database.","pith_inferences":["Because the pipeline treats multi-beam detections as RFI, a genuine narrowband signal bright enough to appear through sidelobes or in overlapping coherent beams would be filtered out; injecting synthetic signals at known beam offsets would directly test this.","The quoted limits use an 8-sigma flux density of 13.92 Jy, while the postprocessing applied an SNR cut of 100; if the higher cut is the true detection threshold, the EIRP limits would be roughly an order of magnitude weaker than stated.","A natural extension is to run the same pipeline on the full survey database and publish per-star EIRP limit maps, turning the technique into a statistical constraint on the prevalence of transmitting civilizations across a large fraction of the sky."],"forward_implications":["The same filter chain can be applied to the 950,000 recorded pointings, converting a database of millions of hourly hits into a short candidate list without human review.","For the nearest stars in the sample, the null result rules out transmitters weaker than about 10^11 W, assuming the single-beam criterion is valid, meaning even modest planetary radars would have been seen.","Because COSMIC observes in commensal mode, expanding the search to the full survey adds no extra telescope time, so the technique can ride along on future large sky surveys.","The pipeline's decision logic is simple enough to be reimplemented or learned by a classifier, so the method could scale to other interferometric arrays."],"supporting_citations":[{"why":"Describes COSMIC's system design, calibration scoring, and beamforming; the new pipeline is the postprocessing layer for that system.","marker":"Tremblay et al. 2024"},{"why":"Supplies the technosignature target catalog and the parallax distances used to convert flux limits into EIRP limits.","marker":"Czech et al. 2021"},{"why":"Defines the sky survey's tile and on-the-fly observing strategy that COSMIC rides along with.","marker":"Lacy et al. 2020"},{"why":"Establishes the tree-summing dedoppler search and the acceleration range parameters used by the real-time search.","marker":"Sheikh et al. 2019a"},{"why":"Introduced the spectral kurtosis estimator that CRICKETS adapts for RFI flagging.","marker":"Nita & Gary 2010"},{"why":"Describes the noise statistics and limitations of the SNR estimates that ARTISTIC relies on.","marker":"Choza et al. 2024"},{"why":"Provides the reference planetary-radar EIRP used to express the survey's sensitivity in transmitter-power units.","marker":"Siemion et al. 2013"}],"fun_headline_variants":["Alien hunt draws blank: 511 stars, zero signals","VLA scans 511 stars, finds no artificial radio","Technosignature search: 511 stars, no hits","No tech signals from 511 stars in VLA survey","Alien radio search around 511 stars comes up empty"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The search's null result depends on treating any signal seen in more than one coherent beam as terrestrial interference, and it also assumes the 8-sigma flux limit of 13.92 Jy, not the actual signal-to-noise cut of 100, is the sensitivity floor.","fun_headline_variants_meta":{"raw":{"variants":["Alien hunt draws blank: 511 stars, zero signals","VLA scans 511 stars, finds no artificial radio","Technosignature search: 511 stars, no hits","No tech signals from 511 stars in VLA survey","Alien radio search around 511 stars comes up empty"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00094,"raw_usage":{"total_tokens":4048,"prompt_tokens":1007,"completion_tokens":3041,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":623,"completion_tokens_details":{"reasoning_tokens":2958}},"tokens_in":623,"tokens_out":3041,"duration_ms":22710,"temperature":1.0,"reasoning_tokens":2958,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T04:25:24.912114+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inject a synthetic narrowband signal into a target's coherent beam and also, at lower amplitude, into an adjacent beam with a known overlap; if the ARTISTIC pipeline discards it as RFI, the single-beam assumption is falsified for realistic spillover. Separately, recompute EIRP_min with an SNR threshold of 100 instead of 8; if the limits rise by a factor roughly equal to 100/8, the published sensitivity range is too optimistic.","supporting_citations":[],"review_version":1}