{"id":"97b3f773-364e-494f-b20e-63116e36b88c","arxiv_id":"1909.00561","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":6,"one_line_summary":"A refined analysis of Murchison Widefield Array observations sets a new upper limit of 3.9 x 10^3 mK^2 on the 21 cm power spectrum at k = 0.20 h Mpc^-1 and z = 7, nearly an order of magnitude better than prior MWA limits.","lead":"Astronomers improved the search for faint radio signals from the early universe, using better data analysis and radio interference removal on existing telescope data. Their new upper limit is the strongest yet from the Murchison Widefield Array, nearly ten times tighter than the previous MWA result.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Abstract's 'noise-dominated' label for the 21-hr limit is contradicted by the paper's own numbers: the quoted k=0.203 N-S bin shows a ~5.5σ excess over thermal noise.","rationale":"Good-faith reading: the paper's purpose is to demonstrate that improved analysis and RFI excision lower the MWA EoR upper limit, and the headline is the 21-hr Δ²≤3.9×10^3 mK² limit. For that claim to hold as stated, the limit should be noise-dominated and a nearly order-of-magnitude improvement. The body explicitly says the all-pointing 21-hr limit is systematic dominated, with a noise-dominated counterpart only in the zenith-pointing subset. The table confirms: the quoted bin has a 5.5σ positive excess over thermal noise, so it is not noise-dominated. This is not a stylistic quibble: 'noise-dominated' changes the physical interpretation and the scaling with integration time. The reader's conditional verdict already identified the abstract/body inconsistency as the main issue. The reader's weakest_assumption about selection bias in the data-informed mask and RFI cuts is legitimate and is acknowledged in §5.2; however, for the specific quoted bin the positive 5.5σ excess means the upper limit is not set by a downward noise fluctuation, so the selection-bias concern is secondary to the mislabeling. A corrected abstract that attributes 'noise-dominated' only to the zenith-pointing limit would preserve the numerical result while changing the headline claim. Therefore the verdict remains conditional; no change to the reader's verdict.","tokens_in":23454,"tokens_out":12259,"duration_ms":370034,"concrete_test":"Using Table B, take the quoted N-S k=0.203 h Mpc^-1 row: measured power Δ²=2.85×10^3 mK², 1σ thermal noise=5.21×10^2 mK², and compute their ratio. If the ratio is ~5.5 (as listed), the 'noise-dominated' label in the abstract is inconsistent with the paper's own numbers; the noise-dominated claim should instead refer to the zenith-pointing subset limit in §5.2.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim as stated in the abstract is that the 21-hr integration produces a noise-dominated upper limit of Δ²≤3.9×10^3 mK² at k=0.20 h Mpc^-1 and z=7. The body directly contradicts this. Section 5.2 states that 'our lowest limits are systematic dominated' and that a similar, noise-dominated limit (Δ²≤3.8×10^3 mK² at k=0.23) is obtained only from the zenith-pointing subset. Table B shows the discrepancy quantitatively: for the quoted N-S bin at k=0.203, the measured power is 2.85×10^3 mK² and the 1σ thermal noise is 5.21×10^2 mK², a ratio of about 5.5. If the bin were noise-dominated, the measured power would be consistent with zero within the noise; instead the 2σ upper limit of 3.89×10^3 mK² is set by this positive excess. The abstract's qualifier is therefore inaccurate. This matters because the paper's own discussion says a systematic-dominated limit would not benefit significantly from further integration, whereas a noise-dominated limit would. The numerical upper limit may survive, but the central claim as written does not.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents an improved Epoch of Reionization 21 cm power spectrum analysis of Murchison Widefield Array Phase I observations, using the FHD/eppsilon pipeline with several precision upgrades, and combines these with SSINS-based RFI identification to select a 21-hour subset of the original 32-hour data set. The central result is a 2σ upper limit of Δ² ≤ 3.9×10³ mK² at k = 0.20 h Mpc⁻¹ and z ≈ 7 in the N–S polarization, which the abstract describes as a noise-dominated limit that improves previous MWA limits by almost an order of magnitude. The paper supports the result with a direct reanalysis of the Beardsley et al. (2016) data set, a comparison against the independent RTS/CHIPS pipeline, an end-to-end simulation that recovers injected EoR signal, and a comparison of observed to expected noise.","tokens_in":23741,"tokens_out":4953,"duration_ms":48140,"significance":"If the reported limit is robust, it would be the best MWA EoR power spectrum upper limit at z ≈ 7, improving on Beardsley et al. (2016) by roughly a factor of 6 in power, and it would be competitive with contemporary LOFAR limits though at different k and redshift. The paper's strengths are substantial: the analysis is built on open-source software and public data; the end-to-end in-situ simulation recovers the input EoR power across a wide range of k-modes; the FHD/eppsilon results are cross-checked against an independent RTS/CHIPS pipeline; and the observed-to-expected noise ratio in §6.3 is close to unity, validating the error propagation. However, the interpretation of the headline limit as noise-dominated is contradicted by the paper's own body and Table 1, and the data-driven choice of foreground-avoidance masks requires a robustness test before the numerical upper limit can be accepted at face value.","major_comments":[{"comment":"The abstract states that the 21-hour integrated limit Δ² ≤ 3.9×10³ mK² at k = 0.20 h Mpc⁻¹ is a \"noise-dominated upper limit,\" but the body explicitly says the opposite. Section 5.2 states that \"our lowest limits are systematic dominated\" and that a noise-dominated limit is obtained only from the zenith-pointing subset, at Δ² ≤ 3.8×10³ mK² and k = 0.23 h Mpc⁻¹. Table 1 quantifies this: for the N–S bin at k = 0.203, the measured power is 2.85×10³ mK², the 1σ thermal noise is 5.21×10² mK², and the 2σ upper limit is 3.89×10³ mK². The measured power is therefore a ~5.5σ positive excess over the thermal noise, so the quoted upper limit is set by a positive systematic excess, not by noise. The final paragraph of §7 repeats the same inconsistency by claiming that \"we are now in the regime where we are noise dominated in our lowest limit.\" The numerical upper limit may be correct, but the central claim as written is not; the abstract, introduction, and conclusion must be revised to distinguish the systematic-dominated 21-hour limit from the noise-dominated zenith-pointing limit.","section":"Abstract, §5.2, §7, Table 1"},{"comment":"The foreground-avoidance mask is chosen using the same data that are then used to produce the upper limit. The k⊥ range of 18–80 λ, the k∥ ≥ 0.15 h Mpc⁻¹ cut, the 15% horizon-slope buffer, and the 40% RFI-occupancy cut in §5.1 are all selected after inspecting the 2D power spectra and SSINS statistics of the actual 678-observation data set. The paper acknowledges a potential selection bias but argues that using only foreground information lowers the degrees of freedom. This argument does not address the possibility that the specific mask thresholds are inadvertently tuned to a favorable noise realization in the EoR window. Because the headline limit is driven by a 5.5σ positive excess rather than by thermal noise, the robustness of the quoted limit to reasonable variations of these thresholds should be demonstrated—for example, with noise-only simulations, bootstrapped mask variations, or a scan over mask parameters. Without such a test, the significance of the reported limit relative to other values in the same table is not fully established.","section":"§5.2"},{"comment":"The claimed improvement over Beardsley et al. (2016) relies on a flux-scale correction factor of 28% in E–W and 23% in N–S, derived from the ratio of mean calibration amplitudes between the KGS/MWACS-based catalog and GLEAM. The comparison in Figure 3 and the factor-of-2.8 improvement in §4.2 are only as reliable as this correction, but no uncertainty on the scale factor is reported or propagated into the comparison. Since the new 21-hour limit uses GLEAM directly, this does not affect that limit by construction, but it does affect the specific quantitative claim of \"improving previous MWA limits by almost an order of magnitude.\" A short statement of the systematic uncertainty in the scale correction, or a sensitivity check on the comparison, would make the relative-improvement claim robust.","section":"§4"}],"minor_comments":[{"comment":"The abstract says \"almost an order of magnitude\" improvement over previous MWA limits; comparing the rebinned Beardsley value of 2.37×10⁴ mK² in §4.2 with 3.9×10³ mK² gives a factor of about 6.1, which is fairly described as almost an order of magnitude, but a reader may expect a factor closer to 10; consider stating the explicit factor.","section":"Abstract and §7"},{"comment":"The RFI-occupancy cut is described as removing observations where \"over 40% of the SSINS samples were identified as contaminated,\" with a footnote that coarse-band edges are excluded from this classification. It would be helpful to state whether this 40% threshold was fixed a priori or selected after inspecting the distribution of occupancy values in the 1029-observation set, since the latter would tie this cut to the same selection-bias concern as the k-space mask.","section":"§5.1"},{"comment":"The observed-to-expected noise ratio is described as \"very close to 1\" with deviations only in poor uv-coverage regions that do not enter the 1D power spectrum. It would improve clarity to give the numerical range of the ratio in the region used for the 1D limits, rather than only a visual statement, since this is the key validation of the error bars that determine the upper limit.","section":"§6.3 and Figure 10"},{"comment":"The RTS/CHIPS comparison applies the FHD/eppsilon binning scheme to the independent pipeline, and the text notes that this may not be optimal for RTS/CHIPS. It would be useful to state explicitly whether the same flux-scale reference (GLEAM) was used in the RTS/CHIPS calibration, so that the comparison is not affected by the catalog-based scale difference discussed in §4.","section":"§6.1"},{"comment":"The table caption lists the 2σ upper limit, lower uncertainty bound, measured power, and thermal noise, but it does not define how the upper limit is constructed from the measured power and variance. A sentence stating that the limit is Δ² + 2σ with a non-negativity prior, as described in §5.2 and Figure 6, would make the table self-contained.","section":"Appendix B and Table 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid observational upper-limit work from the MWA collaboration, with credible validation procedures and a result that is likely correct. My main concern is not the numerical pipeline but the presentation of the headline limit: the abstract's 'noise-dominated' characterization is contradicted by the paper's own Table 1 and body text, and the data-driven mask choice deserves a robustness test. Both issues are fixable in revision. I do not see grounds for rejection, but the current abstract overstates what the 21-hour limit demonstrates."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe thing to know first: the headline number — Δ² ≤ 3.9×10³ mK² at k=0.20 h Mpc⁻¹, z=7 from 21 hr of MWA data — looks real, and it is a genuine improvement over Beardsley et al. 2016 by roughly an order of magnitude. The analysis is careful: they validate with an independent RTS/CHIPS pipeline, inject an EoR signal in end-to-end simulations and recover it, and show observed-to-expected noise ratio near unity. Data and software are public. That is solid, reproducible work, and the new pipeline elements (Blackman-Harris tapered gridding kernel, auto-visibility calibration, average-removal in the frequency transform, SSINS-based observation excision) are real contributions.\n\nBut the abstract is wrong about the nature of the limit. It calls the 21-hr limit \"noise-dominated,\" while Section 5.2 and Appendix B show the opposite: at the quoted k=0.203 N-S bin the measured power is about 5.5σ above thermal noise, so the limit is set by a positive excess and is systematic-dominated. The paper says this clearly in the body, and even offers a separate noise-dominated limit from the zenith-pointing subset. The numerical limit may survive, but the abstract's characterization does not. That needs fixing before publication.\n\nThe softer spot is the one the reader flagged: the foreground-avoidance mask and RFI-occupancy cuts are chosen using the same 2D power spectra and SSINS statistics being analyzed, and the paper acknowledges but does not quantify selection bias. This is a real but manageable concern; a noise-only simulation or bootstrapped mask variations would put a bound on how much the cuts could bias the limit. It does not look load-bearing, but it's worth asking for.\n\nMinor things: the E-W/N-S discrepancy and the unknown systematic at low k⊥ are left unexplained, which is fine for a limits paper, but the discussion leans on \"future work\" a bit.\n\nWho this is for: anyone working on 21 cm EoR power spectra. It is a serious paper that deserves a serious referee. My recommendation: send it out, but require the authors to reconcile the abstract with the body and ideally quantify the mask-selection bias. The result will probably stand; the framing needs to be honest.","headline":"A genuinely improved MWA EoR upper limit with solid validation, but the abstract's 'noise-dominated' claim is contradicted by the paper's own numbers.","tokens_in":24485,"tokens_out":2279,"would_cite":true,"duration_ms":26086,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Improved analysis and faint-RFI excision cut the reionization power-spectrum upper limit to 3,900 mK² at z = 7, almost ten times lower than the previous MWA result.","keywords":["epoch of reionization","21 cm cosmology","power spectrum upper limit","Murchison Widefield Array","foreground avoidance","radio frequency interference","sky-subtracted incoherent noise spectrum","interferometric imaging"],"falsifier":"Re-run the full pipeline on noise-only simulated visibilities using the paper's exact mask ($k_\\perp$ 18-80 $\\lambda$, $k_\\parallel \\geq 0.15$ $h$ Mpc$^{-1}$, wedge slope buffer of 15 percent) and 40 percent occupancy cut; if the distribution of resulting upper limits is centered below the expected 2$\\sigma$ thermal-noise threshold, or if shifting the wedge buffer by $\\pm 5$ percent changes the $k = 0.20$ $h$ Mpc$^{-1}$ limit by more than its 1$\\sigma$ noise, then the mask selection, not thermal noise, sets the reported limit.","tokens_in":23278,"feed_emoji":"📡","tokens_out":8197,"duration_ms":70389,"temperature":0.7,"pith_summary":"This paper claims that the dominant obstacle in the best existing Murchison Widefield Array measurement of 21 cm emission from the Epoch of Reionization was not the instrument but the analysis: reprocessing the same 32 hours of 2013 data with improved calibration, gridding, and power-spectrum estimation lowers the contamination floor by a factor of 2.8. Removing observations contaminated by ultra-faint digital-television radio frequency interference improves the zenith-pointing result by another factor of 3.8. The outcome is a new upper limit on the reionization power spectrum of $\\Delta^2 \\leq 3.9 \\times 10^3$ mK$^2$ at $k = 0.20$ $h$ Mpc$^{-1}$ and $z = 7$ from 21 hours of data, which the paper presents as the lowest published EoR structure limit to date and an improvement of almost an order of magnitude over the previous MWA limit. The paper also shows that the best full-integration limit is still systematic-dominated, while a zenith-only subset reaches a noise-dominated limit, implying pointing or beam errors are the next obstacle.","feed_headline":"Reionization-era hydrogen map limit is cut tenfold","feed_subtitle":"Improved analysis and faint-RFI excision show the 2013 data were systematics-limited, not noise-limited.","key_machinery":"The load-bearing machinery is an image-based power-spectrum pipeline whose two changes do most of the work: a modified gridding kernel, meaning the instrument beam multiplied in image space by the square of a Blackman-Harris window, which smooths the uv-response and suppresses aliasing so that foreground-coupled errors become spectrally smooth in the EoR window; and an interleaved-time cross-power estimator in which visibilities are split into two time samples per uv-cell, so that the power is formed from their cross-multiplication while analytic error propagation and an even-odd observed-noise estimate provide matched uncertainties. A third component, a sky-subtracted incoherent noise spectrum, time-differences visibilities to reveal ultra-faint RFI and is used to excise whole observations rather than flag individual regions. The foreground-avoidance mask with $k_\\perp$ between 18 and 80 $\\lambda$, $k_\\parallel \\geq 0.15$ $h$ Mpc$^{-1}$, and a wedge slope buffer of 15 percent selects the region where these techniques are evaluated.","core_discovery":"The central discovery is a quantitative demonstration that analysis precision and faint RFI, not raw sensitivity, were limiting the MWA's EoR measurement. Starting from the same 1029-observation data set and the same reduction pipeline used in the earlier analysis, the updated pipeline lowers the best-mode 2$\\sigma$ upper limit by a factor of 2.8 in the N-S polarization and 2.1 in the E-W polarization, with the improvement concentrated in the EoR window above the foreground wedge. Applying a new RFI-cataloging method that detects contamination below single-baseline thermal noise removes 311 digital-television-contaminated and 40 high-occupancy observations, yielding 678 observations (21 hr); this excision improves the zenith-pointing limit by a factor of 3.8. The resulting N-S limit at $z \\approx 7$ is $\\Delta^2 \\leq 3.9 \\times 10^3$ mK$^2$ at $k = 0.20$ $h$ Mpc$^{-1}$, and the paper reports it as currently the lowest upper limit on EoR structure in the literature, while noting that the full-integration limit remains systematic-dominated and only the zenith subset is noise-dominated.","pith_inferences":["Editorial inference: if the selection-bias risk is quantified with noise-only simulations or pre-registered blind masks, the true limit could prove somewhat weaker; the paper acknowledges the risk but does not estimate its size, so the factor-of-ten improvement should be read with that uncertainty in mind.","Editorial inference: the documented factor-of-two contamination in the noise from image-space integration implies an immediate sensitivity gain is available by moving power-spectrum estimation to a direct uv-basis or w-projection scheme, without collecting more data.","Editorial inference: applying the same SSINS-based observation excision per pointing and per polarization could isolate the unknown systematic that contaminates the lower-left EoR window in N-S, and could test whether the E-W excess is instrumental or environmental in origin."],"forward_implications":["If the limit is correct, the previous MWA upper limit was set by analysis systematics and faint RFI, not by integration time, so the same data can yield almost an order-of-magnitude better constraints without new observations.","A zenith-only subset gives a noise-dominated limit, so for that pointing additional integration should lower the limit roughly as $1/\\sqrt{t}$; other pointings will not improve until beam-related errors are fixed.","The RFI-excision method that removes entire observations based on sky-subtracted statistics should transfer to other low-frequency arrays and future MWA data, because it catches contamination below single-baseline thermal noise.","The quoted value remains an upper limit: with only 21 hr the measured power is consistent with noise, and a detection requires hundreds of hours, so this result validates the pipeline rather than detecting reionization."],"supporting_citations":[{"why":"Supplies the previous MWA upper limit and the identical 32-hour data set used for the direct comparison, establishing the factor-2.8 baseline.","marker":"Beardsley et al. (2016)"},{"why":"Provides the sky-subtracted incoherent noise spectrum (SSINS) method used to detect ultra-faint RFI and select the 678-observation subset.","marker":"Wilensky et al. (2019)"},{"why":"Describes the FHD/eppsilon pipeline whose precision improvements (modified gridding kernel, auto-visibility calibration, average removal) drive the analysis-systematics reduction.","marker":"Barry et al. (2019)"},{"why":"Supplies the GLEAM calibration catalog and flux scale, changing the absolute power normalization and the sky model.","marker":"Hurley-Walker et al. (2017)"},{"why":"Establishes the foreground-wedge buffer approach used to set the k-space mask for the 1D upper limits.","marker":"Dillon et al. (2015)"},{"why":"Provides the aoflagger-based preprocessing that performs primary RFI flagging before the SSINS excision.","marker":"Offringa et al. (2015)"},{"why":"Gives the competing LOFAR upper limit against which the paper judges its claim of the lowest EoR structure limit in the literature.","marker":"Patil et al. (2017)"},{"why":"Documents analysis signal loss in power-spectrum pipelines, motivating the in-situ signal-recovery validation performed here.","marker":"Cheng et al. (2018)"}],"fun_headline_variants":["Tenfold tighter limit on reionization-era hydrogen","MWA 21-cm EoR limit improves 10x via analysis and RFI cutting","Faint RFI and better analysis shrink EoR limit by factor of 10","EoR power spectrum limit reduced by order of magnitude"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the k-space mask and the 40 percent RFI-occupancy cut were not tuned to the noise realization of the same data they are used to measure; the paper acknowledges this selection-bias risk but does not quantify it.","fun_headline_variants_meta":{"raw":{"variants":["Tenfold tighter limit on reionization-era hydrogen","MWA 21-cm EoR limit improves 10x via analysis and RFI cutting","Faint RFI and better analysis shrink EoR limit by factor of 10","EoR power spectrum limit reduced by order of magnitude"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000296,"raw_usage":{"total_tokens":1750,"prompt_tokens":1008,"completion_tokens":742,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":624,"completion_tokens_details":{"reasoning_tokens":661}},"tokens_in":624,"tokens_out":742,"duration_ms":7263,"temperature":1.0,"reasoning_tokens":661,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T05:45:03.067372+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the full pipeline on noise-only simulated visibilities using the paper's exact mask ($k_\\perp$ 18-80 $\\lambda$, $k_\\parallel \\geq 0.15$ $h$ Mpc$^{-1}$, wedge slope buffer of 15 percent) and 40 percent occupancy cut; if the distribution of resulting upper limits is centered below the expected 2$\\sigma$ thermal-noise threshold, or if shifting the wedge buffer by $\\pm 5$ percent changes the $k = 0.20$ $h$ Mpc$^{-1}$ limit by more than its 1$\\sigma$ noise, then the mask selection, not thermal noise, sets the reported limit.","supporting_citations":[{"cited_title":"P., Hazelton, B","cited_arxiv_id":null,"evidence_quote":"Supplies the previous MWA upper limit and the identical 32-hour data set used for the direct comparison, establishing the factor-2.8 baseline."},{"cited_title":"J., Morales, M","cited_arxiv_id":null,"evidence_quote":"Provides the sky-subtracted incoherent noise spectrum (SSINS) method used to detect ultra-faint RFI and select the 678-observation subset."},{"cited_title":"R., Hancock, P","cited_arxiv_id":null,"evidence_quote":"Supplies the GLEAM calibration catalog and flux scale, changing the absolute power normalization and the sky model."},{"cited_title":"S., Neben, A","cited_arxiv_id":null,"evidence_quote":"Establishes the foreground-wedge buffer approach used to set the k-space mask for the 1D upper limits."},{"cited_title":"H., Yatawatta, S., Koopmans, L","cited_arxiv_id":null,"evidence_quote":"Gives the competing LOFAR upper limit against which the paper judges its claim of the lowest EoR structure limit in the literature."},{"cited_title":"R., Kolopanis, M., et al","cited_arxiv_id":null,"evidence_quote":"Documents analysis signal loss in power-spectrum pipelines, motivating the in-situ signal-recovery validation performed here."}],"review_version":1}