{"id":"fc5b8d2c-aa40-444f-b06e-745f1d51170c","arxiv_id":"2412.20862","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"An ANN recovers the global 21cm signal from the 21cm power spectrum in simulations (mean correlation 0.95; 0.83 with SKA-1 noise), but only within the learned model space.","lead":"This paper trains an artificial neural network on simulated 21cm cosmology data to recover the sky-averaged 21cm signal from the 21cm power spectrum, and tests it with noise levels expected for SKA-1. It works well in simulation, but the method only knows the models it was trained on, so it cannot validate unexpected signals like the EDGES anomaly.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported mean correlation coefficient (CC) does not establish 'accurate recovery' because CC is insensitive to amplitude bias and offset; no absolute error metric is reported on the test set.","rationale":"The paper is a feasibility study with a genuine held-out test set, and the authors properly acknowledge model-dependence (Section 5). However, the evidence for the central claim—'accurately recovers'—consists almost entirely of the correlation coefficient CC (Eq. 8). A high CC does not guarantee small absolute errors; a constant offset or rescaling would preserve CC while making the reconstruction useless for its stated purpose of cross-validating single-dish global-signal measurements. The paper's Fig. 3 (trough depth) is the only aggregate absolute comparison and addresses just one feature. No test-set RMSE or bias is reported, and the low-CC tail in Fig. 4 implies a subset of models where the method may fail badly; their absolute errors are not characterized. The absence of a validation set is a separate concern (as the reader noted), but it is secondary: even a perfectly generalized network can achieve high CC while failing to reproduce the signal amplitude. A simple linear baseline could potentially match the CC, which would undermine the novelty claim and highlight that CC is not a discriminating accuracy measure. My proposed test directly measures reconstruction error and provides a baseline comparison, settling whether the ANN truly recovers the global signal or merely correlates with it. I agree with the reader's overall CONDITIONAL verdict; the concern here reinforces the need for additional validation metrics before the accuracy claim can be accepted.","tokens_in":13761,"tokens_out":7270,"duration_ms":77003,"concrete_test":"Re-analyze the 100-model test set: for each model compute the redshift-resolved error ΔTb(z) = Tb_ANN(z) - Tb_true(z) and report the mean bias, the RMS error, and the maximum absolute error over the full redshift range, for both the noiseless and SKA-1 noise cases. If the RMS error is small (e.g., < 10% of the signal's dynamic range) and the bias is near zero, the CC-based claim is supported; if the bias is sizable or the RMS error is comparable to the signal amplitude, the claim of accurate recovery is not established. Additionally, fit a linear least-squares model (global signal as a linear function of the same power-spectrum input) and compute its CC on the same test set; if the linear baseline achieves CC ≥ 0.95, the nonlinear ANN adds no demonstrated improvement.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that the ANN accurately recovers the global 21cm signal is supported almost exclusively by the correlation coefficient CC (Eq. 8), averaged over the 100 test models. CC measures linear association between the true and predicted signals, not the magnitude of prediction error. A predicted signal that is a linear transformation of the true signal (e.g., recovered = a*true + b with a≠1, b≠0) yields CC=1 while misrepresenting the amplitude and offset of the signal. Figures 2 and 8 show only single examples, and no aggregate bias, scatter, or root-mean-square error (RMSE) is reported for the full test set. Thus the numbers 'mean CC 0.95' and 'mean CC 0.83 with SKA-1 noise' do not by themselves establish accurate reconstruction in the absolute sense needed for cross-validating single-dish observations. Moreover, Fig. 4 reveals a tail of models with CC below 0.6; the absolute errors for these cases are unknown and could be large. This is load-bearing because the entire feasibility demonstration rests on this metric, and a trivial linear baseline (regressing the global signal on the same power-spectrum input) might achieve comparable CC values, which would undercut both the 'nonlinear reconstruction' claim and the interpretability of CC as an accuracy measure.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a method to reconstruct the sky-averaged 21cm global signal from the 21cm power spectrum using a single-hidden-layer artificial neural network. The training and test data are 500 semi-numerical models from the Cohen et al. simulation code, with 400 models used for training and 100 held out for testing. The ANN takes the 21cm power spectrum at fixed wavenumbers (k = 0.1, 0.5, 1.0 Mpc^-1) or over a range of k and redshifts, and outputs the global brightness temperature as a function of redshift. The authors report mean correlation coefficients of 0.95 for the noise-free k=0.1 Mpc^-1 case, 0.83 with SKA-1 thermal noise, and 0.18 at k=1.0 Mpc^-1, concluding that the method accurately recovers the global signal and is robust to realistic noise. The paper acknowledges the model-dependence of the approach and lists several limitations, including the absence of a validation set, lack of uncertainty quantification, and no treatment of foregrounds.","tokens_in":14034,"tokens_out":5387,"duration_ms":51768,"significance":"If the reported accuracy is reliable, this work would provide a computationally inexpensive way to cross-validate 21cm global-signal measurements from single-dish experiments (e.g., EDGES) against power-spectrum measurements from interferometers (e.g., SKA-1), within the framework of standard astrophysical models. The study benefits from using a physically motivated simulation suite, testing multiple wavenumbers, and including a realistic SKA-1 noise model with a clear statement of the inherent model-dependence. However, the quantitative claims as presented are not yet fully supported: the primary accuracy metric (correlation coefficient) is insensitive to amplitude and offset biases, no absolute error metric is reported over the test set, the 1000x-noise robustness claim is based on a single example, and no comparison with a simpler baseline is made. The proof-of-concept is interesting, but the evidence as presented is insufficient to establish the stated accuracy in a way that would be useful for practical cross-validation.","major_comments":[{"comment":"The primary quantitative accuracy metric is the correlation coefficient (CC) defined in Eq. (8). CC is invariant under affine transformations (a*y_true + b) of the predicted signal, so it cannot detect amplitude bias or offset errors. The reported mean CC of 0.95 therefore does not by itself establish accurate recovery of the global signal in an absolute sense. The manuscript reports no absolute error statistic (e.g., RMSE, bias, or scatter) over the 100-model test set; the examples in Figs. 2 and 8 are individual cases, and Fig. 3, while showing trough-depth scatter, lacks a quantitative error summary. Please add absolute error metrics (e.g., RMSE as a function of redshift, mean bias, and scatter) and, ideally, error bars on the recovered light curve (e.g., via ensemble or bootstrap over noise realizations).","section":"Section 4.1, Eq. (8)"},{"comment":"No validation set is used. The assertion that the training MSE plateau at 20,000 iterations \"suggests that our training dataset is sufficiently large to avoid severe over- or under-fitting\" is not an adequate substitute for monitoring validation or test loss during training. Because the network architecture and number of iterations are fixed choices, this may not be fatal, but the claim of generalization to the test set would be strengthened by reporting the test MSE as a function of training iterations, or by using a train/validation/test split. Please provide evidence that the network is not overfitting, e.g., a test-loss curve or comparison of train and test performance.","section":"Section 3.3"},{"comment":"The claim that the global signal can be \"adequately recovered\" when the thermal noise is 1000 times larger than SKA-1 is based on a single example shown in Fig. 8. No aggregate performance metric (e.g., CC distribution or RMSE) is reported for the 1000x-noise scenario, so the robustness claim is not supported. Please provide a distribution of CC or error metrics over the test set for this case.","section":"Section 4.2, Fig. 8"},{"comment":"The CC distribution shows a non-negligible tail with CC below 0.6 (Fig. 4). The manuscript attributes this to anomalous power spectra but does not quantify the absolute errors for these cases. Since the mean CC may be dominated by high-CC models, the practical accuracy for the worst cases is unknown. Please report the RMSE or bias for the low-CC tail and discuss whether these correspond to identifiable regions of parameter space.","section":"Section 4.1, Fig. 4"},{"comment":"The paper claims the ANN captures a nonlinear mapping between the power spectrum and the global signal, but no comparison is made with a simpler baseline, such as a linear regression from the same power-spectrum input to the global signal. Without such a baseline, it is unclear whether the high CC reflects the ANN's nonlinear modeling or simply the linear correlation already present between these observables. Please include a baseline comparison (e.g., linear regression or linear regression with polynomial features) to support the \"nonlinear reconstruction\" claim.","section":"Sections 3.2 and 4.1"}],"minor_comments":[{"comment":"The phrase \"we show the MSE as a function of the number of iterations\" is repeated twice in two consecutive sentences; please remove the duplication.","section":"Section 3.3"},{"comment":"The text below Eq. (5) uses \"wights\" instead of \"weights\"; please correct the typo.","section":"Section 3.2"},{"comment":"Equation (9) is typeset in a garbled manner (e.g., \"d2 Ay Ω t T2 sys ¯n(Lk⊥)Ae\") and the terms are not clearly defined. Please rewrite the equation with clear notation and define each symbol.","section":"Section 4.2, Eq. (9)"},{"comment":"The text states \"at redshifts z = 7.5 − −15\" and \"z = 7.5 − −35\" with double minus signs; these should be written as ranges, e.g., \"z = 7.5–15\" and \"z = 7.5–35\".","section":"Section 4.2"},{"comment":"The paper states that cosmic variance \"contributes significantly at large scales\" at the beginning of the section, but later claims \"For SKA-1, the cosmic variance is negligible.\" Please clarify which scales and signal levels are meant.","section":"Section 4.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of RAA and presents a plausible proof-of-concept for ML-based cross-validation of 21cm observables. However, the quantitative evidence as presented is not yet sufficient for publication: the reliance on correlation coefficient as the sole accuracy metric, the absence of a validation set, the unsupported 1000x-noise robustness claim, and the lack of a baseline comparison all need to be addressed. I would advise the editor to request a revision that adds absolute error metrics, a validation/test-loss analysis, and a baseline comparison before reconsidering the manuscript. The author's explicit acknowledgment of model-dependence is a positive feature, but the framing should be clarified to distinguish in-distribution interpolation from generalization to real data."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper does one concrete thing: it trains a simple ANN on the Cohen et al. semi-numerical simulations to predict the global 21cm brightness temperature from the 21cm power spectrum at fixed k. For k=0.1 h/Mpc, the mean correlation coefficient on a held-out test set is 0.95, dropping to 0.83 when SKA-1-like noise is added. That is a useful proof-of-concept, and the paper is admirably candid about its model dependence and about not using a validation set. The k-dependence result (low k works, k=1.0 h/Mpc fails) is also physically plausible.\n\nThe soft spots are real but not fatal. The main one is the metric: the paper says \"accurately recovers\" based almost entirely on the correlation coefficient, which is insensitive to additive offsets and multiplicative biases. A prediction that is a linear transform of the true signal would still have CC≈1. The paper shows only example light curves and a scatter plot of trough depths; there is no aggregate RMSE, bias, or scatter on the full test set, and no error bars on the recovered curves. Figure 4 shows a tail of models with CC below 0.6—for those, the absolute error could be large. Without a simple baseline—say, linear regression of the global signal on the same power-spectrum inputs—the nonlinearity of the ANN mapping is not actually demonstrated. The lack of a validation set is acknowledged in the text, but it still weakens the claim that the training was sufficient.\n\nOn novelty: the paper cites prior work on recovering one 21cm statistic from another, including Shimabukuro et al. 2022, but never states what the new element is beyond a different observable pair. And there is no code or data release, which makes the ML result hard to reproduce.\n\nWho is this for? People working on 21cm cross-checks between interferometers and single-dish experiments, and ML applications in cosmology. It is not a deep result, but it is a legitimate, small feasibility study. I would send it to peer review with the expectation of major revision: add absolute error metrics, a baseline comparison, a validation curve, and at least a statement on code availability. With those in place, the conclusion about a computationally cheap cross-check would be solid. As is, I would be cautious about citing it for anything quantitative.","headline":"Honest feasibility study of ANN-based 21cm signal reconstruction that is overstated because the 'accurate recovery' claim leans entirely on correlation coefficients, with no absolute error metric or baseline.","tokens_in":14568,"tokens_out":3173,"would_cite":false,"duration_ms":30816,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A neural network trained on 400 simulated universes reconstructs the 21cm global signal from the 21cm power spectrum alone, with mean correlation 0.95 on held-out models and 0.83 under SKA-1 thermal noise.","keywords":["21cm cosmology","global 21cm signal","21cm power spectrum","artificial neural networks","Epoch of Reionization","cosmic dawn","SKA-1 thermal noise","semi-numerical simulations"],"falsifier":"Retrain the identical single-hidden-layer network on a random 300-model subset, keep 100 models for validation and 100 for testing, and watch the validation error during training; if the validation error rises while the training error falls, or if the held-out correlation at $k=0.1\\,\\mathrm{Mpc}^{-1}$ lands well below 0.95, then the claimed generalization is not supported.","tokens_in":13525,"feed_emoji":"📡","tokens_out":9784,"duration_ms":86852,"temperature":0.7,"pith_summary":"Hydrogen's 21cm line can be measured in two complementary ways: single-dish telescopes record the sky-averaged global signal, while interferometers record spatial fluctuations summarized by the power spectrum. This paper argues that a neural network can bridge the two, reconstructing the global signal from power-spectrum measurements even though no direct mathematical relation exists between them. The connection is indirect—both observables respond to the same astrophysical and cosmological parameters—and the network learns that mapping from 400 semi-numerical simulations. On 100 held-out models, the reconstructed global signal matches the truth with mean correlation 0.95 when the input is the power spectrum at $k=0.1\\,h\\,\\mathrm{Mpc}^{-1}$, and with mean correlation 0.83 when realistic SKA-1 thermal noise is added. If correct, this gives interferometric experiments an independent handle on the global signal and a way to cross-check single-dish results without an extra telescope.","feed_headline":"Machine learning recovers the 21cm global signal from power spectra","feed_subtitle":"Trained on simulations, the network maps power-spectrum data to the sky-averaged signal with 0.83 correlation under SKA-1 noise.","key_machinery":"The central object is a feedforward artificial neural network with one hidden ReLU layer, trained by backpropagation with the Adam optimizer for 20,000 iterations on 400 semi-numerical 21cm models. Its input is the 21cm power spectrum at a fixed wavenumber (or, in one experiment, a grid of wavenumbers over EoR redshifts), sampled over 76 redshift bins from $z=7.5$ to 35, and its output is the global brightness-temperature history over the same redshift range. The network serves as a nonlinear regression that captures the implicit, model-dependent relationship between the two observables: because both are produced by the same underlying astrophysical parameters, sufficiently many training models let the network interpolate new combinations. The scale-dependence result is diagnostic of what the network actually uses: larger-scale modes are correlated with the astrophysical processes that shape the sky-average, whereas small-scale modes mostly track local structure.","core_discovery":"On the paper's own terms, the central discovery is that the sky-averaged 21cm brightness temperature—the global signal, which is mathematically the $k=0$ Fourier mode—can be predicted from the 21cm power spectrum at finite wavenumbers using a single-hidden-layer neural network. The prediction succeeds not because small-scale fluctuations average out, but because the ANN learns the model-dependent correlations set by the underlying astrophysical parameters. Quantitatively, using the power spectrum at $k=0.1\\,\\mathrm{Mpc}^{-1}$ recovers the global signal across $z=7.5$ to 35 with a mean correlation coefficient of 0.95 on the 100-model test set; adding SKA-1-level thermal noise lowers the mean to 0.83, and even 1000-times-stronger noise still permits adequate recovery. Recovery quality depends strongly on scale: $k=0.5\\,\\mathrm{Mpc}^{-1}$ gives a mean correlation of 0.88, while $k=1.0\\,\\mathrm{Mpc}^{-1}$ gives only 0.18, because small-scale power spectra lack the three-peaked structure that encodes the astrophysical history.","pith_inferences":["The paper's acknowledged omission of a validation set means the headline 0.95 and 0.83 correlations should be read as provisional until test-side loss curves are reported; checking those curves is the cheapest way to test the claim before real data arrive.","Because the best results come from low-k input, a natural next experiment the paper does not run is to feed the full multi-k, full-redshift power spectrum into the same network; the EoR-only multi-k case already shows mean correlation 0.774, so a complete multi-k input could improve on the single-k numbers.","The wide spread of correlation coefficients (some test models below 0.6 even without noise) points toward a per-model reliability diagnostic: if the power spectrum lacks the usual peak structure, the recovered global signal should be flagged as untrustworthy rather than averaged into a headline number.","If paired with Bayesian or ensemble uncertainty estimates, the network could output a confidence interval per redshift, turning the method from a point predictor into a tool for identifying which cosmic histories leave recoverable imprints."],"forward_implications":["SKA-1's power-spectrum measurement at $k=0.1\\,\\mathrm{Mpc}^{-1}$ could be converted into an estimate of the global 21cm signal, giving interferometers an indirect route to a single-dish observable.","Reconstruction quality falls rapidly with wavenumber, so practical applications should restrict inputs to larger-scale modes; including $k=1.0\\,\\mathrm{Mpc}^{-1}$ data, with its mean correlation of 0.18, would degrade the result.","The method remains useful under realistic noise: with SKA-1 thermal noise the mean correlation is 0.83, and with 1000-times-stronger noise the recovered signal is still adequate, which the paper takes as evidence the approach can work for MWA or LOFAR.","The ANN is model-dependent: if the true global signal contains physics absent from the training simulations, the network will interpret it within the learned parameter space rather than flagging it as new physics, so this is a cross-check of standard-model cosmic history, not an anomaly detector."],"supporting_citations":[{"why":"Supplies the semi-numerical simulation code used to generate the 21cm power spectra and global signals.","marker":"Cohen et al. 2017"},{"why":"Extends that simulation suite to produce the 500 model realizations split into training and test sets.","marker":"Cohen et al. 2018"},{"why":"Provides the backpropagation algorithm used to minimize the training cost function.","marker":"Rumelhart et al. 1986"},{"why":"Gives the single-mode thermal-noise formula used to simulate SKA-1-like noise on the power spectrum.","marker":"McQuinn et al. 2006"},{"why":"Provides the k-shell-averaged thermal-noise expression used in the noise calculations.","marker":"Lidz et al. 2011"},{"why":"Sets the SKA-1 experiment parameters that the noise model is calibrated to.","marker":"Koopmans et al. 2015"},{"why":"Supports the statement that 1000-times-SKA-1 noise is comparable to MWA or LOFAR noise levels.","marker":"Mesinger et al. 2014"},{"why":"Defines the EDGES anomalous absorption trough that the paper argues lies outside the ANN's model-dependent reach.","marker":"Bowman et al. 2018"}],"fun_headline_variants":["ANN recovers 21cm global signal from power spectra","Neural network maps power spectra to sky-averaged 21cm","AI links two 21cm observables for cross-validation","Machine learning rebuilds 21cm global signal from PS","Neural nets unlock global 21cm signal from power spectrum"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The accuracy claim rests on the assumption that a network trained on 400 simulated models, with no separate validation set, generalizes to the 100 held-out models; the paper infers this from the training error flattening out, not from checking the error on untouched data.","fun_headline_variants_meta":{"raw":{"variants":["ANN recovers 21cm global signal from power spectra","Neural network maps power spectra to sky-averaged 21cm","AI links two 21cm observables for cross-validation","Machine learning rebuilds 21cm global signal from PS","Neural nets unlock global 21cm signal from power spectrum"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000645,"raw_usage":{"total_tokens":2995,"prompt_tokens":1009,"completion_tokens":1986,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":625,"completion_tokens_details":{"reasoning_tokens":1911}},"tokens_in":625,"tokens_out":1986,"duration_ms":13723,"temperature":1.0,"reasoning_tokens":1911,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:08:17.054731+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the identical single-hidden-layer network on a random 300-model subset, keep 100 models for validation and 100 for testing, and watch the validation error during training; if the validation error rises while the training error falls, or if the held-out correlation at $k=0.1\\,\\mathrm{Mpc}^{-1}$ lands well below 0.95, then the claimed generalization is not supported.","supporting_citations":[{"cited_title":"2018, MNRAS, 478, 2193","cited_arxiv_id":null,"evidence_quote":"Extends that simulation suite to produce the 500 model realizations split into training and test sets."},{"cited_title":"2017, MNRAS, 472, 1915","cited_arxiv_id":null,"evidence_quote":"Supplies the semi-numerical simulation code used to generate the 21cm power spectra and global signals."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the single-mode thermal-noise formula used to simulate SKA-1-like noise on the power spectrum."},{"cited_title":"R., Oh, S","cited_arxiv_id":null,"evidence_quote":"Provides the k-shell-averaged thermal-noise expression used in the noise calculations."},{"cited_title":"2015, Advancing Astrophysics with the Square Kilometre Array (AASKA14), 1","cited_arxiv_id":null,"evidence_quote":"Sets the SKA-1 experiment parameters that the noise model is calibrated to."}],"review_version":1}