{"id":"9ab24553-66c3-4aca-9e92-7f3cb2e41e93","arxiv_id":"2507.07713","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A neural network trained on CoREAS showers generates AERA radio pulses and reconstructs Xmax with resolution within about 25% of full Monte Carlo, within the trained phase space.","lead":"A neural network trained on about 58,000 CoREAS simulations generates the radio pulses of cosmic-ray air showers in milliseconds, and reconstructs shower-maximum depth with resolution close to the full Monte Carlo. This could remove the main simulation bottleneck in radio-array analyses such as AERA.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Does the 80/20 train/test split separate the 2158 shower parameter sets, or only the 27 iterations of each? If runs from the same parameter set appear in both folds, the reported pulse and Xmax performance is inflated by leakage.","rationale":"The reader's weakest assumption was external validity: CoREAS Monte Carlo is treated as ground truth, so systematics in the emission model would not transfer to real data. That is a legitimate limitation, but for the central claim as stated (a surrogate for CoREAS with comparable Xmax resolution), comparing against CoREAS is the correct benchmark. The more immediate threat is internal validity of the held-out evaluation. The paper's own description in Section 2 creates a concrete ambiguity: 2158 parameter sets times 27 iterations, split as 80/20 'simulations.' If the split is at the run level, test runs share exact input vectors with training runs (the other iterations of the same parameter set), so the network can learn the conditional mean for those inputs. The reported test errors then largely measure thinning noise around a learned mean, not generalization to new geometries or Xmax values. This would inflate both the pulse-quality metrics and the Xmax reconstruction comparison in Figure 7 and Table 1. The issue is checkable, but the paper neither states the split granularity nor releases code and weights. Because this is a possible methodological flaw rather than an established one, I would keep the CONDITIONAL verdict and add an explicit grouped-split condition. I therefore disagree with the reader's choice of the CoREAS ground-truth assumption as the weakest point: the training/test leakage is more fundamental and more immediately testable.","tokens_in":6611,"tokens_out":7892,"duration_ms":96126,"concrete_test":"Perform a grouped split: assign all 27 iterations of each shower parameter set to either the training or test fold (for example 1726/432 parameter sets), retrain from scratch with the same architecture, and recompute the pulse-level metrics in Figure 5 and the bias/resolution table. If the NN Xmax resolution degrades by more than about 5 g/cm2, or if the fluence error distribution shifts upward, the original split was leaking and the comparison is not valid. At minimum, the authors should state explicitly whether the original 80/20 split was grouped by parameter set and, if not, rerun the evaluation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing assumption is that the held-out test data are independent of the training data. Section 2 reports a library of 2158 shower parameter sets, each with 27 iterations, yet says only that '80% of the simulations are used for training and the remaining 20% are used for testing.' If the split is performed on individual CoREAS runs rather than on the 2158 parameter sets, then for a typical test run most of the other 26 iterations of the identical shower (same Xmax, energy, zenith, azimuth, and GDAS atmosphere) sit in the training set. All of those quantities are explicit network inputs, so the model has already seen the exact input vector and can fit the conditional mean pulse for that vector. The test then measures reproduction of an already-seen shower rather than generalization to a new shower geometry. This would directly inflate the pulse correlations, fluence errors, and the Xmax resolution comparison in Figure 7 and Table 1, undermining the central claim that the network can simulate radio pulses for previously unseen events. The paper does not state that the split was grouped by parameter set, and no code or weights are released to verify it.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents a fully connected neural network trained on ~58k CoREAS simulations (2158 shower parameter sets times 27 iterations) to predict 30-80 MHz radio pulses at 240 star-shaped antenna positions for AERA-like geometries. The authors evaluate the surrogate on a held-out 20% of the simulations, comparing pulse shapes, energy fluence, pulse correlations, and a fluence-based Xmax reconstruction against per-event CoREAS simulations. They report fluence agreement within roughly 10% for strong pulses and Xmax resolutions of about 37-40 g/cm2 for the network versus 31-32 g/cm2 for CoREAS at 5-10% noise, concluding that the neural network can serve as a fast surrogate for radio-emission simulation in this phase space.","tokens_in":6852,"tokens_out":7157,"duration_ms":84981,"significance":"If the evaluation is performed on showers whose full parameter sets were excluded from training, the paper is a useful proof of concept: a compact fully connected regressor can reproduce CoREAS pulse shapes and support a downstream Xmax reconstruction at a resolution close to, albeit somewhat worse than, full Monte Carlo, with much faster inference. The authors are appropriately explicit about the limited phase space, the AERA-specific atmosphere and frequency band, and the degradation beyond Xmax = 1000 g/cm2. The main risk is that the reported test performance may not measure generalization to unseen shower parameters, because the paper does not state that the train/test split was grouped by the 2158 underlying parameter sets.","major_comments":[{"comment":"The data split is not defined at the correct level. The library consists of 2158 shower parameter sets, each with 27 iterations, but the text says only that '80% of the simulations are used for training and the remaining 20% are used for testing.' If the split is made per CoREAS run, a typical test run will have 26 other iterations of the identical parameter set (same Xmax, energy, zenith, azimuth, and GDAS atmosphere) in the training sample, and because all of these quantities are explicit network inputs, the model has already been trained on the exact input vector. The pulse correlations in Figure 5 and the Xmax resolution comparison in Figure 7 and Table 1 would then measure reproduction of known parameter points rather than generalization to new showers. Please state explicitly that the split was performed on the 2158 parameter sets (or otherwise demonstrate test independence), and if it was not, repeat the evaluation with a grouped split.","section":"Section 2"},{"comment":"Table 1 reports total bias and resolution as single numbers with no statistical uncertainties, and Figure 7 shows no confidence bands. The sampling is not described beyond 'N=2000 simulations,' and it is not stated how these events relate to the training and test sets. The table shows network resolutions of 39.6 and 37.5 g/cm2 versus CoREAS resolutions of 31.6 and 30.7 g/cm2 at 5% and 10% noise, i.e., about 22-25% worse. The abstract's phrase 'comparable resolution' needs a quantitative justification and an uncertainty estimate; as written, the reader cannot tell whether these differences are statistically significant.","section":"Section 4.4 / Table 1"},{"comment":"The network is deterministic: for a fixed input vector it always returns the same pulse, whereas the 27 CoREAS iterations at fixed shower parameters are stochastic realizations. The paper does not state whether the surrogate is intended to reproduce only the conditional mean pulse or also the event-by-event fluctuation distribution. This matters for the Xmax benchmark, because chi2 templates generated by the network cannot include shower-to-shower fluctuations that are present in CoREAS templates, and it also bears on the interpretation of the pulse-level correlations in Section 4.2. Please discuss this limitation and its effect on the reported resolution.","section":"Sections 3 and 4"}],"minor_comments":[{"comment":"Please define P_ant and P_sim; the S^2 factor in Eq. (1) is inconsistent with the statement that S accounts for multiplicative deviations between fluence measurements and simulations.","section":"Eq. (1)"},{"comment":"The noise model is not fully specified: the width of the Gaussian multiplicative noise is not given, and the additive noise is quoted as '1 muV/m^2' although fluences are in eV/m^2; please state the units and all noise parameters.","section":"Section 4.3"},{"comment":"Please report the criterion and fraction of weak pulses removed as dominated by thinning artifacts; this affects the training distribution and the comparability of the network output with full CoREAS pulses.","section":"Section 2"},{"comment":"The vertical axis label 'N.Coorelation bw Pulses' contains typos; 'bw' should be 'between' and 'Coorelation' should be 'Correlation'.","section":"Figure 5"},{"comment":"The pulse comparison in Figure 3 is qualitative; please state explicitly whether the events shown in Figures 3-5 are drawn from the held-out test set.","section":"Section 4.1"},{"comment":"The model is called 'generative,' but it is a deterministic function without a latent stochastic source; consider using 'surrogate' or 'emulator' in the title and text to avoid overstating the generative aspect.","section":"Section 3"}],"recommendation":"major_revision","confidential_remarks":"This is an ICRC proceedings contribution, so the expected level of detail is that of a conference report; nevertheless, the train/test split ambiguity is a correctness issue rather than a presentation issue because the central claim concerns generalization to new showers. I recommend asking for a clear statement of the grouped split or a re-evaluation with grouped split before acceptance. The absence of released code and weights makes the ambiguity harder to resolve externally."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper does something real and useful—trains a fully connected network to generate 30–80 MHz radio pulses for AERA-like geometries, and benchmarks it for Xmax reconstruction. The idea is not brand new (they cite a LOFAR model), but this is the first demonstration on AERA with a reconstruction benchmark, and the writing is clear and honest about phase-space limits.\n\nWhat I like: the network takes 11 physically meaningful inputs and outputs 512 pulse bins directly. The held-out evaluation, as described, shows fluence agreement within about 10% and pulse correlations above 0.9 for strong signals. The Xmax reconstruction comparison is a fair way to test downstream usability, and they correctly note the 'comparable resolution' caveat: at 5–10% noise, NN resolution is 37–40 g/cm² versus 31–32 for CoREAS—about 25% worse, which is still in the ballpark. They also state the low-Xmax bias is from the parabola-fitting method, not the network, and flag poor performance beyond 1000 g/cm².\n\nThe soft spot: the train/test split. Section 2 says '80% of the simulations are used for training and the remaining 20% for testing.' The library has 2158 shower parameter sets, each with 27 iterations. If the split is done on individual runs, then for each test run, 26 other iterations with the identical input vector (same Xmax, energy, zenith, azimuth, antenna positions) sit in the training set. The network has seen that exact input during training and can memorize the conditional mean pulse. The test then measures reproduction of known showers, not generalization to new geometries. This would inflate the pulse correlations, fluence errors, and Xmax resolution. The paper never states that the split is grouped by parameter set, and no code or weights are released to check. This is the load-bearing issue for the abstract's claim.\n\nOther gaps are minor in comparison: no statistical uncertainties on the bias/resolution numbers, and the validation is entirely internal to CoREAS. The authors acknowledge the latter implicitly by calling it a proof of concept and mentioning fine-tuning for LOFAR.\n\nVerdict: the idea deserves a serious referee. The split ambiguity must be resolved—either re-run with a grouped split or clearly report it. If the grouped split gives similar numbers, this becomes a solid proof of concept. For now I would not cite it in my own work until that is fixed.","headline":"Promising neural surrogate for CoREAS radio pulses whose headline validation may be inflated by a per-run rather than per-shower train/test split.","tokens_in":7380,"tokens_out":2877,"would_cite":false,"duration_ms":30214,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A neural network trained on CoREAS showers can generate radio pulses and reconstruct Xmax with resolution close to full Monte Carlo.","keywords":["radio emission from extensive air showers","neural network surrogate","Xmax reconstruction","CoREAS","AERA","cosmic rays","fluence"],"falsifier":"Run the trained network on real AERA events whose $X_\\mathrm{max}$ is independently known from fluorescence measurements and compare the reconstructed $X_\\mathrm{max}$ values: if resolution or bias is substantially worse than the $37$–$40\\ \\mathrm{g\\,cm^{-2}}$ seen on simulated CoREAS events, the paper's claim of comparable resolution on real data fails.","tokens_in":6384,"feed_emoji":"📡","tokens_out":13937,"duration_ms":121015,"temperature":0.7,"pith_summary":"This paper claims that a fully connected neural network can replace per-event CORSIKA/CoREAS Monte Carlo simulations of radio emission from extensive air showers, at least inside the AERA training phase space. The network takes eleven physical inputs—shower maximum depth, electromagnetic energy, geomagnetic angle, atmospheric density and height at shower maximum, primary energy, arrival direction, and antenna position in the shower plane—and produces 1 ns radio pulses in two polarisations at 240 antenna positions. Pulse fluences agree with CoREAS to within about 10 percent for strong signals, and an $X_\\mathrm{max}$ reconstruction built on the network's pulses has bias and resolution close to the same reconstruction run on full CoREAS simulations ($37$–$40\\ \\mathrm{g\\,cm^{-2}}$ versus $31$–$32\\ \\mathrm{g\\,cm^{-2}}$ at 5–10 percent noise). The reason to care is speed: CoREAS simulations are the dominant computational bottleneck in radio cosmic-ray analysis, while network inference takes only milliseconds per antenna and training takes about a week on a desktop CPU. The authors present this as a proof of concept limited to the phase space covered by the training library.","feed_headline":"Fast neural net matches CoREAS radio pulses in Xmax resolution","feed_subtitle":"11 inputs generate 30-80 MHz pulses in milliseconds, removing the Monte Carlo bottleneck in radio Xmax analysis.","key_machinery":"The carrying mechanism is a fully connected neural network with 11 inputs, 8 hidden layers, about 4 million parameters, and 512 output nodes: 256 time bins at 1 ns resolution for each of the two polarisations. The network is trained directly on CoREAS pulses with an L1 loss, with the weaker $\\vec{v}\\times(\\vec{v}\\times\\vec{B})$ polarisation weighted more heavily so that both polarisations are learned, and ADAM with weight decay is used for minimisation. Because the network outputs the pulse waveform itself at any antenna position in the shower plane, pulse shape information is retained rather than only integral fluence, which matters because pulse shape carries information about shower development.","core_discovery":"The paper's central claim is that radio pulses of extensive air showers can be generated by a trained neural network rather than by summing radiation from individual particle tracks, and that the generated pulses carry enough physical fidelity for $X_\\mathrm{max}$ reconstruction. Trained on roughly 58,000 CoREAS simulations of AERA events (2158 shower configurations with 27 iterations each), the network reproduces 30–80 MHz radio pulses in both polarisations, with fluences that mostly match CoREAS within 10 percent and high pulse-wise correlation for strong signals. Running the same fluence-based $\\chi^2$ reconstruction against both simulation types, the network gives $X_\\mathrm{max}$ resolutions of $37.5$–$39.6\\ \\mathrm{g\\,cm^{-2}}$ at 5–10 percent noise, compared with $30.7$–$31.6\\ \\mathrm{g\\,cm^{-2}}$ for CoREAS, with comparable bias behaviour. The paper concludes that neural-network simulation is a viable tool for radio air-shower analyses and can be fine-tuned to other experiments and frequency bands.","pith_inferences":["If CoREAS's radio-emission model is biased under real AERA atmospheric conditions, the demonstrated comparable resolution may not transfer to measured events; the direct test is applying the network to real data with independently known $X_\\mathrm{max}$ values.","Because the training library follows AERA's measured phase-space distribution, the model is most reliable for typical events, while rare geometries and showers with $X_\\mathrm{max}$ above $1000\\ \\mathrm{g\\,cm^{-2}}$ need resampling or dedicated training sets.","A waveform-generating model may support observables beyond fluence-based $X_\\mathrm{max}$, such as pulse timing, width, or asymmetry, since the network outputs full pulses rather than summary statistics."],"forward_implications":["Radio pulse simulation drops from per-event Monte Carlo runs to milliseconds per antenna, removing the main computational bottleneck in iterating over noisy radio measurements.","$X_\\mathrm{max}$ reconstruction on network pulses matches the resolution of full CoREAS-based reconstruction within the AERA training phase space, enabling fast analyses of many simulated event iterations.","Because the network is small (about 4 million parameters, 19 MB footprint), it could run on FPGAs for near-instant pulse generation and triggering.","The network is differentiable, so it can be combined with gradient-based inference frameworks such as Information Field Theory.","Adding inputs such as the full longitudinal shower profile could extend the same setup to reconstructing the longitudinal development of air showers."],"supporting_citations":[{"why":"CoREAS extension that generates the radio pulses used as training data and as ground truth for the comparison.","marker":"[12]"},{"why":"CORSIKA Monte Carlo code that simulates the air showers whose particle tracks CoREAS converts into radio pulses.","marker":"[13]"},{"why":"AERA analysis approach and simulation library that supplies the training data and the fluence-based $X_\\mathrm{max}$ reconstruction method.","marker":"[7]"},{"why":"Fluence-based $X_\\mathrm{max}$ reconstruction method used by LOFAR that the paper's reconstruction procedure is inspired by.","marker":"[8]"},{"why":"QGSJETII-04 hadronic interaction model used in the training simulations.","marker":"[14]"},{"why":"GDAS atmosphere tool supplying the time-variable atmospheric model used in the AERA simulations.","marker":"[16]"},{"why":"Establishes that pulse shape carries information about shower development, motivating waveform-level comparison rather than fluence only.","marker":"[22]"}],"fun_headline_variants":["Neural net simulates air-shower radio pulses in milliseconds","Generative network matches CoREAS radio pulses for Xmax","Fast neural net gives CoREAS-level Xmax resolution","Neural pulse generator replaces costly Monte Carlo radio","Millisecond air-shower radio pulses via neural net"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The demonstration assumes CoREAS Monte Carlo output is the true radio signal, since the network is trained on CoREAS pulses and benchmarked against CoREAS reconstructions; if CoREAS's emission model is biased for AERA conditions, the comparable resolution may not carry over to measured events.","fun_headline_variants_meta":{"raw":{"variants":["Neural net simulates air-shower radio pulses in milliseconds","Generative network matches CoREAS radio pulses for Xmax","Fast neural net gives CoREAS-level Xmax resolution","Neural pulse generator replaces costly Monte Carlo radio","Millisecond air-shower radio pulses via neural net"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00131,"raw_usage":{"total_tokens":5295,"prompt_tokens":856,"completion_tokens":4439,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":472,"completion_tokens_details":{"reasoning_tokens":4359}},"tokens_in":472,"tokens_out":4439,"duration_ms":32703,"temperature":1.0,"reasoning_tokens":4359,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T18:34:43.898676+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the trained network on real AERA events whose $X_\\mathrm{max}$ is independently known from fluorescence measurements and compare the reconstructed $X_\\mathrm{max}$ values: if resolution or bias is substantially worse than the $37$–$40\\ \\mathrm{g\\,cm^{-2}}$ seen on simulated CoREAS events, the paper's claim of comparable resolution on real data fails.","supporting_citations":[{"cited_title":"Huege, M","cited_arxiv_id":null,"evidence_quote":"CoREAS extension that generates the radio pulses used as training data and as ground truth for the comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CORSIKA Monte Carlo code that simulates the air showers whose particle tracks CoREAS converts into radio pulses."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"AERA analysis approach and simulation library that supplies the training data and the fluence-based $X_\\mathrm{max}$ reconstruction method."},{"cited_title":"Corstanje, S","cited_arxiv_id":null,"evidence_quote":"Fluence-based $X_\\mathrm{max}$ reconstruction method used by LOFAR that the paper's reconstruction procedure is inspired by."},{"cited_title":"Ostapchenko, Nuclear Physics B - Proceedings Supplements 151(2006) 143","cited_arxiv_id":null,"evidence_quote":"QGSJETII-04 hadronic interaction model used in the training simulations."},{"cited_title":"Mitra, A","cited_arxiv_id":null,"evidence_quote":"GDAS atmosphere tool supplying the time-variable atmospheric model used in the AERA simulations."},{"cited_title":"Karastathis, S","cited_arxiv_id":null,"evidence_quote":"Establishes that pulse shape carries information about shower development, motivating waveform-level comparison rather than fluence only."}],"review_version":1}