{"id":"6725fc6b-de8f-4e69-a8c2-925e457a0b41","arxiv_id":"2411.16565","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"On simulated SWGO events, graph neural networks outperform LIC/PINCness observables in gamma/hadron separation and template-based methods in energy resolution.","lead":"This simulation study applies graph neural networks to the planned SWGO water-Cherenkov gamma-ray observatory. The GNNs reject cosmic-ray background and reconstruct gamma-ray energies more accurately than current hand-designed and template-based methods.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The most load-bearing uncertainty is whether the GNN's advantage survives real detector noise: the simulation explicitly omits cosmic-ray-induced noise hits (Sec. 2.1), which will add nodes and edges to the triggered-tank graph the GNN consumes.","rationale":"The reader's weakest_assumption is exactly the condition I would stress-test: the missing cosmic-ray-induced noise hits. This is not a manufactured concern—it is stated by the authors in Sec. 2.1 and listed as future work in Sec. 6. The simulation-scoped comparison itself appears sound: train/test separation is in place, baselines are optimized, and the architecture is documented in detail. The proposed test directly attacks the link between the simulation benchmark and the planned observatory, which is the only major condition needed for the stronger, observatory-level interpretation of the claim. Because that condition is unverified but not disproven, the reader's CONDITIONAL verdict remains appropriate, and I would not change it.","tokens_in":15459,"tokens_out":7403,"duration_ms":72839,"concrete_test":"Using the same CORSIKA/GEANT4 SWGO simulation chain, generate test events with cosmic-ray-induced noise hits overlaid at the per-tank rate expected from the planned site (or, if unavailable, from HAWC/LHAASO single-muon measurements), covering at least 0x, 1x, and 2x that rate. Retrain or fine-tune the GNN and re-optimize the LIC/PINCness cuts on the noisy events. If the GNN's quality factor remains two to three times the baseline and its energy resolution still beats the template method above 200 GeV, the concern is refuted; if the advantage shrinks below the claimed factor, the ideal-simulation assumption is the limiting factor.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central comparison is a simulation study, and internally the benchmark is carefully constructed: train/validation/test split, hyperparameter search, per-bin optimized LIC/PINCness cuts, and error bars are all in place. The weakest link is external validity. Section 2.1 states that the detector simulation 'still lacks cosmic ray-induced noise hits,' and Section 6 defers 'realistic operation conditions, including cosmic ray noise' to future work. Because the GNN graph (Sec. 2.2.1) has one node per triggered tank with charge and time features, random cosmic-ray-induced hits, especially single muons producing large charges, will add false nodes and alter the neighborhood structure used by the EdgeConvolutions. The same noise also affects the LIC/PINCness baselines, but the paper gives no estimate of relative degradation. If the performance gain is an artifact of graph topology in a noise-free simulation, the claimed quality-factor improvement and the energy-resolution gain above 200 GeV may not transfer to SWGO. This does not invalidate the simulation-scoped statement, but it is the condition most load-bearing for the title-level claim about the observatory.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a simulation study of graph neural networks (GNNs) for gamma/hadron separation and energy reconstruction in a candidate SWGO water-Cherenkov detector layout. Events are represented as graphs whose nodes are triggered tanks carrying position, charge, and arrival-time features, connected via k-nearest neighbors. The authors train EdgeConvolution-based GNNs with hyperparameter searches and compare them against the classical LIC/PINCness observables (optimized per energy bin) and a template-based likelihood method. They report a quality factor two to three times higher than the classical observables across the studied energy range, and an energy resolution of about 16% at 17 TeV for zenith angles below 30 degrees, outperforming the template method above about 200 GeV.","tokens_in":15625,"tokens_out":9851,"duration_ms":88198,"significance":"If the results transfer to the real detector, the work provides a strong quantitative case for using GNNs in SWGO event reconstruction. The study is internally well controlled: a 70/10/20 train/validation/test split, 70-trial hyperparameter searches for each task, validation-based model selection, multiple performance metrics, statistical uncertainties following Ref. [50], and per-bin optimization of the LIC/PINCness baselines. The main caveat is that the simulation explicitly omits cosmic-ray-induced noise hits, which will alter the graph topology that the GNN consumes; the relative robustness of the GNN versus the baselines under such noise is not quantified. This limitation is clearly stated in the text but is load-bearing for the title-level claim about the observatory, so a quantitative robustness estimate or a more restricted claim is needed.","major_comments":[{"comment":"The simulation explicitly lacks cosmic-ray-induced noise hits (Sec. 2.1: “still lacks cosmic ray-induced noise hits”), and Sec. 6 defers this to future work. Because the GNN graph has one node per triggered tank and uses k-nearest-neighbor connectivity (Sec. 2.2.1), random noise hits will add nodes and edges and change the local neighborhoods seen by the EdgeConvolution layers. The paper provides no estimate of how much of the reported quality-factor and energy-resolution advantage survives such noise relative to the LIC/PINCness and template baselines, which are also affected but potentially to a different degree. Since the title and framing address the SWGO observatory, this external-validity gap is load-bearing. Please add a quantitative robustness study (for example, injecting cosmic-ray noise hits at expected rates into the simulated events and recomputing all metrics) or explicitly and prominently restrict the claims to the noise-free simulation.","section":"Sec. 2.1 / Sec. 6"},{"comment":"The graph aggregation function is specified inconsistently: Sec. 3.1 states “the sum (used in this work)”, while the architecture tables in Appendix A.2 specify “□j: mean” for every graph layer. Because the aggregation choice determines the network's behavior and thus the reported performance, this contradiction must be resolved. Please state which aggregation was actually used in the final models and correct the text accordingly.","section":"Sec. 3.1 / Tables 2 and 3"},{"comment":"For the energy-reconstruction comparison, the authors apply a dedicated selection that includes a cut on the GNN gamma-score (“The cut value was taken from the previous γ/hadron separation”). It is not stated whether the template-based method is evaluated on the same selected event sample or on its own default selection. If the GNN-score cut is applied before the comparison, it may preferentially retain events that the GNN reconstructs well, biasing the energy-resolution comparison in the GNN's favor. Please clarify whether both methods use the identical event sample and, if the GNN-score cut is applied, discuss its effect on the template method.","section":"Sec. 5 (Data selection)"},{"comment":"The LIC and PINCness baseline cuts are said to be “optimized” for each energy bin “in our dataset”, but the paper does not specify whether this optimization is performed on the training/validation split or on the test set later used for the reported comparisons. To ensure a fair comparison with the GNN (whose hyperparameters are selected on the validation set), please state explicitly which data were used for the baseline cut optimization. If the cuts were tuned on the test set, the comparison is conservative but should still be reported as such.","section":"Sec. 4.3.1"}],"minor_comments":[{"comment":"The abbreviation “knn” should be written as “kNN” or “k-nearest neighbors” when first introduced, and the figure caption should match.","section":"Sec. 2.2.1"},{"comment":"The sentence “This is slightly lower than the trigger threshold of 30 hits investigated in the following” is ambiguous; consider “used in the following analysis” instead of “investigated in the following”.","section":"Sec. 2.1"},{"comment":"The definition of CxPE40 appears after the equation; placing it before the equation would improve readability.","section":"Eq. (4.2)"},{"comment":"The text reports that background rejection improves by “around a factor of two” at low energies and “by one order of magnitude” at high energies, while the quality-factor comparison mentions a factor of three at low energies and five to eight at medium and high energies. These are different metrics, but the juxtaposition may confuse readers; consider adding an explicit statement that the two quantities are not directly comparable.","section":"Sec. 4.3.2"},{"comment":"The yellow marked region in Fig. 11 may be difficult to see in grayscale; please also indicate the ±10% region with dashed lines or similar.","section":"Sec. 5.2"},{"comment":"The exact value of the GNN score cut used in the energy-reconstruction data selection is not stated; please provide it for reproducibility.","section":"Sec. 5"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a well-executed simulation study from a group with prior GNN experience in astroparticle physics. The main concern is whether the missing cosmic-ray noise in the simulation is adequately handled in the framing; the authors should either add a robustness study or visibly restrict the claims. The aggregation-function inconsistency and the unclear baseline optimization/test-set usage are fixable but need attention before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is the first serious GNN benchmark for a wide-field water-Cherenkov gamma observatory, and it is a good one. The benchmark is well controlled: train/val/test split, hyperparameter search on validation, per-bin optimized LIC/PINCness baselines, statistical errors, and honest discussion of limited high-energy statistics. I believe the central claim — that on simulated SWGO events a GNN using position, charge, and arrival time beats the established observables — is supported.\n\nWhat's new: prior GNN work in gamma astronomy targeted IACTs; this extends it to water-Cherenkov arrays and adds the finding that timing information has separation power beyond charge-based observables. The energy reconstruction comparison against the template likelihood method is useful, with ~16% resolution at 17 TeV and stable bias from ~600 GeV upward.\n\nThe soft spots are real but not fatal. The simulation explicitly omits cosmic-ray-induced noise hits (Sec. 2.1). In the real detector, single muons and other noise will add nodes and edges to the triggered-tank graph that the GNN consumes, and nobody knows how much that degrades the advantage relative to LIC/PINCness. This doesn't undermine the simulation-scoped result, but it does mean the title-level claim about the observatory is conditional. The paper says realistic operation conditions including cosmic-ray noise are future work; that is the right next step. No code or data is released, which limits reproducibility but is common for collaboration-internal simulations. The high-energy bins above ~50 TeV have limited statistics; the authors flag it. I don't see a circularity problem — the GNN is evaluated against external baselines on held-out data, and the self-cited earlier GNN work is architectural inspiration, not part of the evaluation.\n\nWho this is for: people working on SWGO/HAWC/LHAASO event reconstruction and on graph-network methods for sparse detector data. It deserves a serious referee. My recommendation would be major revision, not rejection: add a robustness study with simulated cosmic-ray noise hits if possible, or at least a quantitative discussion of how graph topology changes under noise, and consider releasing the trained model or code. As it stands, the paper is a solid simulation study with a clearly stated limitation.","headline":"A carefully done simulation study showing GNNs beat hand-designed gamma/hadron separators for SWGO; the main caveat is that the simulation omits cosmic-ray noise, so the size of the advantage in the real detector is unproven.","tokens_in":16215,"tokens_out":2778,"would_cite":true,"duration_ms":23048,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Modeling air-shower footprints as graphs, a GNN rejects hadronic cosmic-ray background with a quality factor two to three times higher than the LIC/PINCness observables and reconstructs gamma-ray energies more accurately than…","keywords":["graph neural networks","gamma/hadron separation","water Cherenkov detectors","SWGO","energy reconstruction","air showers","deep learning","quality factor"],"falsifier":"Run the same trained GNN and the LIC/PINCness baseline on a simulation that includes cosmic-ray-induced noise hits, or on real SWGO data after commissioning; if the quality-factor advantage drops below the claimed factor of two to three, or the energy resolution no longer beats the template method above 200 GeV, the central claim is refuted.","tokens_in":15232,"feed_emoji":"🔭","tokens_out":9183,"duration_ms":80197,"temperature":0.7,"pith_summary":"The paper sets out to show that graph neural networks (GNNs) classify and reconstruct air showers recorded by a water-Cherenkov gamma-ray observatory better than the hand-designed observables now used by the HAWC experiment. Each event is modeled as a graph whose nodes are triggered tanks carrying position, charge, and arrival time. On simulated events for a candidate SWGO array layout, the authors find that the GNN rejects hadronic background with a quality factor two to three times higher than the optimized combination of LIC and PINCness across the studied energy range. They also report more accurate energy reconstruction than the template-based likelihood method above about 200 GeV, reaching about 16% resolution at 17 TeV for zenith angles below 30 degrees. If these simulation results transfer to the real instrument, the planned southern-hemisphere observatory would gain a large increase in sensitivity to very-high-energy gamma-ray sources.","feed_headline":"GNN lifts gamma-ray sensitivity 2-3x over classic cuts","feed_subtitle":"On simulated SWGO events, the network also improves energy resolution above 200 GeV.","key_machinery":"The central object is the graph representation of the shower footprint: each triggered tank is a node with features $(x, y, \\text{charge}, \\text{arrival time})$, and edges connect each node to itself and to its six nearest spatial neighbors, matching the triangular tank layout. The network uses EdgeConvolution layers, whose kernel function $h_\\Theta$ combines the central node's features with edge differences $x_j - x_i$ and aggregates over neighbors, followed by one DynamicEdgeConvolution layer to capture global structure, global pooling over nodes, and fully connected output layers. This machinery lets a single network handle events with tens to thousands of triggered tanks and exploit spatial, timing, and charge patterns jointly, in contrast to fixed-size image filters.","core_discovery":"The central claim is that a GNN operating on the sparse, irregular footprint of an air shower extracts more discriminative information than the observables currently used by water-Cherenkov experiments. On roughly 370,000 simulated gamma-ray and 440,000 proton events, the GNN trained on tank positions, charge, and arrival time achieves a quality factor two to three times higher than the optimized combination of LIC and PINCness over the whole energy range, and its background contamination at a fixed gamma-ray efficiency of 80% is up to an order of magnitude lower at high energies. For energy reconstruction, the paper reports a stable bias from 600 GeV to about 200 TeV and a resolution that consistently surpasses the template-based likelihood method above 200 GeV, with roughly 16% resolution at 17 TeV for zenith angles below 30 degrees, exceeding the performance requirements set out in the SWGO science case.","pith_inferences":["Because real night-sky and cosmic-ray noise will alter which tanks trigger, the simulation's missing cosmic-ray noise is the most likely place for the claimed advantage to shrink; adding such noise to the simulation and retraining would quantify this.","The graph formulation is layout-agnostic and could transfer to other sparse detector arrays with strongly varying hit multiplicities, such as HAWC, LHAASO, or neutrino detectors.","The paper's own data thin out above 50 TeV, so the high-energy quality-factor gain is the least certain part of the claim and could be pinned down with a larger simulation set.","Attention mechanisms or separate graph clustering in the dense and sparse detector zones, which the authors list as future work, may push performance further, but those gains are not demonstrated here."],"forward_implications":["Source-detection significance scales with the quality factor $Q = \\epsilon_\\gamma / \\sqrt{\\epsilon_p}$, so a two-to-three-fold higher $Q$ translates directly into a two-to-three-fold gain in discovery sensitivity for gamma-ray sources.","Arrival-time information, previously little used for gamma/hadron separation in water-Cherenkov arrays, adds measurable separation power on top of charge and position.","The GNN delivers its separation performance without any quality cuts on the classifier output, simplifying the analysis chain relative to methods that require tuned cuts on LIC and PINCness.","The same graph architecture can be adapted to changes in detector layout or tank design, such as adding muon-tagging capabilities, without redesigning the algorithm."],"supporting_citations":[{"why":"Defines the compactness variable that underlies the LIC baseline observable.","marker":"[7]"},{"why":"Supplies the template-based likelihood reconstruction method used as the energy-resolution baseline.","marker":"[13]"},{"why":"Introduces the PINCness observable and the HAWC simulation package modified for SWGO.","marker":"[37]"},{"why":"Provides the Dynamic Graph CNN and EdgeConvolution operation that forms the core of the GNN.","marker":"[40]"},{"why":"ParticleNet architecture whose design inspires the network structure used here.","marker":"[41]"},{"why":"Documents the current SWGO template-based reconstruction chain that the GNN is compared against.","marker":"[51]"},{"why":"Describes the HAWC reconstruction algorithms and performance that motivate the LIC/PINCness baselines.","marker":"[52]"},{"why":"Shows how to optimize box cuts on LIC and PINCness, the procedure used to set the comparison baseline.","marker":"[55]"}],"fun_headline_variants":["GNN beats classic cuts in gamma-ray shower reconstruction","Graph networks sharpen gamma-ray event selection and energy","GNN improves gamma-ray quality factor and energy resolution","Sparse shower footprints feed GNN for better gamma-ray data","Graph nets outperform template methods for SWGO observatory"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the simulated detector response faithfully represents the planned SWGO array, especially the current absence of cosmic-ray-induced noise hits in the simulation; if real noise changes the pattern of triggered tanks, the GNN's graph structure changes and the claimed performance gain may not transfer.","fun_headline_variants_meta":{"raw":{"variants":["GNN beats classic cuts in gamma-ray shower reconstruction","Graph networks sharpen gamma-ray event selection and energy","GNN improves gamma-ray quality factor and energy resolution","Sparse shower footprints feed GNN for better gamma-ray data","Graph nets outperform template methods for SWGO observatory"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000209,"raw_usage":{"total_tokens":1344,"prompt_tokens":821,"completion_tokens":523,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":437,"completion_tokens_details":{"reasoning_tokens":446}},"tokens_in":437,"tokens_out":523,"duration_ms":4914,"temperature":1.0,"reasoning_tokens":446,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:58:24.121678+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same trained GNN and the LIC/PINCness baseline on a simulation that includes cosmic-ray-induced noise hits, or on real SWGO data after commissioning; if the quality-factor advantage drops below the claimed factor of two to three, or the energy resolution no longer beats the template method above 200 GeV, the central claim is refuted.","supporting_citations":[{"cited_title":"Atkins (Milagro Collaboration) et al","cited_arxiv_id":null,"evidence_quote":"Defines the compactness variable that underlies the LIC baseline observable."},{"cited_title":"Joshi et al","cited_arxiv_id":null,"evidence_quote":"Supplies the template-based likelihood reconstruction method used as the energy-resolution baseline."},{"cited_title":"Leitl, V","cited_arxiv_id":null,"evidence_quote":"Documents the current SWGO template-based reconstruction chain that the GNN is compared against."},{"cited_title":"Alfaro (HAWC Collaboration) et al","cited_arxiv_id":null,"evidence_quote":"Shows how to optimize box cuts on LIC and PINCness, the procedure used to set the comparison baseline."}],"review_version":1}