{"id":"320ad49d-745e-403f-9b25-5a26d27d0a2a","arxiv_id":"2505.03023","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Neural networks can automatically detect Ellerman bombs in high-resolution Hα data, but SDO/AIA four-passband intensity maps alone are insufficient for reliable detection.","lead":"This paper trains neural networks to automatically detect Ellerman bombs, small magnetic reconnection events on the Sun, in high-resolution Hα images from the Swedish 1-m Solar Telescope and in SDO/AIA ultraviolet images. The SST models work well, but the SDO/AIA models cannot reliably separate Ellerman bombs from other bright patches, suggesting that the AIA intensity maps alone are insufficient.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SDO/AIA negative result is not independent of its threshold-derived labels: the benchmark threshold is a variant of the label generator, so the 'not informative enough' conclusion is weaker than stated.","rationale":"The reader's weakest assumption is exactly where I land. The SST claim is reasonably supported: the models are tested on a genuinely new 2024 dataset, the feature-importance analysis recovers the known H-alpha wing signature, and the authors are explicit that the ground truth is threshold-based (Vissers et al. 2019b), so the claim is best read as 'NNs reproduce and generalize threshold-based EB labels.' I do not see a defect there that would overturn the positive result. The load-bearing weakness is the SDO claim. Section 2.2.2 defines SDO labels by thresholding sqrt(1600 Å x 1700 Å) and keeping contours that overlap SST H-alpha masks; Section 3.4 then benchmarks a 1700 Å intensity threshold on those same labels. The threshold baseline is therefore not an independent reference, and the near-tie in F1 is partly a consequence of the label-generation rule. The authors concede this in Sect. 4.3, but the abstract and conclusion still assert that the four AIA passbands are not informative enough. That assertion is plausible and consistent with prior work, but it is not established by the experiment as designed. The requested condition, for example fixed thresholds or an independent manually vetted SDO test set plus uncertainty estimates, matches the reader's conditional verdict. No change to the verdict is needed.","tokens_in":24707,"tokens_out":7807,"duration_ms":90416,"concrete_test":"Regenerate the SDO test-set labels using a fixed, pre-registered threshold (e.g., the Vissers et al. 2019b 5-sigma recipe) instead of per-dataset fine-tuning, and recompute the CNN and 1700 Å threshold baseline F1 scores on this relabeled test set. If the CNN remains within about 0.02 of the threshold baseline with F1 near 0.75, the insufficiency conclusion survives; if the gap or the absolute F1 changes substantially, the original numbers are artifacts of label construction.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing point is the SDO/AIA negative conclusion (Abstract, Sect. 4.2): that the four AIA passbands are not informative enough to accurately classify EBs. That claim rests on SDO ground truth built in Sect. 2.2.2: the authors fine-tune a per-dataset intensity threshold on sqrt(1600 Å x 1700 Å), keep only the resulting bright contours that overlap SST/H-alpha EB masks, and use those as labels. Two consequences follow. First, the positive class is not 'Ellerman bomb as visible in AIA'; it is 'UV brightening that happens to coincide with a threshold-defined H-alpha EB.' EBs lacking UV enhancement are excluded by construction, and UV-bright network points or plage that do not overlap an H-alpha mask are negatives. A model trained on this target can only establish that AIA maps cannot reproduce the overlap-with-H-alpha-mask subset, not that AIA intensity maps intrinsically lack the information to identify EBs. Second, the benchmark in Sect. 3.4 and Fig. 11 is not independent: a 1700 Å brightness threshold is scored against labels that were themselves generated by an intensity threshold on sqrt(1600 x 1700). The near-tie (CNN F1 0.76 vs. threshold 0.75) is therefore partly built into the label definition. The paper acknowledges this in Sect. 4.3, but the abstract and conclusion still state the insufficiency result without that caveat. The SST half of the paper inherits the Vissers et al. threshold definition of EBs, but that is explicit and less damaging because the positive result is about reproducing and generalizing those labels.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a deep-learning pipeline for semantic segmentation of Ellerman bombs (EBs) in two data modalities: high-resolution SST Hα spectral scans and SDO/AIA four-passband images (1600, 1700, 171, 304 Å). For SST, the authors train FNN and CNN models on labels inherited from Vissers et al. (2019b) and report test-set F1 scores up to about 0.9, with the FNN matching the CNN at native resolution, and the CNN clearly outperforming the FNN when the spatial resolution is degraded. For SDO/AIA, the ground truth is constructed by applying a per-dataset fine-tuned intensity threshold to the square root of the 1600×1700 Å product and retaining contours that overlap the SST Hα EB masks; on this target, the CNN reaches F1≈0.77, only slightly above a 2σ intensity threshold on the 1700 Å channel (F1≈0.75). The paper concludes that the four AIA passbands are not informative enough to classify EBs and suggests that temporal information may help. The work also includes permutation feature importance, an application to a new 2024 SST dataset, and a calibration study.","tokens_in":25075,"tokens_out":6453,"duration_ms":68260,"significance":"If the SST result is taken at face value, it is a useful demonstration that a pixel-wise classifier can reproduce expert-label EB segmentation with high fidelity from high-resolution Hα spectra alone, and that spatial context becomes important only as spectral and spatial resolution degrade. The careful ablation design, the balancing patch-selection algorithm, and the evaluation on an unseen dataset are genuine strengths. However, the SST conclusion is tied to the Vissers et al. (2019b) threshold-based EB definition, and the SDO/AIA negative conclusion is weakened by the fact that its ground-truth labels are themselves generated by an intensity threshold on the same passbands used as input. The significance of the SDO claim is therefore conditional on that label definition; the paper partially acknowledges this in Sect. 4.3 but does not carry the caveat into the abstract and conclusions.","major_comments":[{"comment":"The SDO ground-truth labels are constructed in Sect. 2.2.2 by applying a per-dataset fine-tuned intensity threshold to sqrt(1600 Å × 1700 Å) and keeping only the resulting bright contours that overlap SST Hα EB masks. This makes the positive class \"UV brightening that overlaps an Hα-defined EB,\" not an independent definition of an EB. The models are then trained and evaluated on the same four AIA passbands to reproduce this target, so the abstract and Sect. 4.2 conclusion that the four AIA passbands are \"not informative enough\" to classify EBs is established only with respect to this threshold-generated label set. In particular, the near-tie between the CNN (F1=0.76) and the 1700 Å intensity threshold at 2σ (F1=0.75) in Sect. 3.4 and Fig. 11 is not an independent comparison: the labels were themselves produced by an intensity threshold on sqrt(1600×1700). The dependence is acknowledged in Sect. 4.3, but the abstract and Sect. 5 still state the insufficiency result without this caveat. The authors should rephrase the conclusion to state that the four passbands cannot reproduce the threshold-derived overlap labels, and ideally validate on manually labeled or independently defined SDO EB candidates.","section":"Sect. 2.2.2 and Sect. 4.2/Abstract"},{"comment":"The SST ground-truth labels are inherited from Vissers et al. (2019b), which defined EBs via a brightness threshold on Hα wings plus area and lifetime criteria. Because the FNN, which has no spatial context, achieves nearly the same F1 as the CNN at native resolution, the conclusion that \"spatial context is not required to detect EBs\" may largely reflect the fact that the target labels are themselves constructed from per-pixel intensity thresholds on the Hα wings. This is not a flaw in the experimental design, but the abstract and Sect. 4.1 should qualify the claim as being relative to the Vissers et al. definition of EBs, not to a physically independent definition.","section":"Sect. 2.2.1 and Abstract"},{"comment":"The intensity-threshold benchmark is evaluated by scanning thresholds from 1σ to 9σ and reporting the best F1 (0.75 at 2σ) on the test set, while the NN performance is reported at its best probability threshold. Selecting the threshold on the test set gives the threshold method an optimistic bias in the comparison, and the near-tie with the CNN is therefore not on an equal footing. The authors should specify how the threshold was chosen (e.g., on the validation set) or account for the selection bias, because this comparison is used to support the conclusion that the AIA passbands are insufficient.","section":"Sect. 3.4 and Fig. 11"}],"minor_comments":[{"comment":"There are several typographical issues, e.g., \"Futhermore\" in Sect. 2.2.2, \"di fferent\" in multiple places, \"classifiy\" in Sect. 3.1, and inconsistent spacing in \"V oort\" in the author names and text.","section":"General"},{"comment":"The tick labels in the left panel appear disordered and contain what looks like a duplicated \"+1.2\" offset; please check that the wavelength offsets are correctly mapped to the spectral sampling described in Sect. 2.2.1.","section":"Fig. 10"},{"comment":"The per-dataset intensity thresholds used to create the SDO masks are described as \"fine-tuned\" but the numerical values are not reported. Please provide them or specify the tuning criterion so that the ground-truth construction is reproducible.","section":"Sect. 2.2.2"},{"comment":"The text states that detections were selected with \"a minimum linear extent of about 2′′,\" which seems inconsistent with the subarcsecond EB sizes quoted elsewhere and with the simultaneous 0.035 arcsec² area cut; please clarify this criterion.","section":"Sect. 3.1 and Fig. 6"},{"comment":"The interpretation that the SDO model \"applies an intensity threshold over a non-linear combination of the passbands\" is phrased as a speculation; it should be explicitly marked as an interpretation rather than a demonstrated property of the model.","section":"Sect. 4.2"}],"recommendation":"major_revision","confidential_remarks":"The SST portion of the paper is methodologically sound and potentially useful for the community. The main issue is the SDO/AIA negative conclusion: because the SDO labels are generated by an intensity threshold on the same passbands, the claim that the four passbands are intrinsically insufficient is not established. This can be fixed by rephrasing the conclusion and, if possible, by adding an evaluation on an independent label set or a more explicit validation of the label-generation dependence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: this is a useful, honest paper, but the headline negative result about SDO/AIA is softer than the abstract makes it sound. What's actually new is that the authors are the first to apply NN semantic segmentation to Ellerman bombs, and they do it carefully. The SST result—F1 about 0.9 whether you use a CNN or a per-pixel FNN at full spectral resolution, with the CNN pulling ahead only when spatial resolution is degraded—is believable and directly useful. The feature importance analysis confirming that the Hα wings at ±1 Å carry the information is a nice sanity check. They also apply the model to a new, unseen SST dataset and get sensible area statistics. The paper is clearly written, the methods are standard but applied cleanly, and the citations are appropriate.\n\nThe soft spot is the SDO/AIA negative conclusion, and it's a real one. The labels for SDO are produced by fine-tuning an intensity threshold on sqrt(1600×1700) and keeping contours that overlap SST EB masks. That means the models are trained to reproduce that threshold, and the comparison with a 1700 Å intensity threshold in Sect. 3.4 is not an independent benchmark—the near-tie (CNN 0.76 vs threshold 0.75) is partly baked into the label generation. So the conclusion that 'four AIA passbands are not informative enough' is only established for a specific, threshold-defined subset of UV-bright events that coincide with Hα masks. The paper does admit part of this in Sect. 4.3, but the abstract and conclusions state the insufficiency without that caveat. That's the main thing I'd want fixed.\n\nMinor issues: no code or data release, and the metrics are reported without uncertainty quantification—single test set, no error bars. These are minor in an application paper but worth noting. Also, the SDO label generation excludes EBs without UV enhancement by construction, so the positive class is really 'UV-bright threshold events that overlap Hα masks' rather than all EBs.\n\nBottom line: the SST part is a solid, citable result for anyone working on automated EB detection. The SDO part is a useful cautionary tale, but it should be framed as 'this label definition and this model family don't separate EBs from other bright patches' rather than as a general statement about AIA's information content. It deserves peer review, and I'd encourage a referee to push for a softened conclusion, data/code release, and some error bars.","headline":"The SST detection result is solid and citable; the SDO/AIA negative conclusion overreaches because the labels are threshold-derived.","tokens_in":25599,"tokens_out":6475,"would_cite":true,"duration_ms":56436,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Trained neural networks segment Ellerman bombs from SST H-alpha spectra with high fidelity, while SDO/AIA four-passband models cannot reliably separate EBs from other bright patches.","keywords":["Ellerman bombs","semantic segmentation","convolutional neural networks","deep learning","H-alpha line wings","SDO/AIA passbands","feature importance","small-scale magnetic reconnection"],"falsifier":"Re-label the SDO test set without the UV-threshold step—for example, by directly thresholding co-aligned SST H-alpha wing brightness and then training the same four-passband CNN on those labels; if the resulting model exceeds an F1 of roughly 0.85 on held-out frames, the paper's claim that the four static passbands are not informative enough would be falsified. Conversely, if a classifier using only 1600 and 1700 Å light-curve statistics separates H-alpha-confirmed EBs from network bright points with high accuracy, the paper's proposed temporal route would receive direct support.","tokens_in":24559,"feed_emoji":"🌞","tokens_out":7478,"duration_ms":67003,"temperature":0.7,"pith_summary":"This paper sets out to replace hand-tuned intensity thresholds with learned detectors that label every pixel as belonging to an Ellerman bomb or not, working both on high-resolution H-alpha spectra from the Swedish 1-m Solar Telescope and on SDO/AIA images. On SST data the networks are very effective, reaching a balanced precision-recall measure around 0.9, and the H-alpha line wings near ±1 Å carry most of the information, so spatial context is not needed when the spectrum is well resolved. When the spatial and spectral resolution is degraded, spatial context becomes important, and the convolutional model outperforms the pixel-wise model. On SDO/AIA, the same approach does not succeed: the best model reaches only about 0.77, barely better than a simple brightness threshold, and the four passbands together are not informative enough to tell EBs from network bright points and UV bursts. The paper concludes that adding the temporal evolution of bright patches is the most promising route toward a useful SDO-based detector, and a reader should care because a reliable automatic detector would turn a labor-intensive visual classification into a systematic survey of small-scale magnetic reconnection.","feed_headline":"Neural nets find Ellerman bombs in H-alpha, not in SDO alone","feed_subtitle":"A deep-learning detector reaches F1 ~0.9 on SST spectra and suggests SDO needs time-series data.","key_machinery":"The load-bearing mechanism is a semantic-segmentation pipeline built on two deliberately contrasted neural architectures: a fully convolutional network (a CNN that sees image patches and therefore spatial context) and a fully connected network (an FNN that classifies each pixel from its spectral vector alone). Training them on the same balanced patches, then applying a probabilistic calibration step and a threshold, lets the authors attribute any performance gap to the presence or absence of spatial context. The argument is carried further by spatial degradation experiments, which degrade SST data to SDO pixel size while preserving the masks, and by permutation feature importance, which shuffles each input channel and measures the drop in F1. Together these tools convert a practical detection task into a diagnostic about which information—spectral, spatial, or temporal—is genuinely needed.","core_discovery":"The central discovery is a controlled demonstration of what information suffices to detect Ellerman bombs. In SST H-alpha observations at 0.057 arcsec per pixel, both a convolutional network and a per-pixel fully connected network classify EBs with nearly identical performance, showing that the spectral shape alone—especially the H-alpha wings around ±1 Å, identified by permutation feature importance—is enough when spectral resolution is high. Adding the center-to-limb parameter μ does not improve the results. After spatial degradation to SDO-like pixels, the convolutional model clearly beats the pixel-wise model, showing that spatial context substitutes for lost spectral detail. On SDO/AIA, the CNN reaches a maximum F1 of about 0.77 and the FNN about 0.66, and the 1600 Å channel is the dominant input, with 1700 Å, 171 Å, and 304 Å contributing less. A simple 2σ brightness threshold on 1700 Å achieves an F1 of 0.75, very close to the CNN, which the authors read as evidence that the four static intensity maps do not contain enough information to separate EBs from pseudo-EBs, network bright points, and UV bursts. Sample light curves of four bright patches show that H-alpha-confirmed EBs produce stronger short-timescale 1600 Å intensity variations than unrelated bright patches, pointing to temporal information as the key missing ingredient.","pith_inferences":["Because the SDO labels are derived from a UV-intensity threshold that is fine-tuned per dataset, the negative SDO result should be read as 'static AIA maps cannot reproduce these threshold-defined labels'; it does not rule out the possibility that a differently labeled training set, built for example from spectral classification of H-alpha profiles, would be learnable from the same passbands.","The light-curve analysis is based on only four bright patches; a natural extension would be to compute 1600/1700 Å rise rates or fluctuation amplitudes for all SDO CNN candidates and compare their distributions for H-alpha-confirmed EBs versus network bright points, testing the paper's temporal conjecture at scale.","The same CNN-versus-FNN comparison could be applied to other chromospheric diagnostics such as Ca II 8542 Å or H-beta, and to upcoming high-resolution facilities, where the question of whether spectra alone suffice is equally practical.","If the sub-0.1 arcsec² H-alpha-wing enhancements found by the models are genuine EBs, the event-size distribution extends below the previous 0.035 arcsec² cutoff, which would raise the estimated contribution of small reconnection events to the Sun's energy balance; this is a consequence the paper only hints at."],"forward_implications":["SST H-alpha observations with adequate spectral sampling can be processed automatically to produce large, consistent EB catalogs without per-dataset threshold tuning, enabling statistical studies of reconnection occurrence and active-region evolution.","Since the H-alpha wings near ±1 Å dominate detection, future instruments can concentrate spectral sampling there without losing detection power.","At lower spatial resolution, detection depends on spatial context, so wide-field or satellite observations with degraded resolution should include contextual information rather than relying on single-pixel spectra.","SDO/AIA static four-passband maps are insufficient for reliable EB detection; any full-disk EB survey will need temporal information, such as short-timescale light-curve variability, or additional diagnostics.","The trained SDO CNN can serve as a first-stage candidate detector that flags bright patches whose 1600/1700 Å light curves can then be screened for EB-like variability, a pipeline the authors explicitly propose."],"supporting_citations":[{"why":"Supplies the ground-truth SST EB masks, detection criteria, and the earlier attempt to extend EB detection to SDO/AIA passbands.","marker":"Vissers et al. (2019b)"},{"why":"Establishes the H-alpha wing definition of EBs and warns about pseudo-EBs from network bright points, motivating the spectral window choice and the interpretation of SDO failures.","marker":"Rutten et al. (2013)"},{"why":"Provides the reference characterization of EB flame morphology and H-alpha wing spectral signatures used as the physical baseline.","marker":"Watanabe et al. (2011)"},{"why":"Documents the spatial and temporal correlation between H-alpha EBs and co-spatial mid-UV brightenings, underpinning the SDO mask construction and light-curve analysis.","marker":"Vissers et al. (2013)"},{"why":"Provides early evidence that EB-associated mid-UV brightenings vary in time, supporting the paper's temporal-evolution hypothesis.","marker":"Qiu et al. (2000)"},{"why":"Supplies the calibration methodology (isotonic regression) that converts raw network outputs into interpretable probabilities.","marker":"Guo et al. (2017)"},{"why":"Introduces permutation feature importance, the technique used to identify which spectral points and passbands carry detection information.","marker":"Fisher et al. (2018)"},{"why":"Shows that UV bursts produce signatures in both 1600 Å and 1700 Å, providing a documented confusion source for the SDO models.","marker":"Kleint & Panos (2022)"},{"why":"Statistically links H-alpha EBs with bright patches and network magnetic-field concentrations, supporting the discussion of why the SDO models misclassify bright points.","marker":"Tähtinen et al. (2022)"}],"fun_headline_variants":["Deep learning finds Ellerman bombs in H-alpha, SDO needs time","AI spots Ellerman bombs in H-alpha, not in SDO stills","Neural nets detect Ellerman bombs from spectra, not SDO images","H-alpha wings key for deep learning Ellerman bomb detection","Deep learning spots Ellerman bombs better with spectral info"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The SDO conclusion depends on the assumption that the per-dataset UV brightness threshold, applied to the square root of the 1600×1700 Å product and overlaid on SST H-alpha masks, correctly labels SDO pixels as Ellerman bombs; if that label set is wrong, the models learn the wrong target and the claim that AIA passbands are insufficient is not established.","fun_headline_variants_meta":{"raw":{"variants":["Deep learning finds Ellerman bombs in H-alpha, SDO needs time","AI spots Ellerman bombs in H-alpha, not in SDO stills","Neural nets detect Ellerman bombs from spectra, not SDO images","H-alpha wings key for deep learning Ellerman bomb detection","Deep learning spots Ellerman bombs better with spectral info"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000668,"raw_usage":{"total_tokens":3172,"prompt_tokens":1197,"completion_tokens":1975,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":813,"completion_tokens_details":{"reasoning_tokens":1885}},"tokens_in":813,"tokens_out":1975,"duration_ms":13772,"temperature":1.0,"reasoning_tokens":1885,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:01:45.508139+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-label the SDO test set without the UV-threshold step—for example, by directly thresholding co-aligned SST H-alpha wing brightness and then training the same four-passband CNN on those labels; if the resulting model exceeds an F1 of roughly 0.85 on held-out frames, the paper's claim that the four static passbands are not informative enough would be falsified. Conversely, if a classifier using only 1600 and 1700 Å light-curve statistics separates H-alpha-confirmed EBs from network bright points with high accuracy, the paper's proposed temporal route would receive direct support.","supporting_citations":[{"cited_title":"J., Vissers , G","cited_arxiv_id":null,"evidence_quote":"Establishes the H-alpha wing definition of EBs and warns about pseudo-EBs from network bright points, motivating the spectral window choice and the interpretation of SDO failures."}],"review_version":1}