{"id":"296dd923-4a1f-43d8-9406-c2a46a176382","arxiv_id":"2504.15670","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Occlusion sensitivity, not gradient-based saliency, best reveals how a convolutional autoencoder recognizes scintillator pulses; the paper ties the loss minimum to kernel size and estimates 30 ns minimum pulse separation.","lead":"Four explainability methods were compared on a convolutional autoencoder that picks out scintillator pulses, and the paper concludes that an occlusion-based method is the only one that yields usable insight into what the network tracks. The same analysis produces a tentative design rule linking the last convolutional filter size to the signal span the network needs to detect a pulse.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 30 ns separation estimate rests on an unvalidated equivalence between the unmasking loss minimum and two-pulse resolution; without a direct double-pulse test or stated sampling rate, the headline quantitative claim is unsupported.","rationale":"Good-faith reading: the paper's defensible contribution is the qualitative comparison of four xAI methods on a convolutional autoencoder for pulse reconstruction. The layer-output and occlusion-sensitivity plots support the claim that rise and peak regions are most influential and that ReLU output activation yields cleaner attributions for M(18, ReLu). That part is plausible and novel in the small niche. The reader's CONDITIONAL verdict correctly separates this defensible core from the quantitative headline. My stress-test focuses on the load-bearing step: the estimated 30 ns minimum separation. The paper never defines the loss used for occlusion; never states the sampling rate; never demonstrates two-pulse separation for the current models; and never reconciles the 30 ns estimate with the >10 ns claim from earlier work. The unmasking experiment itself is a single-pulse perturbation study, and the leap from 'minimum loss at 18 unmasked samples' to 'two pulses closer than 30 ns cannot be distinguished' is not derived from any model of two-pulse interference, receptive-field math, or an empirical double-pulse test. The kernel-size attribution is additionally confounded by the joint variation of kernel size and activation across the three models. None of these issues undermine the qualitative conclusions; they undermine only the quantitative claims, which should either be removed, explicitly labeled as speculative, or backed by a direct two-pulse measurement. The paper's own concession that gradient-based xAI output is 'subject to further investigation' further narrows the credible contribution to the occlusion-based analysis. Hence I recommend leaving the verdict at CONDITIONAL, unchanged from the reader, since the path to resolution is a missing experiment rather than a fundamental invalidation of the work.","tokens_in":7647,"tokens_out":4450,"duration_ms":39380,"concrete_test":"Run a double-pulse separation study on the same trained models, using the waveform model of Section 2: generate events with two identical pulses at separations dt = 5, 10, 15, 20, 25, 30, 35, 40, and 50 ns, and measure the fraction of events where the modified autoencoder output yields two correctly resolved peaks (e.g., correct count and arrival times). The 30 ns claim stands only if the smallest dt with reliable resolution for M(18, ReLu) matches the loss-minimum-derived value; if the threshold is closer to the 10 ns value from ref [34] or exceeds 30 ns, the interpretive bridge fails. Report the sampling rate used in the simulation so the sample-to-nanosecond conversion is explicit.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim in Section 4 is that the loss-curve minimum in the progressive-unmasking experiment, at 18 unmasked values after the signal rise, implies a minimum resolvable separation of 30 ns between two pulses. This requires unstated premises: (i) the 'total loss' plotted in Figures 5-6 is a defined, meaningful detection metric, but the paper never gives the loss function; (ii) the minimum of this loss as a function of unmasked samples marks the minimal information window needed for reliable pulse identification, rather than an artifact of the masking procedure or the denoising objective; and (iii) the sample count converts directly to nanoseconds via an unstated sampling rate, with 18 samples + 10 ns rise approximately 30 ns. Even granting these, there is no argument that this single-pulse unmasking minimum equals the temporal separation at which two overlapping pulses can be distinguished by the same network. The only empirical anchor, ref [34], reported >10 ns separation for an earlier model, and the present models are not tested on two-pulse events. The kernel-size coincidence is also not uniquely attributable because M(18, ReLu) differs from M(14, lin) in both kernel size and activation function, so the loss-minimum position cannot be assigned to kernel size alone. The paper itself supplies the needed check: Section 2 describes the dataset and the earlier separation claim, but the current models are never evaluated for double-pulse resolution. Until the authors measure two-pulse separation on M(18, ReLu), the 30 ns number is an extrapolation, not a result.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript evaluates four explainable-AI techniques—Integrated Gradients, Vanilla Saliency, SmoothGrad, and Occlusion Sensitivity—on modified convolutional autoencoders that reconstruct simulated scintillating-crystal pulses. Three model variants are considered, differing in the last-filter kernel size (18 vs 14) and the output activation (linear vs ReLU): M(18, lin), M(14, lin), and M(18, ReLU). After presenting layer-output plots and saliency maps, the authors focus on occlusion sensitivity, including a progressive-unmasking experiment in which a single pulse is gradually revealed. They report that the signal rise and peak are the most loss-relevant regions, that M(18, ReLU) reaches the deepest loss minimum after 18 unmasked values, and that this minimum coincides with the last-filter kernel size. They further estimate a minimum resolvable separation of 30 ns between two pulses and conclude that occlusion sensitivity is the most informative xAI method for this application.","tokens_in":7796,"tokens_out":6681,"duration_ms":58783,"significance":"The qualitative observation that occlusion-based attribution highlights the pulse rise and peak, rather than the long decay tail, is a useful demonstration of xAI in a HEP waveform-reconstruction setting, and the progressive-unmasking diagnostic is a promising idea for probing receptive fields. The paper also gives a concrete three-way architecture comparison and states the SmoothGrad hyperparameters, which helps reproducibility of that part. If the quantitative claims were backed by a defined loss, error bars, and a direct two-pulse test, the paper would make a meaningful contribution to applied xAI in calorimetry. As it stands, the contribution is a promising proof-of-concept whose headline numbers are not yet supported.","major_comments":[{"comment":"The 'total loss' plotted in the occlusion-sensitivity figures is never defined. The manuscript does not state which loss function is used, whether the curves are for a single event or averaged over the test set, or how the mask is applied in the progressive-unmasking variant. Without this information the minima in Fig. 6 cannot be interpreted as the 'minimum information required' for detection, and the comparison among models is not reproducible. Please specify the loss, the aggregation, and the event-to-event spread (e.g., mean plus/minus standard deviation or percentiles over many events).","section":"Section 3.4, Figs. 5-6"},{"comment":"The conversion of 18 unmasked values into 30 ns is not justified. The waveforms are described as 1024 ns long, but the sampling rate is never given. If the digitization is one sample per nanosecond, that must be stated explicitly, and the relationship '18 samples + 10 ns rise approximately equals 30 ns' needs an explicit derivation. As written, the headline quantitative result is not reproducible.","section":"Section 4"},{"comment":"The estimate of 30 ns as the minimum separation between two distinguishable signals is not supported by any two-pulse experiment on the three models studied. The only empirical anchor cited, ref. [34], reports that pulses more than 10 ns apart can be separated for an earlier model, which is also in tension with the new 30 ns estimate. The single-pulse unmasking minimum may be related to the receptive field, but no argument is given that it equals the two-pulse resolution limit. A direct double-pulse test, or an explicit validated mapping, is required before the 30 ns claim can stand.","section":"Section 4 vs. Section 2"},{"comment":"The claimed coincidence between the 18-unmasked-value minimum and the last-filter kernel size is under-identified. M(18, ReLU) differs from M(14, lin) in both kernel size and activation function, so the minimum position cannot be attributed to kernel size alone. The comparison of M(18, lin) with M(14, lin) partially isolates the kernel size, but the specific 18-value coincidence is claimed for M(18, ReLU), which is confounded by activation. A control with a single varied factor (for example, adding M(14, ReLU)) is needed to support the kernel-size interpretation.","section":"Section 4"}],"minor_comments":[{"comment":"Integrated Gradients is listed among the tested methods, but no Integrated Gradients results are presented in Section 3; please add the results or revise the claim.","section":"Section 3"},{"comment":"The caption contains a typo ('fuction') and should be corrected to 'function'.","section":"Figure 5 caption"},{"comment":"Equation (1) uses the notation 'M_c(x)' without defining 'M_c' or the subscript 'c'; please define all symbols before using them.","section":"Eq. (1)"},{"comment":"References [24] and [26] are the same paper; please consolidate and remove the duplicate.","section":"References"},{"comment":"The statement that models with shallower minima 'might be better suited' for separating closely spaced signals is speculative and is not tested in the manuscript; consider removing it or supporting it with a two-pulse experiment.","section":"Section 4, final paragraph"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within the journal's scope and the qualitative xAI comparison should be of interest to the HEP-ML community. My main concern is that the headline 30 ns and kernel-size coincidence claims are load-bearing and currently unsupported; they require additional definitions, controls, and a two-pulse study. I would be willing to review a revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Small applied study from the PADME group comparing four xAI methods on a modified convolutional autoencoder for scintillator pulse reconstruction. The genuinely new piece is the progressive-unmasking variant of occlusion sensitivity: masking a whole pulse and incrementally unmasking samples traces how reconstruction loss falls, and the minimum is read as the information window the network needs. That is a useful qualitative idea.\n\nWhat the paper does well: the comparison is clearly illustrated, and the claim that occlusion sensitivity gives more interpretable maps than the gradient-based saliency methods is consistent with the figures. M(18, ReLu) reaching the deepest loss minimum after 18 unmasked samples, coinciding with its last-filter kernel size, is a plausible heuristic linking architecture to effective receptive field. The authors also correctly note that the gradient-based outputs need further investigation.\n\nThe soft spot is the quantitative headline. The 30 ns minimum separation estimate is not backed up: the loss function is never stated, the sampling rate is never given, and the step from 18 samples to 30 ns is simply asserted. The bridge from single-pulse unmasking to two-pulse resolution is never validated; the earlier 10 ns separation claim is never reconciled, and no double-pulse test on M(18, ReLu) is presented. The kernel-size coincidence is also under-identified because M(18, ReLu) differs from M(14, lin) in both kernel size and activation. Integrated Gradients is announced but never shown. No code or data are shipped, and the figures lack error bars.\n\nNone of this kills the qualitative insight, but it means the 30 ns number should not be quoted in any downstream work. The paper would be much stronger with a stated loss function, a stated sampling rate, and a direct two-pulse experiment.\n\nWho is this for? Groups working on xAI for detector signal reconstruction might read it for the unmasking heuristic. It deserves serious peer review because the idea is useful and the presentation is honest, even though the quantitative claim needs major revision. I would send it to review, not desk-reject.","headline":"A useful qualitative xAI heuristic for pulse-reconstruction autoencoders, but the 30 ns separation estimate is an unsupported extrapolation.","tokens_in":8494,"tokens_out":2876,"would_cite":false,"duration_ms":23555,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Occlusion sensitivity reveals what a pulse-reconstruction autoencoder relies on, and points to a roughly 30-nanosecond limit for separating two pulses.","keywords":["explainable AI","xAI","occlusion sensitivity","convolutional autoencoder","pulse reconstruction","scintillating crystal waveforms","signal parameter extraction","double-pulse separation"],"falsifier":"Measure the actual sampling rate of the simulated waveforms (or re-run the unmasking experiment at a stated 1 sample/ns); if 18 samples do not correspond to roughly 20 ns of rise time plus kernel width, the 30 ns separation estimate changes. Alternatively, test M(18, ReLu) on pairs of pulses separated by less than 30 ns: if it can reconstruct both pulses, the estimated minimum separation is not a true limit.","tokens_in":7270,"feed_emoji":"⚡","tokens_out":5299,"duration_ms":48416,"temperature":0.7,"pith_summary":"This paper asks which explainable-AI method actually helps physicists understand what a pulse-reconstruction autoencoder has learned, and whether that understanding can guide detector design. It applies four attribution methods - Integrated Gradients, Vanilla Saliency, SmoothGrad, and Occlusion Sensitivity - to three variants of a convolutional autoencoder trained on simulated scintillator-crystal waveforms. It reports that Occlusion Sensitivity gives the clearest and most usable picture: unmasking a recorded pulse bit by bit produces a sharp minimum in the reconstruction loss, and the position of that minimum aligns with the size of the model's final convolutional filter. From that alignment the paper estimates the minimum time separation at which two pulses can still be told apart, about 30 nanoseconds. The intended payoff is that xAI can turn a black-box pulse finder into a diagnostic tool for architecture choices and detector timing requirements.","feed_headline":"Occlusion maps reveal 30 ns pulse-separation limit","feed_subtitle":"Progressive unmasking of scintillator waveforms shows the autoencoder keys on the rise and peak, not the decay.","key_machinery":"The central object is a modified convolutional autoencoder with an encoder of three 1D convolution layers and a decoder of three transposed-convolution layers, where the final filter has kernel size 18, with variants using kernel size 14 and linear rather than ReLU output. The carrying mechanism is Occlusion Sensitivity in a progressive form: one unmasked sample at a time after the pulse rise, while plotting the reconstruction total loss versus the number of unmasked values. The position of the loss minimum is interpreted as the minimal waveform information required for detection, and its coincidence with the last filter's kernel size is what links architecture to the inferred 30-ns two-pulse separation limit.","core_discovery":"On the paper's own terms, the central discovery is that Occlusion Sensitivity is the xAI method that yields actionable understanding of the modified autoencoder's pulse reconstruction. When a single recorded pulse is progressively unmasked, the total reconstruction loss drops to a sharp minimum and then plateaus; for the model with an 18-sample last-filter kernel and ReLU output, the minimum occurs after 18 samples have been unmasked, matching the kernel size. The paper reads this as the minimal waveform window the model needs for reliable detection and converts it, together with the 10-ns pulse rise time, into an estimated minimum temporal separation of about 30 ns between two distinguishable signals. It also reports that the ReLU output layer confines model activity to pulse arrival positions, while linear output leaves activity across the whole noise-dominated waveform.","pith_inferences":["A direct test of the 30 ns number is to re-run the unmasking experiment at an explicitly stated sampling rate; if the rate is not 1 sample/ns, the inferred separation should scale accordingly.","The kernel-size coincidence suggests a more general diagnostic: for any convolutional pulse finder, the progressive-occlusion minimum could be used to measure receptive field and minimum distinguishable signal spacing without retraining.","Comparing the three models suggests activation function and kernel size are independent tuning knobs; a follow-up could sweep kernel sizes at fixed activation to see whether the loss-minimum position tracks kernel size monotonically.","The same progressive-unmasking procedure could be applied directly to two-pulse events, masking one pulse and measuring how many samples are needed to detect the other, which would give a per-event double-pulse resolution estimate."],"forward_implications":["Occlusion-based unmasking can identify the exact waveform region (rise and peak) that drives a pulse-reconstruction network, so future models can focus training and architecture on that region.","The coincidence between loss-minimum position and the last filter's kernel size gives a way to estimate the model's effective receptive field from explanation output rather than from parameter counting.","The inferred roughly 30-ns minimal two-pulse separation would serve as a design target for calorimeter readout and pile-up rejection requirements.","Models with shallower minima reaching their minimum earlier may be better suited for separating closely spaced pulses, a hypothesis the paper suggests for follow-up.","The last-layer activation function is singled out as a control on whether the model localizes pulses or produces noisy all-event output, which matters for pulse counting and arrival-time determination."],"supporting_citations":[{"why":"Supplies the simulated BGO scintillator-crystal waveforms and the original autoencoder model that this study modifies.","marker":"[33]"},{"why":"Provides the independent dataset and prior evaluation showing that individual pulses more than 10 ns apart can already be separated.","marker":"[34]"},{"why":"Defines SmoothGrad, one of the four xAI methods whose behavior is compared on the three model variants.","marker":"[16]"},{"why":"Defines Occlusion Sensitivity Analysis, the method the paper adapts into progressive unmasking to produce the central loss curve.","marker":"[17]"},{"why":"Defines Integrated Gradients, one of the xAI methods tested and found less immediately interpretable than occlusion.","marker":"[36]"},{"why":"Defines Vanilla Saliency, another xAI baseline whose noisy signal-region output is compared across the models.","marker":"[37]"}],"fun_headline_variants":["Occlusion maps expose 30 ns pulse separation limit","Autoencoder's 18-sample window dictates 30 ns separation","Occlusion sensitivity finds minimal pulse window: 18 samples","ReLU output confines model activity to pulse positions","Progressive unmasking maps kernel-size trigger window"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central quantitative claim assumes that the lowest total-loss point in the progressive-unmasking curve marks the smallest waveform window the network needs, and that the number of samples at that point can be converted into nanoseconds even though the sampling rate of the 1024-sample, 1024-ns waveforms is never stated in the paper.","fun_headline_variants_meta":{"raw":{"variants":["Occlusion maps expose 30 ns pulse separation limit","Autoencoder's 18-sample window dictates 30 ns separation","Occlusion sensitivity finds minimal pulse window: 18 samples","ReLU output confines model activity to pulse positions","Progressive unmasking maps kernel-size trigger window"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000272,"raw_usage":{"total_tokens":1542,"prompt_tokens":762,"completion_tokens":780,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":378,"completion_tokens_details":{"reasoning_tokens":701}},"tokens_in":378,"tokens_out":780,"duration_ms":7752,"temperature":1.0,"reasoning_tokens":701,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:20:32.029592+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the actual sampling rate of the simulated waveforms (or re-run the unmasking experiment at a stated 1 sample/ns); if 18 samples do not correspond to roughly 20 ns of rise time plus kernel width, the 30 ns separation estimate changes. Alternatively, test M(18, ReLu) on pairs of pulses separated by less than 30 ns: if it can reconstruct both pulses, the estimated minimum separation is not a true limit.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the independent dataset and prior evaluation showing that individual pulses more than 10 ns apart can already be separated."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines Occlusion Sensitivity Analysis, the method the paper adapts into progressive unmasking to produce the central loss curve."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines Integrated Gradients, one of the xAI methods tested and found less immediately interpretable than occlusion."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines Vanilla Saliency, another xAI baseline whose noisy signal-region output is compared across the models."}],"review_version":1}