{"id":"372f6126-73af-41a5-8f10-66a3b6f48d35","arxiv_id":"2412.09888","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"gwsnr is a Python package that computes gravitational-wave signal-to-noise ratios and detection probabilities for large compact-binary populations using interpolation, neural networks, and GPU acceleration.","lead":"This paper describes gwsnr, a Python package that speeds up gravitational-wave signal-to-noise calculations for large simulated populations of merging black holes and neutron stars. It combines several known methods (interpolation, neural networks, and GPU acceleration) in one tool, which matters for population and lensing-rate studies that need millions of repeated calculations.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"ANN and partial-scaling P_det accuracy is unvalidated; the paper's own admission that the ANN is poor at rho_opt leaves the central cheap-accurate P_det claim unsupported.","rationale":"The reader's conditional verdict identifies missing benchmarks and the ANN P_det validation gap, but their stated weakest assumption is about the injection-catalogue-derived rho_obs_thr. My concern is complementary and more directly tied to the software's own approximation pathways: even with a perfect threshold, an unvalidated ANN or interpolation scheme can bias P_det. The paper's admission that the ANN is 'poor at estimating rho_opt directly' highlights a real risk that the approximation error is not benign when integrated into selection functions; the hybrid scheme does not automatically fix this unless the approximate pre-filter reliably identifies marginal events. This is the most load-bearing weakness because the package's entire value proposition is accurate P_det at low cost, and no evidence is offered that the approximations preserve accuracy. The standard inner-product math and FINDCHIRP-style partial scaling are well-founded, which I credit; the issue is purely the unvalidated accuracy of the implemented approximate pathways. The proposed concrete test settles this by comparing all pathways against exact inner products on a realistic population; if the agreement is good, the concern is resolved. Hence I do not change the verdict: the manuscript remains conditional pending this validation.","tokens_in":6497,"tokens_out":3203,"duration_ms":38103,"concrete_test":"Run the package's own validation workflow on ~10^4 synthetic BBH events (e.g., IMRPhenomXPHM, O4-like PSD, 2- or 3-detector network): compute rho_opt exactly with lalsuite's noise-weighted inner product; compute P_det from exact rho_opt using the package's statistical model; then compute P_det using partial-scaling, ANN, and hybrid pathways with identical thresholds. Report maximum and mean |Delta P_det| and the fraction of events with |Delta P_det| > 0.01, stratified by true rho_opt, especially within +/-0.5 of the threshold. Also test the hybrid guard band: sample events whose true rho_opt is within the threshold band and check whether the approximate pre-filter correctly flags them for recalculation. If ANN or partial-scaling P_det bias exceeds a few percent near threshold, the claimed accuracy for selection-effect modelling fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that gwsnr delivers accurate P_det for large populations rests on the accuracy of its approximate pathways (Partial Scaling Interpolation, ANN-based P_det, and Hybrid recalculation). The manuscript provides no quantitative validation of any approximate pathway against the exact noise-weighted inner product. Section 'ANN-based P_det Estimation' explicitly concedes the ANN is 'poor at estimating rho_opt directly' and justifies its use for P_det by arguing detectability depends on threshold crossing rather than precise values. That argument is insufficient: P_det is a strong function of rho_opt near the threshold, and the hybrid recalculation scheme can only correct events if the approximate method correctly identifies which events are near threshold. A systematically biased ANN could mis-rank events, causing the hybrid pre-filter to miss true marginal events, biasing the selection function. Similarly, Section 'Partial Scaling Interpolation' asserts 'significant speed-ups' and accurate recovery via spline interpolation, but reports no interpolation error or comparison with direct integration. Without such validation—and without a repository commit hash or runnable benchmarks—the headline claim of enabling efficient and reliable selection-effect modelling and rate estimation is not demonstrated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript describes gwsnr, a Python package for fast signal-to-noise ratio (SNR) and detectability (P_det) calculations for compact-binary gravitational-wave sources. The package offers several computational pathways: a direct noise-weighted inner product with multiprocessing/JAX/MLX acceleration, a partial-scaling interpolation method for non-spinning and aligned-spin systems, an artificial neural network (ANN) for P_det estimation with precessing waveforms, a hybrid scheme that recalculates near-threshold events exactly, statistical models for the observed SNR, and horizon-distance calculations. The central claim is that these tools enable efficient selection-effect modelling, rate estimation, detector-sensitivity studies, and large-scale population simulations. The mathematical formulations are standard (FINDCHIRP scaling, matched-filter inner products, non-central chi-square P_det) and the package appears well-architected, but the paper does not provide quantitative validation of the approximate pathways, which is necessary to support the headline claims of accuracy and speed.","tokens_in":6732,"tokens_out":3292,"duration_ms":36829,"significance":"If the performance claims are borne out, gwsnr would be a practically valuable tool for GW population inference, where repeated SNR and P_det evaluations are computationally expensive. The package bundles several methods under a unified interface, provides JAX/MLX backends, and is publicly available with documented workflows. The use of standard formulas and the disclosure of the ANN's limitations are positive aspects. However, the paper's central contribution—enabling efficient yet accurate selection-effect modelling—rests on the approximate pathways (partial-scaling interpolation, ANN, hybrid recalculation) that are asserted but not validated. Without error metrics, timing benchmarks, or comparisons against exact inner products, the significance of the package as a reliable tool for scientific inference remains undemonstrated. The absence of this validation is the key gap.","major_comments":[{"comment":"This section claims that the interpolation method 'yields significant speed-ups' and that new SNRs are recovered by spline interpolation, but it reports no interpolation error, no grid resolution, no benchmark timings, and no comparison against direct inner-product calculations. This is load-bearing because partial-scaling interpolation is the primary fast pathway for non-spinning and aligned-spin systems. The reader cannot assess whether the speed-up is real or whether the accuracy is adequate for selection-effect calculations over large populations. Please add quantitative results, such as maximum relative error in rho_opt over a test population, grid spacing convergence checks, and wall-clock timing comparisons.","section":"Partial Scaling Interpolation"},{"comment":"The paper states that 'the ANN is poor at estimating rho_opt directly' but that its outputs are 'effective for P_det, since detectability depends on threshold crossing rather than precise values.' This argument is insufficient: P_det is a steep function of rho_opt near the detection threshold, so even moderate rho_opt errors can translate into large P_det errors for marginal events. No quantitative evidence is provided, such as a comparison of ANN-based P_det with direct integration over a representative population, a confusion matrix for threshold crossing, or ROC statistics. Without such metrics, the ANN pathway is unsupported and the abstract's claim of enabling 'reliable selection-effect modelling' is not demonstrated.","section":"ANN-based P_det Estimation"},{"comment":"The hybrid scheme first approximates SNR with partial scaling or the ANN, then recalculates near-threshold events with the exact noise-weighted inner product. The accuracy of this scheme depends critically on the approximate method's ability to identify which events are near threshold. If the approximate method is systematically biased, it may mis-rank events and fail to flag true marginal events, biasing the selection function. The paper does not test this failure mode. Please provide a validation showing, for a test population, the fraction of true near-threshold events that are correctly flagged by the approximate methods, and the resulting bias in P_det before and after the hybrid correction.","section":"Hybrid SNR Recalculation for P_det Estimation"},{"comment":"The P_det estimates rely on precomputed rho_obs_thr values derived from semianalytic injection catalogues following Essick (2023). The paper does not discuss how mismatches between these thresholds and the true FAR-based search selection, or variations across detector configurations and population parameters, would affect the validity of P_det. This is not a flaw in the formula, but it is a load-bearing assumption for the package's use in rate estimation. Please include a sensitivity statement and guidance on when users should compute custom thresholds from their own injection catalogues, as well as the expected impact of threshold uncertainty on P_det.","section":"Statistical Models for P_det"}],"minor_comments":[{"comment":"The abstract mentions 'validation examples' and 'reproducible workflows,' but the full text does not include any validation tables or benchmark results. Please either add the validation to the text or provide explicit pointers to the specific examples in the repository with versioned outputs.","section":"General"},{"comment":"The sentence 'Trainedonlarge ler datasets' is missing spaces and refers to an internal dataset; please clarify whether the training data are publicly available and provide details on the training population, sample size, and network architecture.","section":"ANN-based P_det Estimation"},{"comment":"The numerical method for horizon distance is described in one sentence; please specify the algorithm used for maximising SNR over sky location and solving for d_L, and note any convergence criteria.","section":"Horizon Distance Calculation"},{"comment":"The reference formatting is inconsistent (e.g., 'This' in the article header and some incomplete entries). Please standardise the bibliography.","section":"References"},{"comment":"The relationship between rho_opt_thr, rho_obs_thr, and the user-specified detection threshold is not explicitly distinguished; please define each quantity and clarify the units and parameter dependence.","section":"Notation"}],"recommendation":"major_revision","confidential_remarks":"The paper is a software description rather than a methods paper, and its central claim is about the delivered performance of a tool. The lack of quantitative validation for the approximate pathways is a standard but critical issue in such submissions; the claim that the ANN is 'poor' at rho_opt while being 'effective' for P_det is the kind of statement that needs numbers. The scope of requested revisions is feasible: adding a validation section with error metrics and benchmarks would substantially strengthen the paper. I do not see any internal inconsistency in the formulas, and the topic is within the journal's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take on the gwsnr paper. It's a solid software contribution in the making: a unified Python interface for SNR and P_det computation with multiple acceleration backends, including a partial-scaling interpolation inspired by FINDCHIRP, JAX/MLX GPU support, and an ANN-based P_det estimator with a hybrid exact-recalculation step for signals near threshold. That hybrid scheme is the one genuinely new piece as far as I know, and it's a sensible idea. The mathematics is standard: matched-filter inner products, non-central chi-square detection statistics, and Essick-style threshold estimates. The authors also cite the relevant literature, including Gerosa, Callister, Chapman-Bird, and Essick, and they are upfront about the ANN being poor at estimating rho_opt directly.\n\nThe soft spots are all evidential. There are no benchmarks anywhere: no timing comparisons against direct inner products, no speed-up factors, no parity plots. The paper claims spline interpolation gives 'significant speed-ups' and accurate recovery, but doesn't report interpolation errors. The ANN section justifies using a model that is 'poor at estimating rho_opt' by saying detectability depends on threshold crossing; that argument is insufficient on its own because P_det is steep near threshold and a biased emulator could mis-rank which events need the exact hybrid recalculation. The hybrid step helps but only if the approximate method correctly identifies marginal events. And the reader will look in vain for a repository URL or commit hash in the provided draft, which is a blocking problem for a JOSS submission. These are fixable with a few validation tables, a timing plot, and a link, but they are central to the paper's purpose, not cosmetic.\n\nI also agree with the stress-test note: the central claim that gwsnr enables reliable selection-effect modeling and rate estimation is not demonstrated as written. That said, I don't see a load-bearing flaw in the methods themselves. The formulas are right, the hybrid strategy is defensible, and the package plausibly works. The gap is evidence, not coherence.\n\nI would send this to peer review. The referee should require at least one accuracy comparison of each approximate path against exact inner products (or a convincing aggregated P_det comparison), a benchmark table, and a working repository link. If the authors provide those, this becomes a useful paper for the GW population community. I would not cite it myself until I see the benchmarks.","headline":"A useful-looking GW SNR package whose central efficiency/accuracy claims are not yet backed by benchmarks or validation; deserves a demanding referee, not a desk reject.","tokens_in":7245,"tokens_out":3004,"would_cite":false,"duration_ms":34170,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A Python package makes gravitational-wave signal-to-noise calculations fast enough for large synthetic populations.","keywords":["gravitational waves","signal-to-noise ratio","detection probability","selection effects","compact-binary population","interpolation","neural network","horizon distance"],"falsifier":"Run a large injection campaign with precessing black-hole-binary waveforms through a full matched-filter search pipeline at a fixed false-alarm rate, then compare the recovered fraction as a function of distance with gwsnr's $P_{\\rm det}$ predictions; a systematic offset would show that the semianalytic thresholds do not track true detector selection for that population.","tokens_in":6304,"feed_emoji":"🌊","tokens_out":9641,"duration_ms":88998,"temperature":0.7,"pith_summary":"gwsnr is a Python package that computes the optimal signal-to-noise ratio and detection probability for compact-binary gravitational-wave sources without repeatedly paying the cost of noise-weighted inner products. The paper claims this makes large simulated populations tractable for selection-effect modelling, rate estimation, detector-sensitivity studies, and population simulations. It offers several accelerated paths under one interface: multiprocessing inner products, partial-scaling interpolation for non-spinning and aligned-spin binaries, GPU-accelerated array backends, and an artificial-neural-network estimator for complex precessing waveforms, plus a hybrid scheme that recalculates only marginal events near the detection threshold. The package also models the observed signal-to-noise ratio statistically and derives detection thresholds from semianalytic injection-catalogue sensitivities. If the package works as described, a wide class of gravitational-wave population analyses that previously needed heavy compute can be done quickly and reproducibly.","feed_headline":"Fast SNR for million-event gravitational-wave populations","feed_subtitle":"A Python package computes detection probabilities for compact-binary catalogs without costly inner products.","key_machinery":"The load-bearing object is the partial-scaling interpolation identity $\\rho_{1/2}=D_{\\rm eff}\\,M^{-5/6}\\rho_{\\rm opt}$, where $M$ is chirp mass and $D_{\\rm eff}$ effective distance; it factorizes the costly waveform-dependent part from an easy rescaling, so precomputed spline grids can serve new sources. Around this sits a unified detection-probability pipeline that turns any signal-to-noise estimator into $P_{\\rm det}$ by thresholding an observed signal-to-noise ratio modelled as Gaussian or noncentral chi, with thresholds taken from semianalytic injection-catalogue sensitivities and a hybrid step that applies exact inner products to marginal events near threshold.","core_discovery":"The paper's central claim is that a single code package can replace direct, expensive noise-weighted inner-product signal-to-noise evaluation with a menu of faster approximations while preserving the accuracy needed for detectability estimates. For non-spinning and aligned-spin binaries, a partial-scaling identity stores a distance- and chirp-mass-rescaled signal-to-noise ratio on a parameter grid and recovers new values by interpolation and rescaling. For precessing systems with subdominant modes, a neural-network estimator is used not for precise signal-to-noise values but for reliable threshold-crossing probabilities, and a hybrid mode re-evaluates events near the detection boundary with the full inner product. Detection probability is then computed by comparing an observed signal-to-noise ratio, modelled as Gaussian or noncentral chi distributed, against thresholds estimated from semianalytic injection-catalogue sensitivities. The claimed upshot is that large-scale compact-binary population simulation and selection-effect correction become computationally practical.","pith_inferences":["If $P_{\\rm det}$ is insensitive to small signal-to-noise errors away from threshold, the same network-plus-hybrid architecture could be retrained for next-generation detector designs without reworking the interpolation grids.","The partial-scaling grid's dimensionality, two-dimensional for non-spinning and four-dimensional for aligned-spin binaries, suggests a natural extension to eccentric or higher-dimensional parameter spaces where the rescaling identity would still hold but more grid samples would be needed.","The package's threshold-based detectability inherits the assumption that stationary Gaussian noise and injection-catalogue thresholds approximate real search selection; comparing against a full false-alarm-rate pipeline on nonstationary noise would quantify how much selection-effect estimates could shift.","The pattern of approximating everywhere and computing exactly only near threshold could be reused in other detection-statistics problems, such as lensing or stochastic-background searches, wherever a cheap proxy for the detection statistic exists."],"forward_implications":["Large synthetic compact-binary catalogues can be assigned $\\rho_{\\rm opt}$ and $P_{\\rm det}$ at a fraction of the compute cost of repeated noise-weighted inner products, making population simulations of millions of systems feasible on a single workstation.","Selection-effect corrections in hierarchical Bayesian inference can be computed within the same code path as the signal-to-noise estimates, removing a bottleneck in rate and population inference.","Detector-sensitivity studies can report horizon distances through both analytic rescaling and numerical maximization, cross-checking one method against the other.","The hybrid scheme concentrates exact calculations on events near the detection boundary, so accuracy is retained exactly where $P_{\\rm det}$ changes fastest.","Neural-network-based $P_{\\rm det}$ estimation gives a route to detectability for precessing binaries with subdominant modes, where partial-scaling interpolation is unreliable."],"supporting_citations":[{"why":"supplies the noise-weighted inner-product definition and the FINDCHIRP partial-scaling decomposition that the interpolation method adapts.","marker":"[10]"},{"why":"provides the semianalytic sensitivity-estimate framework from which the package's observed-signal-to-noise detection thresholds are derived.","marker":"[17]"},{"why":"provides differentiable, hardware-accelerated waveform generation that the accelerated signal-to-noise backends rely on.","marker":"[16]"},{"why":"supplies the large event-rate datasets on which the neural-network detectability model is trained.","marker":"[30]"},{"why":"supplies the consistency condition between noise and detection used to justify the package's $P_{\\rm det}$ treatment in hierarchical inference.","marker":"[18]"},{"why":"supplies the hierarchical Bayesian framework with selection effects that motivates the need for fast $P_{\\rm det}$ evaluation.","marker":"[33]"},{"why":"supplies the Gaussian model for observed signal-to-noise ratio centered at the optimal value that one statistical $P_{\\rm det}$ path uses.","marker":"[19]"}],"fun_headline_variants":["Speedy SNR for massive GW catalogs","Fast detection odds for binary mergers via gwsnr","Compute SNR for billions of GW events in a flash","Rapid signal-to-noise for large compact-binary populations","Efficient GW SNR calculations for millions of binaries"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that detectability can be represented by comparing an observed signal-to-noise ratio to thresholds derived from semianalytic injection-catalogue sensitivities; if those thresholds do not match the true false-alarm-rate-based selection for the population or detector being simulated, $P_{\\rm det}$ estimates, and any selection-effect corrections built on them, would be biased even though the signal-to-noise formulas are exact.","fun_headline_variants_meta":{"raw":{"variants":["Speedy SNR for massive GW catalogs","Fast detection odds for binary mergers via gwsnr","Compute SNR for billions of GW events in a flash","Rapid signal-to-noise for large compact-binary populations","Efficient GW SNR calculations for millions of binaries"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000468,"raw_usage":{"total_tokens":2330,"prompt_tokens":939,"completion_tokens":1391,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":555,"completion_tokens_details":{"reasoning_tokens":1317}},"tokens_in":555,"tokens_out":1391,"duration_ms":11853,"temperature":1.0,"reasoning_tokens":1317,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T16:36:58.993116+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a large injection campaign with precessing black-hole-binary waveforms through a full matched-filter search pipeline at a fixed false-alarm rate, then compare the recovered fraction as a function of distance with gwsnr's $P_{\\rm det}$ predictions; a systematic offset would show that the semianalytic thresholds do not track true detector selection for that population.","supporting_citations":[{"cited_title":"FINDCHIRP: An Algorithm for Detection of Gravitational Waves from Inspiraling Compact Binaries","cited_arxiv_id":null,"evidence_quote":"supplies the noise-weighted inner-product definition and the FINDCHIRP partial-scaling decomposition that the interpolation method adapts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the large event-rate datasets on which the neural-network detectability model is trained."}],"review_version":1}