Unsupervised Learning Approach to Anomaly Detection in Gravitational Wave Data

Ammar Fayad

REVIEW 4 major objections 8 minor 2 cited by

Unsupervised Learning Approach to Anomaly Detection in Gravitational Wave Data

T0 review · 4 major / 8 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Trained only on detector noise, a VAE detects gravitational-wave signals as reconstruction-error spikes, reporting AUC 0.89 on ten events and ten noise samples.

desk verdict A clear proof-of-concept for VAE anomaly detection on LIGO noise, but the 0.89 AUC rests on 10 events and 10 noise samples and an unspecified window-aggregation rule, so it cannot support 'reliable detection' yet. read the letter →

arxiv 2411.19450 v2 pith:BJP3ISXG submitted 2024-11-29 gr-qc astro-ph.IMcs.LG

Ammar Fayad This is my paper · ORCID

classification gr-qcastro-ph.IMcs.LG PACS 04.30.-w07.05.Mh

keywords gravitationalwavesanomalydetectionvariationalautoencoderunsupervisedlearningLIGOreconstructionerrorLSTMtime-seriesanalysis

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes that a variational autoencoder trained only on noise-only gravitational-wave detector data can work as an anomaly detector: it reconstructs ordinary noise faithfully and fails to reconstruct gravitational-wave signals, so the pointwise squared reconstruction error spikes where a signal sits. The author applies the idea to whitened, band-passed strain time series from LIGO's H1 and L1 detectors, using an LSTM-based VAE trained on 28,800 noise-only samples. On a mixture of ten gravitational-wave events and ten noise samples, thresholding the anomaly score gives an ROC AUC of 0.89 and an F1 score of 0.857. The broader motivation is that an unsupervised detector needs no waveform templates and could therefore flag both known and unexpected transients.

What carries the argument

The load-bearing object is a variational autoencoder whose encoder and decoder are LSTM networks operating on 100-sample windows of whitened, band-passed (20-400 Hz) strain data. Training maximizes the standard VAE objective (equation 1): the sum over samples of the negative KL divergence (a distance measure between probability distributions) between the approximate posterior and a fixed Gaussian prior, plus the expected log-likelihood of reconstruction, with a latent dimension of eight. At inference the anomaly score is the quadratic reconstruction error between input and output; the probabilistic latent sampling is what distinguishes the VAE from a plain autoencoder and, the paper argues, lets it detect anomalies even when signal and noise share the same mean.

What would settle it

Apply the same noise-trained VAE to many noise-only stretches drawn from times outside the training set and compute the anomaly score: if the false-positive rate at the threshold that gave F1 = 0.857 is far above what the ROC curve implies, or a bootstrap confidence interval for the AUC includes 0.5, the claimed reliable detection is falsified.

Watch

Extended reading notes

Core claim

The central claim is that a VAE trained exclusively on detector noise learns the noise distribution closely enough that its reconstruction error becomes a reliable anomaly statistic for gravitational-wave data. The VAE reconstructs noise-like inputs to near-zero quadratic loss, while a passing gravitational wave is out-of-distribution and produces a clearly elevated loss peak. Using the reconstruction error as an anomaly score and thresholding it yields an AUC of 0.89 and an F1 of 0.857 on the paper's test set of ten GW events and ten noise-only samples. The paper further claims this is the first application of a VAE in an astrophysical context and presents it as a scalable, template-free way to detect new phenomena.

Load-bearing premise

The performance claim depends on the ten gravitational-wave events and ten noise samples in the test set fairly representing real detector conditions; if that test set is not representative, the AUC of 0.89 need not generalize.

Editorial extensions

If this is right

A noise-only-trained VAE can in principle flag any compact transient that departs from the noise distribution, not just waveforms with known templates.
The per-window reconstruction loss can be computed on streaming data, so the method could feed low-latency triggers for follow-up observations.
Retraining for a new detector or observing run requires only noise samples, avoiding the need for labeled gravitational-wave catalogs.
The reported separation (AUC 0.89, F1 0.857) supports using reconstruction error as a detection statistic for gravitational-wave time series.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural next check is to score detector glitches and other non-gravitational transients; if they also produce spikes, the method is best understood as a general transient trigger rather than a gravitational-wave-specific search.
The H1 and L1 anomaly scores could be combined by requiring a coincident spike in both detectors, which would suppress single-detector noise artifacts; the paper presents only per-detector losses.
Because training needs only noise samples, the same pipeline could be applied to other time-series instruments, although the paper does not demonstrate that transfer.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Here's my take on Fayad's arXiv:2411.19450. It's a short, honest proof-of-concept: train a LSTM-based VAE on LIGO noise-only data, then use reconstruction error to flag anomalies like GW signals. The toy example in Fig. 3 is clear and does what it claims. The paper is also frank about its limits—'Due to time constraints' and 'we will leave that to future work'—which I appreciate. The core idea is not new (the paper cites prior autoencoder anomaly detection and even self-cites the author's earlier work), but the specific application to LIGO time series with a VAE and the reported AUC is a new empirical result.

The problem is the evaluation. The AUC of 0.89 is computed on 10 GW events and 10 noise-only samples. With 10 per class, the standard error of the AUC is roughly 0.08–0.1, so the 95% confidence interval probably dips below 0.7. That is not 'reliable detection.' The paper also never states whether the ROC is built from per-window losses (each sample is split into 319 overlapping 100-point windows) or from some per-file aggregation like max or mean loss. Without that, the result isn't reproducible, and treating windows as independent would massively inflate the effective sample size. The 10 events are all confident catalog detections—not the low-SNR or morphologically novel signals an unsupervised search would aim for.

A few smaller issues: the 'first work to use VAE in astrophysics' claim is almost certainly wrong; at minimum it needs a literature check. The section heading 'V AE-GAN' is a leftover from a different idea; no GAN is used. The F1 score of 0.857 inherits the same small-sample problem.

The method itself is sound in principle, and the qualitative spike in Fig. 5 is suggestive. This is exactly the kind of paper that could become a useful contribution after a serious validation study: more events, a clearly defined evaluation protocol, error bars on the AUC, and a baseline comparison (e.g., the vanilla AE from the author's earlier work). As it stands, I'd treat it as a promising preprint, not as a reliable result.

For peer review: I'd send it to a referee rather than desk-reject, because the statistical weaknesses are precisely what a referee should catch, and the core idea is worth engaging with. But the referee should view it as a proof-of-concept in need of major revision, not as a finished detector.

Referee Report

4 major / 8 minor

Summary. The paper proposes an unsupervised anomaly-detection method for gravitational-wave (GW) time series based on a variational autoencoder (VAE) with LSTM encoder/decoder. The VAE is trained only on noise-only LIGO data from the H1 and L1 detectors, and anomaly detection is performed by thresholding the reconstruction error between input and output. The method is evaluated on a test set described as containing 10 GW events and 10 noise-only samples, reporting an AUC of 0.89 and an F1 score of 0.857. The central claim is that this demonstrates reliable, robust detection of GW signals as anomalies, and the authors suggest the method is a scalable unsupervised framework for known and potentially new phenomena.

Significance. The core idea—training a generative model on detector noise alone and using reconstruction error to flag GW signals—is a plausible and potentially useful approach, especially for searching for signals that do not match existing templates. The toy example in Section III is clear, and the qualitative spike shown in Figure 5 for GW150914 is suggestive. If the method could be validated with a rigorous statistical protocol, it would be a meaningful contribution to the growing body of machine-learning-based GW searches. However, the current evidence is not sufficient to support the strong claims in the abstract and conclusion, and the lack of a defined evaluation protocol and uncertainty quantification is a serious limitation. The authors are also transparent that they could not perform further comparisons; this is a strength in terms of honesty but leaves the method's comparative value unestablished.

major comments (4)

[IV] The reported AUC of 0.89 is computed from only 10 GW events and 10 noise samples, with no confidence intervals or statistical significance tests. With n=10 per class, the standard error of an AUC near 0.89 is approximately 0.08–0.10, so the 95% confidence interval would extend below 0.7, meaning the result is not statistically distinguishable from mediocre performance. The claim of 'robust performance' and 'reliable detection' in the abstract and conclusion is therefore not supported by the data. Please provide error bars (e.g., bootstrap or DeLong's method) and ideally evaluate on a larger and more diverse test set, including lower-SNR events.
[III.1 and IV] The manuscript does not specify how the 319 per-window reconstruction errors are aggregated into a single anomaly score for each 4-second sample. The ROC curve in Figure 6 is described as using '10 GW event data and 10 noise-only data,' which implies sample-level scoring, but the aggregation rule (e.g., mean, max, or some other function of the per-window losses) is never stated. This is essential for reproducibility and for the statistical validity of the AUC; if windows were treated as independent, the effective sample size would be inflated by roughly a factor of 319, making the reported AUC meaningless. Please state the exact aggregation procedure.
[IV] The caption of Figure 6 claims that the VAE's performance 'exceeds that of a vanilla AE,' but no results for a vanilla AE are presented—no ROC curve, no AUC, no hyperparameters, and no description of the comparison setup. Without a quantified baseline, the added value of the VAE over a simpler autoencoder is not established. Similarly, the comparison to existing machine-learning or matched-filter searches is deferred to future work; that is acceptable for a preprint, but the claim of superiority should be removed or supported with data.
[IV] The anomaly score is defined only as 'the distance between the input of the VAE and its output' without specifying the exact metric (e.g., mean squared error, mean absolute error, or negative evidence lower bound). The threshold α is also never defined in terms of how it is chosen or varied. This is a reproducibility blocker: a reader cannot implement the method as described without making unclear choices. Please define the loss function and the thresholding procedure explicitly.

minor comments (8)

[Abstract and Conclusion] The word 'reliable' in the abstract and conclusion is too strong given the small test set and lack of uncertainty quantification; consider using 'promising' or 'suggestive' until a larger evaluation is performed.
[III] The heading 'V AE-GAN' is misleading because no generative adversarial network is used or described anywhere in the paper. Please rename the section (e.g., 'Variational Autoencoder') to avoid confusion.
[IV] There is a typo in the sentence reporting the AUC: 'The AUC was found to be 0.890.890.89.' This should be corrected to a single value.
[V] The claim that 'this is the first work to use VAE in the context of astrophysics applications' is a very strong historical claim and is likely incorrect, as VAE-based methods have been used in astrophysics and related fields. Please temper this to a claim about the specific application or provide a thorough literature search to support it.
[II.1] The General Relativity review in Section II.1 is not used in the rest of the paper and could be shortened or moved to an appendix to keep the focus on the machine-learning methodology.
[III.1] The paper does not state how the noise-only training data were selected or whether the 10 noise test samples are independent of the training data (e.g., from different time periods or detector states). Please clarify this to rule out potential data leakage or overly optimistic performance.
[Figure 5] The y-axis label 'loss' is not defined; please specify that it is the reconstruction error (e.g., mean squared error) between the input and VAE output.
[III.1] Reproducibility would benefit from reporting training hyperparameters (learning rate, number of epochs, batch size, optimizer) and the software/hardware environment, as these are not currently given.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the VAE is trained only on noise and the ROC/AUC is an independent empirical evaluation; the sole self-citation is not load-bearing.

full rationale

The derivation chain is: (i) train a VAE on 28,800 noise-only 4-second LIGO samples; (ii) define the anomaly score as the reconstruction error between input and VAE output; (iii) build an ROC curve by thresholding this score on 10 GW events and 10 noise samples; (iv) report AUC = 0.89. The GW labels enter only at the evaluation stage to compute TPR/FPR; they are not used to fit the encoder/decoder or to define the anomaly score. The central hypothesis that out-of-distribution inputs produce high reconstruction error is an architectural assumption, not an input to training, and it is concretely demonstrated rather than defined into existence by Fig. 3's toy example and Fig. 5's GW150914 spike. The only self-citation is [2] (Ibrahim and Fayad), cited alongside external references [1] and [3] for the standard reconstruction-error anomaly-detection heuristic; no load-bearing step reduces to this citation, and the paper's own toy example independently motivates the heuristic. The evaluation's statistical fragility (10 positive and 10 negative samples, 319 overlapping windows per sample with no stated aggregation rule, no error bars on the AUC) is a real correctness/robustness concern, but it is not circularity. Score 2 reflects the presence of a minor, non-load-bearing self-citation; no part of the claimed derivation is equivalent to its input by construction.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim introduces no new physical entities. The free parameters are all architectural or procedural choices. The main unstated burden is the representativeness of the noise training set and the undefined window-score aggregation, both of which are assumptions rather than fitted physical constants.

free parameters (5)

Latent dimension d_z = 8
Chosen by hand; sets the VAE bottleneck and affects how accurately noise is reconstructed.
Sliding window size = 100 samples with 50% overlap
Chosen by hand; defines temporal resolution and number of windows per 4-second sample.
LSTM layer sizes = Encoder (32, 8), decoder (8, 32)
Chosen by hand; architecture capacity affects reconstruction fidelity and anomaly separation.
Anomaly threshold alpha = Not specified in the text
Used to declare anomalies; the ROC curve varies this threshold, but the reported F1 score depends on a specific choice.
Training hyperparameters (learning rate, epochs, batch size) = Not specified
Required to reproduce training; absent from the paper.

assumptions (4)

domain assumption The training set of 28,800 noise-only samples is homogeneous and representative of the detector noise encountered at test time.
If the test noise is nonstationary or contains glitches, the reconstruction-error baseline changes and the anomaly score is unreliable.
domain assumption Gravitational-wave signals are out-of-distribution with respect to the noise distribution, so they produce high reconstruction error.
This is the core detection premise, introduced in Section III; weak signals close to the noise floor may not produce detectable spikes.
ad hoc to paper The per-window reconstruction errors can be aggregated into a single sample-level anomaly score, but the aggregation rule is never stated.
Section III.1 creates 319 windows per sample; Section IV uses one score per sample. The missing rule is load-bearing for reproducing the ROC analysis.
standard math Stochastic gradient ascent on the ELBO (Eq. 1) with the reparameterization trick converges to a useful optimum.
The VAE objective is standard; the paper relies on established training behavior.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Unsupervised Learning Approach to Anomaly Detection in Gravitational Wave Data." pith.science (2026). https://pith.science/paper/BJP3ISXG

@misc{pith2026241119450,
  author       = {Pith},
  title        = {Pith review of: Unsupervised Learning Approach to Anomaly Detection in Gravitational Wave Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BJP3ISXG}},
  note         = {Machine review of arXiv:2411.19450}
}

read the original abstract

Gravitational waves (GW), predicted by Einstein's General Theory of Relativity, provide a powerful probe of astrophysical phenomena and fundamental physics. In this work, we propose an unsupervised anomaly detection method using variational autoencoders (VAEs) to analyze GW time-series data. By training on noise-only data, the VAE accurately reconstructs noise inputs while failing to reconstruct anomalies, such as GW signals, which results in measurable spikes in the reconstruction error. The method was applied to data from the LIGO H1 and L1 detectors. Evaluation on testing datasets containing both noise and GW events demonstrated reliable detection, achieving an area under the ROC curve (AUC) of 0.89. This study introduces VAEs as a robust, unsupervised approach for identifying anomalies in GW data, which offers a scalable framework for detecting known and potentially new phenomena in physics.

Figures

Figures reproduced from arXiv: 2411.19450 by the authors.

**Figure 1.** FIG. 1: Simplified Diagram of the LIGO detector [ [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

**Figure 3.** FIG. 3: Toy example on autoencoders: After training, the autoencoder reconstructs inputs effectively when the test [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: FIG. 4: Illustration of our VAE model with LSTM units [PITH_FULL_IMAGE:figures/full_fig_p003_4.png]

**Figure 5.** Figure 5: FIG. 5: Performance on data from Event GW150914: [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]

**Figure 6.** Figure 6: FIG. 6: ROC curve analysis with 10 positive (GW events) [PITH_FULL_IMAGE:figures/full_fig_p004_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Global Structure in Learned Latent Representations of Confusion-Limited LISA Data
gr-qc 2026-02 accept novelty 5.0 of 10

In a controlled synthetic LISA benchmark, likelihood-based one-class scoring on autoencoder latents (ROC-AUC 0.856) outperforms local manifold-distance scoring (0.766) for detecting resolvable sources.
Template-Free Gravitational Wave Detection with CWT-LSTM Autoencoders: A Case Study of Run-Dependent Calibration Effects in LIGO Data
astro-ph.IM 2025-09 conditional novelty 5.0 of 10

An unsupervised CWT-LSTM autoencoder trained only on LIGO O4 noise reaches 97.0% precision and 96.1% recall on 102 confirmed gravitational wave events, and the study shows cross-run calibration differences distort mul...

Reference graph

Works this paper leans on

8 extracted references · 5 canonical work pages · cited by 2 Pith papers

[1]

Zhou and R

C. Zhou and R. C. Paffenroth, in Proceedings of the 23rd ACM SIGKDD international conference on knowledge dis- covery and data mining(2017) pp. 665–674

work page 2017
[2]

Hierarchical Strategies for Cooperative Multi-Agent Reinforcement Learning

M. Ibrahim and A. Fayad, arXiv preprint arXiv:2212.07397 (2022)

work page Pith review arXiv 2022
[3]

Finke, M

T. Finke, M. Kr¨ amer, A. Morandini, A. M¨ uck, and I. Oleksiyuk, Journal of High Energy Physics (2021)

work page 2021
[4]

B. P. Abbott, R. Abbott, T. Abbott, M. Abernathy, F. Ac- ernese, K. Ackley, C. Adams, T. Adams, P. Addesso, R. X. Adhikari, et al., Physical review letters (2016)

work page 2016
[5]

D. P. Kingma, arXiv preprint arXiv:1312.6114 (2013)

arXiv 2013
[6]

Cuoco, G

E. Cuoco, G. Calamai, L. Fabbroni, G. Losurdo, M. Maz- zoni, R. Stanga, and F. Vetrano, Classical and Quantum Gravity 18, 1727 (2001)

work page 2001
[7]

Hochreiter, Neural Computation MIT-Press (1997)

S. Hochreiter, Neural Computation MIT-Press (1997)

1997
[8]

Raikman, E

R. Raikman, E. A. Moreno, E. Govorkova, E. J. Marx, A. Gunny, W. Benoit, D. Chatterjee, R. Omer, M. Saleem, D. S. Rankin, et al., arXiv preprint arXiv:2309.11537 (2023)

arXiv 2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

Zhou and R

C. Zhou and R. C. Paffenroth, in Proceedings of the 23rd ACM SIGKDD international conference on knowledge dis- covery and data mining(2017) pp. 665–674

work page 2017

[2] [2]

Hierarchical Strategies for Cooperative Multi-Agent Reinforcement Learning

M. Ibrahim and A. Fayad, arXiv preprint arXiv:2212.07397 (2022)

work page Pith review arXiv 2022

[3] [3]

Finke, M

T. Finke, M. Kr¨ amer, A. Morandini, A. M¨ uck, and I. Oleksiyuk, Journal of High Energy Physics (2021)

work page 2021

[4] [4]

B. P. Abbott, R. Abbott, T. Abbott, M. Abernathy, F. Ac- ernese, K. Ackley, C. Adams, T. Adams, P. Addesso, R. X. Adhikari, et al., Physical review letters (2016)

work page 2016

[5] [5]

D. P. Kingma, arXiv preprint arXiv:1312.6114 (2013)

arXiv 2013

[6] [6]

Cuoco, G

E. Cuoco, G. Calamai, L. Fabbroni, G. Losurdo, M. Maz- zoni, R. Stanga, and F. Vetrano, Classical and Quantum Gravity 18, 1727 (2001)

work page 2001

[7] [7]

Hochreiter, Neural Computation MIT-Press (1997)

S. Hochreiter, Neural Computation MIT-Press (1997)

1997

[8] [8]

Raikman, E

R. Raikman, E. A. Moreno, E. Govorkova, E. J. Marx, A. Gunny, W. Benoit, D. Chatterjee, R. Omer, M. Saleem, D. S. Rankin, et al., arXiv preprint arXiv:2309.11537 (2023)

arXiv 2023