REVIEW 3 major objections 6 minor 1 cited by
When Will It Fail?: Anomaly to Prompt for Forecasting Future Anomalies in Time Series
T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read A2P claims to be the first effective solution to Anomaly Prediction — naming the exact future time points of anomalies — and reports state-of-the-art F1 on four real-world datasets.
desk verdict A strong empirical paper on a useful new task; the synthetic-anomaly transfer question is real but not disqualifying, and the threshold-free metrics in the appendix cover part of the gap. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the Anomaly Prompt Pool (APP): a set of learnable key–prompt pairs in which each prompt is a token sequence that encodes an anomaly pattern. A frozen feature extractor with a [CLS] token turns an input into a query, cosine similarity selects the top-N prompts, and their tokens are concatenated onto the input embedding at the embedding level, transforming a normal reconstruction into a synthetic anomaly; the divergence loss widens the gap between synthetic-anomaly features and pseudo-normal features inside the shared transformer backbone. The Anomaly-Aware Forecasting Network plays the complementary role: cross-attention between the anomaly-injected future embedding and the prior embedding yields per-time-step anomaly probability, which reweights the forecasting loss. The shared backbone is what makes the two tasks one model — forecasting and reconstruction branches share transformer layers and are trained jointly — so that at test time, forecasting and anomaly scoring happen in a single forward path.
What would settle it
Train A2P on windows whose anomalies are injected from the five standard types, then evaluate it on held-out test windows whose anomalies are produced by a different mechanism (for example, only point spikes, or slow drifts, or a sixth shape type never synthesized during training) while keeping the baselines on the same test set; if A2P's F1 advantage over forecast-then-detect baselines shrinks or disappears under that shift, the central claim is an artifact of synthetic-to-real similarity. A stricter variant replaces the default tolerance in the F1 metric with exact time-step matching to confirm the reported ordering is not carried by the tolerance window.
Extended reading notes
Core claim
The central claim is that the failure of naive forecast-then-detect pipelines is not a defect of the component models but of training signal: forecasting models trained only on normal data smooth away the very abnormality that must survive into the forecast. A2P therefore couples two mechanisms: Anomaly-Aware Forecasting (AAF), which uses a pre-trained cross-attention network to output per-time-step anomaly probabilities and reweights the forecasting loss so errors at predicted anomaly points are penalized more; and Synthetic Anomaly Prompting (SAP), in which a learnable Anomaly Prompt Pool is queried by a frozen feature extractor, the top-N best-matching prompts are concatenated onto the input embedding, and a divergence loss pushes the resulting synthetic-anomaly features apart from pseudo-normal features. Both components are pre-trained before the main training and frozen at test time, when only the shared transformer backbone is used to forecast, reconstruct, and score anomalies by association discrepancy. The paper reports that A2P outperforms all forecasting-plus-detection baselines on MBA, Exathlon, SMD, and WADI, with its largest advantages at the longest horizon, and that ablations attribute gains to both AAF and SAP, with the divergence loss alone contributing about a 24% F1 improvement on MBA.
Load-bearing premise
The load-bearing assumption is that the synthetic anomalies injected during pre-training (five shape types with learnable magnitude chosen by reconstruction-error regions) resemble the real anomalies that appear in the test windows closely enough that training on them transfers; if the real anomalies are of a different character, A2P learns to recognize its own synthetic artifacts.
Editorial extensions
If this is right
- Anomaly-aware forecasting means future abnormal events can be given exact time points rather than just a near-term red flag, making the prediction actionable for proactive maintenance and clinical monitoring.
- Training-time synthesis of anomalies removes the need for labeled anomaly examples, so Anomaly Prediction becomes applicable to domains where anomalies are rare or unknown in advance.
- The unified shared backbone keeps test-time inference at the cost of a single model, so the prediction capability adds no extra inference overhead over a forecasting model alone.
- The improvement over baselines grows with forecast horizon, with the largest gains reported at the longest horizon, suggesting the approach is suited to long-horizon early warning.
- Both components ablate positively, and the divergence loss is the single largest contributor, indicating that diversity of synthetic anomalies is what drives detection robustness.
Reading between the lines
- Because the Anomaly Prompt Pool and the Anomaly-Aware Forecasting network are frozen after pre-training, the framework cannot re-calibrate if deployment reveals anomaly types never synthesized during training; an online variant that updates the prompt pool with confirmed real anomalies is a natural but untested extension.
- The top-N prompt retrieval doubles as an interpretation channel: the prompts selected for a given window name the anomaly shape the model is bracing for, which could yield explanations of why a specific time step is flagged without any new machinery.
- The learnable injection magnitude suggests the model self-calibrates how abnormal its synthetic anomalies are; a systematic study of how that magnitude tracks each dataset's anomaly ratio could indicate when the approach will need more prompt diversity.
- The reported gains are evaluated with a tolerance window in the F1 metric, so the cleanest head-to-head test of the central claim would be exact time-step matching, which the paper's own tolerance sweep suggests A2P would still win.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Anomaly Prediction (AP), a task in which a model must forecast future time series values and then pinpoint which time steps in the forecasted horizon will be anomalous. The authors propose Anomaly to Prompt (A2P), a unified shared-backbone framework with two main components: Anomaly-Aware Forecasting (AAF), which pre-trains a network to predict anomaly probabilities from synthetically injected anomalies and uses this probability to weight the forecasting loss, and Synthetic Anomaly Prompting (SAP), which learns an Anomaly Prompt Pool (APP) that transforms normal embeddings into synthetic anomalous embeddings during training. At test time, only the shared backbone is used: the model forecasts the future window and then reconstructs it, and the reconstruction error serves as the anomaly score. Experiments on MBA, Exathlon, SMD, and WADI compare A2P with combinations of five forecasting models and three anomaly detectors under an F1 metric with a tolerance window, and report that A2P achieves the highest average F1 at all tested prediction horizons. The paper also provides ablations, additional datasets, additional baselines, and parameter-free VUS metrics in the appendix.
Significance. If the technical claims hold, the paper proposes a genuinely new task formulation and a plausible first solution, with a clean unified architecture and public code. The strengths are the breadth of the empirical study (four main datasets, extra datasets in the appendix, multiple baselines), the inclusion of threshold-free VUS metrics in the appendix, and the ablation study that isolates the contribution of each new component. However, the central claim of 'superiority' rests on an evaluation protocol that uses ground-truth anomaly ratios from the test set to set thresholds, and on an unvalidated assumption that synthetic anomalies used in training are representative of the real anomalies that appear in test windows. These issues affect the interpretation of every reported F1 number, so the significance of the result is real but conditional on addressing them.
major comments (3)
- [Section 4.1, 'Anomaly Threshold' and 'Evaluation Metrics'] The threshold for converting anomaly scores to binary predictions is set using the ground-truth anomaly percentage in the test data, per the paragraph 'The threshold for deciding anomalies from anomaly scores is set by following the widely accepted protocol from (Shen et al., 2020a), adjusting for a percentage of anomalies in the test data.' This leaks test-label information into the evaluation and yields F1 scores that are not fully external. Because all methods receive the same threshold treatment, the relative ranking may be less affected, but the reported absolute F1 values are optimistic and the comparison is not a clean forecast evaluation. The paper should either select the threshold on a validation split, or report main-table results with a threshold-free metric such as VUS-PR/VUS-ROC, which are currently delegated to the appendix. In addition, the default tolerance t=50 is generous relative to the horizons Lout=100-400, and the claim of 'pinpointing' anomaly time points should be supported by metrics at t=0 or t=1 in the main text; the appendix figure varying t is a useful start but is not the headline result.
- [Section 3.3 and Eq. (6)] The core mechanism of the paper is the anomaly-aware weight g(Xin, Xhat_out) in Eq. (6), and the SAP/APP modules that synthesize anomalies for training. Both are trained exclusively on synthetic anomalies injected via the CARLA-style procedure described in Section 3.3. The paper never validates that these synthetic anomalies are representative of the real anomalies that occur in the test windows. This is load-bearing: if the distribution of synthetic anomalies diverges from real anomalies, the learned weight g may emphasize the wrong time steps and the prompt pool may teach the reconstruction module to erase synthetic artifacts rather than real anomalies. The ablations in Tables 2-5 show that the components improve F1 on the same test sets, but this is indirect evidence. The authors should provide direct transfer evidence, for example: (i) the anomaly-probability output of the frozen AAF on real test windows, evaluated as a ranking or AUROC; (ii) a feature-space comparison between real anomaly embeddings and synthetic prompted embeddings; or (iii) an experiment where real anomaly samples (if any) are used to validate or select the prompt pool. Without such evidence, the claim that A2P is an effective AP method remains conditional on an unstated representativeness assumption.
- [Table 1, Exathlon and SMD rows] The claim of consistent superiority over baselines is not statistically supported for some datasets. For example, at Lout=100 on Exathlon, A2P reports F1=18.64±0.16, while PatchTST+AnomalyTransformer reports 18.10±0.24 and FITS+DCdetector reports 17.38±3.26; these differences are within the reported standard deviations. Similar overlap appears in several other rows. With only three random seeds and no significance tests, the statement in Section 4.2 that 'our model consistently outperforms the baselines' is too strong. The authors should report confidence intervals or significance tests, or at least discuss which differences are reliable, before claiming universal superiority.
minor comments (6)
- [Section 4.1, 'Evaluation Metrics'] The definition of F1 with tolerance t is ambiguous: 'the real ground-truth anomaly time points from [i - t, i + t] are considered to be correctly detected before the calculation of F1-score.' It should state explicitly whether a predicted anomaly at time i is counted as a true positive if any ground-truth anomaly falls in the window, and how multiple matches are handled.
- [Section 1 and Related Work] The paper claims to be the first to 'propose a method to deal with the problems of AP,' but You et al. (2024) introduced the AP scenario and is cited as not directly tackling its challenges. To substantiate the 'first effective method' claim, the authors should either compare with a reimplementation of You et al. or explicitly state why it cannot serve as a baseline.
- [Section 4.4 and Figure 7] The statement that the additional computational cost 'is only incurred during training, with no extra overhead at inference time' is not fully precise: at test time A2P runs both the forecasting network and the reconstruction network, as shown in Figure 5, but the comparison to the PatchTST+AnomalyTransformer baseline is fair. The sentence should be rephrased to say there is no overhead relative to training a single forecasting-plus-detection pipeline.
- [Appendix A.2 and Table 7] The notation table lists symbols such as ~X_in and ~X_p_in but the main text uses eX_in and eX_p_in; please unify the notation across the paper.
- [Tables 1, 10, and 12] The column headers 'F AD' and the spaces in 'W ADI' appear to be formatting artifacts; they should be corrected to 'F1' and 'WADI' for readability.
- [Section 3.3, Eq. (4)] The divergence loss has two terms with different signs; the paper should state explicitly that minimizing LD maximizes the cosine-similarity term, thereby pulling selected keys toward normal features, since this is not immediately obvious from the equation alone.
Circularity Check
The forecasting and prompting pipeline is internally consistent; the only test-informed component is the anomaly threshold, which is calibrated on the test anomaly percentage.
-
fitted input called prediction
[Section 4.1, 'Anomaly Threshold']
"Anomaly Threshold. The threshold for deciding anomalies from anomaly scores is set by following the widely accepted protocol from (Shen et al., 2020a), adjusting for a percentage of anomalies in the test data. This approach ensures consistency with established standards for anomaly detection tasks."
The binary anomaly output used in the reported F1 is obtained by thresholding test-time anomaly scores, and the threshold is chosen using the ground-truth anomaly proportion in the test set. Thus a test-label-derived quantity participates in converting scores into predictions, so the reported F1 is not a fully external measure of forecasting ability. However, the anomaly scores themselves come from the model and the threshold does not determine which time steps receive high scores; it only sets the operating point. This is a mild evaluation-protocol leakage rather than a self-definitional derivation, and it does not make the forecast or the anomaly scoring circular.
full rationale
The core A2P pipeline is not circular. AAF is pre-trained with LAAF on synthetic anomalies generated via the CARLA injection scheme, and APP is pre-trained with LD; both are then frozen, and the main training uses LAF and LR only on the shared backbone. At test time, only the forecasting and reconstruction networks are used, with the anomaly score computed from reconstruction error as in Xu et al. (2022). Thus the forecast and anomaly scores do not reduce to the test labels or to any fitted quantity from the evaluation set. The paper's self-citations (e.g., Shin et al. 2023) appear only in related work and are not load-bearing. The single evaluation-level concern is that the anomaly threshold is set using the test data's anomaly percentage, so the headline F1 is calibrated on test labels; this is a standard protocol in anomaly detection and does not force which time steps are flagged, but it does mean the F1 is not a purely external prediction result. Overall, the central derivation is independent of the evaluation labels, so the circularity is minor.
Assumptions & free parameters
free parameters (5)
- Synthetic anomaly injection magnitude =
learnable, no fixed value
- Anomaly Prompt Pool parameters P =
M=10, Lz=5, N=3
- Feature extractor f_ftr weights =
3-layer transformer, [CLS] token
- Anomaly threshold =
percentage of anomalies in test set
- Loss coefficients lambda =
all set to 1
assumptions (3)
- domain assumption The anomaly injection procedure from CARLA (Darban et al., 2025) produces anomalies that are representative of real-world test anomalies.
- domain assumption Reconstruction-based anomaly detection, using the first attention layer's output as a feature distribution, is a valid way to measure anomaly scores.
- domain assumption The datasets used for evaluation (MBA, Exathlon, SMD, WADI) contain anomalies that are predictable from the input window of length 100.
invented entities (2)
-
Anomaly Prompt Pool (APP)
-
Anomaly-Aware Forecasting Network (AAF)
Cite this review
Pith. "Pith review of When Will It Fail?: Anomaly to Prompt for Forecasting Future Anomalies in Time Series." pith.science (2026). https://pith.science/paper/WGYE5KSE
@misc{pith2026250623596,
author = {Pith},
title = {Pith review of: When Will It Fail?: Anomaly to Prompt for Forecasting Future Anomalies in Time Series},
year = {2026},
howpublished = {\url{https://pith.science/paper/WGYE5KSE}},
note = {Machine review of arXiv:2506.23596}
}
read the original abstract
Recently, forecasting future abnormal events has emerged as an important scenario to tackle real-world necessities. However, the solution of predicting specific future time points when anomalies will occur, known as Anomaly Prediction (AP), remains under-explored. Existing methods dealing with time series data fail in AP, focusing only on immediate anomalies or failing to provide precise predictions for future anomalies. To address the AP task, we propose a novel framework called Anomaly to Prompt (A2P), comprised of Anomaly-Aware Forecasting (AAF) and Synthetic Anomaly Prompting (SAP). To enable the forecasting model to forecast abnormal time points, we adopt a strategy to learn the relationships of anomalies. For the robust detection of anomalies, our proposed SAP introduces a learnable Anomaly Prompt Pool (APP) that simulates diverse anomaly patterns using signal adaptive prompt. Comprehensive experiments on multiple real-world datasets demonstrate the superiority of A2P over state-of-the-art methods, showcasing its ability to predict future anomalies. Our implementation code is available at https://github.com/KU-VGI/AP.
Figures
Figures from the paper (7 more)
Forward citations
Cited by 1 Pith paper
-
SC-JEPA: Stabilizing Latent Predictive Learning for Time-Series Anomaly Prediction
MTS-JEPA combines a soft codebook bottleneck with a multi-resolution JEPA objective and reports state-of-the-art early-warning performance on four time-series anomaly-prediction benchmarks.
Reference graph
Works this paper leans on
-
[4]
Mathur, A. P. and Tippenhauer, N. O. Swat: A water treat- ment testbed for research and training on ics security. In 2016 international workshop on cyber-physical systems for smart water networks (CySWater), pp. 31–36. IEEE,
work page 2016
-
[6]
Tuli, S., Casale, G., and Jennings, N. R. Tranad: Deep trans- former networks for anomaly detection in multivariate time series data. arXiv preprint arXiv:2201.07284,
-
[9]
Beatgan: Anomalous rhythm detection using adversarially gener- ated time series
Zhou, B., Liu, S., Hooi, B., Cheng, X., and Ye, J. Beatgan: Anomalous rhythm detection using adversarially gener- ated time series. In IJCAI, volume 2019, pp. 4433–4439,
work page 2019
-
[11]
When Binary Cross Entropy loss was used, the performance was suboptimal. The result implies that driving anomaly probability to be continuous (MSE) rather than discrete (BCE) is better to learn Anomaly-Aware Forecasting Network effectively. For this reason, we finally adopt MSE instead of BCE to learn anomaly probability in AAF. C. Quantitative Results C....
work page 2018
-
[12]
Across these baselines, A2P consistently achieved the highest performance
We combined previously utilized time series forecasting models with time series anomaly detection models such as TranAD (Tuli et al., 2022), BeatGAN (Zhou et al., 2019), and DiffusionAD (Zhang et al., 2025). Across these baselines, A2P consistently achieved the highest performance. C.4. Results on various tolerance In the Anomaly Prediction task, it is cr...
work page 2022
-
[256]
Also, our experiments were executed on single GPU (NVIDIA RTX 3090), implementation library (PyTorch (Paszke et al., 2019)) for fair and exhaustive comparison. Regarding the anomaly detection model for Anomaly Prediction, we set the window size of 100, and used sliced predicted signals to obtain the output of anomaly detection in experiments, including al...
work page 2019
-
[2018]
Exathlon: A benchmark for explainable anomaly detection over time series
Jacob, V ., Song, F., Stiegler, A., Rad, B., Diao, Y ., and Tatbul, N. Exathlon: A benchmark for explainable anomaly detection over time series. arXiv preprint arXiv:2010.05073,
arXiv 2010
-
[2019]
Liao, H., Li, Y ., Wang, C., Guan, Y ., Tam, K., Tian, C., Li, L., Xu, C., and Li, Z. When, where, and what? a novel benchmark for accident anticipation and localization with large language models. arXiv preprint arXiv:2407.16277,
Show all 12 references
-
[2020]
Z., and Koltun, V
Bai, S., Kolter, J. Z., and Koltun, V . An empirical evalua- tion of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271,
-
[2022]
Fits: Modeling time series with 10k parameters
Xu, Z., Zeng, A., and Xu, Q. Fits: Modeling time series with 10k parameters. arXiv preprint arXiv:2307.03756,
-
[2023]
Anomaly prediction: A novel approach with explicit delay and horizon
You, J., Cela, A., Natowicz, R., Ouanounou, J., and Siarry, P. Anomaly prediction: A novel approach with explicit delay and horizon. In 2024 IEEE 20th International Conference on Intelligent Computer Communication and Processing (ICCP), pp. 1–8. IEEE,
2024
-
[2024]
and Ünal, G
Tokgöz, A. and Ünal, G. A rnn based time series approach for forecasting turkish electricity load. In 2018 26th Signal processing and communications applications con- ference (SIU), pp. 1–4. IEEE,
2018
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.