REVIEW 2 major objections 2 minor
IMPACT: Influence Modeling for Open-Set Time Series Anomaly Detection
T0 review · 2 major / 2 minor · reviewed 2026-05-21 · grok-4.3
Pith's one-line read Influence scores from a trained model generate realistic unseen anomalies in time series data while cleaning contaminated training sets.
desk verdict IMPACT uses influence scores to generate pseudo-anomalies and clean contaminated data for open-set time series anomaly detection, but the reliability of those scores on non-convex models remains a question. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
An influence function that scores the impact of each training sample on the learned model, used both to synthesize new anomalies and to select samples for decontamination.
What would settle it
A controlled test in which the anomalies synthesized by IMPACT are fed to a detector and produce no accuracy gain over simple augmentation baselines on standard time-series benchmarks with injected unseen anomalies.
Extended reading notes
Core claim
The paper claims that learning an influence function to estimate the effect of individual training samples on the model allows two things at once: the scores can drive the synthesis of semantically divergent yet realistic unseen anomalies that respect sequential structure, and the same high-influence samples can be repurposed as supervised anomalies to decontaminate the training data, yielding higher accuracy than prior open-set methods across varying contamination rates and settings.
Load-bearing premise
That scores derived from how much each training point changes the model can be turned into new anomaly examples that are both realistic for time series and genuinely different from what was already seen.
Editorial extensions
If this is right
- Detection accuracy rises for both seen and unseen anomaly classes even when training data contains unlabeled anomalies.
- Sequential structure is preserved in the generated pseudo-anomalies, avoiding the trivial patterns produced by prior augmentation techniques.
- High-influence training samples can be directly reused as additional labeled anomalies, reducing the need for extra manual labeling.
- Performance remains stable across different levels of contamination and different open-set configurations.
Reading between the lines
- The same influence-based generation step could be tested on other ordered data such as sensor streams or financial ticks where sequential realism matters.
- If the influence function generalizes, it might reduce reliance on domain-specific augmentation rules in broader open-set learning settings.
- A follow-up experiment could measure how much of the reported gain comes from the decontamination step versus the anomaly synthesis step by ablating each component separately.
- The approach suggests a route to make anomaly detectors more robust without requiring perfectly clean training sets, a common practical constraint.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces IMPACT, a framework for open-set time series anomaly detection (OSAD). It first trains a base anomaly detector and learns an influence function to score the impact of individual training samples on the model. High-influence scores are then used in two ways: (i) to synthesize semantically divergent yet realistic unseen anomalies that preserve sequential structure, and (ii) to identify and repurpose contaminated samples as supervised anomalies for decontamination. Experiments across multiple datasets, OSAD settings, and contamination rates report that IMPACT outperforms existing state-of-the-art methods.
Significance. If the influence-based mechanisms are shown to be reliable, the work addresses a clear gap: standard augmentation techniques fail to preserve temporal dependencies in time series, and contamination handling is rarely addressed in OSAD. The explicit code release supports reproducibility and allows direct inspection of baseline implementations and metric definitions.
major comments (2)
- [§3.2] §3.2 (Influence Function Estimation): The manuscript applies the standard Hessian-vector-product approximation of influence functions to non-convex time-series models (RNNs, transformers, autoencoders). No leave-one-out retraining experiments or correlation analysis between estimated and true influence scores are reported. Because both the anomaly-synthesis step and the decontamination step rest directly on the ranking induced by these scores, this verification is load-bearing for the central performance claims.
- [§5] §5 (Experiments): The superiority claims under varying contamination rates rely on specific choices for baseline implementations, metric definitions, and contamination simulation. The manuscript does not include statistical significance tests (e.g., paired t-tests or Wilcoxon tests across runs) or ablation on the influence-function hyperparameters, leaving open the possibility that reported gains are sensitive to these implementation details.
minor comments (2)
- [§3] Notation for the influence function (Eq. 3) and the anomaly-generation procedure could be clarified with an explicit algorithmic box showing the exact steps from influence scores to synthesized sequences.
- [Figure 3] Figure 3 (qualitative anomaly examples) would benefit from side-by-side comparison with the simple augmentation baselines mentioned in the introduction to illustrate the claimed preservation of sequential structure.
Simulated Author's Rebuttal
We thank the referee for the constructive and detailed feedback. The comments highlight important aspects of influence function validation and experimental rigor that we address below. We have revised the manuscript to incorporate additional analyses where feasible while maintaining the core contributions.
read point-by-point responses
-
Referee: [§3.2] §3.2 (Influence Function Estimation): The manuscript applies the standard Hessian-vector-product approximation of influence functions to non-convex time-series models (RNNs, transformers, autoencoders). No leave-one-out retraining experiments or correlation analysis between estimated and true influence scores are reported. Because both the anomaly-synthesis step and the decontamination step rest directly on the ranking induced by these scores, this verification is load-bearing for the central performance claims.
Authors: We acknowledge that explicit verification of the influence approximations via leave-one-out retraining would strengthen confidence in the rankings. However, full leave-one-out retraining is computationally prohibitive for the RNN and transformer models used across our datasets. The Hessian-vector product approximation follows standard practice in the influence function literature for non-convex settings. To address the concern, we have added a limited correlation study on a smaller synthetic time series dataset where leave-one-out is tractable, along with a discussion of approximation limitations and their potential impact on the anomaly synthesis and decontamination steps. The consistent outperformance across diverse OSAD settings provides indirect empirical support for the utility of the estimated influences. revision: partial
-
Referee: [§5] §5 (Experiments): The superiority claims under varying contamination rates rely on specific choices for baseline implementations, metric definitions, and contamination simulation. The manuscript does not include statistical significance tests (e.g., paired t-tests or Wilcoxon tests across runs) or ablation on the influence-function hyperparameters, leaving open the possibility that reported gains are sensitive to these implementation details.
Authors: We agree that additional statistical validation and hyperparameter sensitivity analysis would improve the robustness of the experimental claims. We have revised the experiments section to include paired t-tests across multiple random seeds for the key performance comparisons under different contamination rates. We have also added an ablation study examining the influence function hyperparameters (damping term and conjugate gradient iterations). These additions confirm that the reported gains remain consistent. We have further clarified the baseline implementations and metric definitions by referencing the exact prior works and code used, which are already released in the repository. revision: yes
Circularity Check
No significant circularity: influence functions drawn from external literature and applied to new task without self-referential reduction.
full rationale
The paper's derivation begins with standard influence-function estimation (from established literature such as Koh & Liang) to score training samples, then applies those scores to generate anomalies and perform decontamination. No equation or step in the abstract or described framework defines the influence scores in terms of the generated anomalies or decontamination outcomes, nor renames a fitted quantity as a prediction. No self-citation load-bearing, uniqueness theorem imported from the same authors, or ansatz smuggled via prior work is present. The central claims remain independent of the paper's own fitted values and rest on externally verifiable influence-function approximations.
Assumptions & free parameters
Cite this review
Pith. "Pith review of IMPACT: Influence Modeling for Open-Set Time Series Anomaly Detection." pith.science (2026). https://pith.science/paper/B3VGXCTI
@misc{pith2026260329183,
author = {Pith},
title = {Pith review of: IMPACT: Influence Modeling for Open-Set Time Series Anomaly Detection},
year = {2026},
howpublished = {\url{https://pith.science/paper/B3VGXCTI}},
note = {Machine review of arXiv:2603.29183}
}
abstract
Open-set anomaly detection (OSAD) is an emerging paradigm designed to utilize limited labeled data from anomaly classes seen in training to identify both seen and unseen anomalies during testing. Current approaches rely on simple augmentation methods to generate pseudo anomalies that replicate unseen anomalies. Despite being promising in image data, these methods are found to be ineffective in time series data due to the failure to preserve its sequential nature, resulting in trivial or unrealistic anomaly patterns. They are further plagued when the training data is contaminated with unlabeled anomalies. This work introduces $\textbf{IMPACT}$, a novel framework that leverages $\underline{\textbf{i}}$nfluence $\underline{\textbf{m}}$odeling for o$\underline{\textbf{p}}$en-set time series $\underline{\textbf{a}}$nomaly dete$\underline{\textbf{ct}}$ion, to tackle these challenges. The key insight is to $\textbf{i)}$ learn an influence function that can accurately estimate the impact of individual training samples on the modeling, and then $\textbf{ii)}$ leverage these influence scores to generate semantically divergent yet realistic unseen anomalies for time series while repurposing high-influential samples as supervised anomalies for anomaly decontamination. Extensive experiments show that IMPACT significantly outperforms existing state-of-the-art methods, showing superior accuracy under varying OSAD settings and contamination rates. Code is available at https://github.com/mala-lab/IMPACT.
Lean theorems connected to this paper
-
IndisputableMonolith/Cost/FunctionalEquation.leanwashburn_uniqueness_aczel unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
learn an influence function that can accurately estimate the impact of individual training samples on the modeling... leverage these influence scores to generate semantically divergent yet realistic unseen anomalies
-
IndisputableMonolith/Foundation/RealityFromDistinction.leanreality_from_one_distinction unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
multi-channel deviation loss... entropy minimization of the latent distribution H(S) ∝ log σ²
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Reviewed May 21, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.