REVIEW 5 major objections 6 minor 1 cited by
SAFE: Self-Supervised Anomaly Detection Framework for Intrusion Detection
T0 review · 5 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read Self-supervised pipeline that turns network flows into images beats top anomaly detectors by up to 26% in F1.
desk verdict A cleanly-written pipeline paper whose headline F1 gains are plausible but not yet substantiated: no code, no error bars, and no control that isolates the t-SNE image layout, which is the one genuinely novel piece. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the vector-to-image mapping. PCA computes each feature's importance as the summed absolute loadings across principal components that explain 95% of the variance, retaining the top $k=31$ features; t-SNE places those features in the plane, and DeepInsight's convex-hull binning assigns each feature to a unique pixel in an $8 \times 8$ grayscale matrix. This construction lets a masked autoencoder treat a network flow as an image and learn spatial correlations between neighboring features. The MAE encoder's latent vector is the representation that LOF scores, and random 75% masking with mean-squared-error reconstruction supplies the label-free training signal.
What would settle it
Take SAFE and shuffle the pixel assignments in the 8-by-8 grid while keeping all other hyperparameters fixed, then retrain and measure F1 on the same four datasets; if F1 does not drop substantially, the spatial layout carries no information and the image-conversion step is not the cause of the reported gains. A complementary check is to run the same MAE+LOF pipeline on the top-31 features in their original vector form: matching performance would show the image transform is unnecessary.
Extended reading notes
Core claim
SAFE's core claim is that a masked autoencoder trained on image-like encodings of normal network flows produces a latent space in which attacks are outliers, even when those attacks were never seen during training. The paper supports this by ranking features with PCA loadings, embedding the top 31 features into an 8-by-8 grid via DeepInsight's t-SNE-based layout, and training the MAE to reconstruct 75%-masked images. The encoder's bottleneck vectors are then fed to LOF, which separates normal from anomalous flows. Across MQTTset, WUSTL-IIoT, X-IIoTID, and Edge-IIoTset, this combination outperforms nine baselines, including graph-based Anomal-E and scale-learning SLAD, and the authors attribute the gain to the layered pipeline of feature selection, spatial mapping, and representation learning.
Load-bearing premise
The load-bearing premise is that placing the selected features on an 8-by-8 grid puts related features next to each other, so the masked autoencoder learns something more useful than it would from the raw feature list; if the pixel layout is arbitrary or dataset-specific, the image conversion adds little and the reported gains would not transfer.
Editorial extensions
If this is right
- Zero-day detection becomes label-free: only normal flows are needed for training, and attack types never seen in training still separate from normal flows in the learned latent space.
- The same fixed configuration (top-31 features, 8x8 grid, 75% masking) transfers across four heterogeneous IoT/IIoT datasets with F1-scores above 91%, so the pipeline is not tied to a single protocol or attack family.
- Per-sample inference cost (0.1819 ms on X-IIoTID) is small relative to the median flow duration of 4.98 ms, supporting real-time deployment.
- The novelty detector matters but is not the whole story: swapping LOF for IF, PCA, SLAD, ICL, or RDP on MQTTset lowers F1 by 1.3 to 11.7 percentage points, so the learned representation helps multiple downstream detectors.
- Consistently top-two precision and top-three recall across datasets indicates the method balances false positives and false negatives, which matters when missing an attack is costlier than a false alarm.
Reading between the lines
- A test the paper does not run is a sensitivity analysis on the image geometry: if randomly permuting the 8x8 pixel positions preserves F1, then spatial arrangement is not the source of the gain; if it collapses, the t-SNE layout is doing real work.
- The same pipeline could be tried on other tabular anomaly domains, such as financial fraud or system logs, whenever feature correlations can be laid out spatially; the paper only demonstrates network traffic.
- The ablation shows feature selection adds only 0.13-1.49% F1, so the bulk of the improvement over baselines likely comes from the image conversion plus MAE rather than PCA ranking; a raw-feature MAE baseline would isolate each contribution.
- Because DeepInsight is fitted on normal training data, the pixel layout is dataset-specific; deployment would need a policy for updating the layout under concept drift, which the paper does not address.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes SAFE, a self-supervised anomaly detection framework for network intrusion detection. The pipeline consists of PCA-based feature ranking and selection, a DeepInsight/t-SNE vector-to-image mapping, masked autoencoder (MAE) pre-training on normal flows, and a Local Outlier Factor (LOF) novelty detector on the MAE latent representations. SAFE is evaluated on MQTTset, WUSTL-IIoT, X-IIoTID, and Edge-IIoTset against three classical novelty detectors and six deep anomaly detection baselines, reporting F1-score improvements of up to 26.2% over SLAD and 23.5% over Anomal-E. The paper also includes ablations for feature selection, novelty detector choice, and an inference-time overhead analysis.
Significance. If the reported results are reproducible, SAFE would be a practically attractive zero-day intrusion detection approach because it trains only on normal data, uses a lightweight novelty detector, and reports very low per-sample inference overhead. The paper has several strengths: it evaluates on four recent IoT/IIoT datasets, includes a meaningful set of baselines, provides ablations for feature selection and detector choice, and reports a concrete overhead comparison. However, the central empirical claim rests on single-run F1 numbers, the code is not released, the most directly related baselines are omitted, and the load-bearing assumption that the t-SNE/DeepInsight image layout provides useful spatial structure is not tested. These issues are fixable with additional experiments and artifact release, so the contribution is potentially sound but not yet established at the level claimed.
major comments (5)
- [Experiments, Table 3] Table 3 reports F1-scores for all methods as single point estimates with no error bars, no number of trials, and no significance tests. Several claimed improvements are tiny in absolute terms (e.g., Edge-IIoTset +0.01% over PCA), and without variance information or statistical testing these differences cannot be distinguished from noise. The authors should run multiple seeds/repetitions, report mean and standard deviation, and perform pairwise significance tests (or at least confidence intervals) for the headline comparisons against SLAD and Anomal-E.
- [Experimental Setup, Image Conversion] The load-bearing premise of Modules 2-3 is that the t-SNE-based DeepInsight layout places semantically related features in neighboring 8x8 cells, so the MAE's convolutional encoder can exploit spatial structure. This premise is not verified. The paper fixes k=31 and an 8x8 grid for all datasets, uses 33 zero-filled cells, and does not report t-SNE seeds or repeated conversions. A random-permutation control (randomly assigning the 31 selected features to grid cells) would directly test whether the spatial layout contributes anything beyond an arbitrary fixed arrangement of features. The authors should also vary k, grid size, mask ratio, and t-SNE initialization to show that the reported performance is not an artifact of these fixed choices.
- [Baselines, Related Works] The two most closely related methods are not included in the comparison: DeepInsight-CNN (Tran et al., 2021), which also converts tabular network data to images using DeepInsight and classifies with a CNN, and Wang et al.'s masked context reconstruction IDS (Wang et al., 2023), which also uses self-supervised masked reconstruction on network data. Without these baselines, the experiments cannot isolate whether SAFE's gains come from the image conversion, the MAE pre-training, the LOF detector, or the combination. Adding these comparisons, even with numbers taken from the original papers where possible, is necessary to support the claim that SAFE improves on the state of the art in SSL-based intrusion detection.
- [Ablation Study, Table 5] Table 5 selects LOF as the final novelty detector based on F1-score computed on MQTTset, and this same detector choice is then applied to all four datasets. If the MQTTset numbers are computed on the same test set used for the headline comparison, this constitutes test-set-based model selection and can inflate the reported gains. The detector choice should be validated on a held-out validation split, or the authors should report the detector selection procedure as part of a nested optimization and show that the ranking is stable across datasets. A related concern is that Optuna hyperparameter tuning for LOF is mentioned but no details are given about which data split was used for tuning, which data was used for the reported F1, and how many optimization trials were run.
- [Experimental Setup, Data Division] The paper states a 60/20/20 train/validation/test split but does not describe how samples are subsampled from datasets with millions of rows, whether the split is stratified by class, or whether the training set is restricted to normal samples as implied by Module 1. The reported precision, recall, and F1 values are therefore hard to interpret without knowing the class distribution in each split and whether duplicate flows were removed. The authors should specify the exact preprocessing, sampling, and split procedure for each dataset, including the number of normal and attack samples in train, validation, and test sets.
minor comments (6)
- [Table 2] The header 'Y ear' should read 'Year'.
- [Proposed Framework, Module 1] The notation R(x_i^train) is defined with an index i that overloads the per-sample index; the definition is understandable but should use a distinct feature index for clarity.
- [Results, Figure 5] Figure 5 combines four datasets and many method labels in two panels, making it difficult to map which marker corresponds to which method and dataset; the caption should explain the grouping and marker conventions, and the panels would be clearer if plotted per dataset.
- [References] References [23] and [24] in the arXiv version both point to SLAD with slightly different titles; the duplicate should be merged into a single reference.
- [Conclusion] The conclusion restates improvements of 26.15% and 23.52% over SLAD and Anomal-E, while the abstract and Table 3 are based on dataset-specific maxima and averages; the text should consistently distinguish 'up to', 'on average', and the specific dataset from which each number comes.
- [General] No code or repository is provided, and the paper does not state whether artifacts will be released; given that the main contribution is an empirically evaluated pipeline, releasing code and processed datasets would substantially increase reproducibility.
Circularity Check
Partial circularity: the MQTTset-based LOF selection is reported as SAFE's MQTTset prediction, and the abstract's 'up to 26.2%' SLAD improvement is computed from that same selected result.
-
fitted input called prediction
[Experiments, Ablation Study, 'SAFE Novelty Detector Selection' (Table 5) and Table 3]
"Table 5 presents the performance of the six top-performing novelty detectors on MQTTset, assuming they were selected in place of LOF. We specifically selected MQTTset for this study due to its status as the largest dataset in our collection. Furthermore, all other parameters in this study were held constant across Modules 1-3 to ensure a fair comparison. Based on our analysis, we found that LOF achieved the highest performance with an F1-score of 91.38%"
The detector choice is made by ranking alternatives on MQTTset test data: Table 5 reports the F1 of six detectors on MQTTset and concludes that LOF 'achieved the highest performance with an F1-score of 91.38%.' That same LOF-based pipeline is then reported in Table 3 as SAFE's MQTTset F1 of 91.38%, and the abstract's headline 'up to 26.2%' improvement over SLAD is the relative gap on this same dataset (91.38 vs 72.44). The MQTTset result is therefore a selected maximum, not an out-of-sample prediction: the choice of detector was optimized against the very benchmark used to claim superiority. If IF had been substituted, the MQTTset F1 would be 90.08%, reducing the SLAD gap to about 24.4%.
full rationale
The paper's technical pipeline is not circular in the equation-level sense: PCA ranking, DeepInsight image mapping, MAE pretraining, and LOF scoring are distinct operations, and no module defines its output in terms of the final F1. The t-SNE/DeepInsight spatial-layout premise is an unverified empirical assumption rather than a circular one; a random-permutation control would test it, and its failure would weaken the mechanism claim without making the derivation circular. The identified circular step is empirical: the novelty detector LOF was selected because it achieved the highest F1 on MQTTset, and the same MQTTset result then anchors the abstract's 'up to 26.2%' improvement over SLAD. That headline figure is therefore a selected maximum rather than an independent prediction. Other datasets' comparisons remain informative, and the self-citations (Gungor et al. 2024) are background/overhead context, not load-bearing. Overall partial circularity: 6.
Assumptions & free parameters
free parameters (6)
- k (number of retained features) =
31
- Image matrix dimension =
8x8
- MAE mask ratio =
75%
- MAE training epochs =
20
- LOF hyperparameters =
Not reported
- PCA explained variance threshold =
95%
assumptions (4)
- domain assumption t-SNE/DeepInsight placement gives spatially meaningful structure to tabular features.
- domain assumption Normal data homogeneity and attack heterogeneity allow a normal-only model to detect unseen attacks.
- domain assumption MAE latent space preserves local density structure so LOF can separate anomalies.
- ad hoc to paper Fixing k=31 and 8x8 image size is adequate across all datasets.
Cite this review
Pith. "Pith review of SAFE: Self-Supervised Anomaly Detection Framework for Intrusion Detection." pith.science (2026). https://pith.science/paper/ZWG6V4O3
@misc{pith2026250207119,
author = {Pith},
title = {Pith review of: SAFE: Self-Supervised Anomaly Detection Framework for Intrusion Detection},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZWG6V4O3}},
note = {Machine review of arXiv:2502.07119}
}
read the original abstract
The proliferation of IoT devices has significantly increased network vulnerabilities, creating an urgent need for effective Intrusion Detection Systems (IDS). Machine Learning-based IDS (ML-IDS) offer advanced detection capabilities but rely on labeled attack data, which limits their ability to identify unknown threats. Self-Supervised Learning (SSL) presents a promising solution by using only normal data to detect patterns and anomalies. This paper introduces SAFE, a novel framework that transforms tabular network intrusion data into an image-like format, enabling Masked Autoencoders (MAEs) to learn robust representations of network behavior. The features extracted by the MAEs are then incorporated into a lightweight novelty detector, enhancing the effectiveness of anomaly detection. Experimental results demonstrate that SAFE outperforms the state-of-the-art anomaly detection method, Scale Learning-based Deep Anomaly Detection method (SLAD), by up to 26.2% and surpasses the state-of-the-art SSL-based network intrusion detection approach, Anomal-E, by up to 23.5% in F1-score.
Figures
Forward citations
Cited by 1 Pith paper
-
CITADEL: Continual Anomaly Detection for Enhanced Learning in IoT Intrusion Detection
CITADEL combines self-supervised masked autoencoders with KL-divergence-based memory selection and a hierarchical buffer to detect IoT intrusions without attack labels while retaining old knowledge.
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
Akiba, T.; Sano, S.; Yanase, T.; Ohta, T.; and Koyama, M. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. arXiv:1907.10902
arXiv 2019
-
[4]
Al Farizi, W. S.; Hidayah, I.; and Rizal, M. N. 2021. Isolation forest based anomaly detection: A systematic literature review. In 2021 8th International Conference on Information Technology, Computer and Electrical Engineering (ICITACEE), 118--122. IEEE
work page 2021
-
[5]
Al-Hawawreh, M.; Sitnikova, E.; and Aboutorab, N. 2022. X-IIoTID: A Connectivity-Agnostic and Device-Agnostic Intrusion Data Set for Industrial Internet of Things. IEEE Internet of Things Journal, 9(5): 3962--3977
work page 2022
-
[6]
Alghushairy, O.; Alsini, R.; Soule, T.; and Ma, X. 2020. A review of local outlier factor algorithms for outlier detection in big data streams. Big Data and Cognitive Computing, 5(1): 1
work page 2020
-
[7]
Almaraz-Rivera, J. G.; Cantoral-Ceballos, J. A.; and Botero, J. F. 2023. Enhancing IoT Network Security: Unveiling the Power of Self-Supervised Learning against DDoS Attacks. Sensors, 23(21): 8701
work page 2023
-
[8]
W.; Layeghy, S.; and Portmann, M
Caville, E.; Lo, W. W.; Layeghy, S.; and Portmann, M. 2022. Anomal-E: A self-supervised network intrusion detection system based on graph neural networks. Knowledge-Based Systems, 258: 110030
work page 2022
Show all 37 references
-
[9]
Cheng, Z.; Zou, C.; and Dong, J. 2019. Outlier detection using isolation forest and local outlier factor. In Proceedings of the conference on research in adaptive and convergent systems, 161--168
2019
-
[10]
A.; et al
Ferrag, M. A.; et al. 2022. Edge-IIoTset: A new comprehensive realistic cyber security dataset of IoT and IIoT applications for centralized and federated learning. IEEE Access, 10: 40281--40306
2022
-
[11]
Gungor, O.; Li, E.; Shang, Z.; Guo, Y.; Chen, J.; Davis, J.; and Rosing, T. 2024. Rigorous Evaluation of Machine Learning-Based Intrusion Detection Against Adversarial Attacks. In 2024 IEEE International Conference on Cyber Security and Resilience (CSR), 152--158. IEEE
2024
-
[12]
Gungor, O.; Rosing, T.; and Aksanli, B. 2024. ROLDEF: RObust Layered DEFense for Intrusion Detection Against Adversarial Attacks. In 2024 Design, Automation & Test in Europe Conference & Exhibition (DATE), 1--6. IEEE
2024
-
[13]
Han, Q.; Zhang, G.; Huang, J.; Gao, P.; Wei, Z.; and Lu, S. 2024. Efficient MAE towards Large-Scale Vision Transformers. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 606--615
2024
-
[14]
He, K.; Chen, X.; Xie, S.; Li, Y.; Doll \'a r, P.; and Girshick, R. 2022. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 16000--16009
2022
-
[15]
R.; Kudva, P.; and Elfadel, I
Karn, R. R.; Kudva, P.; and Elfadel, I. M. 2021. Learning without forgetting: A new framework for network cyber security threat detection. IEEE Access, 9: 137042--137062
2021
-
[16]
R.; Kashif, M.; Jhaveri, R
Khan, A. R.; Kashif, M.; Jhaveri, R. H.; Raut, R.; Saba, T.; and Bahaj, S. A. 2022. Deep learning for intrusion detection and security of Internet of things (IoT): current analysis, challenges, and possible solutions. Security and Communication Networks, 2022(1): 4016073
2022
-
[17]
Liu, B.; Wang, D.; Lin, K.; Tan, P.-N.; and Zhou, J. 2021. RCA: A Deep Collaborative Autoencoder Approach for Anomaly Detection. In Zhou, Z.-H., ed., Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21 , 1505--1511. International Jo...
2021
-
[18]
T.; Ting, K
Liu, F. T.; Ting, K. M.; and Zhou, Z.-H. 2008. Isolation forest. In 2008 eighth ieee international conference on data mining, 413--422. IEEE
2008
-
[19]
Liu, H.; and Lang, B. 2019. Machine learning and deep learning methods for intrusion detection systems: A survey. applied sciences, 9(20): 4396
2019
-
[20]
Ndiour, I.; Ahuja, N.; and Tickoo, O. 2020. Out-of-distribution detection with subspace techniques and probabilistic modeling of features. arXiv preprint arXiv:2012.04250
2020 arXiv
-
[21]
Nguyen, H.; and Kashef, R. 2023. TS-IDS: Traffic-aware self-supervised learning for IoT Network Intrusion Detection. Knowledge-Based Systems, 279: 110966
2023
-
[22]
Shenkar, T.; and Wolf, L. 2022. Anomaly Detection for Tabular Data with Internal Contrastive Learning. In International Conference on Learning Representations
2022
-
[23]
Song, F.; Guo, Z.; and Mei, D. 2010. Feature selection using principal component analysis. In 2010 international conference on system science, engineering design and manufacturing informatization, volume 1, 27--30. IEEE
2010
-
[24]
P.; Nguyen, V
Tran, T. P.; Nguyen, V. C.; Vu, L.; and Nguyen, Q. U. 2021. DeepInsight-Convolutional Neural Network for Intrusion Detection Systems. In 2021 8th NAFOSTED Conference on Information and Computer Science (NICS), 120--125
2021
-
[25]
Vaccari, I.; Chiola, G.; Aiello, M.; Mongelli, M.; and Cambiaso, E. 2020. MQTTset, a new dataset for machine learning techniques on MQTT. Sensors, 20(22): 6578
2020
-
[26]
Van der Maaten, L.; and Hinton, G. 2008. Visualizing data using t-SNE. Journal of machine learning research, 9(11)
2008
-
[27]
Verkerken, M.; D’hooge, L.; Wauters, T.; Volckaert, B.; and De Turck, F. 2022. Towards model generalization for intrusion detection: Unsupervised machine learning techniques. Journal of Network and Systems Management, 30: 1--25
2022
-
[28]
Wang, H.; Pang, G.; Shen, C.; and Ma, C. 2020. Unsupervised Representation Learning by Predicting Random Distances. arXiv:1912.12186
2020 arXiv
-
[29]
Wang, W.; Jian, S.; Tan, Y.; Wu, Q.; and Huang, C. 2023. Robust unsupervised network intrusion detection with self-supervised masked context reconstruction. Computers & Security, 128: 103131
2023
-
[30]
Wu, J.; Chen, X.-Y.; Zhang, H.; Xiong, L.-D.; Lei, H.; and Deng, S.-H. 2019. Hyperparameter optimization for machine learning models based on Bayesian optimization. Journal of Electronic Science and Technology, 17(1): 26--40
2019
-
[31]
Xu, H.; Pang, G.; Wang, Y.; and Wang, Y. 2023 a . Deep Isolation Forest for Anomaly Detection. IEEE Transactions on Knowledge and Data Engineering, 1--14
2023
-
[32]
Xu, H.; Wang, Y.; Wei, J.; Jian, S.; Li, Y.; and Liu, N. 2023 b . Fascinating supervisory signals and where to find them: Deep anomaly detection with scale learning. In International Conference on Machine Learning, 38655--38673. PMLR
2023
-
[33]
Xu, H.; Wang, Y.; Wei, J.; Jian, S.; Li, Y.; and Liu, N. 2023 c . Fascinating supervisory signals and where to find them: deep anomaly detection with scale learning. In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org
2023
-
[34]
Yang, J.; Jiang, X.; Liang, G.; Li, S.; and Ma, Z. 2023. Malicious traffic identification with self-supervised contrastive learning. Sensors, 23(16): 7215
2023
-
[35]
Yue, Y.; Chen, X.; Han, Z.; Zeng, X.; and Zhu, Y. 2022. Contrastive learning enhanced intrusion detection. IEEE Transactions on Network and Service Management, 19(4): 4232--4247
2022
-
[36]
B.; Miani, R
Zarpel \ a o, B. B.; Miani, R. S.; Kawakani, C. T.; and de Alvarenga, S. C. 2017. A survey of intrusion detection in Internet of Things. Journal of Network and Computer Applications, 84: 25--37
2017
-
[37]
Zolanvari, M.; et al. 2021. Wustl-iiot-2021 dataset for iiot cybersecurity research. Washington University in St. Louis, USA
2021
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.