REVIEW 4 major objections 5 minor 14 references
SuperAD: A Training-free Anomaly Classification and Segmentation Method for CVPR 2025 VAND 3.0 Workshop Challenge Track 1: Adapt & Detect
T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read A training-free method claims to beat all trained anomaly detectors on MVTec AD 2.
desk verdict Training-free DINOv2 memory-bank method on MVTec AD 2 looks promising, but the huge F1 numbers are undermined by test-set threshold tuning and category-specific post-processing. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is a memory-bank nearest-neighbor anomaly score computed from multi-scale DINOv2 features. DINOv2 is a self-supervised vision transformer pre-trained on a large unlabeled image corpus; here its ViT-L-14 variant supplies frozen feature maps at four depths. Greedy coreset selection (the PatchCore procedure) picks 16 normal reference images per category so the bank spans the variety of normal appearances. For a test image, each spatial feature vector is matched to its nearest neighbor in the bank at each of the four layers; the distance maps are averaged and upsampled to a final anomaly map. Two additional hand-crafted steps support the core mechanism: a PCA-based binarization of shallow-layer features produces a foreground mask used for Vial and Wallplugs, and closed-region filling is applied for Fabric and Walnuts.
What would settle it
Run the identical pipeline on the MVTec AD 2 private protocol with the segmentation threshold fixed across all categories at a single value not derived from TESTpublic labels, and with the Fabric/Walnuts closed-region filling disabled; if the mean F1 then falls below the best trained baseline, the claimed advantage rests on public-set calibration and category-specific heuristics rather than the training-free representation.
Extended reading notes
Core claim
The central discovery claimed is that nearest-neighbor retrieval against a frozen DINOv2 feature bank is a stronger real-world anomaly segmenter than learned reconstruction, distillation, or flow-based methods when the benchmark contains transparent materials, bulk objects, and lighting variation. For each of the eight categories, 16 normal images are selected greedily to maximize diversity; features from layers 6, 12, 18, and 24 of DINOv2-large are stored as the memory bank. At test time, every patch's nearest-neighbor distance to this bank is computed at each layer, and the four distance maps are averaged and upsampled. The paper attributes the method's robustness under lighting shifts to this matching process: normal regions find similar stored features across lighting conditions, while anomalies do not. Reported per-category gains are large (Can: 17.3% vs 5.0% best prior; Rice: 60.9% vs 19.1%), and the paper additionally reports competitive ClassF1 scores of 70.2% and 74.4% for image-level classification on the two private test sets.
Load-bearing premise
The fixed configuration (16 reference images, DINOv2 layers 6/12/18/24, PCA threshold 1.0, 3x3 morphological kernel, and the category-specific foreground-mask and fill-region post-processing, along with an F1 threshold tuned on the public test set) transfers unchanged to the private test sets and to unseen categories; if any of these choices were fitted to the public set, the claimed margin will not generalize.
Editorial extensions
If this is right
- If these results hold, the primary bottleneck in industrial anomaly detection moves from training algorithms to the quality of pre-trained visual features, since a training-free retriever beats trained detectors.
- The small drop from 47.8% to 43.2% under mixed lighting, versus the large drops for trained methods like MSFlow (21.8% to 9.0%), would indicate that nearest-neighbor matching on robust features degrades gracefully under illumination shift.
- The per-category memory bank of only 16 normal images implies that few-shot deployment on a new product line may need no gradient training, only a handful of defect-free images and a forward pass.
- The near-zero baseline scores on categories such as Wallplugs (0.0%) imply that previously published methods essentially fail at those scenarios, making the reported 13.7% a substantial practical improvement even if far from perfect.
Reading between the lines
- The reported margin probably over-attributes the improvement to training-free matching: two categories (Fabric, Walnuts) receive a fill-closed-regions post-processing that assumes anomalies are enclosed by normal-looking structure, and the F1 decision threshold is tuned on the public test set; a fair transfer test would freeze both.
- A natural extension the authors do not explore is to test the same pipeline on other anomaly benchmarks (e.g., VisA, BTAD) without category-specific post-processing, which would isolate how much of the win comes from DINOv2 features versus the hand-built heuristics.
- The PCA foreground-masking component suggests a cheap way to inject object-level priors into retrieval-based detectors; one could combine it with rough object proposals from a generic segmentation model to avoid per-category manual selection.
- The paper itself documents failure cases (bubbles, specular highlights, missing-type anomalies) that bound the claim; a fair reader should view the quoted margins as category-dependent.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes SuperAD, a fully training-free anomaly classification and segmentation method for the MVTec AD 2 challenge. The method uses a frozen DINOv2-ViT-L-14 backbone to extract features from four layers (6, 12, 18, 24) of a small memory bank of 16 normal reference images selected by greedy coreset sampling, computes per-layer nearest-neighbor anomaly maps, and averages them into a final segmentation map. Category-specific preprocessing is applied to Vial and Wallplugs (PCA-based foreground masking) and to Fabric and Walnuts (filling closed regions). The authors report segmentation F1 scores on TESTpublic, TESTpriv, and TESTpriv,mix and claim that the method consistently outperforms trained baselines such as EfficientAD, PatchCore, and MSFlow.
Significance. If the reported results are verified, the significance is considerable: a training-free memory-bank method built on DINOv2 features would outperform trained anomaly detectors on a challenging real-world benchmark, with practical advantages in deployment cost and generalization. The paper is clearly written, the method is simple and largely reproducible from the text, and the authors are explicit about the threshold-optimization procedure and about typical failure cases, which is a commendable disclosure. However, the central claim is currently supported only by a single evaluation with undisclosed threshold choices, no ablations, and baseline numbers taken from an external paper; the evidence is therefore not yet sufficient to establish the claimed level of superiority.
major comments (4)
- [§2.2.2, Tables 1–2] The F1 decision threshold is explicitly optimized on TESTpublic. Since F1 is a threshold-dependent metric, the reported TESTpriv values (47.8% and 43.2%) may reflect information leakage from the public test set, and the large gap over MSFlow (21.8% and 9.0%) could be partly an artifact of threshold overfitting. Please report how the threshold was chosen (global or per-category), show the sensitivity of F1 to the threshold, or fix the threshold using a validation split from the training data; also provide error bars if any repeated runs were performed.
- [Table 2, §3.2] The claim in Section 3.2 that SuperAD 'consistently outperforms previous methods' is contradicted by the paper's own Table 2: on Fruit Jelly for TESTpriv, MSFlow achieves 47.6% while SuperAD achieves 41.3%. This is a load-bearing discrepancy in the central claim. Please revise the claim to reflect average performance or explain the per-category comparison in detail.
- [§3.1 vs §3.2] The reported numbers are internally inconsistent: Section 3.1 states that the official challenge server gives F1 scores of 47.18% and 42.51% on TESTpriv and TESTpriv,mix, while Table 2 lists 47.8% and 43.2% for the same sets. Please clarify which numbers are the official ones, which are locally recomputed, and why they differ; this ambiguity affects the validity of the headline comparison.
- [§2.1.2, §2.2.1, Table 2] No ablation study isolates the contribution of the core components: the four-layer feature fusion, the greedy coreset selection, the memory bank size of 16, the PCA foreground masking applied only to Vial and Wallplugs, and the closed-region filling applied only to Fabric and Walnuts. The category-specific post-processing appears to have been selected based on observations on the public test set, and without an ablation or a held-out category analysis the reported gains cannot be attributed to the method rather than to the hand-picked post-processing rules.
minor comments (5)
- [Title/Author block] The workshop name appears as 'V AND 3.0' with a spurious space; it should read 'VAND 3.0'.
- [§2.2.1, Eq. (3)] In Equation (3), the left-hand side and the first branch of the right-hand side both use the symbol M_init, which is confusing; please use a different symbol for the corrected mask.
- [§2.1.2] The name 'Patchcore' should be written as 'PatchCore' to match the reference list and standard usage.
- [§2.2.2, §1.2] The metric 'AU-ROC 0.05' is used without a definition, and the notation later switches to 'AucPro0.05' and 'AU-PRO 0.05'; please define these metrics once and use a single consistent notation.
- [Figure 1] The four 'NN Search' blocks in Figure 1 are not labeled with the layer indices (6, 12, 18, 24); adding these labels would make the architecture figure much easier to read.
Circularity Check
No significant circularity: SuperAD is a direct nearest-neighbor pipeline and no claimed result is defined by a fitted input.
full rationale
The paper's central claim is an empirical comparison on MVTec AD 2, not a derivation. The algorithm is fully specified: greedy coreset selection builds a 16-image memory bank from the category training set (Sec. 2.1.2), DINOv2-l/14 features from layers 6, 12, 18, 24 are matched by nearest-neighbor search, and the four similarity maps are averaged (Sec. 2.1.2-2.1.3). No equation in the paper reduces an output to an input by construction; the only fitted quantity is the F1 decision threshold, which is explicitly optimized on TESTpublic (Sec. 2.2.2). That is a threshold-selection protocol issue, not a circular derivation: the TESTpriv and TESTpriv,mix F1 numbers reported in Table 2 were obtained from the challenge server on separate held-out sets, so they are not forced by the TESTpublic threshold fit. The per-category post-processing choices (PCA foreground masking for Vial/Wallplugs, closed-region filling for Fabric/Walnuts) are heuristic and could reflect selection on observed results, but the paper does not define the evaluation result in terms of those choices; they are data-dependent preprocessing, not fitted parameters that determine the metric algebraically. The comparison baselines are external (PatchCore, RD, EfficientAD, MSFlow, SimpleNet, DSR), and the references to DINOv2, PatchCore, and DMAD are standard prior work, not a self-citation chain that supplies the load-bearing content. Therefore, despite legitimate concerns about threshold tuning and undisclosed ablations, no specific circular step can be exhibited, and the honest finding is no significant circularity with score 0.
Assumptions & free parameters
free parameters (6)
- memory_bank_size =
16
- feature_layers =
[6, 12, 18, 24]
- resize_short_side =
672 (448 for Sheet Metal)
- pca_threshold_tau =
1.0
- morphological_kernel_size =
3x3
- f1_decision_threshold =
optimized on TESTpublic
assumptions (5)
- domain assumption DINOv2 features from layers 6/12/18/24 provide sufficient discriminative power to separate normal and anomalous regions via nearest neighbor matching.
- domain assumption Normal regions in a test image find a close neighbor in the 16-image memory bank, while anomalous regions do not.
- domain assumption The first PCA component of DINOv2 features separates foreground from background, and threshold tau=1.0 works across all categories.
- domain assumption Greedy coreset selection on CLS tokens yields a representative memory bank of 16 images.
- domain assumption F1 score on binarized images is the appropriate evaluation metric for MVTec AD 2.
Cite this review
Pith. "Pith review of SuperAD: A Training-free Anomaly Classification and Segmentation Method for CVPR 2025 VAND 3.0 Workshop Challenge Track 1: Adapt & Detect." pith.science (2026). https://pith.science/paper/L5YN4HM3
@misc{pith2026250519750,
author = {Pith},
title = {Pith review of: SuperAD: A Training-free Anomaly Classification and Segmentation Method for CVPR 2025 VAND 3.0 Workshop Challenge Track 1: Adapt & Detect},
year = {2026},
howpublished = {\url{https://pith.science/paper/L5YN4HM3}},
note = {Machine review of arXiv:2505.19750}
}
read the original abstract
In this technical report, we present our solution to the CVPR 2025 Visual Anomaly and Novelty Detection (VAND) 3.0 Workshop Challenge Track 1: Adapt & Detect: Robust Anomaly Detection in Real-World Applications. In real-world industrial anomaly detection, it is crucial to accurately identify anomalies with physical complexity, such as transparent or reflective surfaces, occlusions, and low-contrast contaminations. The recently proposed MVTec AD 2 dataset significantly narrows the gap between publicly available benchmarks and anomalies found in real-world industrial environments. To address the challenges posed by this dataset--such as complex and varying lighting conditions and real anomalies with large scale differences--we propose a fully training-free anomaly detection and segmentation method based on feature extraction using the DINOv2 model named SuperAD. Our method carefully selects a small number of normal reference images and constructs a memory bank by leveraging the strong representational power of DINOv2. Anomalies are then segmented by performing nearest neighbor matching between test image features and the memory bank. Our method achieves competitive results on both test sets of the MVTec AD 2 dataset.
Figures
Reference graph
Works this paper leans on
-
[1]
Efficien- tad: Accurate visual anomaly detection at millisecond-level latencies
Kilian Batzner, Lars Heckler, and Rebecca K ¨onig. Efficien- tad: Accurate visual anomaly detection at millisecond-level latencies. In Proceedings of the IEEE/CVF Winter Confer- ence on Applications of Computer Vision , pages 128–138,
-
[2]
Xuhai Chen, Yue Han, and Jiangning Zhang. April-gan: A zero-/few-shot anomaly classification and segmentation method for cvpr 2023 vand workshop challenge tracks 1&2: 1st place on zero-shot ad and 4th place on few-shot ad.arXiv preprint arXiv:2305.17382, 2023. 2
arXiv 2023
-
[3]
Padim: a patch distribution modeling framework for anomaly detection and localization
Thomas Defard, Aleksandr Setkov, Angelique Loesch, and Romaric Audigier. Padim: a patch distribution modeling framework for anomaly detection and localization. In Inter- national conference on pattern recognition, pages 475–489. Springer, 2021. 2
work page 2021
-
[4]
Anomaly detection via reverse distillation from one-class embedding
Hanqiu Deng and Xingyu Li. Anomaly detection via reverse distillation from one-class embedding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9737–9746, 2022. 5
work page 2022
-
[5]
An image is worth 16x16 words: Trans- formers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 2
arXiv 2010
-
[6]
The mvtec ad 2 dataset: Advanced scenarios for unsupervised anomaly detection
Lars Heckler-Kram, Jan-Hendrik Neudeck, Ulla Scheler, Re- becca K ¨onig, and Carsten Steger. The mvtec ad 2 dataset: Advanced scenarios for unsupervised anomaly detection. arXiv preprint arXiv:2503.21622, 2025. 1, 2
arXiv 2025
-
[7]
Dmad: Dual memory bank for real-world anomaly detection
Jianlong Hu, Xu Chen, Zhenye Gan, Jinlong Peng, Shengchuan Zhang, Jiangning Zhang, Yabiao Wang, Chengjie Wang, Liujuan Cao, and Rongrong Ji. Dmad: Dual memory bank for real-world anomaly detection. arXiv preprint arXiv:2403.12362, 2024. 2
arXiv 2024
-
[8]
Simplenet: A simple network for image anomaly detection and localization
Zhikang Liu, Yiming Zhou, Yuansheng Xu, and Zilei Wang. Simplenet: A simple network for image anomaly detection and localization. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 20402–20411, 2023. 5
work page 2023
Show all 14 references
-
[9]
Dinov2: Learning robust visual features without supervision
Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 2
2023 arXiv
-
[10]
Towards to- tal recall in industrial anomaly detection
Karsten Roth, Latha Pemula, Joaquin Zepeda, Bernhard Sch¨olkopf, Thomas Brox, and Peter Gehler. Towards to- tal recall in industrial anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14318–14328, 2022. 2, 5
2022
-
[11]
Revisiting reverse distillation for anomaly detection
Tran Dinh Tien, Anh Tuan Nguyen, Nguyen Hoang Tran, Ta Duc Huy, Soan Duong, Chanh D Tr Nguyen, and Steven QH Truong. Revisiting reverse distillation for anomaly detection. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 24511–2452...
2023
-
[12]
Wide residual net- works
Sergey Zagoruyko and Nikos Komodakis. Wide residual net- works. arXiv preprint arXiv:1605.07146, 2016. 2
2016 arXiv
-
[13]
Dsr– a dual subspace re-projection network for surface anomaly detection
Vitjan Zavrtanik, Matej Kristan, and Danijel Sko ˇcaj. Dsr– a dual subspace re-projection network for surface anomaly detection. In European conference on computer vision, pages 539–554. Springer, 2022. 5
2022
-
[14]
Msflow: Multiscale flow-based framework for unsupervised anomaly detection
Yixuan Zhou, Xing Xu, Jingkuan Song, Fumin Shen, and Heng Tao Shen. Msflow: Multiscale flow-based framework for unsupervised anomaly detection. IEEE Transactions on Neural Networks and Learning Systems, 2024. 2, 5
2024
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.