REVIEW 3 major objections 5 minor 1 cited by
KKA: Improving Vision Anomaly Detection through Anomaly-related Knowledge from Large Language Models
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read KKA claims that hard anomalies generated from LLM knowledge, rather than random noise, are the key to improving unsupervised vision anomaly detection.
desk verdict KKA is a real empirical pipeline that moves AUC numbers, but the paper doesn't isolate hard-anomaly selection from DPO fine-tuning and regeneration, so the 'key knowledge' attribution is under-supported. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is a three-stage loop. First, Eq. 3 prompts an LLM to write anomaly descriptions in the stylistic mold of sampled normal descriptions, and Eq. 4 renders them into images with a text-to-image model. Second, a confusion evaluator trained by Eq. 5, a Deep-SAD-style hypersphere objective, scores each generated image by distance to the normal cluster, and Eq. 6 keeps only those images whose distance is below the maximum normal distance—these are the hard anomalies. Third, Eq. 7 uses the DPO preference-optimization loss to fine-tune the LLM so its future descriptions favor hard-anomaly text over easy-anomaly text. The cycle repeats, and can run in an add mode that grows the anomaly set or a replacement mode that swaps easy anomalies for hard ones at fixed count.
What would settle it
A control that removes the hard-anomaly selection would settle it: at each iteration, replace the hard-selected anomalies with an equal number of freshly generated anomalies chosen at random, keeping all other KKA steps. If the AUC gains persist, the hard-anomaly filter is not the cause; if they disappear, it is.
Extended reading notes
Core claim
The discovery the paper claims is that 'key knowledge'—LLM knowledge about anomalies that lie close to normal samples—is the lever that moves unsupervised anomaly detection performance. KKA operationalizes this by prompting an LLM to describe anomalies in the textual style of normal samples, rendering those descriptions into images with a text-to-image model, training a confusion evaluator to rank generated images by their distance to the normal distribution, and keeping only the closest-to-normal ones as hard anomalies. It then fine-tunes the LLM with preference optimization on the text of the selected hard anomalies, so each iteration yields harder anomalies. On CIFAR-100 this raises SimpleNet's AUC from 74.62% to 84.04% while generating roughly 5% of the samples SimpleNet uses.
Load-bearing premise
The load-bearing premise is that the anomalies the confusion evaluator labels as 'hard'—those whose distance to the normal cluster is smallest—are exactly the ones that most improve the final detector's boundary, even though the evaluator and the detector are different models.
Editorial extensions
If this is right
- Plugging KKA's generated anomaly set into existing detectors improves AUC across datasets: SimpleNet on UCM-Caption rises from 83.80% to 94.01% in add mode and 94.77% in replacement mode, and ReContrast on CIFAR-100 rises to 87.99%.
- Iterations matter: each round of hard-anomaly selection and LLM fine-tuning improves SAD+KKA (rep) on CIFAR-100 from 70.18% at iteration 0 to 72.26% at iteration 3, showing that updating the anomaly set rather than merely growing it is what helps.
- Hardness, not volume, drives gains: KKA (rep), which keeps the anomaly count fixed, still improves across datasets, and increasing the number of initial easy anomalies has little effect once a sufficient number is present.
- The generated anomalies make training converge faster: ReContrast with KKA reaches about 0.83 accuracy in the first epoch, versus about 0.25 without KKA.
- The benefit carries across different unsupervised detectors: SAD, SimpleNet, and ReContrast all improve, suggesting KKA is a general complement to a detector's own design rather than a fix for one architecture.
Reading between the lines
- A direct control the paper does not report would separate selection from sheer new-data diversity: compare KKA's hard-selected set with a same-size random draw of fresh generations at each iteration, keeping all other steps identical. If the gains persist under random draws, Eq. 6's hard-anomaly filter is not the cause.
- The method currently depends on class labels to define the normal class; a natural extension is to apply the same generation-and-filter loop when the normal class is defined by a reference image set rather than a label, as in industrial inspection.
- Because the LLM provides textual anomaly descriptions, KKA's fine-tuned knowledge may transfer across visual domains: prompting styles adapted to a new normal class could inject hard-anomaly knowledge learned on one category into another.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Key Knowledge Augmentation (KKA), an iterative data-augmentation method for unsupervised visual anomaly detection. KKA uses an LLM to generate anomaly text conditioned on representative normal samples, renders images with Stable Diffusion, trains a Deep SAD-style confusion evaluator to classify generated anomalies as easy or hard, and then iteratively fine-tunes the LLM with DPO while increasing the proportion of hard anomalies in the training set (either by adding samples or by replacing easy ones). Experiments on CIFAR-100, Oxford-102, and UCM-Caption report AUC gains over SAD, SimpleNet, ReContrast, and CMDA, with confidence intervals and a public code release.
Significance. If the reported gains are robust, the paper offers a practical way to inject LLM-derived anomaly knowledge into small detectors while keeping generation cost modest, and it frames an interesting distinction between easy and hard generated anomalies. Strengths include evaluation across three datasets with 95% confidence intervals, integration with multiple detector families, and a released implementation. The central claim, however, depends on an ablation that the paper does not provide, so the significance is currently conditional rather than established.
major comments (3)
- [Section IV.C, Table III] The ablation labeled 'Importance of Key Knowledge' does not isolate the Eq. (6) hard-anomaly selection mechanism. Each iteration simultaneously (a) fine-tunes the LLM by DPO, (b) regenerates images with Stable Diffusion under the updated prompt distribution, and (c) selects hard anomalies via Eq. (6). The reported comparison between KKA(0) and KKA(1/2/3) therefore measures a joint effect, not the effect of key-knowledge selection. The authors should add a control that keeps the LLM and image-generation steps fixed and selects the same number of anomalies uniformly at random from the same generated pool; without this control, the gains could come from improved prompt/image quality rather than from the hardness criterion.
- [Abstract and Table II] The claim that KKA improves SimpleNet from 74.62% to 84.04% 'while generating only about 5% of the samples produced by SimpleNet' is misleading as written. In Table II, the SimpleNet+KKA rows use the original 20,000 SimpleNet-generated anomalies plus 1,000 or 1,200 KKA-generated anomalies, so the total number of generated anomalies is larger than SimpleNet's, not 5% of it. The paper should clarify that KKA adds roughly 5% of SimpleNet's count as additional samples, or provide an experimental setting in which KKA-generated anomalies replace SimpleNet's own anomalies so that the cost comparison is apples-to-apples.
- [Section III.B, Eq. (6)] The hardness criterion is computed with a Deep SAD confusion evaluator θ_C, but the target detectors in the main results are SimpleNet and ReContrast, which use different architectures, objectives, and feature spaces. The paper does not validate that anomalies judged hard by θ_C are also hard for the final detector. At minimum, the authors should compare Eq. (6) selection against uniform-random selection from the same generated anomaly pool while training the target detector; this would also address the cross-architecture transfer concern and make the 'key knowledge' attribution credible.
minor comments (5)
- [Algorithm 1] The loop header reads 'for do' without a termination condition or iteration count; it should specify the number of update iterations or a convergence criterion.
- [Section III.B, Eq. (7)] The DPO loss expression is written as L(θ'_L, θ_L) but the expectation and the paired samples (x^t_n, x^t_ea, x^t_ha) are not fully defined; please specify the data distribution over normal, easy-anomaly, and hard-anomaly text used in the preference pairs.
- [Section III.B, Eq. (5)] The notation in Eq. (5) introduces E(·) and ỹ_j without definitions, and the summation index M is described as the number of anomalies while the loss is applied to a mixture of normal and anomaly samples; please clarify the roles of these symbols.
- [Abstract and Section I] The abstract states that code for 'CMG' is available, but the method and repository name are KKA; this appears to be a typo.
- [Section IV.C, Table III] The conclusion that increasing the proportion of hard anomalies 'significantly enhances' performance is weakened by non-monotonic results on two datasets: for Oxford-102, KKA(3) drops to 92.97% from KKA(2)'s 94.31%, and for UCM-Caption, KKA(3) drops to 73.17% from KKA(2)'s 74.60%. The text acknowledges instability but should temper the claim accordingly.
Circularity Check
No significant circularity: KKA is an empirical hard-anomaly augmentation pipeline evaluated on held-out benchmarks; no prediction reduces to a fitted input or self-citation.
full rationale
The paper's derivation chain is an empirical augmentation pipeline rather than a closed-form derivation. Anomaly candidates are generated by an LLM and Stable Diffusion, a confusion evaluator is trained on normal samples plus those candidates, and Eq. 6 selects candidates close to the evaluator's hypersphere center. The selected anomalies are then used to train a separate detector that is evaluated on held-out test anomalies. No parameter is fitted to the evaluation set, and no reported quantity is a prediction obtained from its own fitted value. The term 'key knowledge' is defined as knowledge about hard anomalies, but that is a terminological label, not a circular derivation: the paper separately ablates the effect of increasing the proportion of hard anomalies at constant dataset size (Table III, Figure 4), which gives the claim independent empirical content. The self-citations ([1], [5], [15], [29]) are used for motivation, baseline context, or general collaboration claims and are not load-bearing for the main AUC improvements. The absence of an ablation that isolates Eq. 6's hardness selection from the DPO and Stable Diffusion updates is a completeness or attribution concern, but it does not make any stated result equivalent to its input by construction. Therefore no specific circular step can be exhibited, and the correct finding is no significant circularity.
Assumptions & free parameters
free parameters (3)
- Number of representative normal samples K
- Update iteration count =
3 (CIFAR-100), 2 (Oxford-102), 2 (UCM-Caption)
- DPO beta and preference pair thresholds
assumptions (5)
- domain assumption LLM prior knowledge can generate anomaly descriptions in the style of the normal text descriptions.
- domain assumption Stable Diffusion renders textual anomalies into images that lie in the same visual manifold as the normal class.
- domain assumption The confusion evaluator's distance criterion identifies hard anomalies that transfer to the downstream detector.
- domain assumption DPO on easy/hard text pairs makes the LLM generate more hard anomalies.
- standard math Mahalanobis distance with cluster-wise means and covariances is a valid measure of normal-representativeness for selecting text descriptions.
Cite this review
Pith. "Pith review of KKA: Improving Vision Anomaly Detection through Anomaly-related Knowledge from Large Language Models." pith.science (2026). https://pith.science/paper/TICGH6UY
@misc{pith2026250214880,
author = {Pith},
title = {Pith review of: KKA: Improving Vision Anomaly Detection through Anomaly-related Knowledge from Large Language Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/TICGH6UY}},
note = {Machine review of arXiv:2502.14880}
}
read the original abstract
Vision anomaly detection, particularly in unsupervised settings, often struggles to distinguish between normal samples and anomalies due to the wide variability in anomalies. Recently, an increasing number of studies have focused on generating anomalies to help detectors learn more effective boundaries between normal samples and anomalies. However, as the generated anomalies are often derived from random factors, they frequently lack realism. Additionally, randomly generated anomalies typically offer limited support in constructing effective boundaries, as most differ substantially from normal samples and lie far from the boundary. To address these challenges, we propose Key Knowledge Augmentation (KKA), a method that extracts anomaly-related knowledge from large language models (LLMs). More specifically, KKA leverages the extensive prior knowledge of LLMs to generate meaningful anomalies based on normal samples. Then, KKA classifies the generated anomalies as easy anomalies and hard anomalies according to their similarity to normal samples. Easy anomalies exhibit significant differences from normal samples, whereas hard anomalies closely resemble normal samples. KKA iteratively updates the generated anomalies, and gradually increasing the proportion of hard anomalies to enable the detector to learn a more effective boundary. Experimental results show that the proposed method significantly improves the performance of various vision anomaly detectors while maintaining low generation costs. The code for CMG can be found at https://github.com/Anfeather/KKA.
Figures
Figures from the paper (3 more)
Forward citations
Cited by 1 Pith paper
-
FAF: A Feature-Adaptive Framework for Few-Shot Time Series Forecasting
A feature-adaptive meta-learning framework for few-shot time series forecasting reports large gains, but its evaluation uses one to nine test tasks per dataset, lacks error bars, and contains numerical and preprocessi...
Reference graph
Works this paper leans on
-
[1]
Improving vision anomaly detection with the guidance of language modality,
D. Chen, K. Pan, G. Dai, G. Wang, Y . Zhuang, S. Tang, and M. Xu, “Improving vision anomaly detection with the guidance of language modality,” IEEE Transactions on Multimedia , 2024
work page 2024
-
[2]
Q. Zhou, S. He, H. Liu, T. Chen, and J. Chen, “Pull & push: Leveraging differential knowledge distillation for efficient unsupervised anomaly detection and localization,” IEEE Transactions on Circuits and Systems for Video Technology, 2022
work page 2022
-
[3]
A simple framework for contrastive learning of visual representations,
T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in International conference on machine learning . PMLR, 2020, pp. 1597–1607
2020
-
[4]
Csi: Novelty detection via contrastive learning on distributionally shifted instances,
J. Tack, S. Mo, J. Jeong, and J. Shin, “Csi: Novelty detection via contrastive learning on distributionally shifted instances,” in 34th Con- ference on Neural Information Processing Systems (NeurIPS) 2020 . Neural Information Processing Systems, 2020
work page 2020
-
[5]
Cross-modal data augmentation for tasks of different modalities,
D. Chen, Y . Zhuang, Z. Shen, C. Yang, G. Wang, S. Tang, and Y . Yang, “Cross-modal data augmentation for tasks of different modalities,” IEEE Transactions on Multimedia , 2023
work page 2023
-
[6]
Simplenet: A simple network for image anomaly detection and localization,
Z. Liu, Y . Zhou, Y . Xu, and R. Wang, “Simplenet: A simple network for image anomaly detection and localization,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 20 402–20 411
work page 2023
-
[7]
Training language models to follow instructions with human feedback,
L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al. , “Training language models to follow instructions with human feedback,” Advances in neural information processing systems , vol. 35, pp. 27 730–27 744, 2022
2022
-
[8]
Chatglm: A family of large language models from glm-130b to glm-4 all tools,
T. GLM, A. Zeng, B. Xu, B. Wang, C. Zhang, D. Yin, D. Rojas, G. Feng, H. Zhao, H. Lai, H. Yu, H. Wang, J. Sun, J. Zhang, J. Cheng, J. Gui, J. Tang, J. Zhang, J. Li, L. Zhao, L. Wu, L. Zhong, M. Liu, M. Huang, P. Zhang, Q. Zheng, R. Lu, S. Duan, S. Zhang, S. Cao, S. Yang, W. L. Tam, W. Zhao, X. Liu, X. Xia, X. Zhang, X. Gu, X. Lv, X. Liu, X. Liu, X. Yang, ...
2024
Show all 37 references
-
[9]
A survey of large language models,
W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y . Hou, Y . Min, B. Zhang, J. Zhang, Z. Dong et al. , “A survey of large language models,” arXiv preprint arXiv:2303.18223, 2023
2023 arXiv
-
[10]
Semantic anomaly detection with large language models,
A. Elhafsi, R. Sinha, C. Agia, E. Schmerling, I. A. Nesnas, and M. Pavone, “Semantic anomaly detection with large language models,” Autonomous Robots, vol. 47, no. 8, pp. 1035–1055, 2023
2023
-
[11]
Harness- ing large language models for training-free video anomaly detection,
L. Zanella, W. Menapace, M. Mancini, Y . Wang, and E. Ricci, “Harness- ing large language models for training-free video anomaly detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 18 527–18 536
2024
-
[12]
Anomaly detection using edge comput- ing in video surveillance system,
D. R. Patrikar and M. R. Parate, “Anomaly detection using edge comput- ing in video surveillance system,” International Journal of Multimedia Information Retrieval, vol. 11, no. 2, pp. 85–110, 2022
2022
-
[13]
An edge computing based anomaly detection method in iot industrial sustainability,
X. Yu, X. Yang, Q. Tan, C. Shan, and Z. Lv, “An edge computing based anomaly detection method in iot industrial sustainability,” Applied Soft Computing, vol. 128, p. 109486, 2022
2022
-
[14]
Adaptive anomaly detection for iot data in hierarchical edge computing,
M. V . Ngo, H. Chaouchi, T. Luo, and T. Q. Quek, “Adaptive anomaly detection for iot data in hierarchical edge computing,” arXiv preprint arXiv:2001.03314, 2020
2001 arXiv
-
[15]
Data shunt: Collaboration of small and large models for lower costs and better performance,
D. Chen, Y . Zhuang, S. Zhang, J. Liu, S. Dong, and S. Tang, “Data shunt: Collaboration of small and large models for lower costs and better performance,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 10, 2024, pp. 11 249–11 257
2024
-
[16]
Deep industrial image anomaly detection: A survey,
J. Liu, G. Xie, J. Wang, S. Li, C. Wang, F. Zheng, and Y . Jin, “Deep industrial image anomaly detection: A survey,” Machine Intelligence Research, vol. 21, no. 1, pp. 104–135, 2024
2024
-
[17]
Towards total recall in industrial anomaly detection,
K. Roth, L. Pemula, J. Zepeda, B. Sch ¨olkopf, T. Brox, and P. Gehler, “Towards total recall in industrial anomaly detection,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 14 318–14 328
2022
-
[18]
Industrial anomaly detection: A comparison of unsupervised neural network architectures,
B. Siegel, “Industrial anomaly detection: A comparison of unsupervised neural network architectures,” IEEE Sensors Letters , vol. 4, no. 8, pp. 1–4, 2020
2020
-
[19]
Financial fraud: a review of anomaly detection techniques and recent advances,
W. Hilal, S. A. Gadsden, and J. Yawney, “Financial fraud: a review of anomaly detection techniques and recent advances,” Expert systems With applications, vol. 193, p. 116429, 2022
2022
-
[20]
Codetect: Financial fraud detection with anomaly feature detection,
D. Huang, D. Mu, L. Yang, and X. Cai, “Codetect: Financial fraud detection with anomaly feature detection,” IEEE Access , vol. 6, pp. 19 161–19 174, 2018
2018
-
[21]
Normality learning in multispace for video anomaly detection,
Y . Zhang, X. Nie, R. He, M. Chen, and Y . Yin, “Normality learning in multispace for video anomaly detection,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 9, pp. 3694–3706, 2020
2020
-
[22]
Spatial-temporal cascade autoencoder for video anomaly detection in crowded scenes,
N. Li, F. Chang, and C. Liu, “Spatial-temporal cascade autoencoder for video anomaly detection in crowded scenes,” IEEE Transactions on Multimedia, vol. 23, pp. 203–215, 2020
2020
-
[23]
Contrastive attention for video anomaly detection,
S. Chang, Y . Li, S. Shen, J. Feng, and Z. Zhou, “Contrastive attention for video anomaly detection,” IEEE Transactions on Multimedia , vol. 24, pp. 4067–4076, 2021
2021
-
[24]
Deep learning for anomaly detection: A review,
G. Pang, C. Shen, L. Cao, and A. V . D. Hengel, “Deep learning for anomaly detection: A review,” ACM computing surveys (CSUR), vol. 54, no. 2, pp. 1–38, 2021
2021
-
[25]
Deep autoencoding gaussian mixture model for unsupervised anomaly detection,
B. Zong, Q. Song, M. R. Min, W. Cheng, C. Lumezanu, D. Cho, and H. Chen, “Deep autoencoding gaussian mixture model for unsupervised anomaly detection,” in International conference on learning representa- tions, 2018
2018
-
[26]
Ssd: A unified framework for self-supervised outlier detection,
V . Sehwag, M. Chiang, and P. Mittal, “Ssd: A unified framework for self-supervised outlier detection,” in International Conference on Learning Representations , 2021. [Online]. Available: https: //openreview.net/forum?id=v5gjXpmR8J
2021
-
[27]
Recontrast: Domain-specific anomaly detection via contrastive reconstruction,
J. Guo, L. Jia, W. Zhang, H. Li et al. , “Recontrast: Domain-specific anomaly detection via contrastive reconstruction,” Advances in Neural Information Processing Systems , vol. 36, 2024
2024
-
[28]
Large language model guided knowledge distillation for time series anomaly detection,
C. Liu, S. He, Q. Zhou, S. Li, and W. Meng, “Large language model guided knowledge distillation for time series anomaly detection,” arXiv preprint arXiv:2401.15123, 2024
2024 arXiv
-
[29]
Improving large models with small models: Lower costs and better performance,
D. Chen, S. Zhang, Y . Zhuang, S. Tang, Q. Liu, H. Wang, and M. Xu, “Improving large models with small models: Lower costs and better performance,” arXiv preprint arXiv:2406.15471 , 2024
2024 arXiv
-
[30]
High- resolution image synthesis with latent diffusion models,
R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High- resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10 684–10 695
2022
-
[31]
Deep semi-supervised anomaly detection,
L. Ruff, R. A. Vandermeulen, N. G ¨ornitz, A. Binder, E. M ¨uller, K.-R. M¨uller, and M. Kloft, “Deep semi-supervised anomaly detection,” in International Conference on Learning Representations . 9
-
[32]
Direct preference optimization: Your language model is secretly a reward model,
R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn, “Direct preference optimization: Your language model is secretly a reward model,” Advances in Neural Information Processing Systems, vol. 36, 2024
2024
-
[33]
Learning multiple layers of features from tiny images,
A. Krizhevsky, G. Hinton et al. , “Learning multiple layers of features from tiny images,” 2009
2009
-
[34]
Generative adversarial text to image synthesis,
S. Reed, Z. Akata, X. Yan, L. Logeswaran, B. Schiele, and H. Lee, “Generative adversarial text to image synthesis,” in International con- ference on machine learning . PMLR, 2016, pp. 1060–1069
2016
-
[35]
Deep semantic understanding of high resolution remote sensing image,
B. Qu, X. Li, D. Tao, and X. Lu, “Deep semantic understanding of high resolution remote sensing image,” in 2016 International conference on computer, information and telecommunication systems (Cits) . IEEE, 2016, pp. 1–5
2016
-
[36]
Ssd: A unified framework for self- supervised outlier detection,
V . Sehwag, M. Chiang, and P. Mittal, “Ssd: A unified framework for self- supervised outlier detection,” in International Conference on Learning Representations
-
[37]
Visualizing data using t-sne
L. Van der Maaten and G. Hinton, “Visualizing data using t-sne.” Journal of machine learning research , vol. 9, no. 11, 2008
2008
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.