REVIEW 3 major objections 6 minor 37 references
CLIP-Flow: A Universal Discriminator for AI-Generated Images Inspired by Anomaly Detection
T0 review · 3 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read A universal fake-image detector can be trained on real images only, using frequency-masked proxies as stand-ins for fakes, and still catch unseen generators—reporting 95.56% mAP on the 19-dataset benchmark.
desk verdict Proxy-image training on CLIP features is a genuinely interesting idea, but the headline benchmark numbers are tainted by test-set model selection; the paper deserves review with major revision. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is a normalizing-flow model applied to features from CLIP's ViT encoder. Features are dimension-reduced and L2-normalized; an affine-coupling bijection maps them to a unit Gaussian, so log-likelihood is tractable. The training objective is unusual: minimize the likelihood of proxy images and optionally maximize the likelihood of natural images, derived from KL divergences in which the proxy distribution is modeled as $K/p_Z(z')$. The proxy images come from random frequency masking—multiplying the Fourier spectrum of a natural image by a random binary mask and inverting—which the paper shows works better than spatial operations such as smoothing, sharpening, noise, and colo
What would settle it
Measure CLIP feature-space distances between (a) natural images, (b) frequency-masked proxies, and (c) outputs of a current high-quality text-to-image generator not in the benchmark. If for that generator the average proxy-to-fake distance is not smaller than the average natural-to-fake distance, the load-bearing premise fails; equivalently, train the proxy-only detector and test it only on that generator—an AP near chance would refute the universality claim.
Extended reading notes
Core claim
The central claim is that a detector never shown an AI-generated image can nevertheless generalize across unseen generators if it is trained as an anomaly scorer over CLIP features rather than as a classifier. Frequency-masked versions of natural images serve as proxy anomalies: the normalizing-flow model is trained to minimize their likelihood (and, in the improved mode, to maximize the likelihood of natural images). At test time the negative log-likelihood under the fitted flow is the anomaly score. On the 19-dataset UnivFD benchmark the authors report 93.85% mAP with proxy-only training and 95.56% mAP with the combined objective, with accuracy 87.30%, without seeing a single AI-generated
Load-bearing premise
The whole approach depends on frequency-masked real images being closer in the detector's feature space to unseen AI-generated images than to natural images; the paper never directly measures this overlap, it only infers it from detection accuracy.
Editorial extensions
If this is right
- A detector can be produced without any generated images, so detection capability can be refreshed by re-running the pipeline on natural images instead of collecting outputs from newly released generators.
- The framework supports both pure unsupervised training (proxy-only, P mode) and a 'self-supervised' mode (natural plus proxy, N+P mode), both using the same anomaly score at test time.
- The reported average mAP of 95.56% exceeds the no-AII supervised baseline OnlyReal (93.18%), and remains much stronger on low-level-vision and perceptual-loss datasets where that baseline struggles.
- On four emerging, unseen generators the same model achieves average AP of 93.32% versus 74.05% for a CLIP-based supervised benchmark, suggesting the proxy-trained score transfers to post-training generator families.
- The complementary-distribution flow objective—maximizing likelihood of one class while minimizing another within a single bijective model—is a reusable component that could apply beyond image forensics.
Reading between the lines
- Not stated in the paper: if the proxy-to-fake transfer holds generally, then the detector's ceiling is set by how well CLIP features separate synthetic artifacts; a feature extractor trained with stronger forensic objectives could raise that ceiling without changing the framework.
- Editorial extension: the same complementary-distribution flow trick should transfer to out-of-distribution detection and open-set recognition, where anomalous samples are similarly hard to collect at training time.
- A testable consequence of the core assumption: measure pairwise distances in the CLIP feature space between natural, proxy, and fake images across many generators. If some unseen generator's outputs sit closer to natural features than to proxy features, the method should fail exactly on that generator.
- The masking-ratio asymmetry between modes (ratio 1.0 for P, 0.1 for N+P) suggests the proxy operation is not a fixed transformation; tuning how artificial the proxy is may be a practical knob for balancing sensitivity and specificity in a target deployment.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes CLIP-Flow, a detector for AI-generated images (AIIs) that does not use any AII during training. It extracts CLIP ViT-L/14 features, applies a learned linear dimension reduction and L2 normalization, and fits a normalizing flow (affine coupling) to the resulting features. As proxies for AIIs, it uses natural images modified by random frequency masking. Two training modes are considered: P mode minimizes the flow log-likelihood of proxy features, and N+P mode additionally maximizes the log-likelihood of natural features. The anomaly score is the negative normalized log-likelihood. Experiments on the 19-dataset UnivFD benchmark report mAP 93.85%/95.56% and accuracy 84.93%/87.30% for P/N+P modes, and an additional evaluation on four recent generators reports an average AP of 93.32%. The central claim is that proxy-image training transfers to unseen generators.
Significance. The idea of using frequency-masked natural images as proxy anomalies for universal AII detection is interesting and potentially useful. The empirical comparison is broad, the paper includes a sanity check on four newer generators, and the code is released. If the generalization claim holds, the method would be a valuable unsupervised alternative to supervised AII detectors. However, the theoretical derivation in Eq. (5) is not sound as written, and the headline numbers are obtained after model selection on the same test benchmark used to claim universality. These issues need to be resolved before the results can be taken at face value.
major comments (3)
- [Section 3.3, Eq. (5)] The assumption that proxy features follow z' ~ K/p_Z(z') with K a normalization constant is not justified. For K/p_Z to be a valid probability density, the integral of 1/p_Z(z') over the feature space must be finite. This is generally false; for example, if p_Z is Gaussian-like, 1/p_Z grows exponentially and is non-integrable. The stated 'mild condition' p_Z(z') != 0 is insufficient. Consequently D_KL[p*_Z' || K/p_Z] is not defined, and the loss in Eq. (6) is not actually derived from KL minimization. The training objective can still be used as a heuristic (minimize proxy likelihood / maximize natural likelihood), but the paper should present it as such or provide a correct derivation.
- [Section 4.1 and Section 4.3 (Tables 3-4, Fig. 4)] The reported headline numbers (N+P: 95.56% mAP, 87.30% accuracy) are not clean out-of-sample estimates. The masking ratio is chosen 'experimentally' in Section 4.1 (1.0 for P, 0.1 for N+P); Table 4 selects the low spectrum band based on best mAP on the 19-dataset benchmark; Fig. 4 and the normalization ablation select DR dimension 128 and feature normalization using test mAP. Thus the model configuration is selected on the same test sets used to claim transfer to unseen generators. The paper should either hold out a validation split disjoint from all reported test generators, or clearly state that the reported numbers are the result of model selection and not unbiased estimates of generalization. Section 4.4 is a useful check, but it uses the already-selected configuration and only the P mode.
- [Section 4.2, Table 2] The text after Table 2 states 'The average accuracy reaches 84.93% for N mode', but Table 2 shows N mode at 52.88% and P mode at 84.93%. This is a critical labeling error: the 84.93% figure corresponds to P mode, not N mode. The abstract and contributions also attribute 84.93% to the proxy-only mode. The text must be corrected and all mode labels made consistent, otherwise readers cannot tell which training mode achieves which result.
minor comments (6)
- [Section 1] Typo: 'CILP-Flow' should be 'CLIP-Flow'.
- [Section 3.2] The statement that frequency-masked proxy images are 'closer to AIIs than to natural images in the feature space' is asserted as an implication of the detection results. A direct feature-space visualization (e.g., t-SNE or nearest-neighbor statistics of proxy, natural, and AII CLIP features) would strengthen this claim.
- [Section 3.3, Eq. (4)] The notation p*_Z is used without a precise definition. Please clarify that it denotes the empirical or true data distribution and explain the constant c1.
- [Section 4.1] The validation set for threshold selection uses frequency-perturbed images with a masking band of (30,100), generated differently from the training proxies. Please clarify how this threshold selection interacts with the test-set model selection and whether the threshold is fixed before seeing test data.
- [Section 3.3] The model is described as 'normalizing flow-like'. Since the architecture is a standard affine-coupling flow with a modified training loss, please clarify in the text what is new about the flow model itself versus the training objective.
- [Table 5] Table 5 reports only P mode on the emerging generators. State whether the N+P mode was also evaluated on these datasets; if not, explain why P mode was chosen for the generalization check.
Circularity Check
Benchmark numbers are partly selected on the same test sets they are used to validate; genuine out-of-sample check on four new generators keeps the central claim from being fully circular.
-
fitted input called prediction
[Section 4.3, 'Spectrum Part for Frequency Masking'; Section 4.1, 'Implementation Details'; Section 4.3, 'Dimension Reduction' and 'Feature Normalization']
"In general, the detection performance on each generative model and with each frequency part is different, and masking the low and high parts has better overall performance than masking the mid part, with the low part masking being slightly better than the high part masking. Therefore, we choose to perform random frequency masking in the low part of the spectrum of natural images to produce proxy images."
The choice of the low-frequency mask (and similarly the masking ratio 1.0/0.1, the DR dimension 128, and feature normalization) is made by comparing mAP on the same 19-dataset UnivFD benchmark that is then reported as the headline result (95.56 mAP / 87.30 acc). Thus the reported number is not an independent evaluation of the chosen configuration: the configuration was selected to maximize mAP on those test sets. The paper uses this mAP to infer that proxies are 'closer to AIIs than to natural images' (Sec. 3.2) and to claim strong generalization. This is a fitted-input-called-prediction pattern: hyperparameters are fit to the benchmark, then the benchmark performance is presented as evidence of transfer.
full rationale
The paper's core derivation (normalizing-flow likelihood with proxy/natural terms) is not circular: the proxy distribution K/p_Z is an explicit modeling assumption, not a derived prediction, and the benchmark comparisons are external. No load-bearing self-citation or equation-level equivalence was found. However, key configuration choices — masking ratio, low-frequency band, DR dimension, and feature normalization — are made by maximizing mAP on the 19-dataset UnivFD benchmark itself, and the same 95.56 mAP / 87.30 acc is then presented as evidence that the proxy criterion implies proxy images lie near AIIs and that the method generalizes. This is a test-set-selection issue: the headline numbers are not clean out-of-sample estimates for the selected configuration. The Section 4.4 evaluation on DALL·E 3, DeepFloyd, RealVisXL, and SD v1.5 is a genuinely held-out check and provides independent support, so the central generalization claim is not forced by construction. Score 4 reflects the partial circularity in the reported benchmark numbers while acknowledging the independent empirical content in the new-generator evaluation.
Assumptions & free parameters
free parameters (3)
- frequency masking ratio =
1.0 (P mode), 0.1 (N+P mode)
- spectrum part for masking =
low
- DR output dimension =
128
assumptions (3)
- ad hoc to paper Proxy features z' follow K/p_Z(z') with K a normalization constant
- domain assumption Frequency-masked proxy images are closer to AIIs than to natural images in CLIP feature space
- domain assumption CLIP ViT-L/14 features retain forensic traces of generated images
Cite this review
Pith. "Pith review of CLIP-Flow: A Universal Discriminator for AI-Generated Images Inspired by Anomaly Detection." pith.science (2026). https://pith.science/paper/GQSVOBBD
@misc{pith2026250809477,
author = {Pith},
title = {Pith review of: CLIP-Flow: A Universal Discriminator for AI-Generated Images Inspired by Anomaly Detection},
year = {2026},
howpublished = {\url{https://pith.science/paper/GQSVOBBD}},
note = {Machine review of arXiv:2508.09477}
}
read the original abstract
With the rapid advancement of AI generative models, the visual quality of AI-generated images (AIIs) has become increasingly close to natural images, which inevitably raises security concerns. Most AII detectors often employ the conventional image classification pipeline with natural images and AIIs (generated by a generative model), which can result in limited detection performance for AIIs from unseen generative models. To solve this, we proposed a universal AI-generated image detector from the perspective of anomaly detection. Our discriminator does not need to access any AIIs and learn a generalizable representation with unsupervised learning. Specifically, we use the pre-trained CLIP encoder as the feature extractor and design a normalizing flow-like unsupervised model. Instead of AIIs, proxy images, e.g., obtained by applying a spectral modification operation on natural images, are used for training. Our models are trained by minimizing the likelihood of proxy images, optionally combined with maximizing the likelihood of natural images. Extensive experiments demonstrate the effectiveness of our method on AIIs produced by various image generators.
Figures
Reference graph
Works this paper leans on
-
[1]
Nilesh A Ahuja, Ibrahima Ndiour, Trushant Kalyanpur, and Omesh Tickoo. 2019. Probabilistic modeling of deep features for out-of-distribution and adversarial detection. InBayesian Deep Learning Workshop. 1–5
work page 2019
-
[2]
Samet Akcay, Amir Atapour-Abarghouei, and Toby P Breckon. 2019. GANomaly: Semi-supervised anomaly detection via adversarial training. InAsian Conference on Computer Vision. 622–637
work page 2019
-
[3]
Weiming Bai, Zhipeng Zhang, Bing Li, Pei Wang, Yangxi Li, Congxuan Zhang, and Weiming Hu. 2021. Robust texture-aware computer-generated image forensic: Benchmark and algorithm.IEEE Transactions on Image Processing30 (2021), 8439–8453
work page 2021
-
[4]
Mu Cai, Hong Zhang, Huijuan Huang, Qichuan Geng, Yixuan Li, and Gao Huang
-
[5]
Lucy Chai, David Bau, Ser-Nam Lim, and Phillip Isola. 2020. What makes fake images detectable? Understanding properties that generalize. InEuropean Con- ference on Computer Vision. 103–120
work page 2020
-
[6]
Thomas Defard, Aleksandr Setkov, Angelique Loesch, and Romaric Audigier
-
[7]
Hanqiu Deng and Xingyu Li. 2022. Anomaly detection via reverse distillation from one-class embedding. InIEEE/CVF Conference on Computer Vision and Pattern Recognition. 9737–9746
work page 2022
-
[8]
InInternational Conference on Pattern Recognition
PaDiM: a patch distribution modeling framework for anomaly detection and localization. InInternational Conference on Pattern Recognition. 475–489
Show all 37 references
-
[9]
Chandler Timm Doloriel and Ngai-Man Cheung. 2024. Frequency masking for universal deepfake detection. InIEEE International Conference on Acoustics, Speech and Signal Processing. 13466–13470
2024
-
[10]
Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. 2017. Density estimation using Real NVP. InInternational Conference on Learning Representations. 1–12
2017
-
[11]
Witherden
Tarik Dzanic, Karan Shah, and Freddie D. Witherden. 2020. Fourier spectrum dis- crepancies in deep network generated images. InAdvances in Neural Information Processing Systems. 3022–3032
2020
-
[12]
Ricard Durall, Margret Keuper, and Janis Keuper. 2020. Watch your up- convolution: CNN based generative deep neural networks are failing to reproduce spectral distributions. InIEEE/CVF Conference on Computer Vision and Pattern Recognition. 7887–7896
2020
-
[13]
Evgeny Gladilin and Roland Eils. 2015. On the role of spatial phase and phase cor- relation in vision, illusion, and cognition.Frontiers in Computational Neuroscience 9 (2015), 1–14
2015
-
[14]
Joel Frank, Thorsten Eisenhofer, Lea Schonherr, Asja Fischer, Dorothea Kolossa, and Thorsten Holz. 2020. Leveraging frequency analysis for deep fake image recognition. InInternational Conference on Machine Learning. 3247–3258
2020
-
[15]
Terro Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2018. Progressive growing of GANs for improved quality, stability, and variation. InInternational Conference on Learning Representations. 1–12
2018
-
[16]
Denis Gudovskiy, Shun Ishizaka, and Kazuki Kozuka. 2022. CFLOW-AD: Real- time unsupervised anomaly detection with localization via conditional normal- izing flows. InIEEE/CVF Winter Conference on Applications of Computer Vision. 98–107
2022
-
[17]
Ivan Kobyzev, Simon JD Prince, and Marcus A Brubaker. 2020. Normalizing flows: An introduction and review of current methods.IEEE Transactions on Pattern Analysis and Machine Intelligence43, 11 (2020), 3964–3979
2020
-
[18]
Terro Karras, Samuli Laine, and Timo Aila. 2019. A Style-based generator archi- tecture for generative adversarial networks. InIEEE/CVF Conference on Computer Vision and Pattern Recognition. 4401–4410
2019
-
[19]
Ji Li and Kai Wang. 2024. Detecting computer-generated images by using only real images. InInternational Conference on Machine Vision. 1–13
2024
-
[20]
Sungwook Lee, Seunghyun Lee, and Byung Cheol Song. 2022. CFA: Coupled- hypersphere-based feature adaptation for target-oriented anomaly localization. IEEE Access10 (2022), 78446–78454
2022
-
[21]
Roy-Chowdhuri, and B.S
Lakshmanan Natraj, Tajuddin Manhar Mohammed, Shivkumar Chandrasekaran, Arjuna Flenner, Amit K. Roy-Chowdhuri, and B.S. Manjunath. 2019. Detecting GAN generated fake images using co-occurrence matrices. InElectronic Imaging. 532:1–6
2019
-
[22]
Bo Liu, Fan Yang, Xiuli Bi, Bin Xiao, Weisheng Li, and Xinbo Gao. 2022. Detecting generated images by real images. InEuropean Conference Computer Vision. 95– 110
2022
-
[23]
Utkarsh Ojha, Yuheng Li, and Yong Jae Lee. 2023. Towards universal fake image detectors that generalize across generative models. InIEEE/CVF Conference on Computer Vision and Pattern Recognition. 24480–24489
2023
-
[24]
Xuelei (Sherry) Ni and Xiaoming Huo. 2007. Statistical interpretation of the importance of phase information in signal and image reconstruction.Statistics & Probability Letters77, 4 (2007), 447–454
2007
-
[25]
Guansong Pang, Chunhua Shen, Longbing Cao, and Anton Van Den Hengel. 2021. Deep learning for anomaly detection: A review.Comput. Surveys54, 2 (2021), 1–38
2021
-
[26]
Oppenheim and Jae Lim
Alan V. Oppenheim and Jae Lim. 1981. The importance of phase in signals.Proc. IEEE69, 5 (1981), 529–541
1981
-
[27]
Weize Quan, Pengfei Deng, Kai Wang, and Dong-Ming Yan. 2024. CGFormer: ViT- based network for identifying computer-generated images with token labeling. IEEE Transactions on Information Forensics and Security19 (2024), 235–250
2024
-
[28]
George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lakshminarayanan. 2021. Normalizing flows for probabilistic modeling and inference.Journal of Machine Learning Research22, 57 (2021), 1–64
2021
-
[29]
Guodong Wang, Shumin Han, Errui Ding, and Di Huang. 2021. Student-teacher feature pyramid matching for unsupervised anomaly detection. InBritish Machine Vision Conference. 1–14
2021
-
[30]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. InInternational Conference on Machine Learnin...
2021
-
[31]
Zhendong Wang, Jianmin Bao, Wengang Zhou, Weilun Wang, Hezhen Hu, Hong Chen, and Houqiang Li. 2023. DIRE for diffusion-generated image detection. In IEEE/CVF International Conference on Computer Vision. 22388–22398
2023
-
[32]
Sheng-Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A Efros. 2020. CNN-generated images are surprisingly easy to spot...for now. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8692–8701
2020
-
[33]
Vitjan Zavrtanik, Matej Kristan, and Danijel Skočaj. 2021. DRÆM - A discrim- inatively trained reconstruction embedding for surface anomaly detection. In IEEE/CVF International Conference on Computer Vision. 8330–8339
2021
-
[34]
Jiawei Yu, Ye Zheng, Xiang Wang, Wei Li, Yushuang Wu, Rui Zhao, and Liwei Wu. 2021. FastFlow: Unsupervised anomaly detection and localization via 2D normalizing flows.arXiv preprint arXiv:2111.07677(2021), 1–11
2021 arXiv
-
[35]
Zu Zhang, Svebor Karaman, and Shih-Fu Chang. 2019. Detecting and simulating artifacts in GAN fake images. InIEEE International Workshop on Information Forensics and Security. 1–6
2019
-
[36]
Vitjan Zavrtanik, Matej Kristan, and Danijel Skočaj. 2022. DSR – A dual subspace re-projection network for surface anomaly detection. InEuropean Conference on Computer Vision. 539–554
2022
-
[2021]
InIEEE/CVF International Conference on Computer Vision
Frequency domain image translation: More photo-realistic, better identity- preserving. InIEEE/CVF International Conference on Computer Vision. 13930– 13940
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.