REVIEW 4 major objections 5 minor 46 references
Robustness in AI-Generated Detection: Enhancing Resistance to Adversarial Attacks
T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read This paper claims that combining DIRE residual maps with adversarial training restores near-pre-attack accuracy for face-forgery detectors under white-box PGD attacks that drive vanilla detectors to 0%.
desk verdict AT+DIRE for AIGC face detection is a sensible idea, but the attack in Algorithm 1 targets raw pixels while the detector is trained on DIRE maps, so the headline robustness numbers are not yet established. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the DIRE residual map, $\mathrm{DIRE}(x_0) = |x_0 - R(I(x_0))|$, the absolute difference between an image and its reconstruction after deterministic DDIM inversion ($I$) and reconstruction ($R$). The map carries the argument because diffusion-generated images reconstruct more faithfully than real images, so residual magnitude separates the two classes even under perturbation. Adversarial training then works on these maps: the detector is trained on concatenated clean and PGD-attacked residual maps, forcing it to treat attack-induced variation as nuisance rather than signal. The inversion-reconstruction chain also acts as a built-in denoiser, which is why the paper attributes both intrinsic robustness and cross-domain stability to DIRE.
What would settle it
Run the trained detector on adversarial images crafted by an attack not seen in training, for example a stronger adaptive attack that differentiably passes through the inversion-reconstruction pipeline at the same epsilon; if accuracy falls well below the reported near-99% while clean accuracy stays high, the central robustness claim holds only for the training attack, not for adversarial robustness generally.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that the failure of face forgery detectors under adversarial attack is not intrinsic: a detector trained in DIRE space on both clean and PGD-attacked samples recovers essentially all of its pre-attack accuracy. In the all-set setting, vanilla ResNet and ViT detectors drop from near 100% clean accuracy to near 0% under attack, while the adversarially trained DIRE detector holds roughly 99.9 to 100% accuracy on CelebA, LFW, Selfie, SFHQ, and SDFace. The paper further claims that adversarial training alone improves but does not fully restore cross-domain accuracy, while the diffusion reconstruction component stabilizes performance on unseen data distributions. The authors interpret DIRE's contribution as a denoising effect: DDIM inversion-reconstruction smooths adversarial perturbations, and the residual maps amplify real-versus-synthetic differences that are invisible in pixel space.
Load-bearing premise
The load-bearing premise is that robustness to the exact PGD attack used in training, with the same L-infinity budget and step size, is a valid proxy for robustness to adversarial attacks in general; the paper does not test stronger, adaptive, or different attacks, so a real adversary using another method could break the defense.
Editorial extensions
If this is right
- In-domain deployment: a face-forgery detector trained in DIRE space with adversarial training keeps roughly 99% accuracy under white-box PGD attacks of the same budget used in training, where the same detector without the defense falls to 0%.
- DIRE alone is a partial defense: even without adversarial training it retains substantial accuracy on some datasets, so reconstruction-error features are a stronger starting point for robust detection than raw pixels.
- Cross-domain robustness is not automatic from adversarial training; the diffusion reconstruction component is the part that prevents accuracy collapse on unseen face datasets.
- The fixed-output collapse observed on non-face natural images means the defense is demonstrated for facial-image distributions, not for general image detection.
Reading between the lines
- The paper's evidence is limited to the PGD attack it trained against; a natural extension is to test adaptive adversaries that back-propagate through the full DDIM inversion-reconstruction, since the reported denoising effect may not survive gradient-aware attacks.
- Because the difference-DIRE visualizations show that real and fake images respond differently to attacks, one could turn the before/after residual difference into an explicit robustness feature or regularizer rather than relying only on implicit concatenated training.
- The method's benefit appears tied to face-like statistics; applying the same pipeline to ImageNet-scale content may require conditioning the reconstruction or augmenting with domain-specific adversarial samples, otherwise the detector can collapse to constant outputs.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper proposes a defense for AI-generated image detection by combining adversarial training with diffusion reconstruction error (DIRE). The method computes DIRE maps for clean and PGD-perturbed images, concatenates them, and trains a classifier (ResNet50 or ViT) on these maps. Experiments are conducted on face datasets (CelebA, LFW, Selfie, SFHQ, SDFace) in all-set and cross-domain settings, measuring accuracy under clean and attacked conditions and a 'robustness score' defined as the ratio of adversarial to clean accuracy. The paper reports large gains from adversarial training on some datasets but also conflicting results in cross-domain settings.
Significance. The topic is timely and important: adversarial robustness of AIGC detectors is under-studied, and DIRE is a plausible feature representation for such detection. The paper contributes visualizations of attack noise and DIRE differences, and it attempts a cross-domain robustness analysis. However, the significance of the claimed improvements is not currently established because the attack definition is inconsistent with the detector input space, the evaluation uses only the same PGD attack as training, and the proposed robustness score can be misleading. These issues are load-bearing for the central claim.
major comments (4)
- [§3.2–3.3, Algorithm 1] Algorithm 1 lines 4–8 generate PGD adversarial examples via ∇_{x'} L(f(x'), y) on raw images, while lines 9–15 define the detector input as x_combined = Concat(DIRE(x0), DIRE(x0')) and train f on that combined representation. Because f is never defined on raw pixels, the attack as written is undefined for the DIRE-based detector; the same inconsistency appears in Eq. (5), which writes f(x) and f(x') on raw inputs. If the attacks were instead generated against a separate pixel-space classifier and only evaluated after the DIRE transformation, the high accuracies in Table 1 (e.g., 99.02% for CelebA ResNet w/ AT) would reflect a non-adaptive adversary. The paper must specify the exact attacked model and, to support the abstract's white-box robustness claim, evaluate adaptive attacks that propagate gradients through the full inversion–reconstruction pipeline.
- [§4.2, §4.4] Training and evaluation use the same PGD attack with the same parameters (Section 4.2, L∞, step size 10), so the robust accuracies in Tables 1–3 may be overfitted to the training attack. Section 4.4.2's conclusion that 'the performance remains comparable' under attack is therefore not evidence of general robustness. I request additional evaluation with unseen attacks (e.g., AutoAttack, FGSM, transfer attacks from other models, and varying epsilon budgets), with results reported per attack.
- [§4.5, Table 2, Table 3] Section 4.5 states that 'models in the w/ AT setting always outperform those in the w/o AT setting,' but this is directly contradicted by the paper's own tables. In Table 2, DIRE on SDFace has adversarial accuracy 78.72% without adversarial training versus 41.06% with it, and in Table 3, DIRE on SFHQ has 95.45% versus 66.42%. Adversarial training can therefore reduce cross-domain robustness in the DIRE setting, which conflicts with the claimed universal benefit and needs a more careful analysis.
- [Eq. (9), Table 2] The robustness score defined in Eq. (9) as Accadv/Accclean can exceed 1; for example, Table 2 reports Selfie DIRE with AT having clean accuracy 25.60% and adversarial accuracy 40.47%, yielding a score of 1.58. This metric rewards a detector whose accuracy improves under attack relative to its poor clean performance, which is misleading. Also, no error bars or multiple-seed results are reported anywhere, making it impossible to assess the statistical reliability of the claimed improvements.
minor comments (5)
- [§4.2] The attack description 'PGD under L∞ norm constraint with a step size of 10' is ambiguous; the perturbation bound epsilon and number of iterations are not reported, and a step size of 10 seems inconsistent with a bounded attack.
- [§1] The claim of being 'the first to focus on this critical issue in AIGC detection' is too strong given existing work on adversarial robustness of deepfake detectors; please qualify the claim with a comparison to related literature.
- [Figures] The figures contain typographical artifacts (e.g., 'NoiseNoise' in Figure 1) and the visualizations would benefit from higher-resolution images and clear labels.
- [Appendix] The text refers to an Appendix for dataset details and additional results, but the Appendix is not included in the manuscript; the submitted version should include it.
- [Author list] The author list includes two email addresses from different institutions; please clarify the corresponding author and ensure the email display is correct.
Circularity Check
No significant circularity: the robustness claims are empirical evaluations, and the same-attack training/evaluation is a scope limitation, not a derivation-level reduction.
full rationale
The paper makes no claim that a fitted value or self-cited theorem forces its results. Adversarial training in Eq. (5) and Algorithm 1 incorporates PGD examples into the training loss, and Table 1 then evaluates on PGD examples; this is a standard training/evaluation protocol rather than a prediction derived from its own inputs. The 'w/o AT' DIRE rows and cross-domain tables provide independent evidence that diffusion reconstruction error contributes robustness beyond the adversarial-training objective. No parameter is fitted and renamed as a prediction, and no load-bearing uniqueness theorem or self-citation is used. Section 5 reports a limitation where adversarial training fails on non-face data, which is a scope caveat rather than a circular step. A separate correctness risk, not circularity, is that Algorithm 1 attacks f(x') in raw-pixel space while f is later defined on concatenated DIRE maps; this domain mismatch could make the reported white-box attack non-adaptive, but it does not make the derivation circular.
Assumptions & free parameters
free parameters (4)
- lambda in Eq. 5 =
1
- PGD step size alpha =
10
- PGD perturbation bound epsilon =
not reported
- PGD iterations T =
not reported
assumptions (4)
- domain assumption DDIM inversion and reconstruction with a pre-trained Stable Diffusion model yields a residual that separates real from generated face images.
- domain assumption White-box PGD under the L-infinity norm is a representative adversary for real-world attacks on AIGC detectors.
- domain assumption The five face datasets are representative and correctly labeled for real vs synthetic faces.
- ad hoc to paper Robustness measured on the training attack distribution transfers to deployment conditions.
Cite this review
Pith. "Pith review of Robustness in AI-Generated Detection: Enhancing Resistance to Adversarial Attacks." pith.science (2026). https://pith.science/paper/NANRAT6R
@misc{pith2026250503435,
author = {Pith},
title = {Pith review of: Robustness in AI-Generated Detection: Enhancing Resistance to Adversarial Attacks},
year = {2026},
howpublished = {\url{https://pith.science/paper/NANRAT6R}},
note = {Machine review of arXiv:2505.03435}
}
read the original abstract
The rapid advancement of generative image technology has introduced significant security concerns, particularly in the domain of face generation detection. This paper investigates the vulnerabilities of current AI-generated face detection systems. Our study reveals that while existing detection methods often achieve high accuracy under standard conditions, they exhibit limited robustness against adversarial attacks. To address these challenges, we propose an approach that integrates adversarial training to mitigate the impact of adversarial examples. Furthermore, we utilize diffusion inversion and reconstruction to further enhance detection robustness. Experimental results demonstrate that minor adversarial perturbations can easily bypass existing detection systems, but our method significantly improves the robustness of these systems. Additionally, we provide an in-depth analysis of adversarial and benign examples, offering insights into the intrinsic characteristics of AI-generated content. All associated code will be made publicly available in a dedicated repository to facilitate further research and verification.
Figures
Reference graph
Works this paper leans on
-
[1]
Lorenzo Baraldi, Federico Cocchi, Marcella Cornia, Alessandro Nicolosi, and Rita Cucchiara. 2025. Contrasting deepfakes diffusion via contrastive learning and global-local similarities. InEuropean Conference on Computer Vision. Springer, 199–216
work page 2025
- [2]
-
[3]
Ali Borji. 2023. Qualitative failures of image generation models and their appli- cation in detecting deepfakes.Image and Vision Computing137 (2023), 104771
work page 2023
-
[4]
Baoying Chen, Jishen Zeng, Jianquan Yang, and Rui Yang. 2024. DRCT: Diffusion Reconstruction Contrastive Training towards Universal Detection of Diffusion Generated Images. InForty-first International Conference on Machine Learning
work page 2024
-
[5]
Jongwook Choi, Taehoon Kim, Yonghyun Jeong, Seungryul Baek, and Jongwon Choi. 2024. Exploiting Style Latent Flows for Generalizing Deepfake Video Detection. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1133–1143
work page 2024
-
[6]
Francesco Croce and Matthias Hein. 2020. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. InInternational conference on machine learning. PMLR, 2206–2216
2020
-
[7]
Prafulla Dhariwal and Alexander Nichol. 2021. Diffusion models beat gans on image synthesis.Advances in neural information processing systems34 (2021), 8780–8794
2021
-
[8]
Alexey Dosovitskiy. 2020. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929(2020)
arXiv 2020
Show all 46 references
-
[9]
Hany Farid. 2022. Lighting (in) consistency of paint by text.arXiv preprint arXiv:2207.13744(2022)
2022 arXiv
-
[10]
Hany Farid. 2022. Perspective (in) consistency of paint by text.arXiv preprint arXiv:2206.14617(2022)
2022 arXiv
-
[11]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets.Advances in neural information processing systems27 (2014)
2014
-
[12]
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples.arXiv preprint arXiv:1412.6572(2014)
2014 arXiv
-
[13]
Gaurav Goswami, Akshay Agarwal, Nalini Ratha, Richa Singh, and Mayank Vatsa. 2019. Detecting and mitigating adversarial perturbations for robust face recognition.International Journal of Computer Vision127 (2019), 719–742
2019
-
[14]
Gaurav Goswami, Nalini Ratha, Akshay Agarwal, Richa Singh, and Mayank Vatsa. 2018. Unravelling robustness of deep learning based face recognition against adversarial attacks. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 32
2018
-
[15]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition. 770–778
2016
-
[16]
Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models.Advances in neural information processing systems33 (2020), 6840–6851
2020
-
[17]
Gary B Huang, Marwan Mattar, Tamara Berg, and Eric Learned-Miller. 2008. La- beled faces in the wild: A database forstudying face recognition in unconstrained environments. InWorkshop on faces in’Real-Life’Images: detection, alignment, and recognition
2008
-
[18]
Mahdi M Kalayeh, Misrak Seifu, Wesna LaLanne, and Mubarak Shah. 2015. How to take a good selfie?. InProceedings of the 23rd ACM international conference on Multimedia. 923–926
2015
-
[19]
Tero Karras. 2019. A Style-Based Generator Architecture for Generative Adver- sarial Networks.arXiv preprint arXiv:1812.04948(2019)
2019 arXiv
-
[20]
Tero Karras, Samuli Laine, and Timo Aila. 2019. A style-based generator ar- chitecture for generative adversarial networks. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 4401–4410
2019
-
[21]
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. 2015. Deep Learning Face Attributes in the Wild. InProceedings of International Conference on Computer Vision (ICCV)
2015
-
[22]
Ruipeng Ma, Jinhao Duan, Fei Kong, Xiaoshuang Shi, and Kaidi Xu. 2023. Ex- posing the fake: Effective diffusion-generated images detection.arXiv preprint arXiv:2307.06272(2023)
2023 arXiv
-
[23]
Aleksander Madry. 2017. Towards deep learning models resistant to adversarial attacks.arXiv preprint arXiv:1706.06083(2017)
2017 arXiv
-
[24]
Takeru Miyato, Andrew M Dai, and Ian Goodfellow. 2016. Adversarial training methods for semi-supervised text classification.arXiv preprint arXiv:1605.07725 (2016)
2016 arXiv
-
[25]
Alexander Quinn Nichol and Prafulla Dhariwal. 2021. Improved denoising diffusion probabilistic models. InInternational conference on machine learning. PMLR, 8162–8171
2021
-
[26]
Weili Nie, Brandon Guo, Yujia Huang, Chaowei Xiao, Arash Vahdat, and Anima Anandkumar. 2022. Diffusion models for adversarial purification.arXiv preprint arXiv:2205.07460(2022)
2022 arXiv
-
[27]
Trevine Oorloff, Surya Koppisetti, Nicolò Bonettini, Divyaraj Solanki, Ben Col- man, Yaser Yacoob, Ali Shahriyari, and Gaurav Bharaj. 2024. AVFF: Audio-Visual Feature Fusion for Video Deepfake Detection. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern R...
2024
-
[28]
William Peebles and Saining Xie. 2023. Scalable diffusion models with trans- formers. InProceedings of the IEEE/CVF International Conference on Computer Vision. 4195–4205
2023
-
[29]
Md Awsafur Rahman, Bishmoy Paul, Najibul Haque Sarker, Zaber Ibn Abdul Hakim, and Shaikh Anowarul Fattah. 2023. Artifact: A large-scale dataset with artificial and factual images for generalizable and robust synthetic image de- tection. In2023 IEEE International Conference on ...
2023
-
[30]
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen
-
[31]
Min Ren, Yuhao Zhu, Yunlong Wang, and Zhenan Sun. 2022. Perturbation inactivation based adversarial defense for face recognition.IEEE Transactions on Information Forensics and Security17 (2022), 2947–2962
2022
-
[32]
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10684–10695
2022
-
[33]
Sergey Sinitsa and Ohad Fried. 2024. Deep Image Fingerprint: To- wards Low Budget Synthetic Image Detection and Model Lineage Analysis. arXiv:2303.10762 [cs.CV] https://arxiv.org/abs/2303.10762
2024 arXiv
-
[34]
Jiaming Song, Chenlin Meng, and Stefano Ermon. 2020. Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502(2020)
2020 arXiv
-
[35]
Chuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu, Ping Liu, and Yunchao Wei. 2024. Rethinking the up-sampling operations in cnn-based generative network for generalizable deepfake detection. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition...
2024
-
[36]
Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. 2024. Visual autoregressive modeling: Scalable image generation via next-scale prediction. arXiv preprint arXiv:2404.02905(2024)
2024 arXiv
-
[37]
tobecwb. 2023. stable-diffusion-face-dataset. https://github.com/tobecwb/stable- diffusion-face-dataset
2023
-
[38]
Zhendong Wang, Jianmin Bao, Wengang Zhou, Weilun Wang, Hezhen Hu, Hong Chen, and Houqiang Li. 2023. Dire for diffusion-generated image detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 22445– 22455
2023
-
[39]
Zekai Wang, Tianyu Pang, Chao Du, Min Lin, Weiwei Liu, and Shuicheng Yan
-
[40]
Haiwei Wu, Jiantao Zhou, and Shile Zhang. 2023. Generalizable synthetic image detection via language-guided contrastive learning.arXiv preprint arXiv:2305.13800(2023)
2023 arXiv
-
[41]
Ying Xu, Philipp Terhöst, Marius Pedersen, and Kiran Raja. 2024. Analyzing Fairness in Deepfake Detection With Massively Annotated Databases.IEEE Transactions on Technology and Society(2024)
2024
-
[42]
Zhiyuan Yan, Yuhao Luo, Siwei Lyu, Qingshan Liu, and Baoyuan Wu. 2024. Tran- scending forgery specificity with latent space augmentation for generalizable deepfake detection. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8984–8994
2024
-
[43]
Daichi Zhang, Zihao Xiao, Shikun Li, Fanzhao Lin, Jianmin Li, and Shiming Ge. 2025. Learning natural consistency representation for face forgery video detection. InEuropean Conference on Computer Vision. Springer, 407–424
2025
-
[44]
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. InPro- ceedings of the IEEE international conference on computer vision. 2223–2232
2017
-
[2022]
Hierarchical text-conditional image generation with clip latents.arXiv preprint arXiv:2204.061251, 2 (2022), 3
2022 arXiv
-
[2023]
InInterna- tional conference on machine learning
Better diffusion models further improve adversarial training. InInterna- tional conference on machine learning. PMLR, 36246–36263
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.