Pith

open record

sign in

arxiv: 2606.25376 · v1 · pith:UEPQ4XJQ · submitted 2026-06-24 · cs.CV

Transferable Attack against Face Swapping in an Extended Space

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-06-25 21:02 UTCgrok-4.3pith:UEPQ4XJQrecord.jsonopen to challenge →

classification cs.CV
keywords face swappingadversarial attacktransferable attackreilluminationidentity extractiondeepfake
0
0 comments X

The pith

The proposed AIR attack combines reillumination and additive perturbations to extend the space of effective adversarial examples against subject-agnostic face swapping models.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces AIR to protect against face swapping by generating adversarial examples that fool identity extraction in subject-agnostic models. It combines reillumination and additive perturbations to extend the attack space, allowing stronger yet natural-looking perturbations. This approach avoids the need for a surrogate model and includes an adaptive translation-invariant operation and illumination control. Experiments across 1000 image pairs on various GAN and diffusion-based models demonstrate higher attack success rates and better image quality than prior methods. A mathematical proof supports the attack space extension.

Core claim

AIR leverages reillumination and additive perturbations to mislead the identity extraction modules in subject-agnostic FS models. By using these two types of perturbations simultaneously, the attack space is extended such that stronger but more visually natural adversarial examples can be identified. An adaptive translation-invariant operation and an illumination control scheme are designed to enhance visual quality while preserving attack effectiveness. Unlike other methods, AIR does not require a surrogate FS model to achieve high transferability, and a mathematical proof is given for the extension of the attack space.

What carries the argument

The Additive Identity attack based on a Relighting function (AIR) that combines reillumination and additive perturbations to extend the attack space.

If this is right

  • AIR achieves higher attack success rates on unseen subject-agnostic FS models including GAN and diffusion-based ones.
  • AIR produces adversarial examples with better visual quality than existing attacks.
  • The method works without training on a surrogate FS model.
  • The attack space extension is supported by a mathematical proof.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • This combination of perturbations could apply to defending against other types of deepfake manipulations.
  • Further optimization of the illumination control might improve performance on specific lighting conditions.
  • Testing the attack on real-world deployed face swapping apps would show practical effectiveness.

Load-bearing premise

Reillumination combined with additive perturbations can reliably mislead identity extraction modules in unseen subject-agnostic face swapping models.

What would settle it

Finding a new subject-agnostic FS model where the AIR attack has a lower success rate than current best attacks on the same test images.

Figures

Figures reproduced from arXiv: 2606.25376 by Adams Wai-Kin Kong, Hong Xu, Jun Xie, Mingzhi Lyu, Yi Huang, Zihao Zhao.

Figure 1
Figure 1. Figure 1: (a) AEs with TAIG under ϵ = 0.02 (left), ϵ = 0.03 (middle) and ϵ = 0.05 (right). (b) AEs with TI. transferability and imperceptibility of adversarial perturbations is a significant challenge when attacking FS models. State-of￾the-art techniques like TAIG [10] have shown effectiveness in enhancing transferability but often introduce visible noise textures, as illustrated in Fig. 1a. Even with TAIG, FS outpu… view at source ↗
Figure 2
Figure 2. Figure 2: The schematic diagram of AIR. Functional Attack (RFA). AIA, operating under the addi￾tive threat model, incorporates a new Adaptive Translation￾Invariant (ATI) operation, enhancing transferability without introducing visible noise textures. RFA, based on the func￾tional threat model, employs a mapping function to alter lighting intensity from various directions while preserving crucial facial attributes su… view at source ↗
Figure 3
Figure 3. Figure 3: Face swapping result of MegaGAN before and after [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Adversarial examples generated by AIR with TI and [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: The filter [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Adversarial examples generated by TI, TAIG, and AIR. [PITH_FULL_IMAGE:figures/full_fig_p011_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: AIA with ATI under ϵ = 0.05 and ϵ = 0.02. The first column is the original images. The second and third columns are the adversarial examples generated with ϵ = 0.02 and ϵ = 0.05, respectively [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Examples of AIR against MegaGAN. The first row is [PITH_FULL_IMAGE:figures/full_fig_p011_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Examples of AIR against FaceShifter. The first row [PITH_FULL_IMAGE:figures/full_fig_p012_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Examples of AIR against SimSwap. The first row is [PITH_FULL_IMAGE:figures/full_fig_p012_11.png] view at source ↗
read the original abstract

Although deep Face Swapping (FS) models may benefit the entertainment industry, they pose severe threats to privacy and security. Existing protections, including deepfake detection and adversarial perturbation, are either passive responses or ineffective to unseen subject-agnostic FS models. In this paper, we propose a transferable attack against subject-agnostic FS models named Additive Identity attack based on a Relighting function (AIR). AIR leverages reillumination and additive perturbations to mislead the identity extraction modules in subject-agnostic FS models. By using these two types of perturbations simultaneously, the attack space is extended such that stronger but more visually natural adversarial examples can be identified. To further enhance the visual quality while preserving the effectiveness of the attack, an adaptive translation-invariant operation and an illumination control scheme are designed for AIR. Unlike other methods, AIR does not require a surrogate FS model to achieve high transferability. In addition, a mathematical proof is given for the extension of the attack space. Extensive experiments using 1000 image pairs across various state-of-the-art subject-agnostic FS models, including GAN and diffusion-based FS models, show that AIR surpasses all existing attacks in terms of both attack success rate and image quality.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper proposes AIR, a transferable adversarial attack on subject-agnostic face-swapping (FS) models that combines reillumination with additive perturbations to extend the attack space, supported by a mathematical proof of the extension; unlike prior methods, AIR requires no surrogate FS model. Experiments on 1000 image pairs across GAN- and diffusion-based state-of-the-art FS models report higher attack success rate and better image quality than existing attacks, with additional components for adaptive translation-invariance and illumination control.

Significance. If the central claims hold, the work is significant because it demonstrates model-agnostic transferability via an extended attack space without surrogate training, together with a mathematical proof and large-scale experiments (1000 pairs) that include both GAN and diffusion FS models. These elements address a practical gap in protecting against unseen FS models and could inform future defenses.

major comments (2)
  1. [Abstract / Mathematical proof] Abstract and mathematical proof section: the claim that the proof establishes model-agnostic extension of the attack space (enabling transfer without surrogate) is load-bearing for the no-surrogate transferability result, yet the proof appears to rest on assumptions (additive separability of illumination and identity features, bounded sensitivity of the extractor) that diffusion-based FS models' iterative denoising and cross-attention mechanisms are likely to violate; explicit verification or counter-examples for diffusion extractors are needed.
  2. [Experiments] Experimental section (1000 image pairs): the reported superiority in attack success rate and image quality over baselines lacks stated metrics, error bars, exact protocol details, or ablation on the reillumination component alone, making it impossible to assess whether the gains are robust or protocol-dependent.
minor comments (2)
  1. [Abstract] Abstract: the phrase 'surpasses all existing attacks' should be qualified with the specific baselines and metrics used.
  2. [Method] Notation: the definitions of the reillumination function and additive perturbation should be introduced with explicit symbols before the proof is referenced.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback on the mathematical proof and experimental reporting. We address each major comment below and will revise the manuscript to incorporate additional verification and details.

read point-by-point responses
  1. Referee: [Abstract / Mathematical proof] Abstract and mathematical proof section: the claim that the proof establishes model-agnostic extension of the attack space (enabling transfer without surrogate) is load-bearing for the no-surrogate transferability result, yet the proof appears to rest on assumptions (additive separability of illumination and identity features, bounded sensitivity of the extractor) that diffusion-based FS models' iterative denoising and cross-attention mechanisms are likely to violate; explicit verification or counter-examples for diffusion extractors are needed.

    Authors: The proof establishes extension of the attack space under the stated assumptions of additive separability between illumination and identity features together with bounded sensitivity of the extractor. While experiments already show strong transfer to diffusion-based FS models, we agree that explicit checks on whether the assumptions hold for their iterative denoising and cross-attention components would strengthen the claim. In revision we will add a dedicated subsection providing empirical verification (or counter-examples) using the identity extractors from the diffusion FS models evaluated in the paper. revision: yes

  2. Referee: [Experiments] Experimental section (1000 image pairs): the reported superiority in attack success rate and image quality over baselines lacks stated metrics, error bars, exact protocol details, or ablation on the reillumination component alone, making it impossible to assess whether the gains are robust or protocol-dependent.

    Authors: We will expand the experimental section to explicitly define the attack success rate and image quality metrics, report standard error bars across repeated trials, detail the full protocol (pair selection, preprocessing, and hyper-parameters for the 1000 pairs), and include a new ablation isolating the reillumination component to quantify its contribution to the observed gains. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation is self-contained

full rationale

The paper presents AIR as an empirical attack combining reillumination and additive perturbations, validated through extensive experiments on 1000 image pairs across GAN and diffusion-based FS models, with a stated mathematical proof for attack-space extension. No equations, fitted parameters, or self-citations are shown in the provided text that reduce the claimed transferability or space extension to inputs by construction. The central results rest on experimental outcomes and an independent proof rather than self-definitional steps or renamed fits.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review provides no explicit free parameters, axioms, or invented entities; the method appears to rely on standard assumptions in adversarial perturbation generation and image relighting.

pith-pipeline@v0.9.1-grok · 5751 in / 1142 out tokens · 19435 ms · 2026-06-25T21:02:34.347994+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

25 extracted references · 4 canonical work pages · 1 internal anchor

  1. [1]

    Fakelocator: Robust localization of gan-based face manipulations,

    Yihao Huang, Felix Juefei-Xu, Qing Guo, Yang Liu, and Geguang Pu, “Fakelocator: Robust localization of gan-based face manipulations,” TIFS, 2022

  2. [2]

    Cmua-watermark: A cross-model universal adver- sarial watermark for combating deepfakes,

    Hao Huang et al., “Cmua-watermark: A cross-model universal adver- sarial watermark for combating deepfakes,” inAAAI, 2022, vol. 36, pp. 989–997

  3. [3]

    Disrupting deepfakes: Adversarial attacks against conditional image translation networks and facial manipulation systems,

    Nataniel Ruiz, Sarah Adel Bargal, and Stan Sclaroff, “Disrupting deepfakes: Adversarial attacks against conditional image translation networks and facial manipulation systems,” inECCV, 2020

  4. [4]

    Visually maintained image disturbance against deepfake face swapping,

    Junhao Dong and Xiaohua Xie, “Visually maintained image disturbance against deepfake face swapping,” inICME, 2021

  5. [5]

    Tafim: Targeted adversarial attacks against facial image manipulations,

    Shivangi Aneja, Lev Markhasin, and Matthias Nießner, “Tafim: Targeted adversarial attacks against facial image manipulations,” inECCV. Springer, 2022, pp. 58–75

  6. [6]

    Dual defense: Adversarial, traceable, and invisible robust watermarking against face swapping,

    Yunming Zhang et al., “Dual defense: Adversarial, traceable, and invisible robust watermarking against face swapping,”TIFS, 2024

  7. [7]

    Attack as the best defense: Nullifying image-to- image translation gans via limit-aware adversarial attack,

    Chin-Yuan Yeh, Hsi-Wen Chen, Hong-Han Shuai, De-Nian Yang, and Ming-Syan Chen, “Attack as the best defense: Nullifying image-to- image translation gans via limit-aware adversarial attack,” inICCV, 2021

  8. [8]

    Initiative defense against facial manipulation,

    Qidong Huang, Jie Zhang, Wenbo Zhou, Weiming Zhang, and Nenghai Yu, “Initiative defense against facial manipulation,” inAAAI, 2021, vol. 35, pp. 1619–1627

  9. [9]

    Restricted black-box adversarial attack against deepfake face swapping,

    Junhao Dong, Yuan Wang, Jianhuang Lai, and Xiaohua Xie, “Restricted black-box adversarial attack against deepfake face swapping,”TIFS, 2023

  10. [10]

    Transferable adversarial attack based on integrated gradients,

    Yi Huang and Adams Wai-Kin Kong, “Transferable adversarial attack based on integrated gradients,” inICLR, 2022

  11. [11]

    High-resolution neural face swapping for visual effects,

    Jacek Naruniec, Leonhard Helminger, Christopher Schroers, and Ro- mann M Weber, “High-resolution neural face swapping for visual effects,” inComputer Graphics F orum, 2020

  12. [12]

    Advancing high fidelity identity swapping for forgery detection,

    Lingzhi Li, Jianmin Bao, Hao Yang, Dong Chen, and Fang Wen, “Advancing high fidelity identity swapping for forgery detection,” in CVPR, 2020

  13. [13]

    Sim- swap: An efficient framework for high fidelity face swapping,

    Renwang Chen, Xuanhong Chen, Bingbing Ni, and Yanhao Ge, “Sim- swap: An efficient framework for high fidelity face swapping,” in Proceedings of the 28th ACM International Conference on Multimedia, 2020

  14. [14]

    One shot face swapping on megapixels,

    Yuhao Zhu, Qi Li, Jian Wang, Cheng-Zhong Xu, and Zhenan Sun, “One shot face swapping on megapixels,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021

  15. [15]

    Diffface: Diffusion-based face swapping with facial guidance,

    Kihong Kim et al., “Diffface: Diffusion-based face swapping with facial guidance,”arXiv preprint arXiv:2212.13344, 2022

  16. [16]

    Diffswap: High-fidelity and controllable face swapping via 3d-aware masked diffusion,

    Wenliang Zhao et al., “Diffswap: High-fidelity and controllable face swapping via 3d-aware masked diffusion,” inCVPR, 2023. 6The appendix is available in the arXiv version of this paper

  17. [17]

    Spatially transformed adversarial examples,

    Chaowei Xiao et al., “Spatially transformed adversarial examples,” in ICLR, 2018

  18. [18]

    Functional adversarial attacks,

    Cassidy Laidlaw and Soheil Feizi, “Functional adversarial attacks,” in NeurIPS, 2019

  19. [19]

    Evading defenses to transferable adversarial examples by translation-invariant attacks,

    Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu, “Evading defenses to transferable adversarial examples by translation-invariant attacks,” in CVPR, 2019

  20. [20]

    Deep single-image portrait relighting,

    Hao Zhou, Sunil Hadap, Kalyan Sunkavalli, and David W Jacobs, “Deep single-image portrait relighting,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2019

  21. [21]

    Progressive Growing of GANs for Improved Quality, Stability, and Variation

    Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen, “Progres- sive growing of gans for improved quality, stability, and variation,”arXiv preprint arXiv:1710.10196, 2017

  22. [22]

    Arcface: Additive angular margin loss for deep face recognition,

    Jiankang Deng, Jia Guo, Xue Niannan, and Stefanos Zafeiriou, “Arcface: Additive angular margin loss for deep face recognition,” inCVPR, 2019

  23. [23]

    Kornia: an open source differentiable computer vision library for pytorch,

    Edgar Riba, Dmytro Mishkin, Daniel Ponsa, Ethan Rublee, and Gary R. Bradski, “Kornia: an open source differentiable computer vision library for pytorch,” inWinter Conference on Applications of Computer Vision, 2020

  24. [24]

    Adv-makeup: A new imperceptible and transferable attack on face recognition,

    Bangjie Yin et al., “Adv-makeup: A new imperceptible and transferable attack on face recognition,”arXiv preprint arXiv:2105.03162, 2021

  25. [25]

    Exploring clip for assessing the look and feel of images,

    Jianyi Wang, Kelvin CK Chan, and Chen Change Loy, “Exploring clip for assessing the look and feel of images,” inAAAI, 2023, vol. 37, pp. 2555–2563. APPENDIX A. Analysis of a Combined Threat Model In AIR, the search for AEs is conducted under a combined threat model that integrates both the additive and functional threat models. This combination results in...