Unlearnable Faces: Privacy Protection Surviving Extraction Pipeline
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-07-08 19:14 UTCglm-5.2pith:ZLNK5LQRrecord.jsonopen to challenge →
The pith
Faces stay unlearnable after crop-and-resize
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central mechanism is coupling the perturbation generation to a differentiable model of the attacker's crop-and-resize transform. When the perturbation is optimized through this transform, the optimization receives no gradient for frequency components the resize will attenuate, so the noise energy concentrates in the surviving low-frequency band. The paper measures this concentration: 93% of the perturbation energy falls in the resize-surviving band when coupled, versus 33% when uncoupled. A factorized ablation confirms that coupling, not localization, is decisive: localization alone leaves the attacker at 25.4% accuracy, while coupling drives it to 2.3%. Because the robustness is a joint
What carries the argument
A differentiable crop-and-resize operator T, composed of a face-box crop and a bilinear resize, inserted into the standard unlearnable-example min-min objective so that the perturbation is optimized through it.
If this is right
- Face-privacy perturbations should be evaluated under the attacker's full extraction pipeline, not at a single fixed resolution, since the crop and resize together neutralize existing protections.
- Coupling perturbation generation to a differentiable model of the downstream transform is a general principle: any preprocessing the attacker applies could be modeled the same way, not just crop-and-resize.
- Because the surviving frequency band is determined by the native face size rather than the attacker's target resolution, a defender can generate at a single resolution without knowing the attacker's input size.
Where Pith is reading between the lines
- The defender assumes knowledge of the face box location and native size at protection time. If the attacker's face detector finds a different box, the mask and crop will be misaligned, and the perturbation may not survive. The paper does not test sensitivity to this misalignment.
- JPEG recompression at quality 85 degrades LPID to 20.8% attacker accuracy, suggesting that platform recompression remains an open challenge for any bounded perturbation.
- The coupling principle could extend to other transforms an attacker might apply, such as grayscale conversion or blur, by adding them to the differentiable operator T.
Load-bearing premise
The defender knows the face box location and native size at protection time. In deployment, the attacker's face detector may find a different box, and the paper does not test what happens when the boxes are misaligned.
What would settle it
If an attacker's face detector produces a box significantly different from the one the defender assumed, the perturbation mask would be misaligned with the actual crop, and the protection could fail.
Figures
read the original abstract
Unlearnable examples keep publicly shared photos from being learned by unauthorized face-recognition models. An imperceptible perturbation, added before sharing, makes any model trained on the protected photos fail on clean faces. The perturbation is crafted on the shared image, however the attacker trains on the face it extracts, cropped and resized to the recognizer input, and under this extraction the protection collapses. We propose LPID, which builds the extraction into the unlearnable-example objective. LPID confines the perturbation to the extracted face region and optimizes it through a differentiable model of the extraction, concentrating its energy in the frequency band the extraction preserves. Because this robustness is a property of the transform rather than of any identity, LPID is re-optimized per album and protects even users it has never seen. LPID attains the lowest attacker accuracy of all methods in every setting we evaluate, holding the attacker below $10\%$ under crop+resize extraction on identities unseen at protection time, while remaining imperceptible at $32.7$\,dB PSNR and $0.161$ LPIPS.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes LPID (Localized, Pipeline-coupled Identity Defense), a method for protecting facial privacy against unauthorized face-recognition model training. The key insight is that existing unlearnable example (UE) methods fail when the attacker applies a crop-and-resize extraction pipeline before training: the crop discards perturbation energy outside the face region, and the resize attenuates high-frequency content. LPID addresses this by (1) confining the perturbation to the face box via a binary mask and (2) optimizing the perturbation through a differentiable model T of the crop-and-resize operator, which steers perturbation energy into the low-frequency band that survives resizing. The method is evaluated under a subject-independent protocol (unseen identities), across four attacker architectures, and with a factorized ablation isolating the contributions of localization versus coupling. The central claim—that coupling the perturbation to the extraction transform is what enables protection to survive crop+resize—is supported by the spectral analysis (93% vs. 33% surviving-band energy) and the ablation (coupling drives attacker accuracy to 2.3% versus 25.4% for localization alone).
Significance. The paper addresses a well-motivated and practically important gap: prior unlearnable example methods are evaluated at fixed resolution, but real face-recognition pipelines crop and resize faces before training. The experimental design is strong in several respects. The subject-independent protocol (identities unseen at protection time) is the correct evaluation for a deployment scenario. The cross-architecture transfer table (Tab. 3: ResNet-18, MobileNet-V2, DenseNet-121, VGG-16) demonstrates that protection is not surrogate-specific. The factorized ablation (Fig. 3b) cleanly isolates coupling from localization. The full-resolution control (Fig. 6) confirms that baseline reimplementations are faithful—UE and REM protect strongly without extraction but collapse under it. The spectral argument is grounded in standard sampling theory and confirmed empirically. The method produces falsifiable predictions (energy concentration ratio, cross-resolution generalization). The JPEG85 limitation (20.8% attacker accuracy) is honestly reported.
major comments (1)
- Sec. 3.2, Eq. (3): The defender's transform T is defined using a known face box (x0, y0, s0). In deployment, the defender publishes the photo and the attacker's face detector determines the box. If the attacker's detector finds a different box, the crop S_box in T will be misaligned with the attacker's actual crop, and the perturbation—whose gradient is shaped through T's specific crop coordinates (Alg. 1, lines 6-7)—may arrive misaligned. The paper's composites (Sec. 5.1) give the defender exact knowledge of the box since the defender created it, and the attacker uses the same box. This is a best-case alignment that real deployments cannot guarantee. The spectral argument (Sec. 4.2) suggests some robustness to spatial shift (low-frequency content survives resize), but this is not verified. A sensitivity analysis to box misalignment (e.g., shifting the attacker's crop by a few pixels or
minor comments (6)
- Sec. 5.1: The composite construction (pasting CASIA-WebFace faces on Places365 backgrounds) is a simplification of real scraped photos. While the full-resolution control (Sec. 5.6) addresses the composite artifact concern for aligned faces, the small-face-in-scene regime could benefit from evaluation on a real-world dataset with naturally occurring small faces.
- Tab. 1: The '+blur' column applies Gaussian blur after resize but the kernel size and sigma are not specified in Sec. 5.1.
- Tab. 2: LSP's max|δ| = 43.7/255 is noted as larger, but the ℓ2 budget value itself is not stated, making the comparison to the ℓ∞ methods less precise.
- Sec. 5.1: The attacker's 30-epoch schedule (extended to 80 for Tab. 3) is stated but the rationale for the extension is not given. A brief note on why 80 epochs were needed for the cross-architecture study would help.
- Fig. 3a: The normalized radial frequency axis would benefit from explicit labeling of what 'normalized' means (relative to the native Nyquist).
- Sec. 4.2: The claim that 'energy placed below the native cutoff s0/2 is kept by any resize that preserves that band' should note that this holds for up-sampling but not for aggressive down-sampling below s0, which could occur if the attacker's target resolution is smaller than the native face size.
Simulated Author's Rebuttal
We thank the referee for the careful and constructive review. The referee correctly identifies the core contribution and experimental strengths of the paper. The single major comment raises a valid concern about box misalignment between the defender's transform T and the attacker's actual crop in real deployment. We agree this is an important gap and will add a sensitivity analysis to box misalignment in the revision. Our spectral argument provides reason to expect partial robustness (low-frequency content is shift-invariant in its passband), but this must be verified empirically, and we will do so.
read point-by-point responses
-
Referee: Sec. 3.2, Eq. (3): The defender's transform T is defined using a known face box (x0, y0, s0). In deployment, the defender publishes the photo and the attacker's face detector determines the box. If the attacker's detector finds a different box, the crop S_box in T will be misaligned with the attacker's actual crop, and the perturbation—whose gradient is shaped through T's specific crop coordinates (Alg. 1, lines 6-7)—may arrive misaligned. The paper's composites (Sec. 5.1) give the defender exact knowledge of the box since the defender created it, and the attacker uses the same box. This is a best-case alignment that real deployments cannot guarantee. The spectral argument (Sec. 4.2) suggests some robustness to spatial shift (low-frequency content survives resize), but this is not verified. A sensitivity analysis to box misalignment (e.g., shifting the attacker's crop by a few pixels or)
Authors: The referee raises a valid and important point. In our current experimental setup, the defender and attacker use the same face box, which is indeed a best-case alignment that real deployments cannot guarantee. We agree that a sensitivity analysis to box misalignment is needed and will add it in the revision. Specifically, we will run experiments where the attacker's crop box is shifted by a random offset (e.g., ±5, ±10, ±15 pixels in each direction) relative to the box the defender used when constructing T, and report attacker accuracy as a function of misalignment magnitude. We note two reasons to expect partial robustness, though neither is a substitute for the empirical test the referee requests. First, LPID's perturbation energy is concentrated in the low-frequency band (ρ_LF = 0.93 vs. 0.33 uncoupled; Fig. 3a), and low-frequency content is comparatively shift-invariant—a small spatial offset changes the phase but not the frequency content, so the resize still preserves it. Second, the perturbation is confined to the face box by the mask m, and face detectors typically produce boxes that overlap substantially even when not identical, so a portion of the perturbation region is retained. However, we acknowledge that substantial misalignment could reduce the effective perturbation energy that survives into the attacker's crop, and the degree of robustness is an empirical question we have not yet answered. We will report the results honestly, including any degradation found, and will discuss the misalignment regime as a limitation if the protection degrades significantly. We will also clarify in the revised text that the current evaluation assumes box alignment and that this is a simplification of the real deployment scenario. revision: yes
Circularity Check
No significant circularity found; the derivation chain is self-contained against external benchmarks.
full rationale
The paper's derivation chain is not circular. The LPID objective (Eq. 2) modifies the standard UE objective (Eq. 1, from [23]) by inserting a differentiable crop+resize operator T (Eq. 3), defined using standard bilinear interpolation with no self-cited ansatz. The spectral argument in Sec. 4.2 — that optimizing through a low-pass filter concentrates perturbation energy in the passband — relies on standard sampling theory cited to [15,37], not to the authors' own prior work. The energy-fraction metric ρLF (Eq. 4) is a measurement of the resulting perturbation, not a prediction renamed from a fit. The main results (Tab. 1) evaluate against an external attacker trained from scratch on four architectures (Tab. 3) distinct from the ResNet-18 surrogate, with unseen identities, so the protection is not fitted to the test condition. The ablation (Fig. 3b) factorizes localization × coupling as an independent experiment. The full-resolution control (Fig. 6) confirms baselines are faithfully implemented. I find no step where a claimed prediction reduces to its inputs by construction, no load-bearing self-citation chain, and no uniqueness theorem invoked from the authors' own work. The one minor observation is that the spectral argument is nearly tautological (optimizing through a low-pass filter naturally places energy in the passband), but this is standard, externally-cited reasoning, not circularity. Score 1 reflects this minor note without any actual circular step.
Axiom & Free-Parameter Ledger
free parameters (6)
- ε (perturbation budget) =
8/255
- K (inner PGD steps) =
20
- α (PGD step size) =
0.8/255
- E (epochs) =
60
- η (outer learning rate) =
0.025
- T target resolution =
224
axioms (5)
- domain assumption The attacker applies a crop+resize transform before training.
- domain assumption The defender knows the face box (x0, y0, s0) at protection time.
- standard math Bilinear resize is a low-pass filter with cutoff at half the smaller of input and output resolutions.
- domain assumption The attacker trains from scratch (no pretrained model).
- domain assumption Platform recompression is at q≈70–85 JPEG quality.
Reference graph
Works this paper leans on
-
[1]
Athalye, A., Engstrom, L., Ilyas, A., Kwok, K.: Synthesizing robust adversarial examples. In: ICML (2018)
work page 2018
- [2]
-
[3]
Cai, J., Xu, G., Li, Z., Fang, R., Pu, R., Wu, D., Lao, Q., Ling, C., Wang, B.: FUSE: Full-spectrum unlearnable examples via spectral equalization. In: ICML (2026)
work page 2026
-
[4]
Cherepanova, V., Goldblum, M., Foley, H., Duan, S., Dickerson, J., Taylor, G., Goldstein,T.:LowKey:Leveragingadversarialattackstoprotectsocialmediausers from facial recognition. In: ICLR (2021)
work page 2021
-
[5]
Chuman, T., Iida, K., Sirichotedumrong, W., Kiya, H.: Image manipulation spec- ifications on social networking services for encryption-then-compression systems. IEICE Trans. Inf. & Syst.E102.D(1), 11–18 (2019)
work page 2019
- [6]
-
[7]
Deng, J., Guo, J., Xue, N., Zafeiriou, S.: ArcFace: Additive angular margin loss for deep face recognition. In: CVPR (2019)
work page 2019
-
[8]
Feng, J., Cai, Q.Z., Zhou, Z.H.: Learning to confuse: Generating training time adversarial data with auto-encoder. In: NeurIPS (2019)
work page 2019
-
[9]
Fowl, L., Goldblum, M., Chiang, P.y., Geiping, J., Czaja, W., Goldstein, T.: Ad- versarial examples make strong poisons. In: NeurIPS (2021)
work page 2021
-
[10]
Fu, S., He, F., Liu, Y., Shen, L., Tao, D.: Robust unlearnable examples: Protecting data against adversarial learning. In: ICLR (2022)
work page 2022
-
[11]
Nature Machine In- telligence2(11), 665–673 (2020)
Geirhos, R., Jacobsen, J.H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., Wichmann, F.A.: Shortcut learning in deep neural networks. Nature Machine In- telligence2(11), 665–673 (2020)
work page 2020
-
[12]
Gong, X., Wang, Y., Chen, Y., Dong, H., Li, Y., Sun, M., Li, S., Wang, Q., Chen, C.: ARMOR: Shielding unlearnable examples against data augmentation. IEEE Trans. Pattern Anal. Mach. Intell.48(4), 3988–4004 (2026)
work page 2026
-
[13]
Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: ICLR (2015)
work page 2015
-
[14]
Günther, M., Cruz, S., Rudd, E.M., Boult, T.E.: Toward open-set face recognition. CVPR Workshops (2017)
work page 2017
-
[15]
Guo, C., Frank, J.S., Weinberger, K.Q.: Low frequency adversarial perturbation. In: UAI (2019)
work page 2019
-
[16]
One-shot Face Recognition by Promoting Underrepresented Classes
Guo, Y., Zhang, L.: One-shot face recognition by promoting underrepresented classes. arXiv preprint arXiv:1707.05574 (2017)
work page internal anchor Pith review Pith/arXiv arXiv 2017
-
[17]
Guo, Y., Zhang, L., Hu, Y., He, X., Gao, J.: MS-Celeb-1M: A dataset and bench- mark for large-scale face recognition. In: ECCV (2016)
work page 2016
-
[18]
Nonlinear Transformations Against Unlearnable Datasets
Hapuarachchi, T., Lin, J., Xiong, K., Rahouti, M., Ost, G.: Nonlinear transforma- tions against unlearnable datasets. arXiv:2406.02883 (2024)
work page internal anchor Pith review Pith/arXiv arXiv 2024
- [19]
- [20]
-
[21]
Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: CVPR (2017)
work page 2017
-
[22]
Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Tech. Rep. 07-49, University of Massachusetts, Amherst (2007) 16 B. Oh et al
work page 2007
-
[23]
Huang, H., Ma, X., Erfani, S.M., Bailey, J., Wang, Y.: Unlearnable examples: Making personal data unexploitable. In: ICLR (2021)
work page 2021
-
[24]
In: International Symposium on Visual Computing (ISVC)
Hukkelås, H., Mester, R., Lindseth, F.: DeepPrivacy: A generative adversarial net- work for face anonymization. In: International Symposium on Visual Computing (ISVC). pp. 565–578 (2019)
work page 2019
-
[25]
Knoche, M., Hormann, S., Rigoll, G.: Cross-quality LFW: A database for analyzing cross-resolution image face recognition in unconstrained environments. In: IEEE FG (2021)
work page 2021
- [26]
-
[27]
Liu, Y., Xu, K., Chen, X., Sun, L.: Stable unlearnable example: Enhancing the robustness of unlearnable examples via stable error-minimizing noise. In: AAAI (2024)
work page 2024
-
[28]
Liu, Z., Zhao, Z., Larson, M.: Image shortcut squeezing: Countering perturbative availability poisons with compression. In: ICML (2023)
work page 2023
-
[29]
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. In: ICLR (2018)
work page 2018
-
[30]
Maximov, M., Elezi, I., Leal-Taixé, L.: CIAGAN: Conditional identity anonymiza- tion generative adversarial networks. In: CVPR (2020)
work page 2020
-
[31]
Ren, J., Xu, H., Wan, Y., Ma, X., Sun, L., Tang, J.: Transferable unlearnable examples. In: ICLR (2023)
work page 2023
- [32]
-
[33]
Sandoval-Segura, P., Singla, V., Geiping, J., Goldblum, M., Goldstein, T., Jacobs, D.W.: Autoregressive perturbations for data poisoning. In: NeurIPS (2022)
work page 2022
-
[34]
Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: A unified embedding for face recognition and clustering. In: CVPR (2015)
work page 2015
-
[35]
Shafahi, A., Huang, W.R., Najibi, M., Suciu, O., Studer, C., Dumitras, T., Gold- stein, T.: Poison frogs! targeted clean-label poisoning attacks on neural networks. In: NeurIPS (2018)
work page 2018
-
[36]
Shan, S., Wenger, E., Zhang, J., Li, H., Zheng, H., Zhao, B.Y.: Fawkes: Protecting privacy against unauthorized deep learning models. In: USENIX Security (2020)
work page 2020
-
[37]
Sharma, Y., Ding, G.W., Brubaker, M.A.: On the effectiveness of low frequency perturbations. In: IJCAI (2019)
work page 2019
-
[38]
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2015)
work page 2015
-
[39]
Sun, W., Liu, Y., Yan, Z., Xu, K., Sun, L.: Medical unlearnable examples: Secur- ing medical data from unauthorized training via sparsity-aware local masking. In: ICML Workshop (2024)
work page 2024
-
[40]
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I.J., Fergus, R.: Intriguing properties of neural networks. In: ICLR (2014)
work page 2014
- [41]
-
[42]
Wang, H., Wu, X., Huang, Z., Xing, E.P.: High-frequency component helps ex- plainthegeneralizationofconvolutionalneuralnetworks.In:CVPR.pp.8684–8694 (2020)
work page 2020
-
[43]
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process.13(4), 600–612 (2004)
work page 2004
-
[44]
In: CVPR (2016) Privacy Protection Surviving Extraction Pipeline 17
Yang,S.,Luo,P.,Loy,C.C.,Tang,X.:WIDERFACE:Afacedetectionbenchmark. In: CVPR (2016) Privacy Protection Surviving Extraction Pipeline 17
work page 2016
- [45]
-
[46]
Learning Face Representation from Scratch
Yi, D., Lei, Z., Liao, S., Li, S.Z.: Learning face representation from scratch. arXiv:1411.7923 (2014)
work page internal anchor Pith review Pith/arXiv arXiv 2014
-
[47]
Yin, B., Wang, W., Yao, T., Guo, J., Kong, Z., Ding, S., Li, J., Liu, C.: Adv- Makeup: A new imperceptible and transferable attack on face recognition. In: IJ- CAI (2021)
work page 2021
-
[48]
Yin, D., Gontijo Lopes, R., Shlens, J., Cubuk, E.D., Gilmer, J.: A fourier perspec- tive on model robustness in computer vision. In: NeurIPS (2019)
work page 2019
-
[49]
Yu, D., Zhang, H., Chen, W., Yin, J., Liu, T.Y.: Availability attacks create short- cuts. In: KDD (2022)
work page 2022
-
[50]
Zhang, K., Zhang, Z., Li, Z., Qiao, Y.: Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process. Lett.23(10), 1499–1503 (2016)
work page 2016
- [51]
-
[52]
Zhang, Z., Zhang, J., Zhang, K., Zhou, W., Zhang, W., Yu, N.: Segue: Side- information guided generative unlearnable examples for facial privacy protection in real world. In: ICASSP (2025)
work page 2025
-
[53]
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., Torralba, A.: Places: A 10 million image database for scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. (2017)
work page 2017
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.