REVIEW 3 major objections 5 minor 54 references
HoneypotNet: Backdoor Attacks Against Model Extraction
T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read Replacing a model's output layer with a honeypot layer lets defenders inject backdoors into stolen substitute models.
desk verdict Genuinely new 'attack as defense' with surprisingly strong hard-label results; the core UAP-transfer assumption is fragile and the paper undersells that fragility. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the honeypot layer plus a universal adversarial perturbation (UAP) used as a trigger. The honeypot layer $H(x)=W\cdot F_{\text{feat}}(x)+b$ replaces the victim's classification head, so it controls only the outputs and adds few parameters. The trigger $\delta$ is found on a lightweight shadow model (ResNet18) and then reinforced by fine-tuning the honeypot layer so that the association 'input plus $\delta$ maps to target class' behaves like a normal function. The bi-level optimization formalizes this: the upper level asks the honeypot layer to keep clean accuracy while mapping triggered inputs to the target class, and the lower level trains the shadow model and updates the trigger. The paper's argument is that adversarial vulnerability transfers: a UAP optimized on the shadow model transfers through the poisonous probability vectors into any substitute model trained on them.
What would settle it
Train a substitute model from HoneypotNet-protected outputs using only hard labels and strong robustification, such as adversarial training or defensive distillation, on a dataset where the paper reports high attack success; if the attack success rate falls to the undefended baseline while clean accuracy stays high, the transferability claim fails. A simpler check is to repeat the KnockoffNets extraction with a vision-transformer substitute, where UAP transfer is typically weak.
Extended reading notes
Core claim
The central claim is that a backdoor can be injected into an extracted substitute model without ever poisoning the attacker's images or retraining the victim. The method replaces the victim's classification layer with a honeypot layer, a single fully connected layer that maps the victim's feature vector to a probability vector. A bi-level optimization loop alternates between training a shadow model on the honeypot layer's outputs, generating a universal adversarial perturbation that makes the shadow model predict the target class, and fine-tuning the honeypot layer to preserve normal accuracy while becoming sensitive to that perturbation. Because the substitute model is trained on the honeypot layer's poisoned probability vectors, it learns the perturbation as a normal input-output association and therefore reproduces the backdoor. The authors report that this works under soft-label extraction and also under the hard-label BlackBox Dissector, with verification accuracy and attack success rate both far above undefended baselines.
Load-bearing premise
The entire defense depends on the assumption that a universal adversarial perturbation computed on a small shadow model will transfer to the substitute model the attacker actually trains, using the poisoned probability vectors as the conduit.
Editorial extensions
If this is right
- A model owner who deploys HoneypotNet can verify ownership by querying a suspect model with triggered inputs: if the suspect predicts the target class at an unusually high rate, it was likely extracted from the protected API.
- The same trigger acts as a reverse attack key, forcing a stolen substitute model to misclassify any triggered input as the target class, which can deter or damage an attacker's deployment.
- The defense requires no retraining of the victim and only a lightweight honeypot layer, so it can be applied to pre-trained models with low computational overhead.
- The method remains effective under hard-label extraction, where the attacker trains on discrete labels rather than full probability vectors.
- Trigger size is a tunable trade-off: larger triggers raise attack success but also begin to perturb the victim's own predictions.
Reading between the lines
- Because the trigger is a UAP, the scheme inherits the known fragility of adversarial transferability: an attacker who trains with a very different architecture (for instance a vision transformer) or uses adversarial training on the substitute may break the backdoor, a risk the paper acknowledges when reporting lower success on DenseNet121.
- The 'attack as defense' framing suggests an escalation dynamic: extraction defenses that poison outputs could push attackers toward hard-label-only or output-filtering strategies, which in turn may make extraction harder or more expensive even when the backdoor fails.
- The UAP-based trigger's resistance to detection and pruning suggests that backdoor defenses based on finding minimal norm patterns may need to be re-evaluated against triggers that align with the model's existing decision boundaries.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes HoneypotNet, a "attack as defense" technique against model extraction. The method replaces the victim model's classification layer with a honeypot layer and uses a shadow model plus a universal adversarial perturbation (UAP) trigger inside a bi-level optimization loop. The honeypot layer is fine-tuned to keep normal predictions while making triggered inputs map to a target class, and the paper claims that a substitute model trained on the honeypot layer's soft-label outputs inherits a backdoor. Experiments on CIFAR10, CIFAR100, CUBS200, and Caltech256 with five extraction attacks, including a hard-label attack, report verification accuracies from 52.29% to 92.61% and attack success rates from 56.99% to 92.35%, with additional tests on trigger size, substitute architecture, backdoor detection, and neuron pruning.
Significance. If the mechanism holds, HoneypotNet introduces a genuinely new defense paradigm: instead of only detecting or watermarking extraction, the defender uses the extraction process itself to implant a controllable backdoor, enabling both ownership verification and an active reverse attack. The paper's strengths are its broad experimental coverage (five extraction attacks, four datasets, multiple substitute architectures, hard-label setting) and its evaluation against two backdoor defenses (Cognitive Distillation and Reconstructive Neuron Pruning). The main weakness is that the load-bearing step, transfer of the UAP-triggered backdoor from the honeypot layer to a substitute trained only on clean pairs, is justified empirically through UAP transferability and shows noticeable sensitivity across architectures. The significance is therefore conditional: the idea is valuable, but the current evidence does not yet establish that the backdoor is reliably inherited beyond the specific training procedures and architecture ranges tested.
major comments (3)
- [Experimental Setup and Table 2] The claim that HoneypotNet preserves the victim model's utility is not directly supported by the main results. Table 2 reports Accc for the substitute models, not for the protected model after the classification layer is replaced by the honeypot layer. The original victim accuracies (91.56%, 71.57%, 77.11%, and 78.44%) are given, but the clean accuracy of the HoneypotNet-protected model is never tabulated. Please add the protected model's clean test accuracy for each dataset and trigger configuration so that the reader can verify that the honeypot layer does not degrade the served model.
- [Eq. (6), trigger masking] The mask operation in Eq. (6) appears inverted. If M is the binarized trigger mask that restricts the trigger to a specific location, then the triggered input should be (1-M)⊙x + M⊙δ, not M⊙x + (1-M)⊙δ as written. As printed, the equation places the trigger everywhere except the masked region. This needs to be corrected or the definition of M clarified, because the trigger generation step is central to the method's reproducibility.
- [Section 'Finetuning the Honeypot Layer', Eqs. (4)-(6), and Table 3] The backdoor transfer mechanism is the load-bearing step: a substitute trained only on clean pairs (x, H(x)) must inherit a trigger that was optimized on a shadow model and never appears in the substitute's training set. The paper asserts this via UAP transferability, but does not provide a controlled study of when transfer fails. Table 3 already shows architecture sensitivity (e.g., DenseNet121 reaches only 51.68% ASR on CIFAR10), and the shadow model and shadow dataset differ from the attacker's architecture and transfer set. Please add ablations over attacker-side training loss (e.g., KL divergence vs. cross-entropy), robust/adversarial training, shadow model architecture, and shadow model ensembles. These experiments would delimit the conditions under which the central claim holds and would separate the contribution of the BLO loop from simply training H to be adversarially vulnerable.
minor comments (5)
- [Algorithm 1, line 5] The loss function L' used in the extraction simulation step is never defined; please specify whether it is cross-entropy, KL divergence, or another objective.
- [Table 1] The symbols '!' and '%' used in Table 1 are not defined in the caption or in the surrounding text, which makes the comparison table hard to interpret.
- [Abstract and Introduction] The abstract says HoneypotNet protects against 'any malicious users' who attempt to extract the victim model, but the experiments cover only the standard soft-label and hard-label extraction pipelines. Please scope the claim to the tested threat model.
- [Figure 3] The curve labeled 'HoneypotNet' in Figure 3 is not defined in the caption; clarify whether it shows the protected model's clean accuracy after the honeypot layer is inserted.
- [Section 'Finetuning the Honeypot Layer'] The term 'universal adversarial perturbation' is used for a trigger optimized on a small verification set Dv and constrained by a fixed mask and location; this is not universal in the original sense of Moosavi-Dezfooli et al. and should be justified or renamed.
Circularity Check
HoneypotNet's backdoor transfer is an externally measured empirical result, not a prediction forced by construction; no load-bearing circularity found.
full rationale
The paper's central mechanism—a UAP δ optimized on a shadow model and a honeypot layer H fine-tuned so that H(x+δ) predicts the target class, with the backdoor expected to transfer to a substitute trained only on clean H(x)—is an empirical design, not a derivation that reduces to its own inputs. The BLO objective in Eqs. (4)-(6) optimizes H and δ on the defender's shadow set, but the reported ASR is measured on substitute models trained from scratch by five different extraction methods, including the hard-label BlackBox Dissector, and across four substitute architectures. The strong variation in results (e.g., DenseNet121 drops to 51.68% ASR on CIFAR10 while VGG16 reaches 97.16%) demonstrates that the evaluation is not a tautology: if the high ASR were forced by construction, architecture and extraction method would not matter. The transferability of the UAP is an explicit empirical assumption, acknowledged in the paper ('Given that our backdoor trigger is based on the transferability of UAP, its effectiveness can vary depending on the model architecture'), and its fragility is a robustness/correctness concern rather than a circularity. No load-bearing claim is justified only by a self-citation: the citations to Huang et al. (2023) and Li et al. (2023a), which share an author with this paper, are used solely as external evaluation tools (backdoor detection and neuron-pruning defenses), not to establish the central transfer mechanism. Thus no circular step is identified.
Assumptions & free parameters
free parameters (4)
- Universal Adversarial Perturbation trigger delta =
Not reported; optimized via Eq. (6)
- Trigger mask size and location =
6x6 for CIFAR10/100; 28x28 for Caltech256/CUBS200; upper-left corner, 4px offset
- Target class y_target =
Last class of each dataset
- Ownership verification threshold =
10%
assumptions (5)
- domain assumption Adversarial perturbations transfer across architectures and training procedures (UAP transferability).
- domain assumption The attacker trains a substitute model on soft-label output vectors H(x) using a standard loss such as cross-entropy or KL divergence.
- domain assumption Only the output layer can be changed, and the victim's training data is inaccessible, so defensive fine-tuning must use a proxy shadow set from a different distribution, CC3M.
- domain assumption Attacker queries are indistinguishable from legitimate queries, so no query filtering is used.
- ad hoc to paper A universal adversarial perturbation can act as a poisoning-free backdoor trigger without explicit injection into training images.
Cite this review
Pith. "Pith review of HoneypotNet: Backdoor Attacks Against Model Extraction." pith.science (2026). https://pith.science/paper/5DJD62XF
@misc{pith2026250101090,
author = {Pith},
title = {Pith review of: HoneypotNet: Backdoor Attacks Against Model Extraction},
year = {2026},
howpublished = {\url{https://pith.science/paper/5DJD62XF}},
note = {Machine review of arXiv:2501.01090}
}
read the original abstract
Model extraction attacks are one type of inference-time attacks that approximate the functionality and performance of a black-box victim model by launching a certain number of queries to the model and then leveraging the model's predictions to train a substitute model. These attacks pose severe security threats to production models and MLaaS platforms and could cause significant monetary losses to the model owners. A body of work has proposed to defend machine learning models against model extraction attacks, including both active defense methods that modify the model's outputs or increase the query overhead to avoid extraction and passive defense methods that detect malicious queries or leverage watermarks to perform post-verification. In this work, we introduce a new defense paradigm called attack as defense which modifies the model's output to be poisonous such that any malicious users that attempt to use the output to train a substitute model will be poisoned. To this end, we propose a novel lightweight backdoor attack method dubbed HoneypotNet that replaces the classification layer of the victim model with a honeypot layer and then fine-tunes the honeypot layer with a shadow model (to simulate model extraction) via bi-level optimization to modify its output to be poisonous while remaining the original performance. We empirically demonstrate on four commonly used benchmark datasets that HoneypotNet can inject backdoors into substitute models with a high success rate. The injected backdoor not only facilitates ownership verification but also disrupts the functionality of substitute models, serving as a significant deterrent to model extraction attacks.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
Chen, H.; Fu, C.; Zhao, J.; and Koushanfar, F. 2021. Proflip: Targeted trojan attack with progressive bit flips. In ICCV
work page 2021
-
[4]
Chen, K.; Lou, X.; Xu, G.; Li, J.; and Zhang, T. 2022. Clean-image backdoor: Attacking multi-label models with poisoned labels only. In ICLR
work page 2022
-
[5]
Chen, X.; Liu, C.; Li, B.; Lu, K.; and Song, D. 2017. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526
arXiv 2017
- [6]
-
[7]
Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; and Bengio, Y. 2014. Generative adversarial nets. In NIPS
work page 2014
-
[8]
J.; Shlens, J.; and Szegedy, C
Goodfellow, I. J.; Shlens, J.; and Szegedy, C. 2015. Explaining and harnessing adversarial examples. In ICLR
work page 2015
Show all 54 references
-
[9]
Griffin, G.; Holub, A.; and Perona, P. 2007. Caltech-256 object category dataset
2007
-
[10]
Gu, T.; Dolan-Gavitt, B.; and Garg, S. 2017. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.06733
2017 arXiv
-
[11]
He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016. Deep residual learning for image recognition. In CVPR
2016
-
[12]
Ho, J.; Jain, A.; and Abbeel, P. 2020. Denoising diffusion probabilistic models. In NeurIPS
2020
-
[13]
M.; and Bailey, J
Huang, H.; Ma, X.; Erfani, S. M.; and Bailey, J. 2023. Distilling Cognitive Backdoor Patterns within an Image. In ICLR
2023
-
[14]
Jha, R.; Hayase, J.; and Oh, S. 2023. Label poisoning is all you need. NeurIPS
2023
-
[15]
A.; Chandrasekaran, V.; and Papernot, N
Jia, H.; Choquette-Choo, C. A.; Chandrasekaran, V.; and Papernot, N. 2021. Entangled watermarks as a defense against model extraction. In USENIX Security Symposium
2021
-
[16]
Jindal, A.; Goyal, V.; Anand, S.; and Arora, C. 2024. Army of Thieves: Enhancing Black-Box Model Extraction via Ensemble based sample selection. In WACV
2024
-
[17]
Juuti, M.; Szyller, S.; Marchal, S.; and Asokan, N. 2019. PRADA: protecting against DNN model stealing attacks. In IEEE EuroS&P
2019
-
[18]
Kariyappa, S.; Prakash, A.; and Qureshi, M. 2021. MAZE: Data-Free Model Stealing Attack Using Zeroth-Order Gradient Estimation. In CVPR
2021
-
[19]
Kariyappa, S.; and Qureshi, M. K. 2020. Defending Against Model Stealing Attacks with Adaptive Misinformation. In CVPR
2020
-
[20]
Karmakar, P.; and Basu, D. 2023. Marich: A Query-efficient Distributionally Equivalent Model Extraction Attack using Public Data. arXiv preprint arXiv:2302.08466
2023 arXiv
-
[21]
Kesarwani, M.; Mukhoty, B.; Arya, V.; and Mehta, S. 2018. Model extraction warning in mlaas paradigm. In ACSAC
2018
-
[22]
Krizhevsky, A.; Hinton, G.; et al. 2009. Learning multiple layers of features from tiny images
2009
-
[23]
Kullback, S.; and Leibler, R. A. 1951. On information and sufficiency. The annals of mathematical statistics
1951
-
[24]
Li, Y.; Lyu, X.; Ma, X.; Koren, N.; Lyu, L.; Li, B.; and Jiang, Y.-G. 2023 a . Reconstructive neuron pruning for backdoor defense. In ICML
2023
-
[25]
Li, Y.; Zhu, L.; Jia, X.; Jiang, Y.; Xia, S.-T.; and Cao, X. 2022. Defending against model stealing via verifying embedded external features. In AAAI
2022
-
[26]
Li, Y.; Zhu, M.; Yang, X.; Jiang, Y.; Wei, T.; and Xia, S.-T. 2023 b . Black-box dataset ownership verification via backdoor watermarking. IEEE TIFS
2023
-
[27]
Lin, Z.; Xu, K.; Fang, C.; Zheng, H.; Ahmed Jaheezuddin, A.; and Shi, J. 2023. QUDA: Query-Limited Data-Free Model Extraction. In ACM Asia CCS
2023
-
[28]
Liu, Y.; Chen, X.; Liu, C.; and Song, D. 2017. Delving into transferable adversarial examples and black-box attacks. In ICLR
2017
-
[29]
Liu, Y.; Lee, W.-C.; Tao, G.; Ma, S.; Aafer, Y.; and Zhang, X. 2019. Abs: Scanning neural networks for back-doors by artificial brain stimulation. In ACM SIGSAC CCS
2019
-
[30]
Liu, Y.; Wen, R.; Backes, M.; and Zhang, Y. 2024. Efficient Data-Free Model Stealing with Label Diversity. arXiv preprint arXiv:2404.00108
2024 arXiv
-
[31]
Lv, P.; Ma, H.; Chen, K.; Zhou, J.; Zhang, S.; Liang, R.; Zhu, S.; Li, P.; and Zhang, Y. 2024. MEA-Defender: A Robust Watermark against Model Extraction Attack. In IEEE S&P
2024
-
[32]
Moosavi-Dezfooli, S.-M.; Fawzi, A.; Fawzi, O.; and Frossard, P. 2017. Universal adversarial perturbations. In CVPR
2017
-
[33]
Orekondy, T.; Schiele, B.; and Fritz, M. 2019 a . Knockoff Nets: Stealing Functionality of Black-Box Models. In CVPR
2019
-
[34]
Orekondy, T.; Schiele, B.; and Fritz, M. 2019 b . Prediction Poisoning: Towards Defenses Against DNN Model Stealing Attacks. In ICLR
2019
-
[35]
Pal, S.; Gupta, Y.; Shukla, A.; Kanade, A.; Shevade, S.; and Ganapathy, V. 2020. ACTIVETHIEF: Model Extraction Using Active Learning and Unannotated Public Data. In AAAI
2020
-
[36]
B.; and Swami, A
Papernot, N.; McDaniel, P.; Goodfellow, I.; Jha, S.; Celik, Z. B.; and Swami, A. 2017. Practical black-box attacks against machine learning. In ACM AsiACCS
2017
-
[37]
Ribeiro, M.; Grolinger, K.; and Capretz, M. A. 2015. Mlaas: Machine learning as a service. In IEEE ICMLA
2015
-
[38]
Rong, D.; Shen, S.; Fu, X.; Qian, P.; Chen, J.; He, Q.; Fu, X.; and Wang, W. 2024. Clean-image Backdoor Attacks. arXiv preprint arXiv:2403.15010
2024
-
[39]
Russakovsky, O.; Deng, J.; Su, H.; Krause, J.; Satheesh, S.; Ma, S.; Huang, Z.; Karpathy, A.; Khosla, A.; Bernstein, M.; et al. 2015. Imagenet large scale visual recognition challenge. IJCV
2015
-
[40]
Sanyal, S.; Addepalli, S.; and Babu, R. V. 2022. Towards data-free model stealing in a hard label setting. In CVPR
2022
-
[41]
S.; and Goldstein, T
Shafahi, A.; Najibi, M.; Xu, Z.; Dickerson, J.; Davis, L. S.; and Goldstein, T. 2020. Universal adversarial training. In AAAI
2020
-
[42]
Sharma, P.; Ding, N.; Goodman, S.; and Soricut, R. 2018. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In ACL
2018
-
[43]
Szegedy, C.; Zaremba, W.; Sutskever, I.; Bruna, J.; Erhan, D.; Goodfellow, I.; and Fergus, R. 2014. Intriguing properties of neural networks. In ICLR
2014
-
[44]
Tan, J.; Zhong, N.; Qian, Z.; Zhang, X.; and Li, S. 2023. Deep Neural Network Watermarking against Model Extraction Attack. In ACM MM
2023
-
[45]
Z.; and Chen, Y
Tang, M.; Dai, A.; DiValentin, L.; Ding, A.; Hass, A.; Gong, N. Z.; and Chen, Y. 2024. MODELGUARD: Information-Theoretic Defense Against Model Extraction Attacks. In USENIX Security Symposium
2024
-
[46]
Tang, R.; Du, M.; Liu, N.; Yang, F.; and Hu, X. 2020. An embarrassingly simple approach for trojan attack in deep neural networks. In ACM SIGKDD
2020
-
[47]
Wah, C.; Branson, S.; Welinder, P.; Perona, P.; and Belongie, S. 2011. The caltech-ucsd birds-200-2011 dataset
2011
-
[48]
Wang, Y.; Li, J.; Liu, H.; Wang, Y.; Wu, Y.; Huang, F.; and Ji, R. 2022. Black-Box Dissector: Towards Erasing-based Hard-Label Model Stealing Attack. In ECCV
2022
-
[49]
Wang, Y.; and Lin, X. 2022. Enhance model stealing attack via label refining. In ICSP. IEEE
2022
-
[50]
Yu, H.; Yang, K.; Zhang, T.; Tsai, Y.-Y.; Ho, T.-Y.; and Jin, Y. 2020. CloudLeak: Large-Scale Deep Learning Models Stealing Through Adversarial Examples. In NDSS
2020
-
[51]
Yuan, X.; Chen, K.; Huang, W.; Zhang, J.; Zhang, W.; and Yu, N. 2024. Data-Free Hard-Label Robustness Stealing Attack. In AAAI
2024
-
[52]
Zhao, S.; Chen, K.; Hao, M.; Zhang, J.; Xu, G.; Li, H.; and Zhang, T. 2023. Extracting Cloud-based Model with Prior Knowledge. arXiv preprint arXiv:2306.04192
2023 arXiv
-
[53]
Zhao, Y.; Deng, X.; Liu, Y.; Pei, X.; Xia, J.; and Chen, W. 2024. Fully Exploiting Every Real Sample: SuperPixel Sample Gradient Model Stealing. In CVPR
2024
-
[54]
Zhou, M.; Wu, J.; Liu, Y.; Liu, S.; and Zhu, C. 2020. DaST: Data-free Substitute Training for Adversarial Attacks. In CVPR
2020
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.