Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

Architectural Backdoors in Deep Learning: A Survey of Vulnerabilities, Detection, and Defense

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read The survey argues that architectural backdoors — malicious logic wired directly into a neural network's computational graph — form a distinct attack class that survives weight resets, data cleansing, and clean retraining.

desk verdict Useful survey of architectural backdoors, but the central 'immune to standard mitigation' claim is overstated and the paper contradicts itself on the real-world evidence. read the letter →

arxiv 2507.12919 v1 pith:FY6QNQSE submitted 2025-07-17 cs.CR

classification cs.CR
keywords architecturalbackdoorsmodelbackdoorcomputationalgraphattackspersistencesupply-chainsecuritycompilertrojansAutoMLvulnerabilitiesdetectionanddefense
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The survey argues that a distinct class of AI attack has been under-examined: architectural backdoors, which wire malicious logic directly into a neural network's computational graph rather than into its training data or weights. Because the trigger's routing and gating logic lives in the graph topology — an extra branch, a gate layer, a rerouted edge — it can survive weight re-initialization, data cleansing, and clean retraining, which the authors characterize as immunity to standard mitigation. The evidence they consolidate includes a controlled study in which a structural Trojan kept a 96.2% attack-success rate after full weight re-initialization plus clean ImageNet retraining, while a classic weight-based trigger fell below 2%; deployment-stage insertions that splice a hidden subgraph into a trained model with over 99% attack success; compiler-level insertions invisible to source audit; and AutoML pipelines steered to produce backdoored architectures on clean data. The paper then maps the attack surface into a twelve-category taxonomy, assesses detection and removal methods against it, and concludes that if the thesis holds, the model supply chain itself is an attack surface that weight- and data-oriented defenses cannot close.

What carries the argument

The load-bearing object is the malicious subgraph, or Model Architectural Backdoor (MAB): a structural addition or modification to the computational graph that stays dormant on benign inputs and overrides normal inference when a trigger fires. Its persistence mechanism is that the trigger's routing and gating logic is encoded in graph topology — extra branches, gate layers, rerouted edges, shareable hidden state — rather than in learned parameters, so the backdoor is weight-agnostic and survives re-initialization, pruning, and clean retraining. The organizing scaffold is a taxonomy that splits the attack surface into four families (sub-network attacks, compiler-based backdoors, AutoML/NAS backdoors, and hybrid multi-stage attacks) and further into twelve subcategories, each with its own threat model, stealth properties, and detection barriers. The taxonomy does the argument's work by showing that persistence is one claim realized through many mechanisms, and by mapping which defense each variant evades.

What would settle it

Two checks would settle the matter. First, insert backdoors from each of the four attack families into a diverse set of architectures, fully re-initialize and clean-retrain them on ImageNet-scale data, and measure attack-success retention: if rates collapse toward zero for most of the set, the immunity claim fails. Second, attempt to trigger the dormant branches in a random sample of the models flagged by the Protect AI scanner: if none of the 352,000 flags can be activated by any input, the real-world urgency claim weakens.

Watch

Extended reading notes

Core claim

The paper's central claim is that a backdoor can be implanted in a third location besides data and weights — the network's architecture — and that this placement changes the security calculus. Once a malicious sub-graph is embedded, the routing or gating logic is hard-wired into the graph topology and is weight-agnostic: re-initializing or retraining the weights leaves those fixed routes untouched, so the backdoor re-emerges as soon as training converges. Reported evidence for this persistence includes a controlled study in which an architectural Trojan retained a 96.2% attack-success rate after full weight re-initialization and clean ImageNet retraining while a weight-based baseline fell below 2%; a deployment-stage attack that splices a hidden subgraph into a trained CNN with over 99% attack success and less than 2% clean-accuracy loss; intermediate-representation insertions made at compile time that escape source-level review; and AutoML reward poisoning that yields backdoored architectures trained entirely on clean data. The authors also survey the detection and mitigation toolbox — static graph inspection, dynamic fuzzing and trigger inversion, explainability and meta-analysis, partial formal verification, subgraph excision, adversarial unlearning, runtime monitoring, and supply-chain assurance — and argue that each technique is either evaded by distributed or stealth triggers or does not yet scale, so no single defense closes the threat.

Load-bearing premise

The load-bearing premise is that persistence results demonstrated on a small set of hand-crafted test architectures generalize to the whole threat class, and that vendor-flagged models are representative of real exploits; the paper itself notes the scanner findings are suspicious architectural backdoors rather than confirmed triggers.

Editorial extensions

If this is right

  • If architectural backdoors persist after clean retraining as claimed, then standard model-repair workflows such as data cleansing, fine-tuning, weight resets, and pruning cannot be trusted to sanitize a suspect model, so removal must become graph-level surgery.
  • Model repositories and export pipelines become critical trust boundaries: logic inserted at ONNX or TensorFlow serialization time is invisible in source code, which means distributed models should be treated as untrusted artifacts until their graphs are compared against a reference.
  • Detection has to be layered — static diffing, dynamic probing, explainability, and bounded formal verification — because each family alone misses distributed, compiler-level, or dormant triggers.
  • Existing backdoor benchmarks are skewed toward poisoned data and weights and understate the structural threat; adding graph-level Trojans of the kind first distributed in TrojAI Round 15 is the direction the field needs, with compiler-time and multi-branch scenarios still untested.
  • Supply-chain assurances — signed reproducible builds, IR differencing, and sandboxed loading of untrusted artifacts — follow directly as the practical defensive corollary, because a tainted toolchain can silently re-insert an excised backdoor.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: if persistence holds at scale, the legal and regulatory line between a model's weights and its executable logic blurs, so model distribution may come to require the same signing, provenance, and liability rules as software binaries.
  • My inference: the persistence claim implies a testable corollary for transfer learning — architectural backdoors should survive downstream domain fine-tuning better than weight backdoors — which the survey lists only as a benchmark gap but existing results already hint at.
  • My inference: the within-batch leakage variant turns architectural backdoors into a privacy threat for shared inference services, so scanning production model hubs for cross-example data-flow edges would quantify how many deployed models contain such structures.
  • My inference: a comparative replication study across attack families — measuring attack-success retention after clean retraining for single-layer, subgraph, and distributed triggers — would show whether immunity is uniform or reserved for the more complex variants.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper surveys architectural backdoors in deep learning, a threat class in which malicious logic is embedded in the model's computational graph rather than in data or weights. It presents a taxonomy of four attack families (design/subgraph, compiler, AutoML/NAS, and hybrid) expanded to twelve subcategories, reviews detection techniques (static graph inspection, dynamic probing, explainability, formal verification), mitigation approaches (subgraph pruning, unlearning, attention distillation, runtime monitoring, supply-chain assurance), and discusses benchmark gaps and a research roadmap. The paper's central claim is that architectural backdoors are immune to standard data- and weight-centric mitigations and persist after weight re-initialization and clean retraining, making them an urgent and under-recognized threat.

Significance. The survey addresses a real gap: prior backdoor surveys concentrate on data and weight poisoning, while structural/compiler-level attacks are scattered across recent papers and technical reports. The manuscript's taxonomy and side-by-side tables (Tables 2, 3, 4, 5, 7) are useful organizing devices, and its discussion of benchmark blind spots (§6) and supply-chain/hardware issues (§5.5, §7.4) is forward-looking. The authors are also appropriately cautious in §2.6 about treating scanner findings as unconfirmed. However, the paper's headline persistence/immunity claim is stated more strongly than its own cited evidence warrants, and two internal contradictions (§2.6 vs. §3.1.2; §4.5 vs. §6.1) undermine the reader's ability to trust the survey's conclusions. With careful revision and calibration of the central claim, this would be a valuable reference for the security community.

major comments (3)
  1. [Abstract, §1, §2.1] The abstract and introduction assert that architectural backdoors are 'immune to standard mitigation techniques like data cleansing, weight resets, or fine-tuning alone' and that they 'persist even after clean retraining.' This claim is contradicted by material the manuscript itself surveys: §4.1 acknowledges that fine-pruning can disable simple single-path backdoors (citing [46]), and §5.1 reports that excising the checkerboard branch reduced attack success from 100% to 2% (citing [6]). The persistence/immunity statement is defensible only for specific subgraph/distributed mechanisms and only against the listed mitigations when used in isolation. Please restate the central claim with explicit scope conditions and define 'standard mitigation' precisely; otherwise the survey's headline conclusion overgeneralizes from a handful of architectures.
  2. [§3.1.2 vs. §2.6] Section 2.6 explicitly says of Protect AI's Guardian results that 'no public analysis has yet demonstrated that the dormant branches can be triggered in practice' and that the paper treats them as 'suspicious architectural backdoors rather than confirmed exploits.' Section 3.1.2, however, calls the Protect AI reports 'the first confirmed examples of such compiler-level insertions in the wild' and states that 'these subgraphs... lay dormant until specific input triggers were received.' These two characterizations are mutually incompatible. The paper must choose one position and adjust the other; the current wording overstates the evidence at the section level while the caveat is relegated to §2.6.
  3. [§6.1 vs. §4.5] Section 4.5 states that TrojAI 'have not extensively tested hidden architectural or compiler-level insertions,' while §6.1 claims that 'TrojAI Round 15 was the first public benchmark to distribute ONNX binaries that contain hidden branches requiring structural detection or neutralization.' If Round 15 introduced structural branches, the earlier statement needs a qualifier (e.g., 'prior rounds' or 'not extensively' in the sense of multi-branch or compiler-time attacks); as written, the two sentences give contradictory information about the benchmark's coverage.
minor comments (6)
  1. [§3.1.2] There is a typo 'More recnt work' that should read 'More recent work,' and the sentence about DynamicQuantizeLinear is missing a period before the following 'Recent findings...' sentence.
  2. [§3.1.3] The acronym 'EV AS' should be written as one token (e.g., 'EVAS') and should be expanded at first use; the current spacing and lack of expansion make the term confusing.
  3. [§7.1] The phrase 'tens of millions of parameters (107)' should be written as '10^7' rather than '107,' which at a glance reads as one hundred seven.
  4. [§4.5] The text defines ASR as 'attack-success-rate reduction,' but ASR conventionally stands for 'attack success rate' itself; please write 'attack success rate (ASR) reduction' or spell out the intended meaning. Also, 'fool-proof' should be 'foolproof.'
  5. [§3.2, Table 2] The relationship between the 'four categories' described in §3 and the 'twelve-subcategory framework' in §3.2 and Table 2 should be clarified at first mention, since the reader initially expects the four categories to be the taxonomy's top level.
  6. [§6.1] The identifier 'PAIT-ONNX-20' appears where the paper elsewhere uses 'PAIT-ONNX-200' (see §3.1.2 and the bibliography); please check and standardize the identifier.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular reasoning found: this survey's claims are inherited from external cited demonstrations, not derived from its own definitions or fitted inputs.

full rationale

This is a survey paper with no original derivation chain, so there is no fitted parameter renamed as a prediction and no first-principles result to reduce to its inputs. The central claim that architectural backdoors evade standard mitigation and persist after clean retraining is asserted in the abstract and Section 1 and is supported by citations to independent empirical studies, including Bober-Irizar et al. [6], Langford et al. [37], HiddenLayer's Shadow Logic [29], and Qi et al. [62]. These are external experimental results rather than assumptions adopted by the survey itself. The only definitional element is Table 1's classification of architectural backdoors as structural modifications 'that persist even under clean retraining,' but this is a taxonomic description of the surveyed attack class, not a derived prediction, and the persistence claim is separately evidenced in Section 2.1 with the reported 96.2% attack-success rate after clean ImageNet retraining. The survey also explicitly hedges the real-world Protect AI examples as 'suspicious architectural backdoors rather than confirmed exploits' in Section 2.6, showing that it does not treat its own framing as proof. Although several load-bearing cited works come from a shared research cluster (e.g., Shumailov, Zhao, Mullins, and Papernot appear in both [6] and [37]), the present authors are not those authors, and the cited works are independently reported empirical demonstrations rather than a self-citation chain that closes the argument. No step in the paper reduces a claimed result to its own definition or to a citation that is itself unverified, so the appropriate finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The survey introduces no new entities, parameters, or measurements. Its claims are second-order summaries of cited attack and defense papers plus vendor reports.

assumptions (3)
  • domain assumption Architectural backdoors can be implanted via architecture design, compiler, or AutoML and persist after clean retraining.
    The survey's central claim rests on demonstrations from Bober-Irizar et al., Langford et al., ImpNet, and the batch-leakage work; these are cited prior works, not re-derived here.
  • domain assumption The threat model grants attackers control of the architecture, compiler/export tool, or AutoML pipeline.
    Sections 2.4 and 3 assume an adversary with such supply-chain access; this is standard in the cited literature but is assumed rather than argued.
  • domain assumption Vendor-reported statistics from Protect AI and HiddenLayer accurately describe real-world model repositories.
    Sections 2.2 and 2.6 rely on vendor blog numbers not independently verified; the paper itself flags them as suspicious rather than confirmed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Architectural Backdoors in Deep Learning: A Survey of Vulnerabilities, Detection, and Defense." pith.science (2026). https://pith.science/paper/FY6QNQSE

@misc{pith2026250712919,
  author       = {Pith},
  title        = {Pith review of: Architectural Backdoors in Deep Learning: A Survey of Vulnerabilities, Detection, and Defense},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FY6QNQSE}},
  note         = {Machine review of arXiv:2507.12919}
}
read the original abstract

Architectural backdoors pose an under-examined but critical threat to deep neural networks, embedding malicious logic directly into a model's computational graph. Unlike traditional data poisoning or parameter manipulation, architectural backdoors evade standard mitigation techniques and persist even after clean retraining. This survey systematically consolidates research on architectural backdoors, spanning compiler-level manipulations, tainted AutoML pipelines, and supply-chain vulnerabilities. We assess emerging detection and defense strategies, including static graph inspection, dynamic fuzzing, and partial formal verification, and highlight their limitations against distributed or stealth triggers. Despite recent progress, scalable and practical defenses remain elusive. We conclude by outlining open challenges and proposing directions for strengthening supply-chain security, cryptographic model attestations, and next-generation benchmarks. This survey aims to guide future research toward comprehensive defenses against structural backdoor threats in deep learning systems.

Figures

Figures reproduced from arXiv: 2507.12919 by the authors.

Figure 1
Figure 1. Three main backdoor insertion points. (a) data poisoning, (b) weight backdoor, and (c) architectural [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Hidden-branch backdoor schematic. (adapted from Bober-Irizar et al. [6]) A light-gray normal path processes inputs conventionally, while a red hidden branch bypasses it. When the trigger pattern is detected, the branch overrides the main features, forcing the attacker-chosen output. 2.2 Motivation for Architectural Threats Architectural backdoors have gained attention for several reasons: • Complex Model Design. As … view at source ↗
Figure 3
Figure 3. Common attack points in the ML supply chain [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: End-to-end threat model for architectural backdoor insertion. A malicious actor can embed a backdoor at three supply-chain stages: tainted architecture code, a compromised NAS pipeline, or a malicious compiler/export tool. Each attack path flows into a seemingly clean …
Figure 5
Figure 5. Figure 5: Separate-path backdoor. An isolated malicious circuit (red dashed path) bypasses normal computa￾tion entirely and forces the output once its trigger fires [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Shared-path backdoor. Malicious computations reside inside the shared layer; the bold red dashed arrow across the top shows the hidden signal, while legitimate processing and label text sit safely below it. Input Main Path Output Trigger A Trigger B [PITH_FULL_IMAGE:f…
Figure 7
Figure 7. Figure 7: Interleaved-path backdoor. A hidden signal (red dashed line) repeatedly diverges from and rejoins the main computation path, with each hop gated by a different partial trigger. All fragments must activate to corrupt the output. Compromised NAS Controller Backdoored Mod…
Figure 8
Figure 8. Figure 8: NAS-based backdoor. A compromised AutoML (NAS) controller emits a backdoored model (red), even when trained on clean data. At inference time, the model behaves normally, unless the hidden trigger is present. 3.1.2 Compiler-Based Backdoors. Modern deep-learning pipeline…
Figure 9
Figure 9. Figure 9: Overall detection pipeline. Defenders can apply one or more methods (static, dynamic, explainability, and (Semi-) formal verification) to reveal suspicious subgraphs. Source Model Graph Malicious Compiler / Export Tool Embedded Malicious Subgraph Compiled Artifact Grap…
Figure 10
Figure 10. Figure 10: Compiler-level insertion and static inspection. A compromised compiler injects a hidden subgraph into the compiled artifact. A subsequent graph-diff step detects the discrepancy by comparing the reference model graph with the potentially compromised artifact. • Struct…
Figure 11
Figure 11. Figure 11: Runtime Backdoor Detection Flow (rotated). Even after offline mitigation, defenders can feed canary inputs or monitor normal traffic for suspicious triggers. If detected, the model can be logged or sandboxed; otherwise, inference continues normally [PITH_FULL_IMAGE:f…
Figure 12
Figure 12. Figure 12: Decision flow for mitigating architectural backdoors. Starting from suspected infiltration, defenders apply static or dynamic checks to pinpoint malicious subgraphs or triggers. They then perform subgraph removal, adversarial unlearning, or attention surgery. If clean…
Figure 13
Figure 13. Figure 13: Conceptual map of open challenges in defending against architectural backdoors. Architectural backdoors present threats that conventional data-poisoning or parameter-centric defenses rarely address, especially when embedded at the compiler or hardware level. The prece…
Figure 14
Figure 14. Figure 14: Research roadmap for architectural-backdoor security.The figure groups open work into four color-banded themes: teal = detection, orange = mitigation, green = benchmarking, and violet = open problems [18, 27, 65]. 8 Conclusion and Outlook [PITH_FULL_IMAGE:figures/ful…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Architectural Backdoors in Vision-Language Model Supply Chains via Representation Steering

    cs.CR 2026-07 conditional novelty 6.0 of 10

    Trigger-gated additive steering vectors embedded in VLM architecture definitions create dormant backdoors that work across VQA, text-to-image, retrieval, and brand/safety biasing without data poisoning.

Reference graph

Works this paper leans on

87 extracted references · 49 canonical work pages · cited by 1 Pith paper

  1. [46]

    Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. 2018. Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks. In Proceedings of the 27th USENIX Security Symposium (USENIX Security ’18) . USENIX Association, Baltimore, MD, USA, 273–290. https://doi.org/10.1007/978-3-030-00470-5_13

  2. [1]

    Gorka Abad, Oguzhan Ersoy, Stjepan Picek, and Aitor Urbieta. 2024. Sneaky Spikes: Uncovering Stealthy Backdoor Attacks in Spiking Neural Networks with Neuromorphic Data. In Proceedings of the 2024 Network and Distributed System Security (NDSS) Symposium . Internet Society, San Diego, CA, USA, 1–20. doi:10.14722/ndss.2024.24334

  3. [2]

    Protect AI. 2025. Six Months of Guardian: 4.47 Million Models Scanned on Hugging Face. https://protectai.com/blog /hugging-face-protect-ai-six-months-in. Accessed 11 Jun 2025

  4. [4]

    Peter Bajcsy and Maxime Bros. 2024. Interactive Simulations of Backdoors in Neural Networks. arXiv:2405.13217. https://arxiv.org/abs/2405.13217 National Institute of Standards and Technology

  5. [5]

    Bazel Project. 2024. Bazel: Reproducible Builds. https://bazel.build/docs/user-manual#reproducible-builds. Online; accessed 22 Apr 2024

  6. [7]

    Castro, Roberto Campbell, Nancy Lau, Octavio Villalobos, Jiaqi Duan, and Alvaro A

    Sebastián R. Castro, Roberto Campbell, Nancy Lau, Octavio Villalobos, Jiaqi Duan, and Alvaro A. Cardenas. 2025. Large Language Models are Autonomous Cyber Defenders. In Proceedings of the IEEE Conference on Artificial Intelligence (CAI) Workshop on Adaptive Cyber Defense . IEEE, Santa Clara, CA, USA, 1–18. https://arxiv.org/abs/2505.04843

  7. [8]

    Bryant Chen, Wilka Carvalho, Nathalie Baracaldo, Heiko Ludwig, Benjamin Edwards, Taesung Lee, Ian Molloy, and Biplav Srivastava. 2019. Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering. In Proceedings of the AAAI Workshop on Artificial Intelligence Safety (SafeAI 2019) co-located with the Thirty-Third AAAI Conference on Artificia...

  8. [9]

    Wei-Jie Chen, Zhi-Hao Li, and Min Zhang. 2025. HGBA: Heterogeneous Graph Backdoor Attack via Relation-Aware Triggers. 21 pages. https://www.arxiv.org/abs/2506.00191

Show all 87 references
  1. [10]

    Fernando Chirigati, Rémi Rampin, Dennis Shasha, and Juliana Freire. 2016. ReproZip: Computational Reproducibility With Ease. In Proceedings of the 2016 ACM SIGMOD International Conference on Management of Data (SIGMOD ’16) . Association for Computing Machinery, San Francisco, ...

  2. [11]

    Edward Chou, Florian Tramèr, Giancarlo Pellegrino, and Dan Boneh. 2020. SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systems. In Proceedings of the 2020 IEEE Security and Privacy Workshops (SPW) . IEEE, San Francisco, CA, USA, 48–54. doi:10 .1109/SPW50...

  3. [12]

    Sheng-Yen Chou, Pin-Yu Chen, and Tsung-Yi Ho. 2023. How to Backdoor Diffusion Models?. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . IEEE / CVF, Vancouver, BC, Canada, 4015–4024. doi:10 .1109/CVPR52729.2023.00391

  4. [13]

    Eleanor Clifford, Ilia Shumailov, Yiren Zhao, Ross Anderson, and Robert Mullins. 2024. ImpNet: Imperceptible and Blackbox-Undetectable Backdoors in Compiled Neural Networks. In Proceedings of the 2024 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) . IEEE Co...

  5. [14]

    Om Suhas Deshmukh, Sankalp Nagaonkar, Achyut Mani Tripathi, and Ashish Mishra. 2024. BadScan: An Ar- chitectural Backdoor Attack on Visual State Space Models. arXiv abs/2411.17283 (2024), 11 pages. https: //arxiv.org/abs/2411.17283 Preprint

  6. [15]

    Andis Draguns, Andrew Gritsevskiy, Sumeet Ramesh Motwani, and Christian Schroeder de Witt. 2024. Unelicitable Backdoors in Language Models via Cryptographic Transformer Circuits. In Advances in Neural Information Processing Systems 37 (NeurIPS 2024) . Curran Associates, Inc., ...

  7. [16]

    Scheirer

    Jacob Dumford and Walter J. Scheirer. 2020. Backdooring Convolutional Neural Networks via Targeted Weight Perturbations. In Proceedings of the 2020 IEEE International Joint Conference on Biometrics (IJCB) . IEEE, Houston, TX, USA, 1–9. doi:10 .1109/IJCB48548.2020.9304875

  8. [17]

    ETH SRI Lab. 2020. ERAN: ETH Robustness Analyzer for Neural Networks. https://github.com/eth-sri/eran. Online; accessed 28 Apr 2025

  9. [18]

    European Parliament and Council of the European Union. 2024. Regulation (EU) 2024/— on Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act) and Amending Certain Union Legislative Acts. Provisional consolidated text adopted 13 March 2024. https://data.consi...

  10. [19]

    Md Omar Faruque, Peter Jamieson, Ahmad Patooghy, and Abdel-Hameed A. Badawy. 2024. Unleashing GHOST: An LLM-Powered Framework for Automated Hardware Trojan Design. arXiv abs/2412.02816 (2024), 11 pages. https://arxiv.org/abs/2412.02816 Preprint

  11. [20]

    Spivak, and Rémy Tuyéras

    Brendan Fong, David I. Spivak, and Rémy Tuyéras. 2019. Backprop as Functor: A Compositional Perspective on Supervised Learning. In Proceedings of the 34th Annual ACM/IEEE Symposium on Logic in Computer Science (LICS 2019) . IEEE, Los Angeles, CA, USA, 1–13. doi:10 .1109/LICS.2...

  12. [21]

    Yuntao Gao, Chengyu Liu, Sanket Bhattad, Min Du, Xia Hu, and Jufeng Yang. 2019. STRIP: A Defence Against Trojan Attacks on Deep Neural Networks. InProceedings of the 35th Annual Computer Security Applications Conference (ACSAC ’19). Association for Computing Machinery, San Jua...

  13. [22]

    Timon Gehr, Matthew Mirman, Dana Drachsler-Cohen, Petar Tsankov, Swarat Chaudhuri, and Martin T. Vechev

  14. [23]

    Kim, Vinod Vaikuntanathan, and Or Zamir

    Shafi Goldwasser, Michael P. Kim, Vinod Vaikuntanathan, and Or Zamir. 2022. Planting Undetectable Backdoors in Machine Learning Models. In Proceedings of the 63rd Annual IEEE Symposium on Foundations of Computer Science (FOCS). IEEE, Denver, CO, USA, 931–942. doi:10.1109/FOCS5...

  15. [24]

    Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. 2017. BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain. arXiv abs/1708.06733 (2017), 14 pages. https://arxiv.org/abs/1708.06733 Preprint

  16. [25]

    Xihe Gu, Greg Fields, Yaman Jandali, Tara Javidi, and Farinaz Koushanfar. 2024. Trojan Cleansing with Neural Collapse. arXiv abs/2411.12914 (2024), 11 pages. https://arxiv.org/abs/2411.12914 Preprint

  17. [26]

    Weinberger

    Chuan Guo, Ruihan Wu, and Kilian Q. Weinberger. 2021. On Hiding Neural Networks Inside Neural Networks. arXiv preprint arXiv:2002.10078, 14 pages. https://arxiv.org/abs/2002.10078 Version 3, May 2021

  18. [27]

    Zhen Guo and Reza Tourani. 2025. DarkMind: Latent Chain-of-Thought Backdoor in Customized LLMs. arXiv abs/2501.18617, 1 (2025), 1–16. arXiv:2501.18617 https://arxiv.org/abs/2501.18617 Preprint, 24 Jan 2025

  19. [28]

    Yingzhe He, Zhili Shen, Chang Xia, Jingyu Hua, Wei Tong, and Sheng Zhong. 2024. SGBA: A Stealthy Scapegoat Back- door Attack against Deep Neural Networks. Computers & Security 136 (2024), 103523. doi:10 .1016/j.cose.2023.103523

  20. [29]

    HiddenLayer. 2024. ShadowLogic: Compiled Model Graph Manipulation for Persistent Backdoors. Technical Advisory Report. https://hiddenlayer.com/research/shadowlogic-backdoors-in-model-graphs

  21. [30]

    IARPA and NIST. 2024. IARPA TrojAI Program — Round 15 Release. https://trojai.nist.gov/TrojAI_Round15.html. Accessed 2025-06-11

  22. [31]

    Hengrui Jia, Sierra Wyllie, Akram Bin Sediq, Ahmed Ibrahim, and Nicolas Papernot. 2025. Backdoor Detection through Replicated Execution of Outsourced Training. In Proceedings of the 3rd IEEE Conference on Secure and Trustworthy Machine Learning (SaTML ’25) . IEEE, Copenhagen, ...

  23. [32]

    Dill, Kyle Julian, and Mykel J

    Guy Katz, Clark Barrett, David L. Dill, Kyle Julian, and Mykel J. Kochenderfer. 2017. Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks. In Proceedings of the 29th International Conference on Computer Aided Verification (CA V) (LNCS, Vol. 10426). Springer, H...

  24. [33]

    Dill, Kyle Julian, and Mykel J

    Guy Katz, Clark Barrett, David L. Dill, Kyle Julian, and Mykel J. Kochenderfer. 2019. Marabou: An Efficient SMT Solver for Verifying Deep Neural Networks. In Proceedings of the 31st International Conference on Computer Aided Verification (CA V). Springer, New York, NY, USA, 443–452

  25. [34]

    Dimitri Kokkonis, Michaël Marcozzi, Emilien Decoux, and Stefano Zacchiroli. 2025. ROSA: Finding Backdoors with Fuzzing . In 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE) . IEEE Computer Society, Los Alamitos, CA, USA, 720–720. https://doi.ieeecompu...

  26. [35]

    Rivera, Heiko Hoffmann, Pingfan Song, Farshad Khorrami, and Richard J

    Soheil Kolouri, Charles E. Rivera, Heiko Hoffmann, Pingfan Song, Farshad Khorrami, and Richard J. Radke. 2021. Meta Neural Trojan Detection. In Proceedings of the 42nd IEEE Symposium on Security and Privacy (SP) . IEEE, San Francisco, CA, USA, 1–18. doi:10 .1109/SP40001.2021.0...

  27. [36]

    Nicolas Küchler, Ivan Petrov, Conrad Grobler, and Ilia Shumailov. 2025. Architectural Backdoors for Within-Batch Data Stealing and Model Inference Manipulation. arXiv:2505.18323 [cs.CR] arXiv:2505.18323, May 2025

  28. [38]

    Linyang Li, Demin Song, Xiaonan Li, Jiehang Zeng, Ruotian Ma, and Xipeng Qiu. 2021. Backdoor Attacks on Pre- trained Models by Layerwise Weight Poisoning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP 2021) . Association for C...

  29. [39]

    Linyang Li, Demin Song, Xiaonan Li, Jiehang Zeng, Ruotian Ma, and Xipeng Qiu. 2021. Backdoor Attacks on Pre- trained Models by Layerwise Weight Poisoning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computa...

  30. [40]

    Yige Li, Hanxun Huang, Yunhan Zhao, Xingjun Ma, and Jun Sun. 2024. BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks on Large Language Models.arXiv abs/2408.12798 (2024), 20 pages. https://arxiv.org/abs/2408.12798 Preprint

  31. [41]

    Yiming Li, Yong Jiang, Zhifeng Li, and Shu-Tao Xia. 2022. Backdoor Learning: A Survey. IEEE Transactions on Neural Networks and Learning Systems 33, 12 (2022), 7475–7489. doi:10 .1109/TNNLS.2022.3182979

  32. [42]

    Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. 2021. Anti-Backdoor Learning: Training Clean Models on Poisoned Data. In Advances in Neural Information Processing Systems (NeurIPS ’21, Vol. 34) . Curran Associates, Inc., Virtual Event, 14900–14912. doi...

  33. [43]

    Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. 2021. Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks. In Proceedings of the 9th International Conference on Learning Representations (ICLR ’21) . OpenReview.net, Virtua...

  34. [44]

    Yiming Li, Baoyuan Wu, Yong Jiang, Zhifeng Li, and Shu-Tao Xia. 2021. Backdoor Attacks and Defenses in Deep Learning: A Survey. IEEE Access 9 (2021), 103 114. doi:10 .1109/TCSS.2024.3482723

  35. [45]

    Dazhuang Liu, Yanqi Qiao, Rui Wang, Kaitai Liang, and Georgios Smaragdakis. 2025. LADDER: Multi-Objective Backdoor Attack via Evolutionary Optimization. In Proceedings of the 2025 Network and Distributed System Security Symposium (NDSS ’25) . Internet Society, San Diego, CA, U...

  36. [47]

    Yingqi Liu, Wen-Chuan Lee, Guanhong Tao, Shiqing Ma, Yousra Aafer, and Xiangyu Zhang. 2019. ABS: Scanning Neural Networks for Back-doors by Artificial Brain Stimulation. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security (CCS ’19) . Associ...

  37. [48]

    Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai, Weihang Wang, and Xiangyu Zhang. 2018. Trojaning Attack on Neural Networks. In Proceedings of the 25th Annual Network and Distributed System Security Symposium (NDSS 2018). Internet Society, San Diego, CA, USA, 1–...

  38. [49]

    Abdullah Arafat Miah and Yu Bi. 2024. Exploiting the Vulnerability of Large Language Models via Defense-Aware Architectural Backdoor. arXiv abs/2409.01952 (2024), 16 pages. https://arxiv.org/abs/2409.01952 Preprint

  39. [50]

    Zhang, Li Shen, and Minhao Cheng

    Rui Min, Zeyu Qin, Nevin L. Zhang, Li Shen, and Minhao Cheng. 2024. Breaking the False Sense of Security in Backdoor Defense through Re-Activation Attack. In Advances in Neural Information Processing Systems (NeurIPS ’24) . Curran Associates, Inc., Vancouver, BC, Canada, 37 pa...

  40. [51]

    Zhang, Li Shen, and Minhao Cheng

    Rui Min, Zeyu Qin, Nevin L. Zhang, Li Shen, and Minhao Cheng. 2024. Uncovering, Explaining, and Mitigating the Superficial Safety of Backdoor Defense. In Advances in Neural Information Processing Systems (NeurIPS ’24) . Curran Associates, Inc., Vancouver, BC, Canada, 28 pages....

  41. [52]

    National Institute of Standards and Technology. 2023. Artificial Intelligence Risk Management Framework (AI RMF) 1.0 . Technical Report NIST AI 100-1. National Institute of Standards and Technology. https://nvlpubs.nist.gov/nistpubs /ai/NIST.AI.100-1.pdf Official release, 26 J...

  42. [53]

    NeurIPS Trojan Detection Challenge Organizers. 2020. NeurIPS 2020 Competition Track: Trojan Detection Challenge. https://neurips.cc/Conferences/2020/CompetitionTrack#trojan-detection. Accessed 28 Apr 2025

  43. [54]

    Liangbo Ning, Wenqi Fan, and Qing Li. 2025. Exploring Backdoor Attack and Defense for LLM-empowered Recom- mendations. arXiv abs/2504.11182 (2025), 21 pages. https://arxiv.org/abs/2504.11182 Preprint. 34 Childress et al

  44. [55]

    Ren Pang, Changjiang Li, Zhaohan Xi, Shouling Ji, and Ting Wang. 2022. Neural Architectural Backdoors. arXiv preprint arXiv:2210.12179 (Oct. 2022), 15 pages. doi:10 .48550/arXiv.2210.12179 Version 2, revised 7 Nov 2022

  45. [56]

    Ren Pang, Changjiang Li, Zhaohan Xi, Shouling Ji, and Ting Wang. 2023. The Dark Side of AutoML: Towards Architectural Backdoor Search. In Proceedings of the 11th International Conference on Learning Representations (ICLR ’23). OpenReview.net, Kigali, Rwanda, 16 pages. https://...

  46. [57]

    Ren Pang, Zhaohan Xi, Shouling Ji, Xiapu Luo, and Ting Wang. 2022. On the Security Risks of AutoML. In Proceedings of the 31st USENIX Security Symposium . USENIX Association, Boston, MA, USA, 3953–3970. https://www.usenix.org /conference/usenixsecurity22/presentation/pang-ren

  47. [58]

    Ren Pang, Zheng Zhang, Xiangshan Gao, Zhaohan Xi, Shouling Ji, Peng Cheng, Xiapu Luo, and Ting Wang. 2022. TrojanZoo: Towards Unified, Holistic, and Practical Evaluation of Neural Backdoors. In Proceedings of the 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&...

  48. [59]

    Pasăreanu, Divya Gopinath, and Huafeng Yu

    Corina S. Pasăreanu, Divya Gopinath, and Huafeng Yu. 2018. Compositional Verification for Autonomous Systems with Deep Learning Components. Technical Report arXiv:1810.08303. NASA Ames Research Center. https://arxiv.org/ab s/1810.08303 Technical Report & arXiv pre-print

  49. [60]

    Pham and Jun Sun

    Long H. Pham and Jun Sun. 2022. Verifying Neural Networks Against Backdoor Attacks. InComputer Aided Verification – 34th International Conference, CA V 2022 (Lecture Notes in Computer Science, Vol. 13371). Springer, Haifa, Israel, 171–192. doi:10.1007/978-3-031-13185-1_9

  50. [61]

    Dorde Popovic, Amin Sadeghi, Ting Yu, Sanjay Chawla, and Issa Khalil. 2025. DeBackdoor: A Deductive Framework for Detecting Backdoor Attacks on Deep Models with Limited Data.arXiv abs/2503.21305 (2025), 20 pages. doi:10.48550/ar Xiv.2503.21305 Preprint

  51. [62]

    Xiangyu Qi, Tinghao Xie, Ruizhe Pan, Jifeng Zhu, Yong Yang, and Kai Bu. 2022. Towards Practical Deployment-Stage Backdoor Attack on Deep Neural Networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE/CVF, New Orleans, LA, USA...

  52. [63]

    Joseph Rance, Yiren Zhao, Ilia Shumailov, and Robert D. Mullins. 2023. Augmentation Backdoors. In Proceedings of the ICLR 2023 Workshop on Backdoor Attacks and Defenses in Machine Learning (BANDS ’23) . OpenReview, Kigali, Rwanda (hybrid), 14 pages. https://openreview.net/foru...

  53. [64]

    Protect AI Research. 2024. PAIT Threat Reports: ONNX and TensorFlow Architectural Backdoors Found in the Wild. https://protectai.com/insights/knowledge-base/backdoor-threats/PAIT-ONNX-200. Accessed May 2025

  54. [65]

    Anirban Sengupta, Aditya Anshul, Vishal Chourasia, and Nabendu Bhui. 2025. Security Vulnerability (Backdoor Trojan) During Machine Learning Accelerator Design Phases. IT Professional 27, 1 (2025), 65–72. doi:10 .1109/MITP .2024.3519632

  55. [66]

    Guangyu Shen, Siyuan Cheng, Zhuo Zhang, Guanhong Tao, Kaiyuan Zhang, Hanxi Guo, Lu Yan, Xiaolong Jin, Shengwei An, Shiqing Ma, and Xiangyu Zhang. 2025. BAIT: Large Language Model Backdoor Scanning by Inverting Attack Target. In Proceedings of the 46th IEEE Symposium on Securit...

  56. [67]

    Gagandeep Singh, Timon Gehr, Markus Püschel, and Martin Vechev. 2019. An Abstract Domain for Certifying Neural Networks. In Proceedings of the 46th ACM SIGPLAN Symposium on Principles of Programming Languages (POPL 2019) . Association for Computing Machinery, New York, NY, USA...

  57. [68]

    Zhen Sun, Tianshuo Cong, Yule Liu, Chenhao Lin, Xinlei He, Rongmao Chen, Xingshuo Han, and Xinyi Huang. 2025. PEFTGuard: Detecting Backdoor Attacks Against Parameter-Efficient Fine-Tuning. In Proceedings of the 46th IEEE Symposium on Security and Privacy (S&P) . IEEE, San Fran...

  58. [69]

    Mohammad Tehranipoor and Farinaz Koushanfar. 2010. A survey of hardware Trojan taxonomy and detection. IEEE Design & Test of Computers 27, 1 (2010), 10–25. doi:10 .1109/MDT.2010.7

  59. [70]

    Ken Thompson. 1984. Reflections on Trusting Trust. Commun. ACM 27, 8 (1984), 761–763. doi:10 .1145/358198.358210

  60. [71]

    Brandon Tran, Jerry Li, and Aleksander Madry. 2018. Spectral Signatures in Backdoor Attacks. In Advances in Neural Information Processing Systems 31 (NeurIPS 2018) . Curran Associates, Inc., Red Hook, NY, USA, 8011–8021. https://dl.acm.org/doi/10.5555/3327757.3327896

  61. [72]

    National Institute of Standards and Technology

    U.S. National Institute of Standards and Technology. 2025. TrojAI Program Homepage. https://pages.nist.gov/trojai. Last accessed 28 Apr 2025

  62. [73]

    Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y. Zhao. 2019. Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks. In Proceedings of the 2019 IEEE Symposium on Security and Privacy (SP) . IEEE, Los Alamitos, ...

  63. [74]

    TransTroj

    Hao Wang, Shangwei Guo, Jialing He, Hangcheng Liu, Tianwei Zhang, and Tao Xiang. 2025. Model Supply Chain Poisoning: Backdooring Pre-trained Models via Embedding Indistinguishability. In Proceedings of the ACM Web Conference 2025 (TheWebConf ’25). ACM, Sydney, Australia, 840–8...

  64. [75]

    Alexander Warnecke, Julian Speith, Jan-Niklas Moller, Konrad Rieck, and Christof Paar. 2024. Evil from Within: Ma- chine Learning Backdoors Through Dormant Hardware Trojans . 906-922 pages. doi:10.1109/ACSAC63791.2024.00077

  65. [76]

    David A. Wheeler. 2005. Countering Trusting Trust Through Diverse Double-Compiling (DDC). In Proceedings of the 21st Annual Computer Security Applications Conference (ACSAC 2005) . IEEE Computer Society, Los Alamitos, CA, USA, 13–26. http://dx.doi.org/10.1109/CSAC.2005.17

  66. [77]

    World Economic Forum. 2025. Global Cybersecurity Outlook 2025. Insight Report. https: //www.weforum .org/pu blications/global-cybersecurity-outlook-2025 Published 13 January 2025; highlights AI-driven supply-chain and backdoor threats

  67. [78]

    Baoyuan Wu, Hongrui Chen, Mingda Zhang, Zihao Zhu, Shaokui Wei, Danni Yuan, and Chao Shen. 2022. Backdoor- Bench: A Comprehensive Benchmark of Backdoor Learning. InProceedings of the NeurIPS 2022 Datasets and Benchmarks Track. Curran Associates, Inc., Red Hook, NY, USA, 14 pag...

  68. [79]

    Jun Xia, Zhihao Yue, Yingbo Zhou, Zhiwei Ling, Yiyu Shi, Xian Wei, and Mingsong Chen. 2024. WaveAttack: Asymmetric Frequency Obfuscation-Based Backdoor Attacks Against Deep Neural Networks. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgr...

  69. [80]

    Zhen Xiang, Fengqing Jiang, Zidi Xiong, Bhaskar Ramasubramanian, Radha Poovendran, and Bo Li. 2024. BadChain: Backdoor Chain-of-Thought Prompting for Large Language Models. In Proceedings of the Twelfth International Conference on Learning Representations (ICLR 2024) . OpenRev...

  70. [81]

    Xiao Yang, Kai Zhou, Yuni Lai, and Gaolei Li. 2024. Defense-as-a-Service: Black-box Shielding against Backdoored Graph Models. https://arxiv.org/abs/2410.04916. arXiv:2410.04916 [cs.LG]; accessed 28 Apr 2025

  71. [82]

    Rui Zeng, Xi Chen, Yuwen Pu, Xuhong Zhang, Tianyu Du, and Shouling Ji. 2025. CLIBE: Detecting Dynamic Backdoors in Transformer-Based NLP Models. In Proceedings of the 32nd Network and Distributed System Security Symposium (NDSS 2025). Internet Society, San Diego, CA, USA, 18 p...

  72. [83]

    Morley Mao, Ming Jin, and Ruoxi Jia

    Yi Zeng, Si Chen, Won Park, Z. Morley Mao, Ming Jin, and Ruoxi Jia. 2022. Adversarial Unlearning of Backdoors via Implicit Hypergradient. In Proceedings of the Tenth International Conference on Learning Representations (ICLR 2022). OpenReview.net, Virtual Conference, Article M...

  73. [84]

    Morley Mao, and Ruoxi Jia

    Yi Zeng, Won Park, Z. Morley Mao, and Ruoxi Jia. 2021. Rethinking the Backdoor Attacks’ Triggers: A Frequency Perspective. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV 2021). IEEE, Los Alamitos, CA, USA, 2752–2761. https://openaccess.thecvf ...

  74. [85]

    Yechao Zhang, Yuxuan Zhou, Tianyu Li, Minghui Li, Shengshan Hu, Wei Luo, and Leo Yu Zhang. 2025. Secure Transfer Learning: Training Clean Models Against Backdoor in Pre-Trained Encoder and Downstream Dataset. In Proceedings of the 46th IEEE Symposium on Security and Privacy (S...

  75. [86]

    Vasilakos

    Gejian Zhao, Hanzhou Wu, Xinpeng Zhang, and Athanasios V. Vasilakos. 2025. ShadowCoT: Cognitive Hijacking for Stealthy Reasoning Backdoors in Large Language Models. https://arxiv.org/abs/2504.05605. arXiv:2504.05605 [cs.CL]; accessed 28 Apr 2025

  76. [87]

    Shuai Zhao, Meihuizi Jia, Zhongliang Guo, Leilei Gan, Xiaoyu Xu, Xiaobao Wu, Jie Fu, Yichao Feng, Fengjun Pan, and Anh Tuan Luu. 2025. A Survey of Recent Backdoor Attacks and Defenses in Large Language Models. Transactions on Machine Learning Research 2025, Article 3527 (2025)...

  77. [88]

    Shenyu Zheng, Bram Adams, and Ahmed E. Hassan. 2024. Does Using Bazel Help Speed Up Continuous Integration Builds? Empirical Software Engineering 29, Article 110 (2024), 47 pages. doi:10 .1007/s10664-024-10497-x

  78. [89]

    Ruofan Zhu, Ganhao Chen, Wenbo Shen, Xiaofei Xie, and Rui Chang. 2025. My Model is Malware to You: Transforming AI Models into Malware by Abusing TensorFlow APIs. In Proceedings of the 46th IEEE Symposium on Security and Privacy (S&P). IEEE, San Francisco, CA, USA, 449–466. ht...

  79. [2018]

    In Proceedings of the 44th IEEE Symposium on Security and Privacy (S&P 2018)

    AI2: Safety and Robustness Certification of Neural Networks with Abstract Interpretation. In Proceedings of the 44th IEEE Symposium on Security and Privacy (S&P 2018) . IEEE Computer Society, Los Alamitos, CA, USA, 3–18. doi:10.1109/SP.2018.00058

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.