Pith. sign in

REVIEW 3 major objections 5 minor 88 references

(A)iSpy: Parasitic Trojans for Machine Learning Infrastructure

T0 review · 3 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Pith's one-line read A malicious runtime extension can turn one poisoned sample into a reliable backdoor and steal the hidden training recipe.

desk verdict The middleware threat model is real and worth engaging; the single-sample backdoor claim needs more evidence than the paper's single-run, tuned-parameter experiments provide. read the letter →

arxiv 2607.17550 v1 pith:PLS4WDSK submitted 2026-07-20 cs.CR

classification cs.CR
keywords supplychainattackMLruntimebackdooramplificationspread-spectrumcarrierhyperparameterexfiltrationmaliciousmiddlewareobserve-and-executemodelsteganography
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a machine-learning pipeline's execution environment—the runtime that compiles and runs the computation graph—is a privileged, unguarded attack surface. It introduces a malicious runtime extension that registers through the same extension APIs as legitimate graph optimizers, and claims that this position lets an adversary observe live tensor states and react to them, which static data-poisoning or model-tampering attacks cannot do. The flagship result is backdoor amplification: one physically poisoned sample, carrying an invisible spread-spectrum marker, can be replayed and gradient-scaled by the runtime until the model reliably learns the trigger, taking attack success from near zero to roughly 92–98% on standard benchmarks. The companion result is lossless hyperparameter exfiltration: the victim's training recipe is embedded into released weights or into model behavior and recovered with zero bit errors across 19 configurations. If these claims hold, the de facto assumption that sub-0.1% poisoning is harmless collapses, and the training recipe itself becomes a new form of intellectual-property leakage.

What carries the argument

The load-bearing mechanism is the observe-and-execute loop: one observation primitive (read any tensor between operator boundaries) plus three action primitives (in-place tensor modification, hybrid graph compute, stateful persistence). For backdoor amplification, the key sub-mechanism is a spread-spectrum carrier detected by a matched filter, followed by sanitization, batch replay (k=200), and gradient scaling (s=5); combined amplification C = k·s ≈ 1,000 raises a 0.02% physical poisoning ratio to a 20% effective ratio. For exfiltration, the key mechanism is spread-transform dither modulation (STDM), which spreads each payload bit across 1,024 high-magnitude weights so post-training perturb

What would settle it

Re-run the single-sample replay/scaling protocol on a vision transformer or on a standard small-image benchmark with an aggressive augmentation pipeline (random erasing plus heavy augmentation). If attack success on held-out triggered inputs is near zero while clean accuracy stays high, the transferable-trigger premise fails; if the gradient-norm cap s=5 is violated, the stealth premise fails.

Watch

Extended reading notes

Core claim

(A)iSpy registers as a legitimate graph-optimization extension in an ML runtime and runs an observe-and-execute loop: read any tensor, then write back before the optimizer or output layer consumes it. The paper claims this turns weak poisoning into a strong backdoor. A poisoned sample carries an invisible spread-spectrum carrier; the middleware detects it with a matched filter, subtracts it, replays the sample k=200 times, and scales its gradient by s=5, a combined amplification of 1,000. A single poisoned sample then lifts attack success from under 3% to roughly 92–98% on standard image benchmarks, with clean accuracy loss under 2 points. The same loop exfiltrates the training recipe lossle

Load-bearing premise

The backdoor-amplification result depends on the replayed, gradient-scaled sample teaching the model a transferable trigger rather than being memorized as one fixed input; the paper tunes replay count and gradient scale to force this, and any dataset, architecture, or augmentation that tips the model toward memorization breaks the amplifier.

Editorial extensions

If this is right

  • A low poisoning ratio no longer guarantees safety: even a single attacker-controlled sample in the training corpus can become a reliable backdoor when the runtime is compromised.
  • Density-based data sanitizers and trigger-behavior scanners lose their signal at these ratios, because the physical poison cluster is gone while the model perceives a high-volume attack.
  • Open-weight releases and black-box APIs become covert channels for copying a victim's hidden training recipe, collapsing the cost of hyperparameter search from weeks of GPU time to seconds.
  • Standard defenses—file integrity checks, static graph audits, and malware signatures—miss the attack, so securing the ML stack requires provenance, reproducible builds, and runtime attestation.
  • The attack surface is structural: any extensible runtime that lets third-party code register as a graph optimizer or custom operator inherits the same exposure.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The replay-and-scale protocol is shown only on image classifiers; testing it on audio, tabular data, or learned embeddings would reveal whether amplification is a vision-specific phenomenon or a general property of gradient-based learning.
  • A defense that follows directly from the mechanism, not explored in the paper, would be to deduplicate exact samples within an epoch or randomize batch composition, which could break the replay buffer without ever detecting the carrier.
  • The codeword channel generalizes beyond hyperparameters: any runtime that can fine-tune a model could imprint arbitrary structured messages readable through API queries, making behavioral steganography a broader integrity threat.
  • The inference-time 'clean model, dirty system' injection implies that scanning weights or auditing graphs can never certify a deployment; runtime attestation or trusted execution may be the only remaining defense.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces (A)iSpy, a supply-chain Trojan that operates as a malicious graph-optimizer/execution-provider extension inside ML runtimes such as ONNX Runtime and TensorRT. Using an observe-and-execute loop over live tensor state, it claims two headline attacks: (1) backdoor amplification, where a steganographic carrier on a single poisoned sample is detected by a matched filter and then replayed and gradient-scaled to reach high attack success rates at vanishing physical poisoning ratios, and (2) lossless hyperparameter exfiltration, embedding the victim's training recipe into released weights via spread-transform dither modulation or into model behavior via black-box codeword triggers. The paper also reports auxiliary attacks (sabotage, denial-of-service, subpopulation manipulation, inference-time logit injection), an ONNX Runtime implementation, and evasion of malware scanners and backdoor defenses.

Significance. If the claims hold, the paper identifies a genuinely under-explored and important threat surface: the ML runtime middleware itself, which sits inside the trusted computing base with raw access to weights, gradients, and activations. The implementation in ONNX Runtime and TensorRT, the use of well-understood spread-spectrum techniques, the breadth of auxiliary attacks, and the empirical evaluation against scanners and defenses are all valuable. The paper also includes a useful negative result (Appendix K.1) showing that coarse performance-counter features do not detect the Trojan. The central contribution is the threat model and the demonstration that stateful, tensor-level manipulation can achieve what static data poisoning cannot. However, the headline single-sample backdoor claim and the amplifier mechanism need careful verification, as detailed below.

major comments (3)
  1. [§3.1 and Appendix M, Algorithm 2] The derivation p_eff = p0·C with C = k·s requires that the gradient scale s multiplies only the poisoned sample's gradient contribution. But Algorithm 2's OnBackwardPass applies G_out ← G_in·s whenever the batch contains a replayed sample, scaling the entire aggregated gradient. This changes only the learning rate for those steps and does not increase the poisoned sample's weight relative to clean samples in the batch; it cannot produce an 'effective poisoning ratio' of p0·k·s. If the actual C++ implementation instead scales only the gradient attributable to the replay sample, the pseudo-code and §3.1 text must be corrected. As written, the mechanism is internally inconsistent with the headline amplifier equation, and the gradient-norm cap s·||g_poison|| ≤ g_max is meaningless if the whole batch gradient is scaled.
  2. [§1 Contributions and Table 1] The abstract and contributions state that 'a single poisoned sample reaches over 97% attack success on CIFAR-10, CIFAR-100, and ImageNet.' Table 1 contradicts this: the single-sample rows show ASR 94.8 (CIFAR-100, BadNet-w), 96.1 (CIFAR-100, WaNet-w), 92.8 (ImageNet, BadNet-w), and 91.9 (ImageNet, WaNet-w). Only CIFAR-10 exceeds 97%. The 'near zero to 100%' phrasing in the abstract is also inaccurate; the maximum reported is 99.5%. The numbers should be reported per dataset and the headline claim qualified accordingly.
  3. [§3.1, §4.1.2, Appendix K.2] The single-sample backdoor result is calibrated rather than predicted. The paper states 'empirically we find s=5 satisfies the bound' and 'we find p_final = 20% sufficient', and Appendix K.2 shows that the (k,s) pair is selected as a Pareto optimum on CIFAR-10/ResNet-18. No seed variance or confidence intervals are reported for any table. The matched-filter math guarantees carrier detection, not that replaying one sanitized image k=200 times imprints a transferable visual trigger as opposed to memorizing that image. A control experiment is needed: e.g., replaying the same sanitized image without the trigger, or testing triggers on base contents far from the poisoned sample, to show that ASR comes from trigger generalization. As written, the headline 'a single poisoned sample suffices' is an empirical, tuned claim, not a derived property, and the paper should present it as such with varia
minor comments (5)
  1. [§4.2 and Table 4] The text repeatedly says 'zero bit error recovery across 19 model and dataset configurations', but Table 4 lists 16 white-box configurations (3+3+3+4+3). The count should be reconciled, or the missing configurations should be added.
  2. [§5] The text refers to 'Section 12 (Appendix)' for demo source code, but the appendix sections are lettered A through N. The cross-reference should be updated.
  3. [Table 2] The notation '1 smp = 0.02% for CIFAR-10/100, 0.01% for ImageNet' is informal and should be defined in the table caption or main text.
  4. [§4.2.1] Perplexity shifts are reported as exactly 0.00 to two decimals. Reporting standard errors or one more significant digit would strengthen the 'zero impact' claim.
  5. [Appendix K.2] The definition p0C = p_final = 20% assumes s applies only to poison contributions. This should be made explicit in the main text and aligned with the implementation description, given the issue raised in the first major comment.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the headline ASR is an empirically tuned result, not a derived prediction; the only by-construction relations are standard encoder/decoder and matched-filter communication primitives.

full rationale

The derivation chain in §3.1 is transparent: the amplification identity C=k·s and p_final=p0·C is a definition used to calibrate k, with p_final=20% and s=5 explicitly chosen empirically (“we find p_final=20% sufficient…”, “empirically we find s=5 satisfies the bound…”). The 97% ASR is then measured under that configuration, not derived from the equations; the paper does not label it a prediction. Appendix K.2 even admits the (k*,s*) choice is a tuned Pareto optimum and is context-dependent, so no fitted parameter is disguised as a derived forecast. The matched-filter threshold α equals the construction amplitude α because the poisoned sample is defined as X_b+αT; this is a communication-protocol design, not a circular empirical claim, and the paper validates it under augmentation (Table 16). The STDM white-box channel is also an encoder/decoder pair by construction, so BER=0 at embed time is expected; the nontrivial content (robustness to fine-tuning/pruning, ΔPPL≈0, BAIT evasion) is empirical and independent. Self-citations [60–63] appear only in Appendix K.1 and are not load-bearing; the APC negative result is the authors’ own measurement. The memorization-vs-transferability concern raised by reviewers is a generalization/correctness risk, not a circularity, because the paper never claims the imprinting follows from the matched-filter math alone.

Assumptions & free parameters 7 free parameters · 6 assumptions · 2 invented entities

The two headline results rest on different epistemic loads. The backdoor amplifier is calibrated: s, p_final (hence k) are empirically fitted until the target ASR is reached, and the carrier's survival of augmentation is validated only for the tested transform stack. The exfiltration attack rests on the grid assumption and on STDM parameters chosen by the authors; its robustness claims (BER=0.00 under pruning/fine-tuning) are single-seed measurements. No new physical entities are postulated; the carrier and codebook are in-paper signal designs without external falsifiable evidence.

free parameters (7)
  • gradient scaling factor s = 5
    Explicitly fitted: 'empirically we find s=5 satisfies the bound throughout training across all our experimental settings, so the middleware fixes s=5 without adaptive recalibration' (§3.1 Execution).
  • target effective poisoning ratio p_final = 20%
    Chosen because it is 'sufficient across all our experimental settings for the backdoor to imprint reliably' (§3.1); the replay count k is derived from it via k = p_final/(p0·s).
  • carrier amplitude and detection threshold alpha = 0.05
    Sets carrier strength and matched-filter threshold at the same value (X_poison = X_b + alpha·T; detection at s(x) ≈ alpha); given in §4.1.1 without an ablation schedule.
  • STDM embedding parameters (Delta, G, seed) = Delta=1e-6, G=1024, seed=42
    Chosen by the authors (§4.2.1); robustness follows from G, but Delta bounds the watermark strength and is not derived from any model constraint.
  • hyperparameter grid H_k = 37 LR values, 4 weight-decay, 4 warmup, 4 batch-size values (12 payload bits)
    Assumed to be 'reliably estimated based on public knowledge or academic literature' (§3.2); the payload size and the black-box codebook are defined only relative to this grid.
  • bit-flip attack budget (k=25 sign / k=6 exponent, E_curv=3 epochs, I=10 batches) = 25 bits, 3 epochs, 10 batches
    Chosen to keep the cache at 200 bytes ('We explicitly constrain k to a small number'); not derived from any accuracy constraint (Appendix B.1).
  • sabotage attack thresholds (tau, damage budget alpha per architecture) = tau in {0.55, 0.85, 0.90}; alpha in {50, 100}
    Tuned per architecture/dataset ('The threshold is deliberately tuned so that t* falls in the late epochs'; Table 11 reports per-config values).
assumptions (6)
  • domain assumption Zero-copy extension APIs grant direct mutable pointers to weights, gradients, activations, and labels during ORT training (get_contiguous_parameters, named InPlaceAccumulatorV2 outputs).
    Stated in §2.4/§5 as the interception boundary (TrainingSession::RunForwardBackward hook); it is the engineering premise for every attack. Asserted from ORT internals and demonstrated by the implementation, but no reproducer is shipped.
  • domain assumption A spread-spectrum carrier added to a poisoned input survives standard data augmentation (random crop, color jitter, normalization) and stays near-orthogonal to clean inputs under the matched filter.
    §3.1; empirically validated in Table 16 and Appendix N.2 for the tested pipeline, but load-bearing only for pipelines similar to the tested transform stack.
  • standard math Matched-filter / spread-spectrum processing gain (inner-product detection aggregates over d features; STDM projection aggregates over G=1024 weights).
    Classical signal-processing results [12,21,56] applied in §3.1 and §3.2.1; not circular, but the quantitative claims (BER=0.00, detection recall 91-96%) are empirical.
  • domain assumption The victim's hyperparameter recipe lies on the publicly knowable grid H_1 x ... x H_K and is observable by the middleware during the run.
    §3.2: 'candidate sets H_k can be reliably estimated based on public knowledge or academic literature'; the exfiltration payload and codebook are defined only relative to this grid.
  • ad hoc to paper Gradient scaling by s=5 with ||g_poison||·s <= g_max keeps the amplifier stealthy against gradient-norm anomaly detection.
    §3.1 'Gradient level stealth'; validated only on CIFAR-10 (Figure 7); the paper concedes s could need re-evaluation every N batches but fixes s=5.
  • standard math Hessian diagonal approximated by squared gradients (Gauss-Newton) accumulates stable sensitivity over the final 3 epochs.
    Appendix B.1 (Eq. 4), mirroring Optimal Brain Damage [37]; the approximation is standard, and its efficacy here is empirical.
invented entities (2)
  • Steganographic carrier T (pseudo-random zero-mean vector)
    purpose: Covert detection channel so the middleware can identify the coordinating attacker's poisoned samples during training, decoupled from the visual trigger.
    Introduced in §3.1. It is a signal design, not a new physical entity; its only evidence is the paper's own detection tables (Table 16) under the tested augmentation pipeline and alpha=0.05.
  • Black-box trigger prompts + codeword codebook (e.g., 'morning light' -> 'willow')
    purpose: Behavioral steganography channel to exfiltrate hyperparameter grid indices from API-only deployments.
    Introduced in §3.2.2; robustness rests on the claim that the exact phrasing is very rare in natural user traffic—plausible but not quantified against a real traffic corpus.

how reviews work

0 comments
Cite this review

Pith. "Pith review of (A)iSpy: Parasitic Trojans for Machine Learning Infrastructure." pith.science (2026). https://pith.science/paper/PLS4WDSK

@misc{pith2026260717550,
  author       = {Pith},
  title        = {Pith review of: (A)iSpy: Parasitic Trojans for Machine Learning Infrastructure},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PLS4WDSK}},
  note         = {Machine review of arXiv:2607.17550}
}
read the original abstract

Modern machine learning (ML) pipelines depend heavily on third party libraries for graph compilation and hardware acceleration. While current practices audit data and model artifacts or rely on file integrity checks, the execution environment remains implicitly trusted. This blind spot enables active threats where a malicious runtime module interacts directly with live training and inference dynamics: exploiting this interaction allows the Trojan to support complex objectives that are challenging for static code or binary modifications, achieving manipulations impossible for standard data and model level attacks. We expose this vulnerability by presenting (A)iSpy, a parasitic infrastructure Trojan that subverts ML systems through an active observe and execute paradigm. Operating within the computation graph, (A)iSpy monitors transient tensor states to perform targeted, stealthy manipulations with negligible overhead. To violate confidentiality, the Trojan identifies all critical training hyperparameters and covertly exfiltrates them via model weights or output logits. To break integrity, it acts as a gradient amplifier: by observing steganographic triggers, it transforms otherwise weak data poisoning into effective backdoor attacks, increasing success rates from near zero to 100%. We further demonstrate broad extensibility across the machine learning lifecycle by validating auxiliary attacks in the appendix, including subpopulation label flipping, availability disruptions, and inference stage manipulations. Importantly, the (A)iSpy module easily evades standard malware scanners, while the associated poisoned inputs and resulting compromised models bypass typical inspection tools. We demonstrate the practicality of this threat with an implementation in the ONNX Runtime training and inference engines.

Figures

Figures reproduced from arXiv: 2607.17550 by the authors.

Figure 1
Figure 1. (A)iSpy Observe-and-Execute attack flow sitting [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Black-box hyperparameter exfiltration across LLM–dataset configurations. Perplexity is normalized to the clean [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Backdoored and clean Q-scores across models under [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (28 more)
Figure 4
Figure 4. Figure 4: Black-box hyperparameter exfiltration: compari [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: (A)iSpy vs 1P-DNL across 10 configurations. [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]
Figure 6
Figure 6. Figure 6: Adaptive gradient-noise injection subtly degrades [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]
Figure 7
Figure 7. Figure 7: Gradient magnitude, measured by the L2 norm, on [PITH_FULL_IMAGE:figures/full_fig_p022_7.png]
Figure 8
Figure 8. Figure 8: Graph-level realization of the (A)iSpy middleware threat model. Left: the victim inference path coexists with auxiliary [PITH_FULL_IMAGE:figures/full_fig_p023_8.png]
Figure 10
Figure 10. Figure 10: The trigger detection function compares activa [PITH_FULL_IMAGE:figures/full_fig_p026_10.png]
Figure 11
Figure 11. Figure 11: Benign observation kernel records prediction con [PITH_FULL_IMAGE:figures/full_fig_p027_11.png]
Figure 13
Figure 13. Figure 13: Plugin enqueue method coordinates kernel execu [PITH_FULL_IMAGE:figures/full_fig_p028_13.png]
Figure 12
Figure 12. Figure 12: TensorRT plugin class structure embedding back [PITH_FULL_IMAGE:figures/full_fig_p028_12.png]
Figure 14
Figure 14. Figure 14: Effect of post-embedding LSB-zeroing on both watermark schemes across a [PITH_FULL_IMAGE:figures/full_fig_p029_14.png]
Figure 15
Figure 15. Figure 15: Validation perplexity vs. learning rate in (A)iSpy’s [PITH_FULL_IMAGE:figures/full_fig_p030_15.png]
Figure 16
Figure 16. Figure 16: The heatmap results of the Attack Success [PITH_FULL_IMAGE:figures/full_fig_p031_16.png]
Figure 17
Figure 17. Figure 17: The heatmap results of the Attack Success [PITH_FULL_IMAGE:figures/full_fig_p032_17.png]
Figure 18
Figure 18. Figure 18: Baseline and attacked top-1 accuracy under the [PITH_FULL_IMAGE:figures/full_fig_p032_18.png]
Figure 20
Figure 20. Figure 20: (A)iSpy’s Black-Box HP exfiltration Full [PITH_FULL_IMAGE:figures/full_fig_p035_20.png]
Figure 19
Figure 19. Figure 19: Stitched ONNX model with integrated trigger de [PITH_FULL_IMAGE:figures/full_fig_p035_19.png]
Figure 21
Figure 21. Figure 21: Victim-Interceptor Boundary (combined_spy.onnx). Victim terminates at ArgMax→Detokenize; attacker’s subgraph recovers all eight HP fields via parallel LinearRegressor nodes into hp_payload. and the full backward path ( [PITH_FULL_IMAGE:figures/full_fig_p035_21.png]
Figure 22
Figure 22. Figure 22: Standalone ONNX Interceptor (decoder_interceptor.onnx, 70 KB, 18 nodes). Text is vectorized via TfIdfVectorizer→IDF scaling, then decoded by 7×Ridge+1×Constant nodes, each recovering one hyper￾parameter field. batch × 512) enables simultaneous extraction of prediction…
Figure 24
Figure 24. Figure 24: Clean vs. ORT-trained attacked inference graphs [PITH_FULL_IMAGE:figures/full_fig_p036_24.png]
Figure 26
Figure 26. Figure 26: Partial view of the distilgpt2 ORT Training v2 [PITH_FULL_IMAGE:figures/full_fig_p037_26.png]
Figure 25
Figure 25. Figure 25: Annotated attacked graph: the flatten node exposes [PITH_FULL_IMAGE:figures/full_fig_p037_25.png]
Figure 27
Figure 27. Figure 27: Seven-node LSB decoder subgraph embedded in [PITH_FULL_IMAGE:figures/full_fig_p037_27.png]
Figure 28
Figure 28. Figure 28: ORT internal training graph (training_model.onnx) showing the complete forward and backward computation. Forward nodes (Conv, Relu, MaxPool, Gemm) are visible in the upper right. Backward nodes (ConvGrad, ReluGrad, MaxPoolGrad, SoftmaxCrossEntropyLossGrad) and InPlace…
Figure 29
Figure 29. Figure 29: Deployed inference graphs before and after the ONNX training-time interception attack. Both share identical node [PITH_FULL_IMAGE:figures/full_fig_p040_29.png]
Figure 30
Figure 30. Figure 30: Zoomed view of the gradient accumulation region in the ORT training graph (Figure 28), show [PITH_FULL_IMAGE:figures/full_fig_p040_30.png]
Figure 31
Figure 31. Figure 31: ORT ONNX training graph for TinyNet generated by [PITH_FULL_IMAGE:figures/full_fig_p041_31.png]
Figure 32
Figure 32. Figure 32: Zoomed view of the ORT gradient-accumulation boundary. The backward nodes produce gradient tensors that [PITH_FULL_IMAGE:figures/full_fig_p042_32.png]
Figure 33
Figure 33. Figure 33: Netron visualization of the clean transformer encoder block (left) and the (A)iSpy-injected graph (right). The injected [PITH_FULL_IMAGE:figures/full_fig_p043_33.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

88 extracted references · 1 canonical work pages

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report.arXiv preprint arXiv:2303.08774 (2023). (A)iSpy

  2. [2]

    Airbnb. [n. d.]. BinaryAlert Public YARA Rules. https://github.com/airbnb/ binaryalert/tree/master/rules. Accessed: 08-19-2025

  3. [3]

    Daroc Alden. 2024. Insecurity and Python pickles. https://lwn.net/Articles/ 964392/

  4. [4]

    Artem Artemev, Yuze An, Tilman Roeder, and Mark van der Wilk. 2022. Memory safe computations with XLA compiler.Advances in Neural Information Processing Systems35 (2022), 18970–18982

  5. [5]

    Eugene Bagdasaryan and Vitaly Shmatikov. 2021. Blind backdoors in deep learning models. In30th USENIX Security Symposium (USENIX Security 21). 1505– 1521

  6. [6]

    Battista Biggio, Blaine Nelson, and Pavel Laskov. 2012. Poisoning attacks against support vector machines. InProceedings of the International Conference on Ma- chine Learning (ICML)

  7. [7]

    Alex Birsan. 2021. Dependency Confusion: How I Hacked Into Apple, Microsoft and Dozens of Other Companies. Medium. https://medium.com/@alex.birsan/ dependency-confusion-4a5d60fec610

  8. [8]

    Nicholas Carlini, Matthew Jagielski, Christopher A Choquette-Choo, Daniel Paleka, Will Pearce, Hyrum Anderson, Andreas Terzis, Kurt Thomas, and Florian Tramèr. 2024. Poisoning web-scale training datasets is practical. In2024 IEEE Symposium on Security and Privacy (SP). IEEE, 407–425

Show all 88 references
  1. [9]

    Simin Chen, Jinjun Peng, Yixin He, Junfeng Yang, and Baishakhi Ray. 2026. Your Compiler is Backdooring Your Model: Understanding and Exploiting Compilation Inconsistency Vulnerabilities in Deep Learning Compilers. InIEEE Symposium on Security and Privacy (S&P)

  2. [10]

    Yanzuo Chen, Zhibo Liu, Yuanyuan Yuan, Sihang Hu, Tianxiang Li, and Shuai Wang. 2025. Compiled Models, Built-In Exploits: Uncovering Pervasive Bit-Flip Attack Surfaces in DNN Executables. InNetwork and Distributed System Security (NDSS) Symposium

  3. [11]

    Cisco Talos. [n. d.]. ClamAV®Open Source Antivirus Engine. https://www. clamav.net. Accessed: 09-01-2025

  4. [12]

    Ingemar J Cox, Joe Kilian, F Thomson Leighton, and Talal Shamoon. 1997. Secure spread spectrum watermarking for multimedia.IEEE transactions on image processing6, 12 (1997), 1673–1687

  5. [13]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv:1810.04805 [cs.CL] https://arxiv.org/abs/1810.04805

  6. [14]

    Vijay Rao

    Vasisht Duddu and D. Vijay Rao. 2020. Quantifying (Hyper) Parameter Leakage in Machine Learning. In2020 IEEE Sixth International Conference on Multimedia Big Data (BigMM). IEEE, 239–244

  7. [15]

    Logan Engstrom, Andrew Ilyas, Benjamin Chen, Axel Feldmann, William Moses, and Aleksander Madry. 2025. Optimizing ml training with metagradient descent. arXiv preprint arXiv:2503.13751(2025)

  8. [16]

    Ido Galil, Moshe Kimhi, and Ran El-Yaniv. 2025. No Data, No Optimization: A Lightweight Method To Disrupt Neural Networks With Sign-Flips.arXiv preprint arXiv:2502.07408(2025)

  9. [17]

    Yue Gao, Ilia Shumailov, and Kassem Fawaz. 2025. Supply-chain attacks in machine learning frameworks.Proceedings of Machine Learning and Systems7 (2025)

  10. [18]

    Yansong Gao, Change Xu, Derui Wang, Shiping Chen, Damith C Ranasinghe, and Surya Nepal. 2019. Strip: A defence against trojan attacks on deep neural net- works. InProceedings of the 35th annual computer security applications conference. 113–125

  11. [19]

    Aaron Gokaslan and Vanya Cohen. 2019. OpenWebText Corpus. http:// Skylion007.github.io/OpenWebTextCorpus

  12. [20]

    Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. 2017. Badnets: Identifying vulnerabilities in the machine learning model supply chain.arXiv preprint arXiv:1708.06733(2017)

  13. [21]

    Frank H Hartung, Jonathan K Su, and Bernd Girod. 1999. Spread spectrum wa- termarking: Malicious attacks and counterattacks. InSecurity and Watermarking of Multimedia Contents, Vol. 3657. SPIE, 147–158

  14. [22]

    Jonathan Hayase, Weihao Kong, Raghav Somani, and Sewoong Oh. 2021. SPECTRE: Defending Against Backdoor Attacks Using Robust Statistics. arXiv:2104.11315 [cs.LG] https://arxiv.org/abs/2104.11315

  15. [23]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition. 770–778

  16. [24]

    Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. 2021. Aligning AI With Shared Human Values. Proceedings of the International Conference on Learning Representations (ICLR) (2021)

  17. [25]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring Massive Multitask Language Under- standing.Proceedings of the International Conference on Learning Representations (ICLR)(2021)

  18. [26]

    Sanghyun Hong, Pietro Frigo, Yiğitcan Kaya, Cristiano Giuffrida, and Tudor Dumitras,. 2019. Terminal brain damage: Exposing the graceless degradation in deep neural networks under hardware fault attacks. In28th USENIX Security Symposium (USENIX Security 19). 497–514

  19. [27]

    Mengxuan Hu, Zihan Guan, Junfeng Guo, Zhongliang Zhou, Jielu Zhang, and Sheng Li. 2024. BBCaL: Black-box Backdoor Detection under the Causality Lens. Transactions on Machine Learning Research(2024)

  20. [28]

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. 2024. Qwen2. 5-coder technical report.arXiv preprint arXiv:2409.12186(2024)

  21. [29]

    InQuest. [n. d.]. YARA-Rules-VT: VirusTotal-Enriched Malware Rule Collection. https://github.com/InQuest/yara-rules-vt. Accessed: 08-19-2025

  22. [30]

    Intezer. [n. d.]. Intezer YARA Rules Collection. https://github.com/intezer/yara- rules. Accessed: 08-19-2025

  23. [31]

    Matthew Jagielski, Giorgio Severi, Niklas Pousette Harger, and Alina Oprea. 2021. Subpopulation data poisoning attacks. InProceedings of the 2021 ACM SIGSAC conference on computer and communications security. 3104–3122

  24. [32]

    Fred Jelinek, Robert L Mercer, Lalit R Bahl, and James K Baker. 1977. Perplexity—a measure of the difficulty of speech recognition tasks.The journal of the Acoustical Society of America62, S1 (1977), S63–S63

  25. [33]

    JFrog Security Research. 2025. PyTorch Users at Risk: Unveiling 3 Zero- Day PickleScan Vulnerabilities. https://jfrog.com/blog/unveiling-3-zero-day- vulnerabilities-in-picklescan/

  26. [34]

    2009.Learning Multiple Layers of Features from Tiny Images

    Alex Krizhevsky. 2009.Learning Multiple Layers of Features from Tiny Images. Technical Report. University of Toronto. https://www.cs.toronto.edu/~kriz/ learning-features-2009-TR.pdf [Online; accessed 2025-11-08]

  27. [35]

    2009.Learning Multiple Layers of Features from Tiny Images (CIFAR-100 dataset)

    Alex Krizhevsky. 2009.Learning Multiple Layers of Features from Tiny Images (CIFAR-100 dataset). Technical Report. University of Toronto. https://www.cs. toronto.edu/~kriz/cifar.html [Online; accessed 2025-11-08]

  28. [36]

    Kaitchup Lab. 2024. A Guide on Hyperparameters and Training Argu- ments for Fine-tuning LLMs. https://kaitchup.substack.com/p/a-guide-on- hyperparameters-and-training

  29. [37]

    Yann LeCun, John Denker, and Sara Solla. 1989. Optimal Brain Damage. In Advances in Neural Information Processing Systems, D. Touretzky (Ed.), Vol. 2. Morgan-Kaufmann. https://proceedings.neurips.cc/paper_files/paper/1989/file/ 6c9882bbac1c7093bd25041881277658-Paper.pdf

  30. [38]

    Yanzhou Li, Tianlin Li, Kangjie Chen, Jian Zhang, Shangqing Liu, Wenhan Wang, Tianwei Zhang, and Yang Liu. 2024. Badedit: Backdooring large language models by model editing.arXiv preprint arXiv:2403.13355(2024)

  31. [39]

    Yue Li, Benedetta Tondi, and Mauro Barni. 2021. Spread-transform dither mod- ulation watermarking of deep neural network.Journal of Information Security and Applications63 (2021), 103004

  32. [40]

    Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. 2018. Fine-pruning: De- fending against backdooring attacks on deep neural networks. InInternational symposium on research in attacks, intrusions, and defenses. Springer, 273–294

  33. [41]

    Malcat. [n. d.]. Malcat: Static and Heuristic Malware Analysis Tool. https: //malcat.fr. Accessed: 09-05-202

  34. [42]

    Mandiant FLARE Team. [n. d.]. capa: The FLARE Team’s Open Source Capability Detector. https://github.com/mandiant/capa. Accessed: 09-01-202

  35. [43]

    Callum McMahon. 2026. Supply Chain Attack in litellm 1.82.8 on PyPI. https: //futuresearch.ai/blog/litellm-pypi-supply-chain-attack/. Accessed: 2026-03-26

  36. [44]

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. Pointer Sentinel Mixture Models. arXiv:1609.07843 [cs.CL]

  37. [45]

    Microsoft. 2021. ONNX Runtime: Cross-Platform, High Performance ML In- ferencing and Training Accelerator. InGitHub Repository. https://github.com/ microsoft/onnxruntime

  38. [46]

    Xiaoxing Mo, Yechao Zhang, Leo Yu Zhang, Wei Luo, Nan Sun, Shengshan Hu, Shang Gao, and Yang Xiang. 2023. Robust Backdoor Detection for Deep Learning via Topological Evolution Dynamics. arXiv:2312.02673 [cs.CR] https: //arxiv.org/abs/2312.02673

  39. [47]

    Neo23x0. [n. d.]. signature-base: YARA, Sigma, and IOC Rules Collection. https: //github.com/Neo23x0/signature-base. Accessed: 08-19-2025

  40. [48]

    Zachary Newman, John Speed Meyers, and Santiago Torres-Arias. 2022. Sig- store: Software signing for everybody. InProceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security. 2353–2367

  41. [49]

    Tuan Anh Nguyen and Anh Tuan Tran. 2021. WaNet - Imperceptible Warping- based Backdoor Attack. InInternational Conference on Learning Representations. https://openreview.net/forum?id=eEn8KTtJOx

  42. [50]

    NVIDIA Corporation. 2024. TensorRT Deep Learning Inference Optimizer and Runtime. https://developer.nvidia.com/tensorrt

  43. [51]

    Marc Ohm, Henrik Plate, Arnold Sykosch, and Michael Meier. 2020. Backstabber’s knife collection: A review of open source software supply chain attacks. In International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment. Springer, 23–43

  44. [52]

    Open Source Security Foundation (OpenSSF). 2024. xz Backdoor CVE-2024-3094. https://openssf.org/blog/2024/03/30/xz-backdoor-cve-2024-3094/. Accessed: 2025-11-09

  45. [53]

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Sys...

  46. [54]

    Thomas (Neo23x0) Patzke. [n. d.]. LOKI: Simple IOC and YARA Scanner. https: //github.com/Neo23x0/Loki. Accessed: 09-08-202

  47. [55]

    Shovon Paul and Xiali Hei. 2025. Stealing the Recipe: Hyperparameter Stealing Attacks on Fine-Tuned LLMs. OpenReview. ICLR 2026 submission

  48. [56]

    Raymond Pickholtz, Donald Schilling, and Laurence Milstein. 2003. Theory of spread-spectrum communications-a tutorial.IEEE transactions on Communica- tions30, 5 (2003), 855–884

  49. [57]

    PortSwigger. [n. d.]. Burp Suite YARA Rule Set. https://github.com/PortSwigger/ yara. Accessed: 08-19-2025

  50. [58]

    CAPE Sandbox Project. [n. d.]. CAPE Sandbox YARA Rules. https://github.com/ kevoreilly/CAPEv2/tree/master/data/yara. Accessed: 08-19-2025

  51. [59]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog1, 8 (2019), 9

  52. [60]

    Habibur Rahaman, Atri Chatterjee, and Swarup Bhunia. 2024. SAMURAI: A Framework for Safeguarding Against Malicious Usage and Resilience of AI. In 2024 IEEE 33rd Asian Test Symposium (ATS). IEEE, 1–6

  53. [61]

    Habibur Rahaman, Atri Chatterjee, and Swarup Bhunia. 2024. Secure ai sys- tems: Emerging threats and defense mechanisms. In2024 IEEE 33rd Asian Test Symposium (ATS). IEEE, 1–6

  54. [62]

    Habibur Rahaman, Atri Chatterjee, and Swarup Bhunia. 2026. SAMURAI: Run- time Attack Detection in AI Accelerators Using AI Performance Counters.IEEE Transactions on Circuits and Systems for Artificial Intelligence(2026)

  55. [63]

    Habibur Rahaman, Sudipta Paria, Atri Chatterjee, and Swarup Bhunia. 2026. Evolving Landscape of Attacks on AI Hardware and Robust Defenses. In2026 27th International Symposium on Quality Electronic Design (ISQED). IEEE, 1–6

  56. [64]

    Adnan Siraj Rakin, Zhezhi He, and Deliang Fan. 2020. Bit-flip attack: Crushing neural network with progressive bit search. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

  57. [65]

    Adnan Siraj Rakin, Zhezhi He, Jingtao Li, Fan Yao, Chaitali Chakrabarti, and Deliang Fan. 2021. T-bfa: Targeted bit-flip adversarial weight attack.IEEE Transactions on Pattern Analysis and Machine Intelligence44, 11 (2021), 7928– 7939

  58. [66]

    ReversingLabs. [n. d.]. ReversingLabs YARA Rules: Ransomware, Backdoor, Infos- tealer, Trojan, Virus Signatures. https://github.com/reversinglabs/reversinglabs- yara-rules. Accessed: 08-19-2025

  59. [67]

    Lutz Roeder. 2022. Netron: Visualizer for neural network, deep learning, and machine learning models. doi:10.5281/zenodo.5854961 Accessed: 2025-08-19

  60. [68]

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al

  61. [69]

    Amit Sabne. 2020. Xla: Compiling machine learning for peak performance

  62. [70]

    Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Dis- tilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter.arXiv preprint arXiv:1910.01108(2019)

  63. [71]

    Andrei Semenov et al . 2025. Benchmarking Optimizers for Large Language Model Pretraining.arXiv preprint arXiv:2509.01440(2025)

  64. [72]

    Ali Shafahi, W Ronny Huang, Mahyar Najibi, Octavian Suciu, Christoph Studer, Tudor Dumitras, and Tom Goldstein. 2018. Poison frogs! targeted clean-label poisoning attacks on neural networks.Advances in neural information processing systems31 (2018)

  65. [73]

    Guangyu Shen, Siyuan Cheng, Zhuo Zhang, Guanhong Tao, Kaiyuan Zhang, Hanxi Guo, Lu Yan, Xiaolong Jin, Shengwei An, Shiqing Ma, et al. 2025. Bait: Large language model backdoor scanning by inverting attack target. In2025 IEEE Symposium on Security and Privacy (SP). IEEE, 1676–1694

  66. [74]

    Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional net- works for large-scale image recognition.arXiv preprint arXiv:1409.1556(2014)

  67. [75]

    SLSA SLSA. 2024. Supply-chain Levels for Software Artifacts

  68. [76]

    Fnu Suya, Saeed Mahloujifar, Anshuman Suri, David Evans, and Yuan Tian. 2021. Model-targeted poisoning attacks with provable convergence. InInternational Conference on Machine Learning. PMLR, 10000–10010

  69. [77]

    ThreatLabz. 2025. Malicious PyPI Packages Deliver SilentSync RAT. https://www.zscaler.com/blogs/security-research/malicious-pypi-packages- deliver-silentsync-rat Zscaler Security Research

  70. [78]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971(2023)

  71. [79]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yas- mine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288(2023)

  72. [80]

    Binghui Wang and Neil Zhenqiang Gong. 2018. Stealing Hyperparameters in Machine Learning. In2018 IEEE Symposium on Security and Privacy (SP). IEEE, 36–52

  73. [81]

    Bolun Wang, Yuanshun Yao, Shawn Shan, Hui Li, Bimal Viswanath, Haitao Zheng, and Ben Y. Zhao. 2019. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. InProceedings of the IEEE Symposium on Security and Privacy (S&P)

  74. [82]

    2020.{DeepHammer}: Depleting the intelligence of deep neural networks through targeted chain of bit flips

    Fan Yao, Adnan Siraj Rakin, and Deliang Fan. 2020.{DeepHammer}: Depleting the intelligence of deep neural networks through targeted chain of bit flips. In 29th USENIX Security Symposium (USENIX Security 20). 1463–1480

  75. [83]

    Yara-Rules Project. [n. d.]. Community Malware YARA Rules. https://github. com/Yara-Rules/rules. Accessed: 08-19-2025

  76. [84]

    Boyang Zhang, Xinlei He, Yun Shen, Tianhao Wang, and Yang Zhang. 2023. A plot is worth a thousand words: model information stealing attacks via scientific plots. In32nd USENIX Security Symposium (USENIX Security 23). 5289–5306

  77. [85]

    bad hyperparameters

    Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022. Opt: Open pre-trained transformer language models.arXiv preprint arXiv:2205.01068 (2022). (A)iSpy A Ethical Considerations Whi...

  78. [87]

    clean model, dirty system

    and target ID (𝑦𝑡𝑎𝑟𝑔𝑒𝑡 ) resolved, the intervention proceeds: For at- tacks on training engine, (A)iSpy overwrites the ground-truth label 𝑦 with𝑦𝑡𝑎𝑟𝑔𝑒𝑡 for the subpopulation samples (𝑚= 1). This poisons the decision boundary, teaching the model to associate the subpop- ulation...

  79. [88]

    The attack latches at epoch 6 and remains permanently active for 21,114 batches thereafter

    The baseline model achieves59.90%test accuracy, whereas the sabotaged model achieves58 .76%, yielding an absolute accuracy drop of1.14percentage points. The attack latches at epoch 6 and remains permanently active for 21,114 batches thereafter. The injected noise standard devi...

  80. [2015]

    Imagenet large scale visual recognition challenge.International Journal of Computer Vision115, 3 (2015), 211–252

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.