Pith. sign in

REVIEW 3 major objections 5 minor 120 references

SparSEEty: Extracting Tokens from Sparsity-Exploiting LLM Serving Systems via Deterministic Side Channels

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Sparsity-boosted LLM serving leaks full prompts and responses to the cloud host, even inside Intel TDX.

desk verdict A credible new attack that turns binary neuron-activation traces leaked through page faults into full prompt/response recovery; the core result holds up, but the writeup must clarify the per-neuron page-layout assumption and release artifacts. read the letter →

arxiv 2608.02995 v1 pith:5BZP6NW7 submitted 2026-08-04 cs.CR cs.AI

Yongwan Jo , Jinyoung Park , Euihyun Lee , Dokyung Song This is my paper · ORCID
classification cs.CRcs.AI
keywords side-channelattacktokenextractionLLMservingactivationsparsityconfidentialcomputingIntelTDXpagefaultneuronoracle
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the sparsity optimizations used to speed up LLM serving create a deterministic side channel that leaks the identity of every input and output token, even when the model runs inside a confidential VM. It builds a neuron-activation oracle from page faults triggered by selective accesses to down-projection weights, then inverts the resulting binary activation traces to reconstruct the user's prompt and the model's response. The authors show that monitoring just 100 first-layer neurons—roughly 0.015% to 0.028% of all FFN neurons—is enough to reach near-perfect reconstruction (BLEU >0.95) across five models and five datasets, at an inference overhead of 3.7% to 7.2%. If correct, the result overturns the assumption that confidential-computing enclaves protect user prompts from the host.

What carries the argument

The load-bearing object is the neuron-activation oracle: a mapping from page-fault events to a binary vector indicating which of the monitored FFN neurons were activated for each token, assembled by arming the guest-physical pages that hold each neuron's down-projection weights. Two further mechanisms carry the attack: entropy-based selective monitoring, which picks the ~100 first-layer neurons whose joint activation pattern is most discriminative across tokens, and likelihood-guided vocabulary search, which resolves the ambiguity of binary patterns by testing tokens in descending order of their next-token probability from a full forward pass.

What would settle it

Run the same attack against a sparsity-exploiting serving system that packs two neurons' down-projection weights into a single 4 KiB page, or uses a transposed weight matrix that interleaves neurons across pages; if page-fault traces can no longer be attributed to individual neurons, reconstruction accuracy should collapse, showing that the oracle rather than the inversion is the limiting step.

Watch

Extended reading notes

Core claim

SparSEEty establishes that input-dependent weight accesses in sparsity-exploiting LLM serving systems are a reliable, reversible side channel. During sparse down projection, the system reads only the columns of the down-projection weight matrix whose neurons were activated, and because each neuron's weights occupy dedicated page-aligned memory (8 KiB for Llama-2-7b), a page fault on that page reports whether that neuron fired. The attacker identifies the guest physical addresses of these weight pages by chaining block-I/O, page-allocation, and page-fault oracles, then uses chained page arming to capture binary activation traces for a selected subset of first-layer neurons without single-stepping. Offline, the attacker inverts each trace autoregressively: a full forward pass ranks candidate tokens by next-token probability, and the first candidate whose binary trace matches the observation is selected. With 100 monitored neurons chosen by joint entropy, this guided search reconstructs both prompts and responses with BLEU scores above 0.95 across OPT-6.7b, ReluLLaMA-7B, Nemotron-3-8B-Base-4k, Llama-2-7b, and Gemma-7b, and remains effective when a private LoRA adapter is applied if the neuron budget grows to 400.

Load-bearing premise

The attack works only if each neuron's down-projection weights sit in their own page-aligned memory, so that a page fault unambiguously means that neuron fired; the paper verifies this for the PowerInfer/Llama-2 layout but not for all sparsity-exploiting stacks.

Editorial extensions

If this is right

  • A host-side adversary of a confidential VM can recover the full conversation—user prompt and generated response—without query access to the LLM service.
  • Monitoring 100 of tens of thousands of neurons is enough, so the attack stays covert: 3.7% to 7.2% inference overhead and a small per-token page-fault count.
  • The attack transfers across activation functions (ReLU, ReLU^2, SwiGLU, GeGLU) when sparsity is natural or induced by magnitude thresholding.
  • Private LoRA adapters do not block reconstruction; the attacker needs only the base-model weights and 400 monitored neurons.
  • Simply encrypting the model file at rest does not help, since block-level layout is preserved; defenses must randomize or hide the weight-page layout.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Any optimization that turns token identity into a memory-access pattern—MoE expert routing, speculative decoding outcomes, KV-cache hits—may be invertible through the same oracle-plus-search recipe, so the result reads as a template for auditing LLM serving optimizations inside TEEs.
  • The page-fault oracle depends on a per-neuron page-aligned weight layout; a serving system that packs or transposes weights to defeat page-granular monitoring could break this specific attack, but the same activation information may still leak through cache-line or off-chip bus channels.
  • Because the inversion uses only binary activations of the first layer, the attack could plausibly be extended to sparse kernels on GPUs or NPUs that expose access patterns through shared memory or I/O, not just CPU page faults.
  • The 3.7–7.2% overhead is a detection signal: a defender inside the guest could look for anomalous CVM exits or page-fault rates during inference, though detecting a determined host is hard.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents SparSEEty, an attack that recovers tokens from a sparsity-exploiting LLM serving system running inside an Intel TDX confidential VM. The attack combines page-fault, block I/O, and page-allocation side channels to identify the guest physical addresses of down-projection weight pages, then uses a chained page-arming technique to record which of a selected subset of first-layer FFN neurons are activated for each token. The resulting binary activation traces are inverted autoregressively by ranking vocabulary candidates with the victim model's own next-token probabilities and selecting the first candidate whose exact binary trace matches the observation. The evaluation covers five models with different activation functions, five datasets, and 100 prompts per dataset, reporting BLEU scores above 0.95 with 100 monitored neurons and inference overhead of 3.7% to 7.2%. Additional experiments address robustness to private LoRA adapters and partial CPU offloading of cold neurons.

Significance. If the central claims hold, SparSEEty demonstrates a serious privacy risk: sparsity-exploiting inference optimizations inside a confidential VM leak input-dependent neuron-weight accesses, and those binary activation traces can be inverted to reconstruct a user's full prompt and response. The paper has notable strengths: a concrete end-to-end attack implementation, a realistic host-side TDX adversary model, a broad evaluation across five models and five datasets, explicit overhead measurements, ablation studies for neuron selection and guided search, and a discussion of mitigations and limitations. The principal weakness is that the observability claim rests on a weight-layout assumption that is not validated for all evaluated models, and the attack is demonstrated only on a modified PowerInfer configuration. Consequently, the paper's general claim over 'sparsity-exploiting LLM serving systems' is currently broader than the evidence supports.

major comments (3)
  1. [§V-B, Table II] The observability claim in §V-B is that 'By monitoring either of the two 4KiB pages, SparSEEty can reliably determine whether each neuron was activated.' This requires each neuron's down-projection row to occupy whole, page-exclusive 4KiB pages. Table II reports that Gemma-7b has a neuron size of 6,144 bytes, i.e., 1.5 pages. In a standard row-major packed layout, a page in the interior of the matrix contains the tail of one neuron and the head of the next, so a page fault is an OR of activations from two or more neurons. Algorithm 1's line 12 requires exact equality between the observed and candidate binary traces, so any OR-collision breaks reconstruction. The manuscript does not state whether PowerInfer pads rows to 8KiB for Gemma-7b, nor does it show that an exclusive page can be selected for every monitored neuron in this model. This is load-bearing for the claim of near-100% accuracy 'across all models and datasets that were evaluated' (§VIII-B).
  2. [§VIII-A, Abstract] The attack is instantiated on a single serving system, described as 'PowerInfer' modified with magnitude-based thresholding. The abstract and title generalize the claim to 'sparsity-exploiting LLM serving systems,' but other serving stacks (e.g., Deja Vu, PowerInfer-2, LLM in a Flash) may store down-projection matrices transposed, interleave neurons within pages, or gather active rows into a scratch buffer. Under any of those layouts, the page-fault oracle does not provide per-neuron activation information, and the attack's core mechanism fails. The authors should either demonstrate the attack on at least one alternative layout or explicitly restrict the paper's claims to the evaluated PowerInfer-style row-major layout.
  3. [§VII, §VIII] The online phase relies on an intricate arrangement of chained page arming, double-turnstile barriers, and per-token fault attribution, but no code or artifact is released, and the correctness of the chain is validated only indirectly through final BLEU scores. Releasing the implementation or providing a trace-level validation that each page fault is correctly attributed to a specific neuron and token position would substantially strengthen the end-to-end claim and allow independent verification of the page-to-neuron exclusivity assumption.
minor comments (5)
  1. [Table V] The reconstructed Nemotron output contains '&', which appears to be an HTML escaping artifact and should be rendered as an ampersand.
  2. [Fig. 6, Fig. 7] The legends and line styles may be difficult to distinguish in grayscale; distinct markers per model would improve readability.
  3. [§VIII-B] BLEU scores are reported as averages over 100 prompts; including standard deviations or box plots would better support the word 'consistently' in the paper's claim.
  4. [§V-A] The parameter k=16,000 used when arming pages near the high watermark is introduced without a sensitivity analysis; a brief discussion of how dependent GPA discovery is on this value would be useful.
  5. [§V-C, §VI] The paper uses 'first layer' and 'first FFN layer' interchangeably; it would be clearer to specify that this means the first Transformer layer's FFN sublayer, not the embedding layer.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SparSEEty's inversion matches side-channel traces against the victim model's own deterministic forward pass, with neuron selection on a public corpus and no fitted parameters in the central claim.

full rationale

The paper's derivation chain is self-contained. The inversion (Algorithm 1, Section VI-B) takes observed binary activation traces from the page-fault oracle and searches the vocabulary by recomputing first-layer activation patterns for candidate tokens with the victim LLM's forward pass, ranking candidates by the model's own next-token probabilities. This is an inverse-map/dictionary procedure, not a definitional identity: the observed traces are independent side-channel measurements, and the candidate traces are recomputed from the model weights. Neuron selection (Section V-C) is performed offline on the Wikipedia corpus before any attack traces are collected, so it is not fitted to the target data; the reported accuracy is a held-out transfer result. The only self-citation in the manuscript is [38] (ASGARD), which appears in a list of related work on DNN serving in TEEs and is not load-bearing for any security claim. The paper's stated limitations, such as the page-aligned per-neuron layout precondition in Section V-B and the unknown-LoRA scenario in Section VIII-C, are empirical preconditions or acknowledged scope restrictions, not circular inputs. No equation or fitted parameter is reused as a 'prediction' by construction.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claim rests on several domain assumptions: the specific serving configuration (PowerInfer-style sparse down projection on CPU in a CVM), the page-aligned per-neuron weight layout, full offline access to base weights, the informativeness of first-layer binary activations, and the usefulness of LM priors for resolving ambiguity. The paper states most of these explicitly in Section III, and empirically validates them for the evaluated systems. There are no invented entities or new physical postulates. The free parameters are implementation constants, not fitted to the token-extraction outcome.

free parameters (2)
  • k (pages armed below high watermark) = 16,000
    Set based on empirical observation in Section V-A to catch the page where LLM weights land after block I/O bounce. Not fitted to the token-extraction outcome, but hand-tuned.
  • Number of monitored first-layer neurons = 100 (up to 400 for LoRA and offloading cases)
    Chosen as the setting that reaches BLEU >0.95 across evaluated datasets (Section VIII-B, Fig. 6). This is a scan parameter, not a fitted constant in a derivation, but it is tuned to the target accuracy.
assumptions (5)
  • domain assumption Each neuron's down projection weights occupy dedicated page-aligned (4KiB or larger) memory so a page fault isolates a single neuron's activation
    Invoked in Section V-B to justify per-neuron observability; verified only via Llama-2-7b's 8KiB per-neuron size.
  • domain assumption The victim serving system uses activation-sparsity-based sparse down projection on the CPU inside a CVM, with model parallelism and batched prefill (PowerInfer-style)
    Threat model in Section III; the entire attack relies on this specific serving configuration.
  • domain assumption Attacker has full offline access to the base LLM weights
    Assumed in Section III in line with prior work; used in Section VI to compute candidate activation patterns and next-token probabilities.
  • domain assumption The model's next-token probabilities conditioned on previously reconstructed tokens rank the true token sufficiently high for the guided search to find a matching activation pattern
    The inversion (Section VI-B) depends on the LM prior resolving binary-pattern ambiguity; the paper verifies this empirically but it is not a proven property.
  • domain assumption First-layer FFN binary activations preserve enough token-specific information for reconstruction
    Motivated in Sections V-C and VI-A by referencing prior inversion work; required for the 100-neuron result.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SparSEEty: Extracting Tokens from Sparsity-Exploiting LLM Serving Systems via Deterministic Side Channels." pith.science (2026). https://pith.science/paper/5BZP6NW7

@misc{pith2026260802995,
  author       = {Pith},
  title        = {Pith review of: SparSEEty: Extracting Tokens from Sparsity-Exploiting LLM Serving Systems via Deterministic Side Channels},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5BZP6NW7}},
  note         = {Machine review of arXiv:2608.02995}
}
read the original abstract

Modern large language models (LLMs) exhibit activation sparsity, wherein only a subset of their neurons is activated for given input tokens. Researchers have leveraged this property to optimize LLM serving systems by omitting weight accesses and computations pertaining to inactive neurons. Unfortunately, however, such optimizations create input-dependent weight accesses, which can be leaked over side channels. We present SparSEEty, a new token extraction attack that exploits input-dependent neuron weight accesses introduced by sparsity-exploiting LLM serving systems. SparSEEty first constructs a neuron-activation oracle using neuron weight access side channels during LLM inference, and then inverts the activation traces to reconstruct the input tokens, forming an end-to-end token extraction attack. We instantiate SparSEEty against an LLM serving system protected inside an Intel TDX confidential virtual machine (CVM), addressing three key challenges: (i) constructing a neuron-activation oracle using a combination of side channels exposed by CVMs, (ii) reducing inference-time overheads of neuron activation monitoring for covertness, and (iii) accurately inverting partial binary activation traces back to tokens. Our evaluation shows that SparSEEty can reconstruct both prompt and response tokens with consistently high BLEU scores (>0.95) across various models and datasets, while incurring monitoring overheads of 3.7% to 7.2%.

Figures

Figures reproduced from arXiv: 2608.02995 by the authors.

Figure 1
Figure 1. High-level overview of the SparSEEty attack. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Finding private GPAs of down projection weights. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Page-arming chain used to monitor the activations of [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Distribution of activation frequencies (fraction of tokens activating each neuron) on the Wikipedia dataset. Red dashed [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Wall-clock inference time before & after SparSEEty’s monitoring. Results were obtained while each LLM was processing [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Reconstruction accuracy across five datasets, with varying numbers of monitored neurons. Measured by BLEU scores. [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Reconstruction accuracy obtained after (a) adapting [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Reconstruction accuracy obtained via (a) random neuron [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Effectiveness of SparSEEty’s guided search in terms of [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

120 extracted references · 63 canonical work pages

  1. [1]

    Toy models of superposition,

    N. Elhage, T. Hume, C. Olsson, N. Schiefer, T. Henighan, S. Kravec, Z. Hatfield-Dodds, R. Lasenby, D. Drain, C. Chen, R. Grosse, S. Mc- Candlish, J. Kaplan, D. Amodei, M. Wattenberg, and C. Olah, “Toy models of superposition,”arXiv preprint arXiv:2209.10652, 2022

    arXiv 2022

  2. [2]

    Steering large language model activations in sparse spaces,

    R. Bayat, A. Rahimi-Kalahroudi, M. Pezeshki, S. Chandar, and P. Vincent, “Steering large language model activations in sparse spaces,” arXiv preprint arXiv:2503.00177, 2025

    arXiv 2025

  3. [3]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” inAdvances in Neural Information Processing Systems (NeurIPS), 2017

    2017

  4. [4]

    Deja Vu: Contextual sparsity for efficient LLMs at inference time,

    Z. Liu, J. Wang, T. Dao, T. Zhou, B. Yuan, Z. Song, A. Shrivastava, C. Zhang, Y . Tian, C. Reet al., “Deja Vu: Contextual sparsity for efficient LLMs at inference time,” inProceedings of the International Conference on Machine Learning (ICML), 2023

    2023

  5. [5]

    PIT: Optimization of dynamic sparse deep learning models via permutation invariant transformation,

    N. Zheng, H. Jiang, Q. Zhang, Z. Han, L. Ma, Y . Yang, F. Yang, C. Zhang, L. Qiu, M. Yanget al., “PIT: Optimization of dynamic sparse deep learning models via permutation invariant transformation,” inProceedings of the ACM Symposium on Operating Systems Principles (SOSP), 2023

    2023

  6. [6]

    LLM in a flash: Efficient large language model inference with limited memory,

    K. Alizadeh, S. I. Mirzadeh, D. Belenko, S. Khatamifard, M. Cho, C. C. Del Mundo, M. Rastegari, and M. Farajtabar, “LLM in a flash: Efficient large language model inference with limited memory,” inProceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2024

    2024

  7. [7]

    PowerInfer: Fast large language model serving with a consumer-grade GPU,

    Y . Song, Z. Mi, H. Xie, and H. Chen, “PowerInfer: Fast large language model serving with a consumer-grade GPU,” inProceedings of the ACM Symposium on Operating Systems Principles (SOSP), 2024

    2024

  8. [8]

    PowerInfer-2: Fast large language model inference on a smartphone,

    Z. Xue, Y . Song, Z. Mi, L. Chen, Y . Xia, and H. Chen, “PowerInfer-2: Fast large language model inference on a smartphone,”arXiv preprint arXiv:2406.06282, 2024

    arXiv 2024

Show all 120 references
  1. [9]

    Telling your secrets without page faults: Stealthy page table-based attacks on enclaved execution,

    J. Van Bulck, N. Weichbrodt, R. Kapitza, F. Piessens, and R. Strackx, “Telling your secrets without page faults: Stealthy page table-based attacks on enclaved execution,” inProceedings of the USENIX Security Symposium (Security), 2017

    2017

  2. [10]

    Controlled-channel attacks: Determin- istic side channels for untrusted operating systems,

    Y . Xu, W. Cui, and M. Peinado, “Controlled-channel attacks: Determin- istic side channels for untrusted operating systems,” inProceedings of the IEEE Symposium on Security and Privacy (IEEE S&P), 2015

    2015

  3. [11]

    HIDE: An infrastructure for efficiently protecting information leakage on the address bus,

    X. Zhuang, T. Zhang, and S. Pande, “HIDE: An infrastructure for efficiently protecting information leakage on the address bus,” in Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2004

    2004

  4. [12]

    An off-chip attack on hardware enclaves via the memory bus,

    D. Lee, D. Jung, I. T. Fang, C.-C. Tsai, and R. A. Popa, “An off-chip attack on hardware enclaves via the memory bus,” inProceedings of the USENIX Security Symposium (Security), 2020

    2020

  5. [13]

    Transparent domain extensions: Breaking Intel TEE implementations via DDR5 memory bus interposition,

    J. Chuang, A. Seto, N. Berrios, S. van Schaik, C. Garman, and D. Genkin, “Transparent domain extensions: Breaking Intel TEE implementations via DDR5 memory bus interposition,” inProceedings of the IEEE Symposium on Security and Privacy (IEEE S&P), 2026, to appear

    2026

  6. [14]

    Unsupervised feature selection towards pattern discrimination power,

    W. Seo and J. Lee, “Unsupervised feature selection towards pattern discrimination power,” inProceedings of the Conference on Uncertainty in Artificial Intelligence (UAI), 2024

    2024

  7. [15]

    Understanding deep image representa- tions by inverting them,

    A. Mahendran and A. Vedaldi, “Understanding deep image representa- tions by inverting them,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015

    2015

  8. [16]

    Prompt inversion attack against collaborative inference of large language models,

    W. Qu, Y . Zhou, Y . Wu, T. Xiao, B. Yuan, Y . Li, and J. Zhang, “Prompt inversion attack against collaborative inference of large language models,” inProceedings of the IEEE Symposium on Security and Privacy (IEEE S&P), 2025

    2025

  9. [17]

    Depth gives a false sense of privacy: LLM internal states inversion,

    T. Dong, Y . Meng, S. Li, G. Chen, Z. Liu, and H. Zhu, “Depth gives a false sense of privacy: LLM internal states inversion,” inProceedings of the USENIX Security Symposium (Security), 2025

    2025

  10. [18]

    Language model inversion,

    J. X. Morris, W. Zhao, J. T. Chiu, V . Shmatikov, and A. M. Rush, “Language model inversion,” inProceedings of the International Conference on Learning Representations (ICLR), 2024

    2024

  11. [19]

    Linux kernel virtual machine

    Open Virtualization Alliance, “Linux kernel virtual machine.” [Online]. Available: https://www.linux-kvm.org

  12. [20]

    QEMU, a fast and portable dynamic translator,

    F. Bellard, “QEMU, a fast and portable dynamic translator,” in Proceedings of the USENIX Annual Technical Conference, FREENIX Track, 2005

    2005

  13. [21]

    LoRA: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  14. [22]

    The lazy neuron phenomenon: On emergence of activation sparsity in Transformers,

    Z. Li, C. You, S. Bhojanapalli, D. Li, A. S. Rawat, S. J. Reddi, K. Ye, F. Chern, F. Yu, R. Guo, and S. Kumar, “The lazy neuron phenomenon: On emergence of activation sparsity in Transformers,” inProceedings of the International Conference on Learning Representations (ICLR), 2023

    2023

  15. [23]

    ReLU 2 wins: Discovering efficient activation functions for sparse LLMs,

    Z. Zhang, Y . Song, G. Yu, X. Han, Y . Lin, C. Xiao, C. Song, Z. Liu, Z. Mi, and M. Sun, “ReLU 2 wins: Discovering efficient activation functions for sparse LLMs,”arXiv preprint arXiv:2402.03804, 2024

    2024 arXiv

  16. [24]

    ReLU strikes back: Exploiting activation sparsity in large language models,

    S. I. Mirzadeh, K. Alizadeh-Vahid, S. Mehta, C. C. del Mundo, O. Tuzel, G. Samei, M. Rastegari, and M. Farajtabar, “ReLU strikes back: Exploiting activation sparsity in large language models,” inProceedings of the International Conference on Learning Representations (ICLR), 2024

    2024

  17. [25]

    Efficient LLM inference using dynamic input pruning and cache-aware masking,

    M. Federici, D. Belli, M. V . Baalen, A. Jalalirad, A. Skliar, B. Major, M. Nagel, and P. Whatmough, “Efficient LLM inference using dynamic input pruning and cache-aware masking,” inProceedings of the Conference on Machine Learning and Systems (MLSys), 2025

    2025

  18. [26]

    Training-free activation sparsity in large language models,

    J. Liu, P. Ponnusamy, T. Cai, H. Guo, Y . Kim, and B. Athiwaratkun, “Training-free activation sparsity in large language models,” inPro- ceedings of the International Conference on Learning Representations (ICLR), 2025

    2025

  19. [27]

    Intel ® Software Guard Extensions programming reference,

    Intel, “Intel ® Software Guard Extensions programming reference,”

  20. [28]

    SEVurity: No security without integrity: Breaking integrity-free memory encryption with minimal assumptions,

    L. Wilke, J. Wichelmann, M. Morbitzer, and T. Eisenbarth, “SEVurity: No security without integrity: Breaking integrity-free memory encryption with minimal assumptions,” inProceedings of the IEEE Symposium on Security and Privacy (IEEE S&P), 2020

    2020

  21. [29]

    Exploiting unprotected I/O operations in AMD’s secure encrypted virtualization,

    M. Li, Y . Zhang, Z. Lin, and Y . Solihin, “Exploiting unprotected I/O operations in AMD’s secure encrypted virtualization,” inProceedings of the USENIX Security Symposium (Security), 2019

    2019

  22. [30]

    TDXdown: Single-stepping and instruction counting attacks against Intel TDX,

    L. Wilke, F. Sieck, and T. Eisenbarth, “TDXdown: Single-stepping and instruction counting attacks against Intel TDX,” inProceedings of the ACM Conference on Computer and Communications Security (CCS), 2024

    2024

  23. [31]

    TDXRay: Microarchitectural side-channel analysis of Intel TDX for real-world workloads,

    T. Hornetz, H. Yavarzadeh, A. Cheu, A. Gascon, L. Gerlach, D. Moghimi, P. Schoppmann, M. Schwarz, and R. Zhang, “TDXRay: Microarchitectural side-channel analysis of Intel TDX for real-world workloads,” inProceedings of the IEEE Symposium on Security and Privacy (IEEE S&P), 202...

    2026

  24. [32]

    AMD SEV-SNP,

    AMD, “AMD SEV-SNP,” 2020. [Online]. Available: https://www.amd. com/content/dam/amd/en/documents/epyc-business-docs/white-papers/ SEV-SNP-strengthening-vm-isolation-with-integrity-protection-and-more. pdf

    2020

  25. [33]

    Intel® Trust Domain Extensions,

    Intel, “Intel® Trust Domain Extensions,” 2022. [Online]. Available: https: //cdrdv2-public.intel.com/690419/TDX-Whitepaper-February2022.pdf

    2022

  26. [34]

    Intel® Trust Domain Extensions (Intel® TDX) module base archi- tecture specification,

    ——, “Intel® Trust Domain Extensions (Intel® TDX) module base archi- tecture specification,” 2025. [Online]. Available: https://cdrdv2-public. intel.com/853286/intel-tdx-module-base-spec-348549006.pdf

    2025

  27. [35]

    QEMU system emulation user’s guide

    “QEMU system emulation user’s guide.” [Online]. Available: https://www.qemu.org/docs/master/system/index.html

  28. [36]

    DarkneTZ: Towards model privacy at the edge using trusted execution environments,

    F. Mo, A. S. Shamsabadi, K. Katevas, S. Demetriou, I. Leontiadis, A. Cavallaro, and H. Haddadi, “DarkneTZ: Towards model privacy at the edge using trusted execution environments,” inProceedings of the Annual International Conference on Mobile Systems, Applications, and Service...

    2020

  29. [37]

    Guaran- TEE: Towards attestable and private ML with CCA,

    S. Siby, S. Abdollahi, M. Maheri, M. Kogias, and H. Haddadi, “Guaran- TEE: Towards attestable and private ML with CCA,” inProceedings of the Workshop on Machine Learning and Systems (EuroMLSys), 2024

    2024

  30. [38]

    ASGARD: Protecting on- device deep neural networks with virtualization-based trusted execution environments,

    M. Moon, M. Kim, J. Jung, and D. Song, “ASGARD: Protecting on- device deep neural networks with virtualization-based trusted execution environments,” inProceedings of the Network and Distributed System Security Symposium (NDSS), 2025

    2025

  31. [39]

    PipeLLM: Fast and confidential large language model services with speculative pipelined encryption,

    Y . Tan, C. Tan, Z. Mi, and H. Chen, “PipeLLM: Fast and confidential large language model services with speculative pipelined encryption,” in Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2025

    2025

  32. [40]

    TZ-LLM: Protecting on-device large language models with Arm TrustZone,

    X. Wang, J. Shi, Z. Zhao, Y . Yu, Z. Hua, and J. Gu, “TZ-LLM: Protecting on-device large language models with Arm TrustZone,” inProceedings of the ACM European Conference on Computer Systems (EuroSys), 2026

    2026

  33. [41]

    Enabling more private generative AI,

    Google, “Enabling more private generative AI,”

  34. [42]

    Confidential inference via trusted virtual machines,

    Anthropic, “Confidential inference via trusted virtual machines,”

  35. [43]

    Confidential AI,

    Microsoft, “Confidential AI,” 2023. [Online]. Available: https://learn. microsoft.com/en-us/azure/confidential-computing/confidential-ai

    2023

  36. [44]

    Confidential LLM inference: Performance and cost across CPU and GPU TEEs,

    M. Chrapek, M. Copik, E. Mettaz, and T. Hoefler, “Confidential LLM inference: Performance and cost across CPU and GPU TEEs,” inProceedings of the IEEE International Symposium on Workload Characterization (IISWC), 2025

    2025

  37. [45]

    Megatron-LM: Training multi-billion parameter language models using model parallelism,

    M. Shoeybi, M. Patwary, R. Puri, P. LeGresley, J. Casper, and B. Catanzaro, “Megatron-LM: Training multi-billion parameter language models using model parallelism,”arXiv preprint arXiv:1909.08053, 2019

    1909 arXiv

  38. [46]

    I know what you asked: Prompt leakage via KV-cache sharing in multi-tenant LLM serving,

    Y . Z. Guanlong Wu, Zheng Zhang, “I know what you asked: Prompt leakage via KV-cache sharing in multi-tenant LLM serving,” in Proceedings of the Network and Distributed System Security Symposium (NDSS), 2025

    2025

  39. [47]

    I know what you said: Unveiling hardware cache side-channels in local large language model inference,

    Z. Gao, J. Hu, F. Guo, Y . Zhang, Y . Han, S. Liu, H. Li, and Z. Lv, “I know what you said: Unveiling hardware cache side-channels in local large language model inference,” inProceedings of the USENIX Security Symposium (Security), 2025

    2025

  40. [48]

    Learning to embed categorical features without embedding tables for recommendation,

    W.-C. Kang, D. Z. Cheng, T. Yao, X. Yi, T. Chen, L. Hong, and E. H. Chi, “Learning to embed categorical features without embedding tables for recommendation,” inProceedings of the ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 2021

    2021

  41. [49]

    Efficient memory side-channel protection for embedding generation in machine learning,

    M. Umar, A. P. Marathe, M. D. Gupta, S. J. Ghosh, G. E. Suh, and W. Xiong, “Efficient memory side-channel protection for embedding generation in machine learning,” inProceedings of the IEEE Interna- tional Symposium on High Performance Computer Architecture (HPCA), 2025

    2025

  42. [50]

    virtio: Towards a de-facto standard for virtual I/O devices,

    R. Russell, “virtio: Towards a de-facto standard for virtual I/O devices,” SIGOPS Operating Systems Review, vol. 42, no. 5, pp. 95–103, Jul. 2008

    2008

  43. [51]

    Virtual I/O device (VIRTIO) version 1.2,

    “Virtual I/O device (VIRTIO) version 1.2,” 2022. [Online]. Available: https://docs.oasis-open.org/virtio/virtio/v1.2/virtio-v1.2.pdf

    2022

  44. [52]

    Implementing dm-verity,

    Google, “Implementing dm-verity,” 2024. [Online]. Available: https: //source.android.com/docs/security/features/verifiedboot/dm-verity

    2024

  45. [53]

    Intel trust domain extensions (TDX) security review,

    E. Aktas, C. Cohen, J. Eads, J. Forshaw, and F. Wilhelm, “Intel trust domain extensions (TDX) security review,” 2023. [Online]. Available: https://services.google.com/fh/files/misc/intel tdx - full report 041423.pdf

    2023

  46. [54]

    Shadow in the cache: Unveiling and mitigating privacy risks of KV-cache in LLM inference,

    Z. Luo, S. Shao, S. Zhang, L. Zhou, Y . Hu, C. Zhao, Z. Liu, and Z. Qin, “Shadow in the cache: Unveiling and mitigating privacy risks of KV-cache in LLM inference,” inProceedings of the Network and Distributed System Security Symposium (NDSS), 2026

    2026

  47. [55]

    LLMmap: Fingerprinting for large language models,

    D. Pasquini, E. M. Kornaropoulos, and G. Ateniese, “LLMmap: Fingerprinting for large language models,” inProceedings of the USENIX Security Symposium (Security), 2025

    2025

  48. [56]

    Reverse engineering convolutional neural networks through side-channel information leaks,

    W. Hua, Z. Zhang, and G. E. Suh, “Reverse engineering convolutional neural networks through side-channel information leaks,” inProceedings of the Annual Design Automation Conference (DAC), 2018

    2018

  49. [57]

    Cache telepathy: Leveraging shared resource attacks to learn DNN architectures,

    M. Yan, C. W. Fletcher, and J. Torrellas, “Cache telepathy: Leveraging shared resource attacks to learn DNN architectures,” inProceedings of the USENIX Security Symposium (Security), 2020

    2020

  50. [58]

    DeepTheft: Stealing DNN model architectures through power side channel,

    Y . Gao, H. Qiu, Z. Zhang, B. Wang, H. Ma, A. Abuadbba, M. Xue, A. Fu, and S. Nepal, “DeepTheft: Stealing DNN model architectures through power side channel,” inProceedings of the IEEE Symposium on Security and Privacy (IEEE S&P), 2024

    2024

  51. [60]

    TDXploit: Novel techniques for single-stepping and cache attacks on Intel TDX,

    F. Rauscher, L. Wilke, H. Weissteiner, T. Eisenbarth, and D. Gruss, “TDXploit: Novel techniques for single-stepping and cache attacks on Intel TDX,” inProceedings of the USENIX Security Symposium (Security), 2025

    2025

  52. [61]

    Downey,The little book of semaphores

    A. Downey,The little book of semaphores. Green Tea Press, 2008, vol. 2, no. 2

    2008

  53. [62]

    A law of next-token prediction in large language models,

    H. He and W. J. Su, “A law of next-token prediction in large language models,”Phys. Rev. E, 2025

    2025

  54. [63]

    OPT: Open pre-trained Transformer language models,

    S. Zhang, S. Roller, N. Goyal, M. Artetxe, M. Chen, S. Chen, C. Dewan, M. Diab, X. Li, X. V . Lin, T. Mihaylov, M. Ott, S. Shleifer, K. Shuster, D. Simig, P. S. Koura, A. Sridhar, T. Wang, and L. Zettlemoyer, “OPT: Open pre-trained Transformer language models,” 2022

    2022

  55. [64]

    Sparse large language models with ReLU activation,

    SpaseLLM Team, “Sparse large language models with ReLU activation,” 2023. [Online]. Available: https://huggingface.co/ SparseLLM/ReluLLaMA-7B

    2023

  56. [65]

    Nemotron-3-8B-Base-4k,

    NVIDIA, “Nemotron-3-8B-Base-4k,” 2023. [Online]. Available: https://huggingface.co/nvidia/nemotron-3-8b-base-4k

    2023

  57. [66]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V . Goswami, N. Goyal, A. Hartshorn, S. Ho...

    2023 arXiv

  58. [67]

    Gemma: Open models based on Gemini research and technology,

    Gemma Team, “Gemma: Open models based on Gemini research and technology,”arXiv preprint arXiv:2403.08295, 2024

    2024 arXiv

  59. [68]

    Skytrax airline reviews,

    E. Danisman, “Skytrax airline reviews,” 2019. [Online]. Available: https: //www.kaggle.com/datasets/efehandanisman/skytrax-airline-reviews

    2019

  60. [69]

    MedAlpaca–an open-source collection of medical conversational AI models and training data,

    T. Han, L. C. Adams, J.-M. Papaioannou, P. Grundmann, T. Oberhauser, A. L ¨oser, D. Truhn, and K. K. Bressem, “MedAlpaca–an open-source collection of medical conversational AI models and training data,”arXiv preprint arXiv:2304.08247, 2023

    2023 arXiv

  61. [70]

    Neural legal judgment prediction in English,

    I. Chalkidis, I. Androutsopoulos, and N. Aletras, “Neural legal judgment prediction in English,” 2019. [Online]. Available: https: //arxiv.org/abs/1906.02059

    2019 arXiv

  62. [71]

    Private prompts,

    J. Morris, “Private prompts,” 2023. [Online]. Available: https: //huggingface.co/datasets/jxm/private prompts

    2023

  63. [72]

    System prompt leakage,

    G. Chua, “System prompt leakage,” 2024. [Online]. Available: https://huggingface.co/datasets/gabrielchua/system-prompt-leakage

    2024

  64. [73]

    Extracting prompts by inverting LLM outputs,

    C. Zhang, J. X. Morris, and V . Shmatikov, “Extracting prompts by inverting LLM outputs,” inProceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024

    2024

  65. [74]

    The early bird catches the leak: Unveiling timing side channels in LLM serving systems,

    L. Song, Z. Pang, W. Wang, Z. Wang, X. Wang, H. Chen, W. Song, Y . Jin, D. Meng, and R. Hou, “The early bird catches the leak: Unveiling timing side channels in LLM serving systems,”arXiv preprint arXiv:2409.20002, 2025

    2025

  66. [75]

    System prompt extraction attacks and defenses in large language models,

    B. C. Das, M. H. Amini, and Y . Wu, “System prompt extraction attacks and defenses in large language models,”arXiv preprint arXiv:2505.23817, 2025

    2025 arXiv

  67. [76]

    LaMP: When large language models meet personalization,

    A. Salemi, S. Mysore, M. Bendersky, and H. Zamani, “LaMP: When large language models meet personalization,” inProceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2024

    2024

  68. [77]

    Democratizing large language models via personalized parameter-efficient fine-tuning,

    Z. Tan, Q. Zeng, Y . Tian, Z. Liu, B. Yin, and M. Jiang, “Democratizing large language models via personalized parameter-efficient fine-tuning,” inProceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024

    2024

  69. [78]

    From persona to personalization: A survey on role-playing language agents,

    J. Chen, X. Wang, R. Xu, S. Yuan, Y . Zhang, W. Shi, J. Xie, S. Li, R. Yang, T. Zhuet al., “From persona to personalization: A survey on role-playing language agents,”arXiv preprint arXiv:2404.18231, 2024

    2024 arXiv

  70. [79]

    Personalized generation in large model era: A 15 survey,

    Y . Xu, J. Zhang, A. Salemi, X. Hu, W. Wang, F. Feng, H. Zamani, X. He, and T.-S. Chua, “Personalized generation in large model era: A 15 survey,” inProceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2025

    2025

  71. [80]

    llama.cpp: LLM inference in C/C++,

    G. Gerganov, “llama.cpp: LLM inference in C/C++,” 2025. [Online]. Available: https://github.com/ggml-org/llama.cpp

    2025

  72. [81]

    Using the Linux kernel Tracepoints

    “Using the Linux kernel Tracepoints.” [Online]. Available: https: //www.kernel.org/doc/Documentation/trace/tracepoints.txt

  73. [82]

    Linux extended BPF (eBPF) tracing tools,

    B. Gregg, “Linux extended BPF (eBPF) tracing tools,” 2018. [Online]. Available: http://www.brendangregg.com/ebpf.html

    2018

  74. [83]

    BLEU: a method for automatic evaluation of machine translation,

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “BLEU: a method for automatic evaluation of machine translation,” inProceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2002

    2002

  75. [84]

    Wikimedia downloads

    Wikimedia Foundation. Wikimedia downloads. [Online]. Available: https://dumps.wikimedia.org

  76. [85]

    dm-crypt,

    The kernel development community, “dm-crypt,” 2024. [Online]. Avail- able: https://www.kernel.org/doc/html/v6.7/admin-guide/device-mapper/ dm-crypt.html

    2024

  77. [86]

    Software protection and simulation on oblivious RAMs,

    O. Goldreich and R. Ostrovsky, “Software protection and simulation on oblivious RAMs,”Journal of the ACM (JACM), vol. 43, no. 3, pp. 431–473, 1996

    1996

  78. [87]

    Raccoon: Closing digital side-channels through obfuscated execution,

    A. Rane, C. Lin, and M. Tiwari, “Raccoon: Closing digital side-channels through obfuscated execution,” inProceedings of the USENIX Security Symposium (Security), 2015

    2015

  79. [88]

    HOP: Hardware makes obfuscation practical,

    K. Nayak, C. W. Fletcher, L. Ren, N. Chandran, S. V . Lokam, E. Shi, and V . Goyal, “HOP: Hardware makes obfuscation practical,” inProceedings of the Network and Distributed System Security Symposium (NDSS), 2017

    2017

  80. [89]

    FlexGen: high-throughput generative inference of large language models with a single GPU,

    Y . Sheng, L. Zheng, B. Yuan, Z. Li, M. Ryabinin, B. Chen, P. Liang, C. R ´e, I. Stoica, and C. Zhang, “FlexGen: high-throughput generative inference of large language models with a single GPU,” inProceedings of the International Conference on Machine Learning (ICML), 2023

    2023

  81. [90]

    InfiniGen: Efficient generative inference of large language models with dynamic KV cache manage- ment,

    W. Lee, J. Lee, J. Seo, and J. Sim, “InfiniGen: Efficient generative inference of large language models with dynamic KV cache manage- ment,” inProceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2024

    2024

  82. [91]

    Improving throughput-oriented LLM inference with CPU computations,

    D. Park and B. Egger, “Improving throughput-oriented LLM inference with CPU computations,” inProceedings of the International Conference on Parallel Architectures and Compilation Techniques (PACT), 2024

    2024

  83. [92]

    DeepCache: Revisiting cache side-channel attacks in deep neural networks exe- cutables,

    Z. Liu, Y . Yuan, Y . Chen, S. Hu, T. Li, and S. Wang, “DeepCache: Revisiting cache side-channel attacks in deep neural networks exe- cutables,” inProceedings of the ACM Conference on Computer and Communications Security (CCS), 2024

    2024

  84. [93]

    Phantom: Practical oblivious computation in a secure processor,

    M. Maas, E. Love, E. Stefanov, M. Tiwari, E. Shi, K. Asanovic, J. Kubiatowicz, and D. Song, “Phantom: Practical oblivious computation in a secure processor,” inProceedings of the ACM Conference on Computer and Communications Security (CCS), 2013

    2013

  85. [94]

    FLUSH+RELOAD: A high resolution, low noise, L3 cache side-channel attack,

    Y . Yarom and K. Falkner, “FLUSH+RELOAD: A high resolution, low noise, L3 cache side-channel attack,” inProceedings of the USENIX Security Symposium (Security), 2014

    2014

  86. [95]

    Flush+Flush: a fast and stealthy cache attack,

    D. Gruss, C. Maurice, K. Wagner, and S. Mangard, “Flush+Flush: a fast and stealthy cache attack,” inProceedings of the International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment (DIMVA), 2016

    2016

  87. [96]

    Activation functions considered harmful: Recovering neural network weights through controlled channels,

    J. Spielman, D. Oswald, M. Ryan, and J. Van Bulck, “Activation functions considered harmful: Recovering neural network weights through controlled channels,” inProceedings of the International Symposium on Research in Attacks, Intrusions and Defenses (RAID), 2025

    2025

  88. [97]

    Hy- perTheft: Thieving model weights from TEE-shielded neural networks via ciphertext side channels,

    Y . Yuan, Z. Liu, S. Deng, Y . Chen, S. Wang, Y . Zhang, and Z. Su, “Hy- perTheft: Thieving model weights from TEE-shielded neural networks via ciphertext side channels,” inProceedings of the ACM Conference on Computer and Communications Security (CCS), 2024

    2024

  89. [98]

    Reverse-engineering deep neural networks using floating-point timing side-channels,

    C. Gongye, Y . Fei, and T. Wahl, “Reverse-engineering deep neural networks using floating-point timing side-channels,” inProceedings of the ACM/EDAC/IEEE Design Automation Conference (DAC), 2020

    2020

  90. [99]

    Relocate-V ote: Using sparsity information to exploit ciphertext side- channels,

    Y . Yan, W. Huang, I. Grishchenko, G. Saileshwar, A. Mehta, and D. Lie, “Relocate-V ote: Using sparsity information to exploit ciphertext side- channels,” inProceedings of the USENIX Security Symposium (Security), 2025

    2025

  91. [100]

    InputSnatch: Stealing input in LLM services via timing side-channel attacks,

    X. Zheng, H. Han, S. Shi, Q. Fang, Z. Du, Q. Guo, and X. Hu, “InputSnatch: Stealing input in LLM services via timing side-channel attacks,”arXiv preprint arXiv:2411.18191, 2024

    2024 arXiv

  92. [101]

    MoEfication: Transformer feed-forward layers are mixtures of experts,

    Z. Zhang, Y . Lin, Z. Liu, P. Li, M. Sun, and J. Zhou, “MoEfication: Transformer feed-forward layers are mixtures of experts,” inFindings of the Association for Computational Linguistics: ACL 2022, 2022

    2022

  93. [102]

    Ex- ploiting activation sparsity with dense to dynamic-k mixture-of-experts conversion,

    F. Szatkowski, B. W ´ojcik, M. Pi ´orczy´nski, and S. Scardapane, “Ex- ploiting activation sparsity with dense to dynamic-k mixture-of-experts conversion,” inAdvances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  94. [103]

    Buffer overflow in mixture of experts,

    J. Hayes, I. Shumailov, and I. Yona, “Buffer overflow in mixture of experts,”arXiv preprint arXiv:2402.05526, 2024

    2024 arXiv

  95. [104]

    Stealing user prompts from mixture of experts,

    I. Yona, I. Shumailov, J. Hayes, and N. Carlini, “Stealing user prompts from mixture of experts,”arXiv preprint arXiv:2410.22884, 2024

    2024 arXiv

  96. [105]

    MoEcho: Exploiting side-channel attacks to compromise user privacy in mixture-of-experts LLMs,

    R. Ding, T. Xu, X. Shen, A. A. Ding, and Y . Fei, “MoEcho: Exploiting side-channel attacks to compromise user privacy in mixture-of-experts LLMs,” inProceedings of the ACM Conference on Computer and Communications Security (CCS), 2025

    2025

  97. [106]

    Fast inference from Trans- formers via speculative decoding,

    Y . Leviathan, M. Kalman, and Y . Matias, “Fast inference from Trans- formers via speculative decoding,” inProceedings of the International Conference on Machine Learning (ICML), 2023

    2023

  98. [107]

    Privacy risks of speculative decoding in large language models,

    J. Wei, A. Abdulrazzag, T. Zhang, A. Muursepp, and G. Saileshwar, “Privacy risks of speculative decoding in large language models,”arXiv preprint arXiv:2411.01076, 2024

    2024

  99. [108]

    Effective prompt extraction from language models,

    Y . Zhang, N. Carlini, and D. Ippolito, “Effective prompt extraction from language models,” inProceedings of the Conference on Language Modeling (COLM), 2024

    2024

  100. [109]

    PLeak: Prompt leak- ing attacks against large language model applications,

    B. Hui, H. Yuan, N. Gong, P. Burlina, and Y . Cao, “PLeak: Prompt leak- ing attacks against large language model applications,” inProceedings of the ACM Conference on Computer and Communications Security (CCS), 2024

    2024

  101. [110]

    Machine learning with confidential computing: A systematization of knowledge,

    F. Mo, Z. Tarkhani, and H. Haddadi, “Machine learning with confidential computing: A systematization of knowledge,”ACM Computing Survey (CSUR), vol. 56, no. 11, Jun. 2024

    2024

  102. [111]

    ShadowNet: A secure and efficient on-device model inference system for convolutional neural networks,

    Z. Sun, R. Sun, C. Liu, A. R. Chowdhury, L. Lu, and S. Jha, “ShadowNet: A secure and efficient on-device model inference system for convolutional neural networks,” inProceedings of the IEEE Symposium on Security and Privacy (IEEE S&P), 2023

    2023

  103. [112]

    No privacy left outside: On the (in-)security of TEE-shielded DNN partition for on-device ML,

    Z. Zhang, C. Gong, Y . Cai, Y . Yuan, B. Liu, D. Li, Y . Guo, and X. Chen, “No privacy left outside: On the (in-)security of TEE-shielded DNN partition for on-device ML,” inProceedings of the IEEE Symposium on Security and Privacy (IEEE S&P), 2024

    2024

  104. [113]

    SOTER: Guarding black-box inference for general neural networks at the edge,

    T. Shen, J. Qi, J. Jiang, X. Wang, S. Wen, X. Chen, S. Zhao, S. Wang, L. Chen, X. Luo, F. Zhang, and H. Cui, “SOTER: Guarding black-box inference for general neural networks at the edge,” inProceedings of the USENIX Annual Technical Conference (ATC), 2022

    2022

  105. [114]

    Occlumency: Privacy-preserving remote deep-learning inference using SGX,

    T. Lee, Z. Lin, S. Pushp, C. Li, Y . Liu, Y . Lee, F. Xu, C. Xu, L. Zhang, and J. Song, “Occlumency: Privacy-preserving remote deep-learning inference using SGX,” inProceedings of the Annual International Conference on Mobile Computing and Networking (MobiCom), 2019

    2019

  106. [115]

    sNPU: Trusted execution environments on integrated NPUs,

    E. Feng, D. Feng, D. Du, Y . Xia, and H. Chen, “sNPU: Trusted execution environments on integrated NPUs,” inProceedings of the ACM/IEEE Annual International Symposium on Computer Architecture (ISCA), 2024

    2024

  107. [116]

    Confidential Compute on NVIDIA Hopper H100,

    NVIDIA, “Confidential Compute on NVIDIA Hopper H100,”

  108. [117]

    Confidential inference systems: Design principles and security risks,

    Pattern Labs and Anthropic, “Confidential inference systems: Design principles and security risks,” 2025. [Online]. Available: https://assets.anthropic.com/m/c52125297b85a42/original/ Confidential Inference Paper.pdf 16

    2025

  109. [2014]

    Available: https://www.intel.com/content/dam/develop/ external/us/en/documents/329298-002-629101.pdf

    [Online]. Available: https://www.intel.com/content/dam/develop/ external/us/en/documents/329298-002-629101.pdf

  110. [2023]

    Available: https://images.nvidia.com/aem-dam/en-zz/ Solutions/data-center/HCC-Whitepaper-v1.0.pdf

    [Online]. Available: https://images.nvidia.com/aem-dam/en-zz/ Solutions/data-center/HCC-Whitepaper-v1.0.pdf

  111. [2024]

    Available: https://developers.googleblog.com/en/ enabling-more-private-gen-ai

    [Online]. Available: https://developers.googleblog.com/en/ enabling-more-private-gen-ai

  112. [2025]

    Available: https://www.anthropic.com/research/ confidential-inference-trusted-vms

    [Online]. Available: https://www.anthropic.com/research/ confidential-inference-trusted-vms

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.