REVIEW 3 major objections 6 minor 26 references
Distilled Large Language Model in Confidential Computing Environment for System-on-Chip Design
T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read This paper claims that for distilled LLMs below roughly 3 billion parameters, Intel TDX confidential execution runs inference faster than an unsecured CPU-only setup, while keeping model and data inside an encrypted enclave.
desk verdict TDX-beats-CPU claim is unconvincing due to unmatched baseline resources; the quantization data is the solid part. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism is Intel TDX, a VM-level trusted execution environment that allocates a private memory region to a secure Trust Domain and encrypts data crossing the enclave boundary. Inside that enclave, the paper relies on two properties of small models: distillation (DeepSeek-R1's 1.5B/7B/14B variants keep reasoning ability at low parameter counts) and post-training quantization (Q4/Q8 integer weights). These properties keep the whole model and workload resident in TDX private memory, avoiding the memory-swap overhead that larger models trigger, so the TDX CPU configuration can run small models faster than the CPU-only baseline the authors measured.
What would settle it
Run the same DeepSeek-R1-1.5B model in a CPU-only container configured with the same 62 cores and 510 GB as the TDX VM, pinned to the same NUMA nodes and free of other host workloads; if TDX no longer achieves about 25.67 tokens/s against a CPU-only 10.25, the claimed enclave speedup collapses.
Extended reading notes
Core claim
The central discovery, as the authors state it, is that Intel TDX's CPU execution can beat plain CPU execution for small distilled models: DeepSeek-R1-1.5B runs at 25.67 tokens/s in the TDX VM versus 10.25 tokens/s CPU-only, and similar advantages appear for Llama3.2-1B/3B and Gemma2-2B. The advantage fades with model size, DeepSeek-7B is slower in TDX (6.42 vs 8.53 tokens/s), so the benefit is specific to sub-3B models. They also find that quantization compounds the effect: Q4 DeepSeek roughly triples tokens/s relative to FP16, and Q4 storage is about 30% of FP16. These patterns hold on HWSec models fine-tuned for hardware security, where TDX exceeds CPU-only in every case they report.
Load-bearing premise
The speed comparison assumes the CPU-only baseline is a fair, resource-equivalent control, but the paper does not state its core count, memory limit, or isolation from other host workloads, so the TDX advantage could be an artifact of a weaker baseline.
Editorial extensions
If this is right
- For sub-3B distilled models, confidential execution is not a performance tax: TDX can roughly double CPU-only throughput, so security can be added at no speed penalty.
- Q4/Q8 quantization is the practical deployment choice in TDX, combining up to 3x speedup over FP16 with storage of roughly 30-50% of FP16, which fits enclave memory limits.
- The TDX advantage disappears at 7B and 14B scales, where memory bandwidth dominates, so larger confidential models require GPU-in-enclave support that current TDX does not offer.
- GPU acceleration remains 8-14x faster than TDX only because it moves data out of the enclave in plaintext; the paper's central trade-off is security versus speed.
- Fine-tuned HWSec hardware-security models follow the same trend, indicating that domain-adapted confidential LLM deployment in SoC verification is feasible.
Reading between the lines
- The TDX-versus-CPU gap is most plausibly a configuration effect (62 cores and 510 GB in the TDX VM versus an unspecified host container) rather than an effect of the security machinery itself; a matched-resource rerun would separate these causes.
- If the speedup survives controlled reruns, confidential cloud providers could offer secure small-model inference for latency-tolerant CAD workloads without GPU allocation, lowering cost.
- The same benchmark could be run on AMD SEV and ARM TrustZone to test whether the small-model advantage is generic to VM-level TEEs or specific to TDX.
- A direct extension would profile memory bandwidth and cache behavior of the 1.5B model inside TDX to identify whether the gain comes from ISA optimizations or from reduced memory pressure.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper empirically evaluates lightweight LLM inference in an Intel TDX confidential-computing environment versus CPU-only and CPU-GPU baselines, using Ollama to measure tokens/s for DeepSeek-R1 distilled variants (1.5B, 7B, 14B), Llama 3.2, Gemma 2, and several HWSeC fine-tuned models. The authors report that for sub-3B models TDX achieves higher throughput than CPU-only execution (e.g., DeepSeek 1.5B at 25.67 vs 10.25 tokens/s), that quantization Q4/Q8 gives up to 3x speedup over FP16, and that these trends hold for domain-specific HWSeC models. They conclude that lightweight distilled LLMs can be deployed efficiently and securely in confidential computing for SoC design workflows.
Significance. If the throughput comparison were properly controlled, this would be a useful empirical data point: it would show that confidential-execution overhead need not make small-model inference dramatically slower than an insecure CPU baseline, and the quantization results would be practically relevant for memory-constrained TEE deployments. The paper also addresses a topical gap, since most TEE-LLM work focuses on SGX or large models, whereas TDX and distilled sub-3B models are less studied. The strengths are the breadth of models tested (DeepSeek, Llama, Gemma, HWSeC domain models) and the explicit reporting of raw tokens/s values and model storage sizes. However, the headline TDX-beats-CPU claim is not established because the CPU-only baseline is not resource-equivalent, and the HWSeC validation claims accuracy improvements from a table that contains only throughput. The paper would need additional experiments and careful re-scoping before its central conclusions can be accepted.
major comments (3)
- [Section IV-A, Table I, Section V-A] The central claim that TDX outperforms CPU-only execution for sub-3B models is confounded by unequal resources. Table I specifies the TDX configuration as "62 CPU cores and 510GB DRAM" but describes CPU Only only as "Runs on the host machine without GPU acceleration enabled," with no vCPU count, memory limit, NUMA placement, container constraints, or host-isolation details in Section IV-A. The 25.67 vs 10.25 tokens/s comparison for DeepSeek 1.5B in Table II, and its attribution to "TDX's optimized CPU ISA configuration" in Section V-A, therefore conflates TDX-specific effects with core/memory allocation and host contention. The reversal at 7B (CPU-only 8.53 vs TDX 6.42) is consistent with a resource-allocation explanation rather than a TDX ISA effect. The authors should rerun with a CPU-only container constrained to the same cores, memory, and Ollama thread settings as the TDX VM, and should report whether the advantage survives.
- [Section V-C, Table III] The text states that "The results shown in Table III confirm that these domain-specific HWSeC models achieve improved accuracy and usability in chip design and verification benchmarks, outperforming general LLMs on hardware-centric tasks." Table III contains only throughput in tokens/s and throughput ratios; no accuracy metrics, no comparison to general LLMs on hardware benchmarks, and no benchmark scores are reported. This sentence is unsupported by the presented data. Either add the accuracy results or rephrase the claim to refer only to throughput.
- [Section V-A, Tables II and III] All performance numbers are reported as single point values from Ollama's built-in logging, with no repeated runs, error bars, or statistical tests. Because the paper's headline conclusions are ratios (e.g., "more than twice," "9x," "20x"), single measurements do not establish that the observed differences are reliable. At minimum, the authors should report multiple repetitions with standard deviations and a simple significance test for the key TDX-vs-CPU comparisons.
minor comments (6)
- [Abstract and Section I contributions] The abstract and the third contribution bullet state that LLM performance in TEEs exceeds CPU-only execution without qualification, but the paper's own results show this holds only for sub-3B models; the 7B and 14B cases reverse or narrow the gap. Please qualify the claim to the sub-3B distilled-model regime.
- [Section I contributions and References [12]] The contribution bullet claims "the first evaluation of a distilled LLM within a TEE," yet reference [12] is a prior evaluation of DeepSeek in a confidential computing environment by the same authors. Please reconcile this claim, for example by clarifying what is new relative to [12].
- [Section V-A, Figure 3] Figure 3 is described as performance "ratioed to the CPU performance," but the caption does not define which CPU baseline is used (CPU-only, TDX, or normalized value) or how the ratio is computed. Please define the normalization explicitly.
- [Section V-D] The statement "8-bit DeepSeek-14B model performs better than 16-bit DeepSeek-7B" is confusing, and the following sentence "the DeepSeek model sees only a slight gain (2.15 to 2.17 tokens/s)" needs context: which models and quantization levels are being compared, and at which parameter sizes? Please state the comparison clearly with the corresponding numbers.
- [Tables II and III] The column headers in Tables II and III contain typesetting artifacts such as "GP U T DX" and "CP U T DX" that should be reformatted as GPU/TDX and CPU/TDX, and the dimensionless ratio columns should be defined in the captions.
- [Throughout] The domain-specific model family is referred to inconsistently as "HWSeC," "HWSec," and "KSU HWSEC"; please choose one consistent notation. Also, no code or data availability statement is provided, which limits reproducibility of the benchmark.
Circularity Check
No significant circularity: the benchmark results are measured against external models and workloads, and the only self-citation is not load-bearing.
full rationale
The paper is an empirical measurement study. Its headline claims, such as the TDX environment reaching 25.67 tokens/s for DeepSeek-1.5B versus 10.25 tokens/s for CPU-only (Table II, Section V-A), are direct readings from Ollama's logging for external models and workloads; no derived constant or fitted parameter is renamed as a prediction. The quantization speedups and storage ratios (Section V-D, Table IV) are arithmetic consequences of reduced precision and reported measured throughput, not fitted results. The only self-citation, reference [12], is a prior evaluation of DeepSeek in a confidential environment; it is cited for background motivation and does not supply any equation, fitted value, or benchmark number used in the present measurements. The paper's CPU-only baseline is underspecified relative to the TDX VM (Table I gives TDX '62 CPU cores and 510GB DRAM' but describes CPU Only only as 'Runs on the host machine without GPU acceleration enabled'), which threatens the causal attribution of the reported speedup to TDX-specific optimizations. That is an experimental-control weakness, not a circular-reasoning weakness. Likewise, the paper's own acknowledged limitations about plaintext CPU-GPU communication and lack of full GPU support inside TDX are substantive engineering constraints, not circular steps. No claim in the derivation chain reduces by construction to its inputs, so the circularity score is 0.
Assumptions & free parameters
free parameters (1)
- TDX VM resource allocation (62 vCPUs, 510GB DRAM) =
62 CPU cores, 510 GB DRAM
assumptions (4)
- domain assumption Ollama's built-in logging yields accurate, comparable tokens/s measurements across TDX, CPU-only, and GPU-CPU environments.
- ad hoc to paper The CPU-only Docker baseline is an appropriate performance reference even though its resource limits are not specified.
- domain assumption Q4/Q8 quantization in Ollama preserves the models' task behavior, making tokens/s the relevant comparison.
- domain assumption HWSeC models are representative of confidential SoC design workloads.
Cite this review
Pith. "Pith review of Distilled Large Language Model in Confidential Computing Environment for System-on-Chip Design." pith.science (2026). https://pith.science/paper/KVBC453G
@misc{pith2026250716226,
author = {Pith},
title = {Pith review of: Distilled Large Language Model in Confidential Computing Environment for System-on-Chip Design},
year = {2026},
howpublished = {\url{https://pith.science/paper/KVBC453G}},
note = {Machine review of arXiv:2507.16226}
}
read the original abstract
Large Language Models (LLMs) are increasingly used in circuit design tasks and have typically undergone multiple rounds of training. Both the trained models and their associated training data are considered confidential intellectual property (IP) and must be protected from exposure. Confidential Computing offers a promising solution to protect data and models through Trusted Execution Environments (TEEs). However, existing TEE implementations are not designed to support the resource-intensive nature of LLMs efficiently. In this work, we first present a comprehensive evaluation of the LLMs within a TEE-enabled confidential computing environment, specifically utilizing Intel Trust Domain Extensions (TDX). We constructed experiments on three environments: TEE-based, CPU-only, and CPU-GPU hybrid implementations, and evaluated their performance in terms of tokens per second. Our first observation is that distilled models, i.e., DeepSeek, surpass other models in performance due to their smaller parameters, making them suitable for resource-constrained devices. Also, in the quantized models such as 4-bit quantization (Q4) and 8-bit quantization (Q8), we observed a performance gain of up to 3x compared to FP16 models. Our findings indicate that for fewer parameter sets, such as DeepSeek-r1-1.5B, the TDX implementation outperforms the CPU version in executing computations within a secure environment. We further validate the results using a testbench designed for SoC design tasks. These validations demonstrate the potential of efficiently deploying lightweight LLMs on resource-constrained systems for semiconductor CAD applications.
Figures
Reference graph
Works this paper leans on
-
[12]
Evaluating the performance of the deepseek model in confidential computing environment,
B. Dong and Q. Wang, “Evaluating the performance of the deepseek model in confidential computing environment,” arXiv preprint arXiv:2502.11347, 2025
arXiv 2025
-
[1]
Overview on signing and whitelisting for intel® software guard extensions (intel® sgx) enclaves,
Intel Corporation, “Overview on signing and whitelisting for intel® software guard extensions (intel® sgx) enclaves,” https://www.intel.com/content/dam/develop/external/us/en/documents/ overview-signing-whitelisting-intel-sgx-enclaves.pdf, Jan. 2019
work page 2019
-
[2]
Privacy-Preserving Inference in Machine Learning Services Using Trusted Execution Environments
K. G. Narra, Z. Lin, Y . Wang, K. Balasubramaniam, and M. Annavaram, “Privacy-preserving inference in machine learning services using trusted execution environments,” arXiv preprint arXiv:1912.03485 , 2019
work page Pith review arXiv 1912
-
[3]
{SOTER}: Guarding black-box inference for general neural networks at the edge,
T. Shen, J. Qi, J. Jiang, X. Wang, S. Wen, X. Chen, S. Zhao, S. Wang, L. Chen, X. Luo et al. , “ {SOTER}: Guarding black-box inference for general neural networks at the edge,” in 2022 USENIX Annual Technical Conference (USENIX ATC 22) , 2022, pp. 723–738
work page 2022
-
[4]
Z. Sun, R. Sun, C. Liu, A. R. Chowdhury, L. Lu, and S. Jha, “Shad- ownet: A secure and efficient on-device model inference system for convolutional neural networks,” in 2023 IEEE Symposium on Security and Privacy (SP) . IEEE, 2023, pp. 1596–1612
work page 2023
-
[5]
Intel® trust domain extensions,
Intel Corporation, “Intel® trust domain extensions,” https: //cdrdv2-public.intel.com/690419/TDX-Whitepaper-February2022.pdf, Feb. 2022
work page 2022
-
[6]
Language models are few-shot learners,
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al. , “Language models are few-shot learners,” Advances in Neural Information Processing Systems (NeurIPS), vol. 33, pp. 1877–1901, 2020. [Online]. Available: https://arxiv.org/abs/2005.14165
arXiv 1901
-
[7]
Introducing gemini: Our most capable ai model,
G. DeepMind, “Introducing gemini: Our most capable ai model,” 2023. [Online]. Available: https://www.deepmind.com/blog/ introducing-gemini
work page 2023
Show all 26 references
-
[8]
Understanding oversubscribed memory management for deep learning training,
M. Lin and H. Jeon, “Understanding oversubscribed memory management for deep learning training,” in Proceedings of the 5th Workshop on Machine Learning and Systems , ser. EuroMLSys ’25. New York, NY , USA: Association for Computing Machinery, 2025, p. 46–55. [Online]. Availabl...
2025
-
[9]
Llm4sechw: Leveraging domain-specific large language model for hardware debug- ging,
W. Fu, K. Yang, R. G. Dutta, X. Guo, and G. Qu, “Llm4sechw: Leveraging domain-specific large language model for hardware debug- ging,” in 2023 Asian Hardware Oriented Security and Trust Symposium (AsianHOST). IEEE, 2023, pp. 1–6
2023
-
[10]
Socurellm: An llm-driven approach for large-scale system-on-chip security verification and policy generation,
S. Tarek, D. Saha, S. K. Saha, M. Tehranipoor, and F. Farahmandi, “Socurellm: An llm-driven approach for large-scale system-on-chip security verification and policy generation,” Cryptology ePrint Archive , 2024
2024
-
[11]
Deepseek llm: Scaling open-source language models with longtermism,
X. Bi, D. Chen, G. Chen, S. Chen, D. Dai, C. Deng, H. Ding, K. Dong, Q. Du, Z. Fu et al. , “Deepseek llm: Scaling open-source language models with longtermism,” arXiv preprint arXiv:2401.02954 , 2024
2024 arXiv
-
[13]
Amd memory encryption: Sev, sme, and sev-es,
AMD, “Amd memory encryption: Sev, sme, and sev-es,” 2018. [On- line]. Available: https://www.amd.com/content/dam/amd/en/documents/ epyc-business-docs/white-papers/memory-encryption-white-paper.pdf
2018
-
[14]
Building a secure system using trustzone technology,
ARM, “Building a secure system using trustzone technology,”
-
[15]
Drgpum: Guiding memory optimization for gpu-accelerated applications,
M. Lin, K. Zhou, and P. Su, “Drgpum: Guiding memory optimization for gpu-accelerated applications,” in Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3 , ser. ASPLOS 2023. New York, NY , USA...
2023
-
[16]
Exploring parallel implemen- tation of sphincs+ using advanced vector extensions (avx) sets,
Y . Zhou, K. Rajasekaran, and Q. Wang, “Exploring parallel implemen- tation of sphincs+ using advanced vector extensions (avx) sets,” in 2025 26th International Symposium on Quality Electronic Design (ISQED) , 2025, pp. 1–8
2025
-
[17]
Forest: Access-aware gpu uvm management,
M. Lin, Y . Feng, G. Cox, and H. Jeon, “Forest: Access-aware gpu uvm management,” in Proceedings of the 52nd Annual International Symposium on Computer Architecture , ser. ISCA ’25. New York, NY , USA: Association for Computing Machinery, 2025, p. 137–152. [Online]. Available:...
2025
-
[18]
Marvel: Multi-agent rtl vulnerability extraction using large language models,
L. Collini, B. Ahmad, J. Ah-kiow, and R. Karri, “Marvel: Multi-agent rtl vulnerability extraction using large language models,” arXiv preprint arXiv:2505.11963, 2025
2025
-
[19]
Spiced: Syntactical bug and trojan pattern identification in a/ms circuits using llm-enhanced detection,
J. Chaudhuri, D. Thapar, A. Chaudhuri, F. Firouzi, and K. Chakrabarty, “Spiced: Syntactical bug and trojan pattern identification in a/ms circuits using llm-enhanced detection,” in 2024 IEEE Physical Assurance and Inspection of Electronics (PAINE) . IEEE, 2024, pp. 1–7
2024
-
[20]
Threatlens: Llm- guided threat modeling and test plan generation for hardware security verification,
D. Saha, H. A. Shaikh, S. Tarek, and F. Farahmandi, “Threatlens: Llm- guided threat modeling and test plan generation for hardware security verification,” arXiv preprint arXiv:2505.06821 , 2025
2025 arXiv
-
[21]
Is chatgpt a general-purpose natural language processing task solver?
C. Qin, A. Zhang, Z. Zhang, J. Chen, M. Yasunaga, and D. Yang, “Is chatgpt a general-purpose natural language processing task solver?” arXiv preprint arXiv:2302.06476 , 2023
2023 arXiv
-
[22]
Can chatgpt understand too? a comparative study on chatgpt and fine-tuned bert,
Q. Zhong, L. Ding, J. Liu, B. Du, and D. Tao, “Can chatgpt understand too? a comparative study on chatgpt and fine-tuned bert,” arXiv preprint arXiv:2302.10198, 2023
2023 arXiv
-
[23]
A survey of large language models,
W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y . Hou, Y . Min, B. Zhang, J. Zhang, Z. Dong et al. , “A survey of large language models,” arXiv preprint arXiv:2303.18223, vol. 1, no. 2, 2023
2023 arXiv
-
[24]
A generalize hardware debugging approach for large language models semi-synthetic, datasets,
W. Fu, S. Li, Y . Zhao, K. Yang, X. Zhang, Y . Jin, and X. Guo, “A generalize hardware debugging approach for large language models semi-synthetic, datasets,” IEEE Transactions on Circuits and Systems I: Regular Papers, 2024
2024
-
[25]
The case for 4-bit precision: k- bit inference scaling laws,
T. Dettmers and L. Zettlemoyer, “The case for 4-bit precision: k- bit inference scaling laws,” in International Conference on Machine Learning. PMLR, 2023, pp. 7750–7774
2023
-
[2020]
Available: https://documentation-service.arm.com/static/ 5f212796500e883ab8e74531
[Online]. Available: https://documentation-service.arm.com/static/ 5f212796500e883ab8e74531
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.