Pith. sign in

REVIEW 5 major objections 9 minor 61 references

Per-layer automated search cuts FHE transformer approximation depth by about a third in under an hour, with little accuracy loss.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-30 21:14 UTC pith:XMGKXP4Y

load-bearing objection Solid systems paper: per-layer HE approximation search is real and the depth/latency wins hold; the accuracy-parity claim is thinner than the abstract sells, especially on LLaMA. the 5 major comments →

arxiv 2607.23478 v1 pith:XMGKXP4Y submitted 2026-07-26 cs.CR cs.AIcs.LG

ATLAS: Automated Approximation of Transformers for Efficient Homomorphic Inference in One Hour

Jianhang Xie , Sicheng Tan , Vishnu Naresh Boddeti , Zhichao Lu This is my paper
classification cs.CR cs.AIcs.LG
keywords fully homomorphic encryptionCKKStransformerssecure inferencepolynomial approximationmulti-objective optimizationmultiplicative depthNSGA-II
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Private inference of transformers under fully homomorphic encryption is dominated by the cost of polynomial stand-ins for softmax, normalization, and activations. Today those stand-ins are hand-tuned once per function and reused on every layer, which wastes multiplicative depth because layers do not need the same precision. ATLAS treats the choice of iteration counts and polynomial degrees as a multi-objective search over depth and end-to-end error, allowing each layer its own settings. A two-stage evolutionary procedure first finds good uniform configurations, then relaxes them layer by layer, with cheap surrogates so the whole search finishes in under an hour even for a 32-layer model. On BERT, ViT, and LLaMA3-8B the resulting recipes cut total approximation depth by roughly a third and end-to-end ciphertext latency by about a fifth to a quarter versus strong hand-tuned baselines, while matching or slightly beating downstream accuracy. The method is a post-processing step on frozen weights and is meant to sit on top of existing FHE packing and matrix-multiply stacks.

Core claim

Uniform, expert-chosen approximation hyperparameters leave large unused depth budgets across transformer layers. Framing per-layer iteration counts and polynomial degrees as a multi-objective combinatorial problem, and solving it with a two-stage NSGA-II search plus cheap MAE surrogates, yields heterogeneous configurations that reduce total approximation multiplicative depth by about 35% and FHE latency by about 20–25% versus iterative-softmax and reproduced THOR/NEXUS recipes, with negligible task accuracy loss, in under one hour of search.

What carries the argument

Two-stage multi-objective evolutionary search (NSGA-II): Stage 1 optimizes a single shared configuration across all layers; Stage 2 seeds the full per-layer space from that Pareto set and refines heterogeneous settings, accelerated by data-subset MAE and early-layer MAE surrogates plus distributed evaluation.

Load-bearing premise

That ranking configurations by cleartext mean absolute error on a tiny proxy set (and early-layer error in stage one) is reliable enough that the depth–error Pareto front transfers to real downstream accuracy and to actual encrypted latency under the authors’ packing and bootstrapping stack.

What would settle it

Run the same search objectives on a held-out architecture or task and check whether the depth–proxy-MAE Pareto front still ranks configurations by true task accuracy and by measured end-to-end FHE latency; a large inversion (high proxy rank, large accuracy drop or no latency gain) would break the claim.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Existing FHE transformer systems can adopt ATLAS configs as a drop-in post-process without retraining, repacking, or changing crypto parameters.
  • Hand-tuned uniform recipes (fixed softmax iterations, Newton/Goldschmidt counts, activation degree) systematically overspend depth on many layers.
  • Task- and architecture-specific Pareto fronts become practical: different GLUE tasks or vision vs language prefer different per-layer budgets.
  • Search finishes fast enough (under an hour for LLaMA3-8B) that approximation configuration can be part of ordinary deployment, not a one-off expert effort.
  • The approach is complementary to packing and matrix-multiply work: depth savings stack with those orthogonal speedups.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If proxy-MAE ranking generalizes, the same two-stage pattern could automate other FHE hyperparameter families (bootstrapping placement thresholds, slot packing choices) that today are still hand-set.
  • The large fraction of invalid (NaN) configs suggests future search could add cheap validity filters or learned feasibility models before full evaluation.
  • Per-layer slack implies that “safe” domain constants (softmax max, activation range) might also be layer-adaptive rather than global hard-codes.
  • As open weights and FHE stacks standardize, automated approximation search may become a default compiler pass for private inference rather than a research prototype.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 9 minor

Summary. The paper presents ATLAS, a post-processing framework that replaces hand-tuned, layer-uniform polynomial approximation hyperparameters in FHE transformer inference (softmax iterations/degrees, Newton/Goldschmidt invsqrt steps, activation polynomial degree) with per-layer configurations found by a two-stage NSGA-II search over (multiplicative depth, cleartext MAE). Stage 1 searches a 10-variable layer-uniform space with early-layer and data-subset surrogates; Stage 2 refines per-layer over the full 10L-variable space. On BERT-base, ViT-base, and LLaMA3-8B, evaluated on a Phantom-FHE/NEXUS-End2End-derived CKKS system on RTX 4090, the authors report ~35% (BERT/LLaMA) and ~29% (ViT) depth reduction and ~20–28% end-to-end latency reduction versus an iterative-softmax baseline (Cho et al., CCS 2024), with near-baseline task accuracy, and search wall-clock under one hour. Ablations in Fig. 10 support the two-stage joint design and quantify the surrogate/distributed speed-ups.

Significance. If the results hold, this is a useful and practical contribution: per-layer approximation configuration is a genuine, unexploited degree of freedom in FHE transformer inference, and the paper demonstrates that it yields real, directly measured savings. Credit is due on several points: (i) the depth, bootstrapping-count, and latency claims are measured end-to-end on an actual GPU CKKS stack rather than extrapolated from depth formulas; (ii) the two-stage design and each acceleration technique are ablated (Fig. 10a/b) rather than asserted; (iii) the analysis in §4.3 (Fig. 8/9) yields transferable insight — e.g., BERT attention scores needing only k=3 softmax iterations — that is valuable independent of the search method; (iv) code is released, and the reproduction caveats for NEXUS/THOR are disclosed unusually honestly. The main weaknesses are not in the efficiency claims but in the evidentiary basis for "negligible accuracy loss," particularly on LLaMA, and in the robustness of hard-coded softmax max-constants calibrated on a single sample.

major comments (5)
  1. [§4.2.3, Table 1c] The LLaMA accuracy claims rest on evaluations too small to support them. MMLU is evaluated on 64 sampled items from the abstract-algebra subset, so IS at 31.25% (20/64) vs. ATLAS-M2 at 32.81% (21/64) is a single-item difference; a 95% Wilson interval around 31.25% on n=64 spans roughly ±11pp. HumanEval differences (~2pp) correspond to ~3 problems, and GSM8K deltas are ~0.2pp. The claims 'matching or exceeding baseline accuracy' (§4.2.3) and 'improves MMLU to 32.81%' are therefore statements about eval noise. Since 'negligible accuracy loss' on LLaMA is one of the abstract's headline claims and one of three architectures demonstrating generality, this needs either a substantially larger evaluation (full MMLU subset or several thousand items, with confidence intervals reported) or an explicit softening of the LLaMA accuracy language to 'within eval noise at current evaluation size.'
  2. [§3.5, Fig. 5, App. E] The search ranks all 24,144 configurations by cleartext MAE on a 10-sample proxy, but the surrogate validation for BERT does not cover the operating point: the reported Kendall τ≈0.85 for BERT is at subset size 50, while the text states size 10 was used 'for all three models.' BERT's τ at size 10 is never reported and is presumably lower; the ViT (0.936) and LLaMA (0.905) values at size 10 suggest BERT is the outlier, which is precisely the model where the final accuracy margins are thinnest (S4 at 93.0% vs. IS 93.2%). Additionally, the LLaMA τ in App. E is computed on only 8 NetworkProblem configurations, which is too few for a stable rank-correlation estimate. Please report τ at the actually-used subset sizes for all three models with an adequate number of sampled configurations, and state whether any search decisions would change under a larger proxy.
  3. [App. B.3, §4.3 (Fig. 8), Table 2] The per-layer softmax max-constants (Eq. 13–15) are calibrated on a single sample per model and injected into every evaluation, cleartext and FHE. The search objective never stresses these constants, and the tiny proxy/eval sets make violation unlikely to be observed. This interacts directly with the discovered configurations: the k=3 softmax found for BERT (Fig. 8) implies an effective input domain of roughly [−38.8, 0], and the exp approximation itself is fit on (−8, 0) (Table 2); if deployment activations produce x_i − c below −38.8 (e.g., because the true row max exceeds the single-sample constant c), the approximation fails silently. The BERT SST-2 results are on a fixed dev set with the same constants, so this risk is untested. At minimum the paper needs (i) an empirical distribution of per-layer attention-score ranges over a larger corpus showing headroom under the chosen c and k,
  4. [§4.2.1, Table 1a] The comparison against NEXUS† and THOR† in Table 1a is against degenerate baselines: reproduced NEXUS achieves 49.1% on SST-2 (near chance) and THOR† 52.4% on QNLI. The abstract and §4.2.1 then claim ATLAS outperforms NEXUS and THOR in both accuracy and efficiency, but an efficiency comparison against a configuration that fails its accuracy task is not meaningful — NEXUS's 948-depth recipe at usable accuracy might look quite different. The authors' conjecture (weight-specific calibration of hard-coded domains) is plausible but untested; a concrete check, e.g., recalibrating NEXUS's GELU interval and softmax constant to the open-source weights and re-measuring, would either substantiate the generalizability argument or reveal that the comparison should be restricted to IS. As written, the NEXUS/THOR columns of Table 1a should be framed strictly as a domain-sensitivity finding, not as a co
  5. [§3.1 (Eq. 6), §4.2] The search objective (Eq. 6) is output-deviation MAE, while the reported outcome is task accuracy, a thresholded function of the outputs. Configurations near-tied in proxy MAE need not preserve accuracy ordering, and this is only checked post hoc by evaluating the Pareto front on downstream tasks. The steep fronts in Fig. 6/7 partially mitigate this concern, but the paper should state how many Pareto configurations were task-evaluated, whether the (depth, proxy-MAE) front and the (depth, accuracy) front contain the same configurations, and whether any high-HV search solutions failed downstream. This is the load-bearing transfer step of the whole method and currently rests on implicit evidence.
minor comments (9)
  1. [Table 1c] Table 1c reports only depth and accuracy for LLaMA; #Boot and end-to-end latency are missing for the LLaMA rows even though latency reduction is a headline claim. Please add these, as Fig. 2-style numbers for LLaMA are needed to back the '~25% latency' claim for that architecture.
  2. [§2.3, App. C] In §2.3, 'From the NEXUS [47] codebase' cites the reproduction repo [47] rather than the NEXUS paper [54]; the same pattern recurs in §2.3 and §C. Please make the paper/code citation split consistent.
  3. [§4.1 Baselines] The BERT baseline pairs iterative softmax with a degree-511 GELU, which is described as 'high-precision' but is arguably not the strongest uniform baseline for the latency comparison (degree 255 as in ARION, or the degree-127 polynomials the search itself converges to, would be more competitive). A short justification of why degree 511 is the right IS reference point would strengthen the 35%/25% headline figures.
  4. [§4.4, Fig. 10a, App. A] Fig. 10a's grid-search baseline is labeled '21,000 FEs' reaching HV ~310, but the grid resolution per operator group is not specified; without it the 'lower-bound baseline' is hard to interpret. Also, the HV reference point and objective scaling are not reported anywhere (they affect all HV numbers); please include them.
  5. [§4.1] The 10-sample proxy subsets are 'randomly sampled' (§4.1); please state whether subsets are fixed across the search, how many seeds were tried, and the sensitivity of the final Pareto set to the subset draw. One or two repeated searches with different subsets would address this cheaply given the 30-minute budget.
  6. [Fig. 7] Fig. 7 (b)–(d) y-axes show accuracy on MMLU/HumanEval/GSM8K over very narrow ranges; given the n=64 issue, error bars or item counts should accompany these plots to avoid over-reading one-item differences.
  7. [§3.4.2] §3.4.2: the Stage-1/Stage-2 budget split (1/10 vs 9/10) is introduced parenthetically; a brief ablation or sensitivity note on this ratio would be useful since it is a free parameter of the method.
  8. [Algorithm 1, Table 2] Algorithm 1, line 1 fixes the exp polynomial degree at 15; Table 2 lists fixed input intervals per iteration j. Please clarify whether these intervals (inherited from [16]) remain valid under the per-layer constants c of App. B.3, which shift the pre-softmax inputs.
  9. Typos/style: 'ATLAS tractable' (§3.5, 'make the ATLAS tractable'); 'each of the L Transformer layer' (§3.2); abstract's '10^84'/'10^225' formatting; Fig. 3(b) y-axis label is truncated in the rendered figure.

Circularity Check

0 steps flagged

No significant circularity: ATLAS is empirical multi-objective search with externally measured depth, latency, and task accuracy.

full rationale

The paper does not present a first-principles derivation that collapses into its inputs. It formulates per-layer approximation hyperparameter selection as multi-objective minimization of multiplicative depth and cleartext MAE (Eq. 6), solves it with two-stage NSGA-II, then reports measured FHE latency and downstream task accuracy against fixed baselines (iterative softmax, reproduced NEXUS/THOR). Multiplicative depth is a deterministic function of the chosen degrees/iterations; MAE and task metrics are evaluated against a frozen cleartext model and external benchmarks (GLUE, ImageNet, MMLU/GSM8K/HumanEval). Surrogates (10-sample MAE, early-layer MAE) only accelerate ranking during search; final claims rest on post-hoc full evaluations, not on renaming the surrogate as the result. Citations to Cho et al., NEXUS, THOR, and AutoFHE supply operator recipes or related CNN work; none load-bear a uniqueness theorem or force the reported depth/latency gains by construction. Mild engineering choices (one-sample softmax max-constants, proxy subsets) affect validity risk, not circularity. Score 0; steps empty.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 1 invented entities

The claim rests on standard CKKS/HE cost structure, published nonlinear approximation templates, and evolutionary multi-objective search, plus several engineering choices (proxy metrics, bounds, FE budgets, fixed softmax maxima) that are not derived from first principles. No new physical entities; the contribution is a configuration method and measured trade-offs.

free parameters (6)
  • NSGA-II Stage-1/Stage-2 budgets (pop 48×50 gen; pop 96×225 gen; total 24144 FEs) = 2448 + 21696 FEs
    Hand-chosen evaluation budget that defines how thoroughly the combinatorial space is explored; reported HV and Pareto quality depend on it.
  • Proxy subset size for MAE = 10
    Fixed to 10 samples for BERT/ViT/LLaMA during search; ranking fidelity is empirical (Kendall τ), not proven.
  • Early-layer proxy indices (BERT L4, ViT L7, LLaMA L9) = layer 4 / 7 / 9
    Chosen from τ-vs-speedup curves; Stage-1 objective uses these truncated MAEs.
  • Per-variable integer bounds (softmax p_j, Newton/Goldschmidt ν,γ, activation p_act) = as in §3.2
    Search box upper bounds follow Cho et al. / NEXUS defaults (e.g., p≤7, ν≤4, γ≤2, p_act≤9); they truncate the true design space.
  • Per-layer softmax max-constants c = model-specific vectors
    Hard-coded from one cleartext sample per model (App. B.3); all configs inherit these constants.
  • HV reference point and MAE-vs-depth objective scaling
    Pareto/HV comparisons depend on reference and objective definitions used inside PyMOO.
axioms (5)
  • domain assumption CKKS multiplicative depth (plus bootstrapping when levels are insufficient) dominates nonlinear FHE cost and is a valid primary efficiency objective.
    Stated in §1–§2 and used to replace latency in Eq. 5–6; latency still depends on packing, MM, and bootstrap placement.
  • domain assumption Iterative softmax (Cho et al.) with Chebyshev invsqrt/exp, Newton–Goldschmidt invsqrt, and Chebyshev GELU/SiLU are adequate approximation families.
    Search only tunes hyperparameters inside these templates (§3.2, Alg. 1–2), not alternative functional forms.
  • ad hoc to paper NSGA-II with two-point crossover and integer step mutation is a sufficient optimizer for the stated NP-hard integer multi-objective problem.
    No optimality guarantee; justified by HV growth vs grid/sequential baselines (§3.4, Fig. 10).
  • domain assumption Cleartext polynomial simulation of approximations predicts relative FHE behavior for ranking configurations.
    Search never runs FHE inside the loop (§3.3–3.5); final tables use FHE only for selected Pareto points.
  • standard math Standard multi-objective Pareto dominance and hypervolume compare configuration quality.
    NSGA-II / HV machinery from Deb et al. and Zitzler & Thiele.
invented entities (1)
  • ATLAS two-stage LayerProblem→NetworkProblem encoding no independent evidence
    purpose: Make 10L-variable per-layer approximation search tractable via uniform warm-start then heterogeneous refinement.
    Methodological construct, not a physical entity; validated only by search-efficiency and downstream tables in this paper.

pith-pipeline@v1.2.0-grok45-kimik3 · 45878 in / 3739 out tokens · 74472 ms · 2026-07-30T21:14:37.642700+00:00 · methodology

0 comments
read the original abstract

Fully homomorphic encryption (FHE) provides strong cryptographic guarantees for private inference, but deploying transformer models under FHE remains prohibitively expensive. A key bottleneck is that non-linear operations such as softmax, normalization, and activation must be replaced with polynomial approximations compatible with the CKKS scheme, and the multiplicative depth consumed by these approximations dominates inference cost. Recent frameworks have advanced approximation techniques, yet all rely on manually configured approximation hyperparameters (e.g., number of iterations, polynomial degree), applied uniformly across all layers. While convenient, this uniform-configuration approach is overly rigid: different layers can tolerate different levels of approximation error without degrading predictive accuracy, and uniform configurations cannot exploit this variability to reduce latency. Allowing each layer to adopt its own configuration, however, causes the search space to explode with model depth, reaching roughly $10^{84}$ configurations for BERT/ViT (12 layers) and $10^{225}$ for LLaMA3 (32 layers), rendering manual exploration practically impossible. We present ATLAS, an automated framework that configures per-layer approximation settings by formulating the problem as a multi-objective optimization over latency and predictive accuracy. The resulting problem is inherently difficult: 1) competing objectives over a large decision space (120 or 320 variables for BERT/ViT or LLaMA3); 2) expensive evaluation, as each configuration takes 70-1,000 seconds even in cleartext; and 3) sparse optimization signals, as 35-50% of candidate configurations yield numerically invalid solutions. ATLAS addresses these challenges through a two-stage optimization strategy that progressively relaxes layer-wise constraints, combined with surrogate models to accelerate evaluation.

Figures

Figures reproduced from arXiv: 2607.23478 by Jianhang Xie, Sicheng Tan, Vishnu Naresh Boddeti, Zhichao Lu.

Figure 1
Figure 1. Figure 1: ATLAS can automatically design polynomial approx￾imation configurations for FHE compatible transformers. ATLAS solutions span the trade-off between accuracy and la￾tency for ciphertext inference and can be deployed on the cloud server to satisfy a range of customer requirements. Keywords Fully Homomorphic Encryption, CKKS Scheme, Transformers, Se￾cure Inference, Multi-Objective Optimization 1 Introduction … view at source ↗
Figure 2
Figure 2. Figure 2: Runtime breakdown of end-to-end FHE BERT [ [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Key challenges in searching for FHE approximation [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: ATLAS is a two-stage process. Stage 1 identifies a set of efficient layer-uniform approximations, and Stage 2 refines these into heterogeneous layer-wise configurations to further improve the depth–accuracy trade-off. • Expensive evaluation. Each candidate configuration must be instantiated as a model and evaluated to obtain its MAE and depth. Even in the cleartext polynomial setting used during search, a … view at source ↗
Figure 5
Figure 5. Figure 5: MAE at an early layer as a surrogate for MAE at the [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Pareto fronts discovered by ATLAS on three tasks of BERT. Each point represents a non-dominated 𝝀 configuration, plotted in the space of multiplicative depth vs. downstream task accuracy. The iterative softmax baseline is shown as a reference. Dashed horizontal lines indicate the accuracy of the underlying cleartext model. capture, and what ATLAS discovers automatically. Table 1a summa￾rizes the results of… view at source ↗
Figure 7
Figure 7. Figure 7: Pareto fronts discovered by ATLAS on ViT and three tasks of LLaMA. Each point represents a non-dominated 𝝀 configuration, plotted in the space of multiplicative depth vs. downstream task accuracy. The iterative softmax baseline is shown as a reference. Dashed horizontal lines indicate the accuracy of the underlying cleartext model. 4.2.2 ViT. The IS baseline achieves a top-1 accuracy of 80.32% with a laten… view at source ↗
Figure 8
Figure 8. Figure 8: Per-layer visualization of the Pareto configs of [PITH_FULL_IMAGE:figures/full_fig_p010_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Per-layer visualization of the Pareto configs of ATLAS for ViT shown in Figure 7a [PITH_FULL_IMAGE:figures/full_fig_p010_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Search efficiency of ATLAS on BERT. (a) Comparison of optimisation strategies in terms of hypervolume achieved for a given number of function evaluations. (b) Impact of the acceleration techniques introduced in Section 3.5: each curve adds one more technique on top of the previous. Comparison of Optimization Strategies. Figure 10a tracks the HV of the Pareto front as a function of the number of FEs for fi… view at source ↗
Figure 11
Figure 11. Figure 11: Illustration of the hypervolume in two-dimensional minimization case. B FHE Inference System Implementation We develop our GPU-accelerated FHE inference system based on our extensions of an open-source codebase, i.e., NEXUS-End2End [47]. Our experiment platform is a modified version RTX 4090 with 48GB. The encoder of BERT, decoder of LLaMA, and vision encoder of ViT can be run in constrained 40GB GPU base… view at source ↗
Figure 12
Figure 12. Figure 12: MAE at an early layer as a surrogate for MAE at [PITH_FULL_IMAGE:figures/full_fig_p016_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Accuracy–Latency Pareto fronts discovered by [PITH_FULL_IMAGE:figures/full_fig_p017_13.png] view at source ↗
Figure 15
Figure 15. Figure 15: Accuracy–Latency Pareto front discovered by [PITH_FULL_IMAGE:figures/full_fig_p017_15.png] view at source ↗
Figure 14
Figure 14. Figure 14: Per-layer visualization of the Accuracy–Latency [PITH_FULL_IMAGE:figures/full_fig_p017_14.png] view at source ↗
Figure 16
Figure 16. Figure 16: Per-layer visualization of the Accuracy–Latency [PITH_FULL_IMAGE:figures/full_fig_p017_16.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

61 extracted references · 5 linked inside Pith

  1. [1]

    Wei Ao and Vishnu Naresh Boddeti. 2024. AutoFHE: Automated adaption of CNNs for efficient evaluation over FHE. InProceedings of the 33rd USENIX Security Symposium. 2173–2190

    2024

  2. [2]

    Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. Layer normaliza- tion.arXiv preprint arXiv:1607.06450(2016)

    Pith/arXiv arXiv 2016

  3. [3]

    Youngjin Bae, Jaehyung Kim, Damien Stehlé, and Elias Suvanto. 2024. Bootstrap- ping small integers with CKKS. InInternational Conference on the Theory and Application of Cryptology and Information Security. Springer, 330–360

    2024

  4. [4]

    Jean-Philippe Bossuat, Christian Mouchet, Juan Troncoso-Pastoriza, and Jean- Pierre Hubaux. 2021. Efficient bootstrapping for approximate homomorphic encryption with non-sparse keys. InAnnual International Conference on the Theory and Applications of Cryptographic Techniques. 587–617

    2021

  5. [5]

    Zvika Brakerski. 2012. Fully homomorphic encryption without modulus switch- ing from classical GapSVP. InAnnual Cryptology Conference. 868–886

    2012

  6. [6]

    Zvika Brakerski, Craig Gentry, and Vinod Vaikuntanathan. 2014. (Leveled) fully homomorphic encryption without bootstrapping.ACM Transactions on Computation Theory (TOCT)6, 3 (2014), 1–36

    2014

  7. [7]

    Mark Chen, Jerry Tworek, Heewoo Jun, et al. 2021. Evaluating Large Language Models Trained on Code.arXiv preprint arXiv:2107.03374(2021)

    Pith/arXiv arXiv 2021

  8. [8]

    Jung Hee Cheon, Wonhee Cho, Jaehyung Kim, and Damien Stehlé. 2023. Ho- momorphic multiple precision multiplication for CKKS and reduced modulus consumption. InProceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security. 696–710

    2023

  9. [9]

    Jung Hee Cheon, Hyeongmin Choe, Minsik Kang, Jaehyung Kim, Seonghak Kim, Johannes Mono, and Taeyeong Noh. 2025. Grafting: decoupled scale factors and modulus in RNS-CKKS. InProceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security. 1098–1112

    2025

  10. [10]

    Jung Hee Cheon, Kyoohyung Han, Andrey Kim, Miran Kim, and Yongsoo Song

  11. [11]

    Jung Hee Cheon, Kyoohyung Han, Andrey Kim, Miran Kim, and Yongsoo Song. 2018. A full RNS variant of approximate homomorphic encryption. In International Conference on Selected Areas in Cryptography. 347–368. https: //link.springer.com/chapter/10.1007/978-3-030-10970-7_16

    doi:10.1007/978-3-030-10970-7_16 2018

  12. [12]

    Jung Hee Cheon, Guillaume Hanrot, Jongmin Kim, and Damien Stehlé. 2025. SHIP: A shallow and highly parallelizable CKKS bootstrapping algorithm. In Annual International Conference on the Theory and Applications of Cryptographic Techniques. Springer, 398–428

    2025

  13. [13]

    Jung Hee Cheon, Minki Hhan, Seungwan Hong, and Yongha Son. 2019. A Hybrid of Dual and Meet-in-the-Middle Attack on Sparse and Ternary Secret LWE.IEEE Access7 (2019), 89497–89506

    2019

  14. [14]

    Jung Hee Cheon, Andrey Kim, Miran Kim, and Yongsoo Song. 2017. Homo- morphic encryption for arithmetic of approximate numbers. InInternational conference on the theory and application of cryptology and information security. Springer, 409–437

    2017

  15. [15]

    Ilaria Chillotti, Nicolas Gama, Mariya Georgieva, and Malika Izabachène. 2020. TFHE: fast fully homomorphic encryption over the torus.Journal of Cryptology 33, 1 (2020), 34–91

    2020

  16. [16]

    Wonhee Cho, Guillaume Hanrot, Taeseong Kim, Minje Park, and Damien Stehlé

  17. [17]

    Hyeongmin Choe, Jaehyung Kim, Damien Stehlé, and Elias Suvanto. 2025. Lever- aging discrete CKKS to bootstrap in high precision. InProceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security. 1083–1097

    2025

  18. [18]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training Verifiers to Solve Math Word Problems.arXiv preprint arXiv:2110.14168(2021)

    Pith/arXiv arXiv 2021

  19. [19]

    Leo de Castro, Daniel Escudero, Adya Agrawal, Antigoni Polychroniadou, and Manuela Veloso. 2025. EncryptedLLM: Privacy-Preserving Large Language Model Inference via GPU-Accelerated Fully Homomorphic Encryption. InProceedings of the International Conference on Machine Learning

    2025

  20. [20]

    K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. 2002. A fast and elitist multiobjec- tive genetic algorithm: NSGA-II.IEEE Transactions on Evolutionary Computation 6, 2 (2002), 182–197. doi:10.1109/4235.996017

    arXiv 2002

  21. [21]

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 248–255

    2009

  22. [22]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 4171–4186

    2019

  23. [23]

    Ye Dong, Wen-jie Lu, Yancheng Zheng, Haoqi Wu, Derun Zhao, Jin Tan, Zhicong Huang, Cheng Hong, Tao Wei, and Wenguang Chen. 2023. Puma: Secure inference of llama-7b in five minutes.arXiv preprint arXiv:2307.12533(2023). https: //arxiv.org/abs/2307.12533

    arXiv 2023

  24. [24]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. InProceedings on the International Conference on Learning Repres...

    2021

  25. [25]

    Austin Ebel, Karthik Garimella, and Brandon Reagen. 2025. Orion: A Fully Homomorphic Encryption Framework for Deep Learning. InProceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems. 734–749

    2025

  26. [26]

    Zahra Ghodsi, Akshaj Kumar Veldanda, Brandon Reagen, and Siddharth Garg

  27. [27]

    1964.Applications of division by convergence

    Robert E Goldschmidt. 1964.Applications of division by convergence. Ph. D. Dissertation. Massachusetts Institute of Technology

    1964

  28. [28]

    Abellan, David Kaeli, and Ajay Joshi

    Seyda Nur Guzelhan, Lohit Daksha, Carlos Agulló Domingo, Gilbert Jonatan, John Kim, Jose L. Abellan, David Kaeli, and Ajay Joshi. 2026. ELLMo: Packing- and Depth-Aware Encrypted Transformer Inference.Cryptology ePrint Archive, Paper 2026/198(2026). https://eprint.iacr.org/2026/198

    2026

  29. [29]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring Massive Multitask Language Un- derstanding. InProceedings of the International Conference on Learning Represen- tations

    2021

  30. [30]

    Nandan Kumar Jha, Zahra Ghodsi, Siddharth Garg, and Brandon Reagen. 2021. Deepreduce: Relu reduction for fast private inference. InProceedings of the Inter- national Conference on Machine Learning. PMLR, 4839–4849

    2021

  31. [31]

    Andrey Kim, Ahmet Can Mert, Anisha Mukherjee, Aikata Aikata, Maxim Deryabin, Sunmin Kwon, Hyung Chul Kang, and Sujoy Sinha Roy. 2024. Explor- ing the advantages and challenges of fermat NTT in FHE acceleration. InAnnual International Cryptology Conference. Springer, 76–106

    2024

  32. [32]

    Eunsang Lee, Joon-Woo Lee, Junghyun Lee, Young-Sik Kim, Yongjune Kim, Jong- Seon No, and Woosuk Choi. 2022. Low-complexity deep convolutional neural networks on fully homomorphic encryption using multiplexed parallel con- volutions. InProceedings of the International Conference on Machine Learning. 12403–12422

    2022

  33. [33]

    Eunsang Lee, Joon-Woo Lee, Jong-Seon No, and Young-Sik Kim. 2021. Minimax approximation of sign function by composite polynomial for homomorphic comparison.IEEE Transactions on Dependable and Secure Computing19, 6 (2021), 3711–3727

    2021

  34. [34]

    Junghyun Lee, Eunsang Lee, Joon-Woo Lee, Yongjune Kim, Young-Sik Kim, and Jong-Seon No. 2023. Precise approximation of convolutional neural networks for homomorphically encrypted data.IEEE Access11 (2023), 62062–62076

    2023

  35. [35]

    Lawrence Lim, Vikas Kalagi, Divyakant Agrawal, and Amr El Abbadi. 2025. Tricycle: Private Transformer Inference with Tricyclic Encodings.Cryptology ePrint Archive, Paper 2025/1200(2025). https://eprint.iacr.org/2025/1200

    2025

  36. [36]

    Wen-jie Lu, Zhicong Huang, Zhen Gu, Jingyu Li, Jian Liu, Cheng Hong, Kui Ren, Tao Wei, and WenGuang Chen. 2025. BumbleBee: Secure Two-party Inference Framework for Large Transformers. InProceedings of the 32nd Annual Network and Distributed System Security Symposium

    2025

  37. [37]

    Maas, Raymond E

    Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning Word Vectors for Sentiment Analysis. InProceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Dekang Lin, Yuji Matsumoto, and Rada Mihalcea (Eds.). Association for Computational Lingu...

    2011

  38. [38]

    Jungho Moon, Dongwoo Yoo, Xiaoqian Jiang, and Miran Kim. 2025. THOR: Secure transformer inference with homomorphic encryption. InProceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security. 3765–3779

    2025

  39. [39]

    Qi Pang, Jinhao Zhu, Helen Möllering, Wenting Zheng, and Thomas Schneider

  40. [40]

    Dongjin Park, Eunsang Lee, and Joon-Woo Lee. 2025. Powerformer: Efficient and High-Accuracy Privacy-Preserving Language Model with Homomorphic Encryption. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics. 11090–11111

    2025

  41. [41]

    Hanjun Park, Byeong-Seo Min, Jiheon Woo, Min-Wook Jeong, Jongho Shin, Yongwoo Lee, Young-Sik Kim, and Yongjune Kim. 2026. Efficient Softmax Refor- mulation for Homomorphic Encryption via Moment Generating Function.arXiv preprint arXiv:2602.01621(2026)

    Pith/arXiv arXiv 2026

  42. [42]

    Hongyuan Qu and Guangwu Xu. 2023. Improvements of homomorphic secure evaluation of inverse square root. InInternational Conference on Information and Communications Security. Springer, 110–127

    2023

  43. [43]

    In2024 IEEE Symposium on Security and Privacy (SP)

    BOLT: Privacy-Preserving, Accurate and Efficient Inference for Transform- ers. In2024 IEEE Symposium on Security and Privacy (SP). 4753–4771

  44. [44]

    Donghwan Rho, Taeseong Kim, Minje Park, Jung Woo Kim, Hyunsik Chae, Ernest K Ryu, and Jung Hee Cheon. 2025. Encryption-Friendly LLM Architecture. InProceedings of the International Conference on Learning Representations

    2025

  45. [45]

    SEAL. 2020. Microsoft SEAL is an easy-to-use and powerful homomorphic encryption library. https://github.com/Microsoft/SEAL

    2020

  46. [46]

    Wenting Zheng Srinivasan, PMRL Akshayaram, and Popa Raluca Ada. 2020. Delphi: A cryptographic inference service for neural networks. InProceedings of the 29th USENIX Security Symposium. 2505–2522

    2020

  47. [47]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners.OpenAI blog 1, 8 (2019), 9

    2019

  48. [48]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971(2023). https://arxiv.org/abs/2302.13971

    Pith/arXiv arXiv 2023

  49. [49]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need.Advances in neural information processing systems30 (2017)

    2017

  50. [50]

    Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019. GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. InProceedings on the International Conference on Learning Representations

    2019

  51. [51]

    Zhengyuan Su. 2025. Reproducing NEXUS with Phantom bootstrapping. https: //github.com/timzsu/NEXUS-End2End

    2025

  52. [52]

    Linhan Yang, Jingwei Chen, Wangchen Dai, Shuai Wang, Wenyuan Wu, and Yong Feng. 2025. ARION: Attention-Optimized Transformer Inference on Encrypted Data.Cryptology ePrint Archive, Paper 2025/2271(2025). https://eprint.iacr.org/ 2025/2271

    2025

  53. [53]

    Biao Zhang and Rico Sennrich. 2019. Root mean square layer normalization. In Proceedings of the Advances in Neural Information Processing Systems

    2019

  54. [54]

    Jiawen Zhang, Xinpeng Yang, Lipeng He, Kejia Chen, Wen-jie Lu, Yinghao Wang, Xiaoyang Hou, Jian Liu, Kui Ren, and Xiaohu Yang. 2025. Secure transformer inference made non-interactive. InProceedings of the 32nd Annual Network and Distributed System Security Symposium

    2025

  55. [55]

    Hao Yang, Shiyu Shen, Wangchen Dai, Lu Zhou, Zhe Liu, and Yunlei Zhao. 2024. Phantom: A cuda-accelerated word-wise homomorphic encryption library.IEEE Transactions on Dependable and Secure Computing21, 5 (2024), 4895–4906

    2024

  56. [56]

    Itamar Zimerman, Moran Baruch, Nir Drucker, Gilad Ezov, Omri Soceanu, and Lior Wolf. 2024. Converting transformers to polynomial form for secure inference over homomorphic encryption. InProceedings of the International Conference on Machine Learning. 62803 – 62814

    2024

  57. [57]

    Zitzler and L

    E. Zitzler and L. Thiele. 1999. Multiobjective evolutionary algorithms: a com- parative case study and the strength Pareto approach.IEEE Transactions on Evolutionary Computation3, 4 (1999), 257–271. doi:10.1109/4235.797969 A Hypervolume For a set of solutions S and a reference point 𝒓, the hypervol- ume [57] HV(S,𝒓) measures the portion of the objective s...

    arXiv 1999

  58. [59]

    Linru Zhang, Xiangning Wang, Jun Jie Sim, Zhicong Huang, Jiahao Zhong, Huax- iong Wang, Pu Duan, and Kwok-Yan Lam. 2026. MOAI: Module-optimizing architecture for non-interactive secure transformer inference. InProceedings of the 14th International Conference on Learning Representations

    2026

  59. [2018]

    InAnnual Inter- national Conference on the Theory and Applications of Cryptographic Techniques

    Bootstrapping for approximate homomorphic encryption. InAnnual Inter- national Conference on the Theory and Applications of Cryptographic Techniques. 360–384

  60. [2020]

    InProceedings of the Advances in Neural Information Processing Systems, Vol

    Cryptonas: Private inference on a relu budget. InProceedings of the Advances in Neural Information Processing Systems, Vol. 33. 16961–16971

  61. [2024]

    InProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security

    Fast and accurate homomorphic softmax evaluation. InProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security. 4391–4404

    2024