{"id":"b7521747-a793-47ae-9b32-0f10e06bdf85","arxiv_id":"2505.10315","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A structured survey of private transformer inference, comparing MPC- and HE-based methods and showing non-linear layers dominate overhead.","lead":"This paper reviews ways to run transformer AI models on a server without the server being able to read the user's private input. It organizes these privacy-protecting methods into categories and compares their speed, accuracy, and data-transfer costs.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1's cross-paper comparison rests on unverified attribution and non-commensurable setups, so the survey's central comparative conclusions are only as sound as a primary-source audit.","rationale":"The reader's conditional verdict identifies the same central weakness: the cross-paper comparison in Table 1 depends on commensurability and correct attribution. I agree with that assessment and with the conditional recommendation because the survey still provides a useful taxonomy and a broad organizational structure that can be repaired. My agreement is partial rather than full because the reader's specific claim that Zeng is duplicated for two blocks is not exactly what appears in the supplied text: the table contains separate rows for [Zheng et al., 2023] and [Zeng et al., 2023]. The deeper issue remains valid, however, because [Zheng et al., 2023] appears only in the table and not in the narrative, and the relationship between the BERT row and the PRIMER reference is not made explicit. The mismatch between Section 6.2's characterization of Hao et al. as ASS-based and Table 1's MPC+HE classification is another internal inconsistency that a reader cannot resolve from the paper alone. None of these issues invalidate the survey's organizational value, and they are the kind of mechanical but substantive errors that a careful revision can fix. The novelty claim of being the first PTI survey cannot be verified internally, but that is a separate external question rather than an internal correctness problem. I see no basis for moving the verdict to reject or unverified; the appropriate outcome is to accept the paper conditionally on a corrected and audited Table 1.","tokens_in":14049,"tokens_out":4232,"duration_ms":39960,"concrete_test":"Reconstruct the disputed rows of Table 1 from primary sources: (1) open the PRIMER paper [Zheng et al., 2023] and record its BERT-Tiny/Base/Large communication, runtime, and accuracy values; (2) do the same for MPCViT [Zeng et al., 2023] on CCT; compare those numbers with Table 1 to see whether the current attribution swaps or duplicates a block; and (3) for the runtime ordering claim in Section 6.3, extract the CPU model, thread count, and network parameters from Gupta et al., Pang et al., and Zhang et al. and recompute the comparison under a single normalized hardware and network model. If either the PRIMER row is misattributed or the ordering flips after normalization, the survey's comparative conclusions need revision.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The survey's central value proposition is the structured comparison in Table 1, and that table is the least secure part of the argument. Three conditions must hold for the comparison to support the Section 6 conclusions: (i) every row correctly reports the cited primary source; (ii) columns are measured under comparable conditions; (iii) the taxonomy labels (MPC, HE, MPC+HE) match the cited papers' actual technique. The paper itself concedes in Section 6.3 that because most studies are not open source, \"differences in experimental platforms may still affect the runtime performance.\" The table mixes network setups, including 5 Gbps/1 ms, 9.4 Gbps/0.05 ms, 100 Mbps/2.3 ms, 3 Gbps/0.8 ms, 1 Gbps/0.5 ms, and 100 Mbps/80 ms, and it does not report CPU model or thread counts, so the runtime ordering in Section 6.3 (MPC-only about 1.84 s vs hybrid about 185 s vs HE about 857 s) is not established as a technique effect rather than a hardware or network effect. There are also internal inconsistencies: Section 6.2 characterizes [Hao et al., 2022] as \"using ASS\" while Table 1 labels it MPC+HE; and the only place [Zheng et al., 2023] (PRIMER) appears is a table row, with the text using [Zeng et al., 2023] (MPCViT) for CCT experiments, so a reader cannot tell from the paper alone whether the BERT-Tiny row is correctly attributed. These issues are correctable, but until a primary-source audit is done, the headline comparative conclusions must be treated as conditional.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript surveys private transformer inference (PTI) in the machine-learning-as-a-service setting, covering works from 2022 to 2025. It organizes the literature with a taxonomy of linear layers (matrix multiplication, embedding, feed-forward) versus non-linear layers (Softmax, GELU, LayerNorm) and of the underlying cryptographic primitives (MPC, HE, and hybrid MPC+HE). The paper also presents Table 1, which aggregates reported communication volumes, runtimes, accuracy values, and accuracy losses across roughly fifteen studies, and uses this table in Section 6 to draw conclusions about communication overhead, runtime ordering, and approximation accuracy. Section 7 lists future directions: GPU acceleration, evaluation on generation tasks, and real-world practicality. The paper claims to be the first survey focused specifically on PTI.","tokens_in":14315,"tokens_out":8362,"duration_ms":77213,"significance":"If its comparative claims survive a primary-source audit, this survey would be a useful entry point: the linear/non-linear taxonomy is clear, the table consolidates numbers that are otherwise scattered across many papers, and the discussion of non-linear-layer bottlenecks is grounded in concrete percentages from the cited works. The paper is also explicit about the semi-honest threat model and about the fact that most PTI studies are not open source. The contribution is organizational and comparative rather than technical; no new protocols, proofs, or reproducible artifacts are offered. The value is therefore gated by the accuracy and commensurability of Table 1 and by the internal consistency of the technique labels attached to each row.","major_comments":[{"comment":"The classification of [Hao et al., 2022] is internally inconsistent. Section 3.3 describes it as 'using HE', Section 6.2 describes it as 'using ASS', and Table 1 labels it 'MPC+HE'. Because Section 6.2 uses the communication figure 280.99 GB to support a claim about MPC-based studies, and Section 6.3 uses the same row in the runtime ordering, the primitive attribution is load-bearing for the survey's comparative conclusions. Please check the primary source and align all three statements.","section":"§3.3, §6.2, Table 1"},{"comment":"The central runtime comparison is not commensurable as presented. Table 1 mixes network configurations (9.4 Gbps/0.05 ms, 5 Gbps/1 ms, 3 Gbps/0.8 ms, 1 Gbps/0.5 ms, 100 Mbps/2.3 ms, 100 Mbps/80 ms, and 44 Mbps/40 ms) and does not report CPU model or thread counts. Section 6.3 acknowledges that 'differences in experimental platforms may still affect the runtime performance,' but the following paragraph still concludes that MPC-only solutions are faster than hybrid and HE-only solutions, citing 1.84 s versus 185 s versus 857 s. This ordering is not established as a technique effect rather than a hardware/network effect. Please restrict the comparison to commensurable settings, or explicitly re-label the ordering as reflecting heterogeneous reported setups and remove the causal attribution to the cryptographic technique.","section":"Table 1, §6.3"},{"comment":"The BERT-family block in Table 1 is attributed to [Zheng et al., 2023] (PRIMER), but the prose never cites Zheng et al.; the reference-list entry for [Zheng et al., 2023] appears only in the table. The CCT rows are attributed to [Zeng et al., 2023] (MPCViT). A reader therefore cannot verify from the manuscript alone whether the BERT row's numbers (for example MRPC: 0.9 GB, 10.6 s) come from PRIMER, or whether the row is a mislabeled duplicate of another source. Please add a prose citation for [Zheng et al., 2023] and audit the row against the primary source.","section":"Table 1, References"}],"minor_comments":[{"comment":"The novelty claim that 'surveys focusing on PTI do not exist so far' is supported only by two adjacent surveys; a broader, systematic related-work search would substantiate a 'first survey' claim.","section":"§1, §7"},{"comment":"The phrase 'three NLU tasks and nine corresponding corpora' is confusing; GLUE has nine tasks grouped into three families (single-sentence, similarity/paraphrase, and inference). Please rephrase.","section":"§6.1"},{"comment":"For several HE-only rows the communication column is blank or contains '-'; the caption should state that communication overhead was not reported in those studies.","section":"Table 1"},{"comment":"The reference [Zhang et al., 2023] contains a typo in the title: 'efficiefnt' should be 'efficient'.","section":"References"},{"comment":"The ethical statement says 'Our experiments use only open-source datasets,' but this paper is a survey and performs no experiments; please rephrase to describe the survey's data sources.","section":"Ethical Statement"},{"comment":"The statement 'RSS requires at least three parties' is too broad; the replicated secret-sharing variant used in the cited works requires three parties under an honest-majority assumption, not secret sharing in general.","section":"§4.1"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern that Table 1 duplicates [Zeng et al., 2023] does not fully land in the version I reviewed: the table as provided contains one [Zheng et al., 2023] block for BERT-family models and one [Zeng et al., 2023] block for CCT. The deeper problem is that [Zheng et al., 2023] is never cited in the text, so the attribution remains unverifiable from the manuscript alone. The load-bearing defects are the Hao et al. primitive inconsistency across sections and the runtime commensurability issue in Table 1 and Section 6.3."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a competent and genuinely useful survey of private transformer inference, and it earns a serious referee. The taxonomy (linear vs non-linear layers, MPC vs HE) organizes a live subfield; Table 1 gives researchers a concrete map of 2022-2025 methods, models, datasets, and reported numbers; and the prose mostly tracks the cited papers. It is synthesis rather than new science, which is fine for a survey.\n\nWhat I would credit: the paper correctly identifies where the cost lives. Non-linear layers dominate PTI overhead, and the survey documents this across MPC, HE, and hybrid systems. The technical summaries of Softmax/GELU/LayerNorm approximations are accurate enough for a newcomer. The honest discussion of platform-dependent runtime and the failed Q&A example are good signs.\n\nSoft spots. The central comparison in Section 6 is weaker than the rest. The authors admit in 6.3 that most studies are not open source and that hardware differences affect runtimes, yet the table still implies a ranking (MPC ~1.84s, hybrid ~185s, HE ~857s). Those numbers come from different network setups and unknown CPU/thread counts, so the ordering is not demonstrated as a technique effect. That is a major caveat, not a fatal one. There is also an internal inconsistency: Section 6.2 calls Hao et al. (2022) an ASS-based method while Table 1 labels it MPC+HE; one of these is wrong and it changes the communication comparison.\n\nOne point where the stress-test overshoots: the claim about a duplicated [Zeng et al., 2023] block in Table 1 does not hold up on inspection. The table has a separate PRIMER [Zheng et al., 2023] row and a separate MPCViT [Zeng et al., 2023] row. The real issue is subtler: PRIMER appears only in the table, never in the text, so no reader can verify the attribution without going to the source. The paper also gives no systematic search protocol, so the 'first PTI survey' claim is plausible but unproven.\n\nWho gets value: a grad student starting in private inference, or a researcher who wants a compact map of the area. It should not be treated as an authoritative benchmark table until the entries are audited.\n\nRecommendation: send to peer review. Ask for a fixed Hao classification, a normalization or caveat column for the runtime comparison, explicit mention of PRIMER in the text, and a short selection protocol for included papers.","headline":"A useful but uneven first PTI survey: the taxonomy and comparative table earn their place, but the cross-paper runtime ranking needs an audit before it should be quoted.","tokens_in":14944,"tokens_out":3866,"would_cite":true,"duration_ms":35080,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims to be the first survey of private transformer inference, organizing 2022–2025 solutions by a linear/non-linear layer taxonomy and comparing MPC, HE, and hybrid approaches.","keywords":["private transformer inference","secure multi-party computation","homomorphic encryption","machine learning as a service","taxonomy","non-linear layers","Softmax approximation","privacy-preserving inference"],"falsifier":"A literature search for any earlier survey dedicated specifically to private transformer inference would settle the novelty claim, and re-running the Table 1 BERT-Base comparisons on a single hardware platform with the original code would settle whether the cross-paper numbers are comparable.","tokens_in":13761,"feed_emoji":"🔐","tokens_out":7589,"duration_ms":68310,"temperature":0.7,"pith_summary":"The paper claims to be the first survey devoted specifically to Private Transformer Inference, covering work from 2022 to 2025. It argues that every transformer layer can be sorted into linear or non-linear cryptographic operations, and that this split explains the design space: matrix multiplications are natively supported by MPC and homomorphic encryption, while Softmax, GELU, and LayerNorm dominate the cost and force accuracy-versus-efficiency trade-offs. The survey organizes existing protocols into MPC-only, HE-only, and hybrid categories and compares them on communication, runtime, and accuracy across BERT, GPT, Vision Transformer, and LLaMA models. A careful reader would care because the comparison gives a map of where the field stands and where the remaining bottlenecks are.","feed_headline":"First survey maps private transformer inference's cost trade-offs","feed_subtitle":"MPC is fast but chatty; homomorphic encryption is quiet but slow; non-linear layers are the bottleneck.","key_machinery":"The load-bearing mechanism is the binary taxonomy of transformer layers in cryptographic contexts: linear layers (embedding, attention MatMul, feed-forward) versus non-linear layers (Softmax, GELU, LayerNorm). This split determines which cryptographic tools apply and where overhead concentrates; the paper reports that non-linear layers account for more than 85% of runtime in one MPC baseline and about 60% in one hybrid baseline. The companion evaluation grid (communication, runtime, accuracy) is what makes the taxonomy yield an explicit comparison across studies.","core_discovery":"On its own terms, the paper's discovery is that private transformer inference can be usefully reviewed as a problem of matching cryptographic primitives to layer types. Linear layers (embedding, attention matrix multiplications, feed-forward networks) are treated by secure multiplication techniques such as Beaver triples, replicated secret sharing, or SIMD-encoded homomorphic operations; non-linear layers (Softmax, GELU, LayerNorm) are handled by substitution, polynomial approximation, lookup tables, or iterative reciprocal-square-root methods. The paper asserts that no earlier survey had this PTI-specific focus, and it uses its taxonomy to produce a cross-paper comparison table. The central pattern that emerges is a three-way trade-off: MPC-only schemes are fastest but communication-heavy, HE-only schemes are non-interactive and communication-light but slow, with bootstrapping a major cost, and hybrid schemes sit between; accuracy losses come mainly from non-linear approximations and can sometimes be repaired by knowledge distillation.","pith_inferences":["If the linear/non-linear split generalizes, private inference for non-transformer sequence models with data-dependent gating would likely show the same pattern: linear projections are easy to secure, while activation and normalization functions dominate the cost.","The table's attributions should be independently checked before relying on its numbers: [Zheng et al., 2023] appears in the reference list but never in the text, while two rows are labeled [Zeng et al., 2023], so at least one row may belong to the wrong paper.","The non-interactive HE direction suggests a practical MLaaS pattern where a client uploads one ciphertext and disconnects; if GPU-accelerated HE matures, private inference could become a drop-in API with predictable communication."],"forward_implications":["A newcomer to PTI can use the taxonomy to decide whether their bottleneck is communication (choose HE or hybrid) or latency (choose MPC with function secret sharing).","Further speedups should target non-linear layers first, since they are the dominant cost in both MPC and HE settings.","HE-only systems already allow non-interactive inference; reducing bootstrapping cost is the clearest path to practical server-only private inference.","Knowledge distillation is an effective but expensive repair for accuracy loss from crypto-friendly approximations, while refined polynomial approximations reduce the loss without retraining.","GPU acceleration and evaluation on generation tasks remain open problems, since current benchmarks are mostly classification tasks."],"supporting_citations":[{"why":"Defines the transformer/attention architecture that the surveyed PTI protocols must secure.","marker":"[Vaswani et al., 2017]"},{"why":"Foundational secure two-party computation protocol that underpins the MPC category.","marker":"[Yao, 1982]"},{"why":"Introduces fully homomorphic encryption, the basis of the HE category and its bootstrapping overhead.","marker":"[Gentry, 2009]"},{"why":"MPCFormer provides the MPC baseline with aggressive Softmax/GELU substitutions and reports non-linear layers taking over 85% of runtime.","marker":"[Li et al., 2022]"},{"why":"IRON is the hybrid MPC+HE baseline whose 280.99 GB BERT-Base communication and roughly 60% non-linear time anchor the comparison.","marker":"[Hao et al., 2022]"},{"why":"SIGMA is the fastest MPC-only baseline (1.84 seconds on BERT-Base) and the main evidence for MPC's latency advantage.","marker":"[Gupta et al., 2023]"},{"why":"NEXUS is the non-interactive HE-only baseline with 0.16 GB communication and 857 seconds runtime, used for the HE trade-off.","marker":"[Zhang et al., 2025]"},{"why":"BOLT is the hybrid baseline showing refined polynomial approximations that cut accuracy loss to 1.18% on STS-B.","marker":"[Pang et al., 2024]"},{"why":"PUMA supplies the LLaMA-7B MPC result and the observed generation-task failure that motivates the scalability discussion.","marker":"[Dong et al., 2023]"}],"fun_headline_variants":["Survey maps crypto trade-offs for private transformer inference","Private transformer inference: match crypto to layer type","PTI: MPC fast but chatty, HE slow but quiet","New taxonomy for private transformer inference in MLaaS"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The survey's utility rests on two assumptions: that it is genuinely the first survey focused on private transformer inference, and that the runtimes, communication volumes, and accuracies it tabulates from different, mostly non-open-source papers were measured under comparable conditions and are correctly attributed.","fun_headline_variants_meta":{"raw":{"variants":["Survey maps crypto trade-offs for private transformer inference","Private transformer inference: match crypto to layer type","PTI: MPC fast but chatty, HE slow but quiet","New taxonomy for private transformer inference in MLaaS"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000251,"raw_usage":{"total_tokens":1500,"prompt_tokens":833,"completion_tokens":667,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":449,"completion_tokens_details":{"reasoning_tokens":604}},"tokens_in":449,"tokens_out":667,"duration_ms":6409,"temperature":1.0,"reasoning_tokens":604,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:11:05.621085+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A literature search for any earlier survey dedicated specifically to private transformer inference would settle the novelty claim, and re-running the Table 1 BERT-Base comparisons on a single hardware platform with the original code would settle whether the cross-paper numbers are comparable.","supporting_citations":[{"cited_title":"Protocols for secure computa- tions","cited_arxiv_id":null,"evidence_quote":"Foundational secure two-party computation protocol that underpins the MPC category."},{"cited_title":"Fully homomorphic encryption using ideal lattices","cited_arxiv_id":null,"evidence_quote":"Introduces fully homomorphic encryption, the basis of the HE category and its bootstrapping overhead."},{"cited_title":"Iron: Pri- vate inference on transformers","cited_arxiv_id":null,"evidence_quote":"IRON is the hybrid MPC+HE baseline whose 280.99 GB BERT-Base communication and roughly 60% non-linear time anchor the comparison."},{"cited_title":"SIGMA: secure GPT inference with function secret sharing","cited_arxiv_id":null,"evidence_quote":"SIGMA is the fastest MPC-only baseline (1.84 seconds on BERT-Base) and the main evidence for MPC's latency advantage."},{"cited_title":"Secure transformer inference made non-interactive","cited_arxiv_id":null,"evidence_quote":"NEXUS is the non-interactive HE-only baseline with 0.16 GB communication and 857 seconds runtime, used for the HE trade-off."},{"cited_title":"BOLT: Privacy- preserving, accurate and efficient inference for transform- ers","cited_arxiv_id":null,"evidence_quote":"BOLT is the hybrid baseline showing refined polynomial approximations that cut accuracy loss to 1.18% on STS-B."}],"review_version":1}