{"id":"4bf5c3f9-f410-4a58-8e2c-5c3197c1e903","arxiv_id":"2412.01042","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"TruncFormer statically places truncations in private LLM inference so all nonlinear operations reduce to adds, multiplies, and truncations, cutting estimated truncation latency by up to about 1.92x versus PUMA without hurting accuracy.","lead":"TruncFormer converts language models into versions whose private, encrypted inference needs only additions, multiplications, and truncations, with truncation locations chosen automatically. It reports up to 1.92x lower estimated truncation latency than the PUMA baseline while keeping model accuracy, although full private inference remains too slow for practical use.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1's post-truncation width update is incorrect for multiplications, so the static truncation schedule may under-truncate and overflow; this is more load-bearing than the latency-emulation issue.","rationale":"The reader's identified weakest assumption, the unvalidated plaintext-emulation latency model, is a real concern for the quantitative speedup claims. However, I found a more direct and more fundamental problem: Algorithm 1's internal width update rule is inconsistent for multiplication. This attacks the core correctness claim, not just the magnitude of the speedups. The pseudocode's while-loop recomputes worst case using an addition rule after truncating an operand, which is mathematically wrong for products. In the concrete 52-bit-by-52-bit case, it would stop after one truncation and output 53 bits when the actual product still needs 65 bits, overflowing the 64-bit field. If the implementation is correct, the paper must correct the pseudocode and ideally include a formal invariant; if the implementation follows the pseudocode, the reported accuracy results are difficult to explain and need re-examination. Because this is likely a fixable specification error rather than a fundamental invalidation of the truncation-scheduling idea, I do not recommend changing the conditional verdict, but the conditions for acceptance should include correcting and verifying Algorithm 1. I give credit where due: the paper honestly reports that PI remains impractical and releases its code, which makes a concrete verification test feasible.","tokens_in":14866,"tokens_out":5786,"duration_ms":55944,"concrete_test":"Run the released TruncFormer code on a minimal 52-bit × 52-bit multiplication with F=64 and k=13, and also simulate Algorithm 1 exactly as written on that DAG. If either reports an output width of 53 rather than the correct 65, the algorithm under-truncates. Additionally, inspect the multiplication width-propagation code to see whether it computes k + other_width or max(k, other_width)+1; the implementation and pseudocode must match the corrected bound.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing issue is internal to the central construction, not the benchmarking methodology. Algorithm 1 in Section 3 computes the output bit-width of a multiplication as E1+E2, then, inside the `while worst case > F` loop, it truncates an operand to k and recomputes `worst case ← max(E1,E2)+1`. That recomputation is the rule for addition only; for multiplication, after truncating one operand to k bits, the output width is k + (the other operand's width). Example with F=64 and k=13: a product of two 52-bit operands starts at 104 bits; truncating one operand to 13 gives a 65-bit product, which still exceeds the field, but the pseudocode computes max(13,52)+1 = 53 and stops, recording a 53-bit output. Subsequent nodes then inherit an underestimated width, so the static schedule can miss needed truncations and overflow can occur. This directly contradicts the abstract's claim that the framework 'statically determines where to compute truncations wherever overflows can occur.' If the released code uses the correct k + other rule, then Algorithm 1 as written is wrong and needs correction; if it follows the pseudocode, the accuracy results are surprising and may reflect a different width accounting. Either way, the paper's central algorithm is not yet adequately specified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TruncFormer, a framework for private LLM inference that statically schedules truncation operations in fixed-point arithmetic, avoiding the common practice of truncating after every multiplication. The framework replaces LLM nonlinearities with differentiable approximations built from additions, multiplications, and truncations, and determines truncation locations from a bit-width analysis of the model graph. The authors evaluate accuracy on Llama-7B and Gemma-2B across coding, math, commonsense, and perplexity benchmarks, and report truncation-latency speedups of 0.98x--1.92x over a self-implemented PUMA baseline. The core claim is that truncation scheduling, rather than bespoke nonlinearity approximations, is the key lever for faster private LLM inference.","tokens_in":15050,"tokens_out":8736,"duration_ms":81227,"significance":"If the static truncation scheduling is correct, the paper identifies a valuable and relatively unexplored optimization axis for private inference: deciding at compile time where truncations are actually needed, instead of after every multiplication. The accuracy results are broad and mostly close to plaintext, the operation-level breakdown is informative, and the authors have open-sourced their code. However, the central algorithm as written has a correctness bug in multiplication bit-width accounting, and the quantitative speedup claims rest on unit-test timings summed over the model graph rather than measured end-to-end private inference. These issues are fixable, but they currently prevent the paper's central claims from being fully supported.","major_comments":[{"comment":"The inner while loop recomputes `worst case <- max(E1,E2)+1` after truncating an operand, which is the addition rule and is incorrect for multiplication. After truncating one operand of a multiplication to k bits, the output width is k plus the other operand's width, not max(E1,E2)+1. For example, with F=64 and k=13, a product of two 52-bit operands has worst-case width 104 bits; truncating one operand to 13 bits leaves a 65-bit product, which still exceeds the field, but the pseudocode computes max(13,52)+1=53, stops, and records E3=53. This under-estimates downstream bit widths and contradicts the abstract's claim that truncations are inserted wherever overflows can occur. Please correct the multiplication rule and, if the released code mirrors the pseudocode, re-run the bit-width analysis and experiments; if the code uses the correct k+other rule, state that explicitly.","section":"Algorithm 1 (Section 3)"},{"comment":"All reported speedups are ratios of unit-test timings summed over the model graph, not measured end-to-end private inference. The PUMA baseline is the authors' reimplementation inside their own framework rather than the published PUMA system. Truncation time may dominate (Table 3), but the headline 0.98x--1.92x figures should be labeled as truncation-time speedups under a unit-test cost model, and the authors should report a sensitivity analysis over non-truncation overheads (communication, preprocessing, protocol orchestration) before claiming end-to-end latency improvements.","section":"Section 4.1, Tables 2 and 4"},{"comment":"The title and contribution 3 claim private inference with \"adds/multiplies and truncations alone,\" but Section 3 states that differentiable functions can be represented \"with only truncations and comparisons as the out-of-field operations.\" If comparisons remain in the approximated graphs, their cost must be included and the claim should be revised; if not, the text should explain how comparisons are avoided.","section":"Abstract and Contribution 3"}],"minor_comments":[{"comment":"The variable `iterations` in `while iterations <= Niter` is never initialized; please specify the starting value and the loop increment.","section":"Algorithm 2 (Section 3)"},{"comment":"Table 3 should state explicitly that the percentages are for the TruncFormer model under the unit-test cost model, and should indicate the field size and input bit-width used.","section":"Table 3"},{"comment":"The paper says \"takes any transformer model,\" but experiments cover only Llama-7B and Gemma-2B; please soften the claim or add evidence for architectural generality.","section":"Abstract and Contributions"},{"comment":"The algorithm title contains the typo \"Trunctions,\" and the notation `V <- (operation, E1, E2, E3)` and `H <- (V,H)` should be clarified so that the graph update is unambiguous.","section":"Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The static truncation direction is worth pursuing, and the breadth of the accuracy evaluation is a strength. The key correctness risk is Algorithm 1: please verify whether the released implementation's bit-width accounting matches the corrected multiplication rule, since this directly affects the overflow guarantee and could change the experimental results. The quantitative comparison to PUMA is also weakened by the reimplementation and by the absence of end-to-end measurements; ask the authors to present the figures as unit-test-based estimates or provide a stronger validation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's core idea—decoupling truncations from multiplications and placing them statically—is worth taking seriously, and the observation that truncations dominate modeled latency (90–99%) is a useful reframing for the field. Adapting CrypTen's approximations to 64/128-bit fields is a concrete additional contribution. The accuracy results on Llama-7B and Gemma-2B are close to plaintext, and the authors honestly acknowledge that PI is still impractical. Those are real merits.\n\nThe load-bearing problem is Algorithm 1. For a multiplication, after truncating one operand to k bits, the output width is k plus the other operand's width, not max(E1,E2)+1 as the pseudocode states. With F=64 and k=13, two 52-bit operands produce a 104-bit product; truncating one to 13 gives 65 bits, which still overflows, but the algorithm computes max(13,52)+1 = 53, sees that as within the field, and stops. The schedule can therefore miss needed truncations. The abstract's claim of statically determining truncations \"wherever overflows can occur\" is not supported by the written algorithm. If the released code uses the correct rule, the pseudocode is wrong and needs fixing; if the code follows the pseudocode, the accuracy results are surprising and need explanation. Either way, the central algorithm is not adequately specified.\n\nThe methodology also has a soft spot: the speedups come from summing unit-test timings of scalar garbled-circuit truncations and beaver-triple multiplications, not from end-to-end private inference runs. That can miss communication overhead, pipeline stalls, and protocol interactions. The PUMA baseline is reimplemented inside their own framework rather than using the actual system. These issues could change the quantitative speedups, though the direction—fewer truncations should help—is plausible.\n\nMinor inconsistency: the title and abstract say \"only truncations\" (with adds/multiplies), but Section 2 says comparisons are also out-of-field operations. Worth cleaning up.\n\nWho is this for? Researchers working on MPC-based LLM inference, especially those deciding where to spend optimization effort. It deserves a serious referee: the idea is novel and the flaws are fixable. A good review would require correcting Algorithm 1, clarifying the code path, and either running one end-to-end PI configuration or clearly labeling the numbers as projected estimates. I'd send it to review rather than desk-reject.","headline":"The core idea of statically scheduling truncations is promising, but Algorithm 1's width update is wrong for multiplications and the latency claims rest on unit tests—worth a serious referee, not a desk reject.","tokens_in":15685,"tokens_out":2859,"would_cite":false,"duration_ms":25301,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TruncFormer shows that private LLM inference can run on additions, multiplications, and statically scheduled truncations alone, cutting latency by up to 1.92x while matching plaintext accuracy.","keywords":["private inference","secure multiparty computation","large language models","truncation scheduling","fixed-point arithmetic","garbled circuits","nonlinearity approximation","transformer inference"],"falsifier":"Run a true end-to-end private inference of Llama-7B and Gemma-2B at 64- and 128-bit fields using the TruncFormer schedule, with actual secret-shared multiplications and garbled-circuit truncations, and compare total wall-clock time against the summed unit-test estimate; if protocol-level overheads materially change the ratio to the truncate-every-multiply baseline, the claimed speedups will not hold.","tokens_in":14604,"feed_emoji":"🔐","tokens_out":10893,"duration_ms":91352,"temperature":0.7,"pith_summary":"This paper claims that the main latency cost in private LLM inference is not the exotic nonlinear functions that receive the most attention, but the routine truncation that follows every multiplication in a fixed-size cryptographic field. TruncFormer therefore turns private inference into a scheduling problem: examine the model's arithmetic graph once, compute worst-case bit growth, and insert a truncation only where the result could overflow the field. On Llama-7B and Gemma-2B, the method matches plaintext accuracy on nearly all tested benchmarks while cutting the dominant truncation cost by up to 1.92x relative to the standard truncate-after-every-multiplication recipe. A sympathetic reader would care because this points at a single operation that consumes 90-99% of private-inference time, giving the field one lever to pull instead of a moving target of new activation functions.","feed_headline":"Truncation scheduling speeds private LLM inference up to 1.92x","feed_subtitle":"Truncations eat 90-99% of private inference time; scheduling them instead of chasing new approximations is the new lever.","key_machinery":"The load-bearing object is a directed acyclic graph whose vertices are the model's additions and multiplications and whose edges carry worst-case bit widths. Starting from an input representation of $k$ bits, the scheduler sets each addition's output to $\\max(m,n)+1$ bits and each multiplication's output to $m+n$ bits; if that exceeds the field size $F$, it inserts a truncation that resets the larger input to $k$ bits and repeats. Because the graph is known before any private computation, the resulting truncation list is static and data-independent. The same pass is paired with fixed-point encoding at scale $M=2^n$ and with Newton-Raphson and limit approximations for softmax, GeLU, SiLU, and layer norm, modified with lower initial guesses and more iterations so they converge at 64- and 128-bit field sizes.","core_discovery":"The paper's central discovery is that the standard private-inference practice of truncating after every multiplication is wasteful, and that a static pass over the model's arithmetic graph can identify the only multiplications whose results can overflow the fixed field. For each addition the worst-case bit width is $\\max(m,n)+1$; for each multiplication it is $m+n$; when that bound exceeds the field size, the scheduler resets the larger input to the input bit-width and records a truncation. Iterating this over the whole transformer produces a data-independent truncation schedule, and every out-of-field nonlinearity is replaced by iterative approximations built from additions, multiplications, and truncations. On Llama-7B and Gemma-2B this preserves accuracy on the vast majority of downstream benchmarks, with one acknowledged perplexity outlier, while cutting private-inference truncation latency by up to roughly 1.9x compared with a reimplementation of the previous state of the art.","pith_inferences":["The paper leaves open the natural next lever: choosing field size per layer or per operation instead of one global 64- or 128-bit field, since each truncation costs more in larger fields and a mixed schedule could beat both uniform choices.","The same static analysis should transfer to other differentiable models beyond transformers; the paper only demonstrates it on two LLM families, but the DAG scheduler is architecture-agnostic.","The paper's own operation breakdown predicts where speedups will show up in a real system: long-prompt, single-token settings gain most from softmax truncation savings, while activation-heavy autoregressive generation gains less, and this is a testable prediction for any end-to-end implementation."],"forward_implications":["Because truncations consume 90-99% of private-inference time in the measured models, reducing the number of truncations is a larger lever than speeding any single nonlinearity approximation.","New differentiable activation functions no longer require bespoke private-inference protocols: they can be approximated by additions, multiplications, and truncations and then scheduled by the same static pass.","The truncation schedule is computed once per model and is data-independent, so it adds no per-query cryptographic cost.","Field size interacts with scheduling: going from 64 to 128 bits removes many truncations but roughly doubles each truncation's cost, yielding only 0.98x-1.04x net speedups, so the choice of field size is part of the optimization.","The method preserves plaintext accuracy on nearly all tested benchmarks, so the latency savings are not bought with degraded model quality."],"supporting_citations":[{"why":"Supplies the two-party private-inference protocol structure, secret sharing for in-field operations and garbled circuits for out-of-field operations, that TruncFormer's latency model follows.","marker":"(Mishra et al., 2020)"},{"why":"The state-of-the-art LLM private-inference baseline whose softmax and GeLU/SiLU optimizations TruncFormer reimplements to obtain accuracy and speedup comparisons.","marker":"(Dong et al., 2023)"},{"why":"Source of the iterative approximations for exponentials, reciprocals, and inverse square roots that TruncFormer adapts to higher bit precision.","marker":"(Knott et al., 2021)"},{"why":"Garbled-circuit toolkit used to benchmark the scalar truncation latencies at 64 and 128 bits that generate the reported speedups.","marker":"(Wang et al., 2016)"},{"why":"Provides the garbled-circuit benchmarking methodology used to measure evaluator-side truncation latency.","marker":"(Mo et al., 2023)"},{"why":"Supports the premise that truncation is an out-of-field operation, motivating the need to minimize truncation count.","marker":"(Li et al., 2023)"},{"why":"Supplies the remaining out-of-field operations used in the reconstructed baseline, defining the comparison point for non-softmax and non-GeLU components.","marker":"(Ma et al., 2023)"}],"fun_headline_variants":["TruncFormer: Smarter truncation scheduling for faster private inference","Private LLM inference gets a 1.9x speedup with TruncFormer's static truncations","Scheduling truncations, not chasing approximations, speeds private LLM inference","TruncFormer: Only truncate when needed to speed up private LLM inference","Static truncation planning boosts private LLM inference latency"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that timing a single garbled-circuit truncation and a single secret-shared multiplication in isolation, then adding the costs over the whole model, faithfully predicts how long true end-to-end private inference will take.","fun_headline_variants_meta":{"raw":{"variants":["TruncFormer: Smarter truncation scheduling for faster private inference","Private LLM inference gets a 1.9x speedup with TruncFormer's static truncations","Scheduling truncations, not chasing approximations, speeds private LLM inference","TruncFormer: Only truncate when needed to speed up private LLM inference","Static truncation planning boosts private LLM inference latency"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000569,"raw_usage":{"total_tokens":2698,"prompt_tokens":953,"completion_tokens":1745,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":569,"completion_tokens_details":{"reasoning_tokens":1645}},"tokens_in":569,"tokens_out":1745,"duration_ms":11001,"temperature":1.0,"reasoning_tokens":1645,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T04:45:13.270296+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a true end-to-end private inference of Llama-7B and Gemma-2B at 64- and 128-bit fields using the TruncFormer schedule, with actual secret-shared multiplications and garbled-circuit truncations, and compare total wall-clock time against the summed unit-test estimate; if protocol-level overheads materially change the ratio to the truncate-every-multiply baseline, the claimed speedups will not hold.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the two-party private-inference protocol structure, secret sharing for in-field operations and garbled circuits for out-of-field operations, that TruncFormer's latency model follows."}],"review_version":1}