REVIEW 4 major objections 6 minor 14 references
Towards Provable (In)Secure Model Weight Release Schemes
T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read TaylorMLP, a 'secure weight release' scheme, fails all three formal security goals because its published Taylor coefficients are efficiently invertible to the original model weights.
desk verdict Solid TaylorMLP break plus a useful formal framework, but the claimed WRec-insecurity result needs a concrete distance to close the gap between game and experiment. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The paper's argument runs on two connected mechanisms. The first is a game-based security framework in the code-based style: each goal is an experiment between challenger and adversary, security is a negligible advantage bound, and the two implication theorems are proved by contrapositive reduction (build an EffiInf adversary from a WRec adversary, and a W-IND adversary from a WRec adversary). The second is the derivative-ratio identity $f_{a,b}(x)=\mathrm{Act}^{(a)}(x)/\mathrm{Act}^{(b)}(x)$, which is what makes the TaylorMLP attack work. Since the released coefficient tensors satisfy $(a!\Theta_{i,a})/(b!\Theta_{i,b}) = f_{a,b}((z_0+b)[j])$ elementwise, each position contributes one scalar equation in one unknown pre-activation value. Inverting $f_{a,b}$ numerically solves for $z_0+b$, the bias $b$ follows, and $W_i$ is recovered by direct division, turning the released encoding into a recoverable system.
What would settle it
Run the attack on a released TaylorMLP checkpoint and compute the matrix-norm distance between the recovered and original weights; if that distance is larger than the threshold used by the paper's formal 'same weight' test, the attack wins the per-element statistics but does not win the formal weight-recovery game as written.
Extended reading notes
Core claim
On its own terms, the central discovery is that the TaylorMLP release format is algebraically invertible rather than merely hard to invert. For each MLP row the scheme publishes $\Theta_{i,n}=W_i \odot \mathrm{Act}^{(n)}(z_0+b)/n!$, and equating two different orders $a$ and $b$ gives $(a!\Theta_{i,a})/(b!\Theta_{i,b}) = f_{a,b}((z_0+b)[j])$ with $f_{a,b}(x)=\mathrm{Act}^{(a)}(x)/\mathrm{Act}^{(b)}(x)$. Because $f_{a,b}$ is invertible on nearly every pre-activation value for SiLU/GeLU, a numerical root-finder recovers $z_0+b$ elementwise, then $b$, then the full weight $W_i$. Across OPT and Llama-2 checkpoints the attack recovers 99.76% to 100% of parameters with <1% per-element relative error, running in under ten minutes for Llama-2-13B at an estimated $0.22 of cloud compute. The paper therefore classifies TaylorMLP as failing weight-recovery security and, by its two implication theorems, also failing efficient-inference security and weight-indistinguishability security.
Load-bearing premise
The formal verdict that TaylorMLP is insecure presupposes that recovering nearly all parameters with tiny per-element errors counts as 'the same weight' under the paper's distance test, and the paper never fixes that test's threshold.
Editorial extensions
If this is right
- TaylorMLP should no longer be treated as a protective release format, because a modest CPU attacker can reconstruct its weights from the published coefficients.
- Since efficient-inference security and weight-indistinguishability security both imply weight-recovery security, TaylorMLP's failure is not limited to one game: it fails all three goals at once.
- The measured attack cost is low enough to be routine: under $0.01 for OPT-125M, about $0.22 for Llama-2-13B, with runtimes from about 20 seconds to under 10 minutes.
- A weight release scheme whose transformation is an elementwise algebraic function of the weights cannot claim security until the corresponding recovery game is analyzed, and the paper's definitions give a concrete template for doing so.
Reading between the lines
- Our inference: the same ratio-of-released-tensors trick is likely to transfer to any release scheme that stores elementwise products of hidden weights with functions of one shared unknown scalar, so the attack's scope may extend beyond TaylorMLP itself.
- Our inference: because the paper leaves the 'same weight' threshold unspecified, its formal verdict is only as strong as that threshold; instantiating the distance and reporting full-matrix norm errors would make the attack's formal status unambiguous.
- Our inference: if such elementary algebra defeats TaylorMLP, the middle ground of 'protected but locally usable' weights may require genuinely keyed or randomized transformations and explicit hardness reductions, rather than deterministic algebraic encodings.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper proposes game-based security definitions for model weight release schemes (weight recovery, efficient inference, and weight indistinguishability), proves two implication relations among these notions, and applies the framework to TaylorMLP. The authors present an algebraic attack that reconstructs MLP weights from TaylorMLP's released Taylor coefficients by inverting ratios of activation-function derivatives, and they report experiments on OPT and Llama-2 models with per-parameter recovery rates between 99.76% and 100% under a 1% relative-error threshold, running in under ten minutes on a CPU. The paper concludes that TaylorMLP fails to provide weight-recovery-security and, via the stated implications, also fails the other two security notions, and it closes with design principles for future weight release schemes.
Significance. The practical attack is an important negative result for TaylorMLP as a weight-obfuscation mechanism: the released coefficients are an elementwise transformation of the original weights, and the paper demonstrates at scale that this transformation can be inverted at negligible cost. The proposed definitional framework is a useful step toward formalizing weight release security, and the blueprint in Section 5 is sensible. However, the central formal claim is not yet established by the reported evidence: Definition 3.2 leaves the distance and threshold for the Same predicate unspecified, the experiments report only per-parameter relative-error counts rather than a matrix norm, and the reductions in Section 3.3 rely on unstated assumptions about the Same predicate. These gaps are fixable, so the result is better treated as a strong practical insecurity result with a formal claim that needs tightening.
major comments (4)
- [§3.1 and Definition 3.2, with §4.4] The formal insecurity conclusion is not supported by the experiments as reported. Definition 3.2 defines WRec-security via AdvWRec = Pr[Same(W*, W)], but the surrounding text leaves Dist and the threshold for Same unspecified; Section 3.1 only gives an example and says delta can be chosen for some specific model types. The experiments in Table 1 report the percentage of parameters with relative error below 1%, which is neither a matrix norm nor an instantiation of the Same predicate. For OPT-1.3B, 1,003,519 of 402,653,184 parameters have relative error at least 1%, so under a max-relative-error threshold of 10^-3 the recovered matrix would not satisfy Same(W*, W). Under an L2 or Frobenius norm the attack might well win, but no such distance is computed. Please instantiate Dist and delta explicitly, report Dist(W*, W) for each model (e.g., ||W* - W||_2 / ||W||_2 and ||W* - W||_max / ||W||_max), and state the resulting advantage in the instantiated game before claiming in Section 4.4 that TaylorMLP fails to provide weight-recovery-security.
- [§3.2, Definition 3.2, and §4.1] The security notion itself is not well-defined for TaylorMLP because there is no security parameter. Definition 3.2 requires the advantage to be negligible with regard to the security parameter of the scheme, but for TaylorMLP KGen returns empty keys and Section 3.1's remark that security parameters are implicitly determined by model weight size does not specify how epsilon scales with any parameter. Since Release is deterministic and keyless, an adversary who can invert it has advantage 1 for every model size, so the notion of a negligible function has no clear referent. The paper should either specify a security parameter and a scaling of epsilon, or explicitly restrict the claim to the informal insecurity of the scheme rather than a formal WRec-security violation.
- [§3.3, proofs of EffiInf -> WRec and W-IND -> WRec] The two reduction proofs assume properties of the Same predicate that are not stated. In the W-IND reduction, the proof says that because Same(W0, W1) is false, either Same(W0, W) or Same(W1, W) holds for the recovered W. With approximate equality defined by a distance threshold, this need not hold: W can be within distance delta of both W0 and W1 if W0 and W1 are close but not Same, or it can be within delta of neither. Similarly, the EffiInf reduction uses correctness to conclude that Run(W, x) is Same to Run'(pk, W', x), but if W is only approximately equal to the original weight, this inference requires an additional robustness assumption on the distance or on the inference outputs. Please add a formal condition on Same (e.g., that the delta-ball around W0 and W1 are disjoint) and prove the implications under that condition.
- [§4.3, Table 1, and Appendix A] The success metric conflates practical recovery with the formal claim. Table 1 reports Recovered Ratio as the fraction of parameters with relative error below 1%, but for OPT-1.3B this still leaves 1,003,519 parameters above the 1% threshold, and for the Llama-2 rows the 100.00% figures are not accompanied by any error magnitudes or distribution. Appendix A concedes that the derivative-ratio f_{a,b} is numerically unstable on outlier values. This is consistent with a practical attack, but it is in tension with the claim of weight recovery as defined by Same(W*, W), and it also means the paper never quantifies how close the full reconstructed matrix is to the original. Please report the overall error distribution (max, mean, and quantiles of per-element relative error) and the matrix-norm distances requested above, and clarify how the numerical failures in Appendix A affect the recovered-parameter counts.
minor comments (6)
- [§3.2, Definitions 3.2-3.4] The word paramter appears in the phrase security paramter in all three definitions; this should be corrected to parameter.
- [§3.2, Definition 3.4] Line 4 of GameW-IND, W1 <- Train(D) not Same(W0, W1), is not a well-formed sampling procedure; the joint distribution over (W0, W1) should be specified explicitly, for example by sampling W0 first and then rejecting W1 until not Same(W0, W1), with care that this conditioning is well-defined.
- [§4.2, Eq. (16)] The notation 1/Act^(n)(z0+b) denotes elementwise reciprocal, but this is not stated; adding an explicit elementwise notation (such as a Hadamard inverse) would prevent ambiguity, especially since Eq. (19) and Eq. (20) rely on elementwise division.
- [§4.3, Figure 2] The caption says weights are concatenated along the x-axis while the axis is labeled Input Dimension (8192); this is confusing because the x-axis appears to be a concatenation over layers, not an input dimension. Please clarify the axes and the aggregation used in the heat map.
- [§4.3, attack cost paragraph] The instance type m8g.8xlarge is cited without a region or exact configuration; please provide the precise instance type and pricing source used to compute the dollar costs in Table 1.
- [Appendix A] The appendix describes numerical instability but does not state how the attack selects among the available pairs (a, b) of derivative orders when one pair is unstable; a short description of the selection heuristic would improve reproducibility.
Circularity Check
No circularity: the TaylorMLP attack is a parameter-free algebraic inversion of the scheme's own public Release transform, checked against ground-truth weights; the security definitions and reductions are external and standard.
full rationale
The central derivation is self-contained against an external target. Section 4.2 starts from Equation 11, Theta_{i,n} = W_i ⊙ Act^{(n)}(z0+b)/n!, which is the paper's own restatement of TaylorMLP's Release algorithm, and solves algebraically for W_i via ratios (a!)Theta_{i,a}/(b!)Theta_{i,b} = f_{a,b}((z0+b)[j]). Nothing is fitted to experimental data; the attack uses the released coefficients and public z0, and Table 1 compares recovered parameters to ground-truth model weights with a stated per-element relative-error criterion. The security definitions in Section 3 adapt Bellare-Rogaway/Shoup game-based methodology and Goldwasser-Micali indistinguishability, and the Section 3.3 implications are standard contrapositive reductions, not premises that already contain TaylorMLP's insecurity. The target paper TaylorMLP (Wang et al., 2024) is external, not a self-citation chain, and no uniqueness theorem or ansatz is imported from the authors' prior work. The one weakness noted in the manuscript, that Same(W*,W) in Definition 3.2 is left with an unspecified distance while experiments report per-parameter relative error below 1%, is a formalization and instantiation gap in the insecurity claim rather than a circular dependency; it does not make the derivation equivalent to its inputs. Accordingly, the paper's attack derivation is not circular.
Assumptions & free parameters
assumptions (5)
- domain assumption The ratio of activation derivatives f_{a,b}(x) = Act^{(a)}(x)/Act^{(b)}(x) is invertible for most pre-activation values and the inversion is numerically stable in the model's native precision.
- domain assumption The released weight W' leaks the value z0 = (zmax+zmin)/2 as part of its format, so the attacker knows the expansion point.
- domain assumption In the EffiInf-to-WRec reduction, the cost inequality Cost(Run') >> Cost(Run) + Cost(A) holds for every adversary A.
- domain assumption Correctness holds with probability 1, i.e., Run'(pk,W',x) produces the same output as Run(W,x) for the Same threshold.
- domain assumption Negligible functions are defined with respect to a fixed 'security parameter' implicitly given by model weight size.
Cite this review
Pith. "Pith review of Towards Provable (In)Secure Model Weight Release Schemes." pith.science (2026). https://pith.science/paper/HUG236NW
@misc{pith2026250619874,
author = {Pith},
title = {Pith review of: Towards Provable (In)Secure Model Weight Release Schemes},
year = {2026},
howpublished = {\url{https://pith.science/paper/HUG236NW}},
note = {Machine review of arXiv:2506.19874}
}
read the original abstract
Recent secure weight release schemes claim to enable open-source model distribution while protecting model ownership and preventing misuse. However, these approaches lack rigorous security foundations and provide only informal security guarantees. Inspired by established works in cryptography, we formalize the security of weight release schemes by introducing several concrete security definitions. We then demonstrate our definition's utility through a case study of TaylorMLP, a prominent secure weight release scheme. Our analysis reveals vulnerabilities that allow parameter extraction thus showing that TaylorMLP fails to achieve its informal security goals. We hope this work will advocate for rigorous research at the intersection of machine learning and security communities and provide a blueprint for how future weight release schemes should be designed and evaluated.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774,
-
[6]
Li, S., Yao, L., Gao, J., Zhang, L., and Li, Y . Double- i watermark: Protecting model copyright for llm fine- tuning.arXiv preprint arXiv:2402.14883,
-
[8]
Slip: Securing llms ip using weights decomposition.arXiv preprint arXiv:2407.10886,
Refael, Y ., Hakim, A., Greenberg, L., Aviv, T., Lokam, S., Fishman, B., and Seidman, S. Slip: Securing llms ip using weights decomposition.arXiv preprint arXiv:2407.10886,
-
[9]
Llama 2: Open foundation and fine- tuned chat models.arXiv preprint arXiv:2307.09288,
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y ., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine- tuned chat models.arXiv preprint arXiv:2307.09288,
-
[10]
Taylor unswift: Secured weight release for large language models via Taylor expansion
Wang, G., Chuang, Y .-N., Tang, R., Zhong, S., Yuan, J., Jin, H., Liu, Z., Chaudhary, V ., Xu, S., Caverlee, J., and Hu, X. Taylor unswift: Secured weight release for large language models via Taylor expansion. In Al- Onaizan, Y ., Bansal, M., and Chen, Y .-N. (eds.),Proceed- ings of the 2024 Conference on Empirical Methods in Natural Language Processing,...
work page 2024
-
[11]
doi: 10.18653/v1/2024.emnlp-main
Association for Compu- tational Linguistics. doi: 10.18653/v1/2024.emnlp-main
-
[14]
Yan, B., Li, K., Xu, M., Dong, Y ., Zhang, Y ., Ren, Z., and Cheng, X. On protecting the data privacy of large language models (llms): A survey.arXiv preprint arXiv:2403.05156,
-
[393]
CoTGuard: Using Chain-of-Thought Triggering for Copyright Protection in Multi-Agent LLM Systems
URL https://aclanthology.org/2024. emnlp-main.393/. Wen, Y ., Guo, J., and Huang, H. Cotguard: Using chain-of- thought triggering for copyright protection in multi-agent llm systems.arXiv preprint arXiv:2505.19405,
work page Pith review arXiv 2024
Show all 14 references
-
[1984]
doi: https://doi.org/10.1016/0022-0000(84)90070-9
ISSN 0022-0000. doi: https://doi.org/10.1016/0022-0000(84)90070-9. URL https://www.sciencedirect.com/ science/article/pii/0022000084900709. Guo, F., Susilo, W., Chen, X., Jiang, P., Lai, J., and Zhao, Z. Sok: Research motivations of public-key cryptography. Cryptology ePrint Archive,
-
[2004]
Puma: Secure inference of llama-7b in five minutes.arXiv preprint arXiv:2307.12533,
Dong, Y ., Lu, W.-j., Zheng, Y ., Wu, H., Zhao, D., Tan, J., Huang, Z., Hong, C., Wei, T., and Chen, W. Puma: Secure inference of llama-7b in five minutes.arXiv preprint arXiv:2307.12533,
-
[2007]
Privacy in large language models: Attacks, defenses and future directions.arXiv preprint arXiv:2310.10383,
Li, H., Chen, Y ., Luo, J., Wang, J., Peng, H., Kang, Y ., Zhang, X., Hu, Q., Chan, C., Xu, Z., et al. Privacy in large language models: Attacks, defenses and future directions.arXiv preprint arXiv:2310.10383,
-
[2023]
Qwen technical report.arXiv preprint arXiv:2309.16609,
Bai, J., Bai, S., Chu, Y ., Cui, Z., Dang, K., Deng, X., Fan, Y ., Ge, W., Han, Y ., Huang, F., et al. Qwen technical report.arXiv preprint arXiv:2309.16609,
-
[2024]
Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437,
Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C., et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437,
-
[2025]
Mark your llm: Detecting the misuse of open-source large language models via watermarking.arXiv preprint arXiv:2503.04636,
Xu, Y ., Liu, A., Hu, X., Wen, L., and Xiong, H. Mark your llm: Detecting the misuse of open-source large language models via watermarking.arXiv preprint arXiv:2503.04636,
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.