Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Membership Inference Risks in Quantized Models: A Theoretical and Empirical Study

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper derives that the worst-case membership inference security of a quantized model is asymptotically governed by a single rate constant r_Q — the squared gap between the two lowest quantized losses divided by their loss variance —…

desk verdict Useful empirical tool, but the theory overclaims: Theorem 3.3 only gives an upper bound on the MIS gap, so the quantizer-ranking rule is not proven. read the letter →

arxiv 2502.06567 v2 pith:TWYNWEAK submitted 2025-02-10 stat.ML cs.LG

Eric Aubinais , Philippe Formont , Pablo Piantanida , Elisabeth Gassiat This is my paper · ORCID
classification stat.MLcs.LG MSC 60F1068T07
keywords membershipinferencequantizationsecuritypost-trainingprivacylargedeviationsmoderateloss-gapprocess
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that the privacy of a quantized machine learning model against membership inference attacks, including attacks not yet invented, is asymptotically controlled by one simple quantity: the gap between the average losses of the two best quantized models, relative to their per-sample variance. The authors prove that, as the training set grows, the Membership Inference Security (MIS, defined as twice one minus the best achievable attack accuracy) is of order 1 − $e^{{−n r_Q}}$, so a larger r_Q means an asymptotically more private quantizer. Because computing the true r_Q over an exponentially large codebook is infeasible, they propose an estimator that reads the validation losses of quantized checkpoints along a single training trajectory, and they show empirically that this estimator ranks quantizers in the same order as an expensive attack-based baseline at roughly a tenth of the compute. On synthetic and molecular property-prediction data, the ranking reveals that sparsity, not bit width, drives privacy, and that regression tasks have almost no privacy-performance trade-off.

What carries the argument

The load-bearing object is the loss-gap process: for each quantized model θ_k, the per-sample difference ℓ(θ_k,z) − ℓ(θ_1,z) between its loss and the best quantized model's loss. The MIS is expressed as 1 minus a total-variation distance, which the proof bounds by the probability that the empirical mean of these gaps falls in the 'wrong' region Ω^c = {∃k : x_k < −1}—the event that some worse quantized model beats the best one on the training data. In the size-adaptive case the gaps are rescaled by δ^n_2 and √n; a central limit theorem in Hilbert space turns the problem into a moderate-deviation rate, and the infimum of the quadratic form x^T Λ^+x/2 over the boundary of Ω^c reduces, via a Schur-complement identity, to 1/($2σ^{2}$). This reduction is what makes the final rate depend only on the two lowest loss gaps and the scaled loss variances, which is exactly what Algorithm 1 estimates.

What would settle it

Take a small model whose quantized codebook can be enumerated exactly, compute r_Q from the full codebook, and compare it with Algorithm 1's trajectory-based estimate on the same data; a flip in the ranking against the baseline MIS discriminator would show that trajectory coverage fails. Alternatively, adversarially permute the codebook so the two lowest-loss quantized models lie far from any SGD checkpoint and check whether the estimated r_Q still orders quantizers correctly.

Watch

Extended reading notes

Core claim

The central claim is Theorem 3.3: for a size-adaptive quantizer whose minimal loss gap δ^n_2 tends to 0 while √n δ^n_2 tends to infinity, the MIS of the induced algorithm satisfies lim −1/(n(δ^n_2)^2) log(1 − MIS_n) ≥ 1/($2σ^{2}$), with $σ^{2}$ = max_k lim (δ^n_2/δ^n_k)^2 Var(ℓ(θ^n_k,z) − ℓ(θ^n_1,z)). Equivalently, MIS = 1 − $e^{{−n r_Q(1+o(1))}}$ with r_Q = (δ^n_2)^2/($2σ^{2}$), so two quantizers can be compared asymptotically by their r_Q values without running any attack. The paper also gives a fixed-quantizer version (Theorem 3.1) where the rate is a constant depending only on the loss distribution over the codebook, and a practical estimator (Algorithm 1) that computes r_Q from validation losses of quantized checkpoints collected along training. The empirical sections argue that this estimate correlates strongly with a baseline MIS discriminator and that the resulting rankings are stable after about 20 training runs.

Load-bearing premise

The estimator assumes that the quantized models seen along one training trajectory include or approximate the two lowest-loss quantized models and the checkpoint that maximizes the scaled loss variance, so that the two terms in Algorithm 1 match the true codebook extremes; the paper offers only the histograms in Figure 10 as evidence, not a guarantee.

Editorial extensions

If this is right

  • Quantizers can be ranked by privacy without running a single membership inference attack, because the asymptotic exponent r_Q is a property of the loss distribution over the quantized codebook.
  • Fewer bits do not automatically mean more privacy: empirically Sign (1 bit) is less private than the sparse 1.58b 33%, and the most private method is the sparsest one (1.58b 90%), so sparsity dominates bit width.
  • Privacy and performance trade off sharply on classification—the most private quantizers drop relative AUROC to roughly 80–90%—while on regression tasks there is essentially no viable trade-off, since 2-bit quantization yields negative R².
  • The r_Q estimator stabilizes after about 20 training runs and costs roughly an hour versus about ten hours for the attack-based baseline, making privacy audits of quantizers practical in moderate-scale settings.
  • Because MIS is defined against the most powerful attack, the ranking is meant to hold against unknown future attacks as well, not just current state-of-the-art MIAs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The trajectory-coverage assumption is the fragile point: if the true two best quantized models never appear near the training trajectory, Algorithm 1's mval[2]−mval[1] and max term will misestimate r_Q; one can test this directly by exhaustively enumerating a small model's codebook and comparing the full r_Q with the trajectory estimate.
  • The theory suggests r_Q is a quantity one could optimize during quantization-aware training: enlarging the gap δ_2 and shrinking the scaled variance σ^2 would push the privacy exponent up, effectively training for privacy as a differentiable objective.
  • Since the result depends only on the loss distribution over a discrete set of models, the same rate formula should apply to other discrete compression schemes, such as pruning masks or hashed weight sharing, whenever the compressed model set is finite.
  • A natural next step is relating r_Q to differential privacy: the exponent n r_Q gives an asymptotic privacy guarantee against the membership-inference game, and comparing it with ε,δ guarantees of the same algorithm would clarify how quantization-based and noise-based privacy protections interact.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies membership inference security (MIS) of post-training quantization procedures for empirical-risk-minimizing learning algorithms. For a fixed quantizer and for size-adaptive quantizers, the authors prove asymptotic bounds on the leakage rate 1 - MIS_n in terms of the average-loss gaps and loss variabilities of the quantized codebook. They then introduce an estimator r_Q computed from the losses of quantized checkpoints along a single training trajectory, and use this estimator to rank quantization methods by privacy. The ranking is validated against a discriminative baseline MIS estimator on synthetic Gaussian data and on molecular property-prediction tasks, and a privacy-performance trade-off is reported for real-world drug-discovery models.

Significance. If the central ranking claim were fully supported, the paper would be a valuable practical contribution: it would provide a computationally cheap privacy indicator for quantization procedures, grounded in a theoretical analysis of the strongest possible membership inference attacks, with experiments on a realistic application domain. The paper has real strengths: the proofs in Appendix C are coherent, the estimator is inexpensive compared with training attack classifiers, the experiments span synthetic and real-world settings, and the authors provide code. However, the main theoretical result as stated proves only a one-sided bound on the leakage rate, while the text converts that bound into an exact asymptotic order and a quantizer-ranking rule. The ranking rule is the paper's central practical claim, so the gap between theorem and interpretation is load-bearing. The empirical Spearman correlations provide evidence for r_Q as a heuristic, but they do not repair the missing theoretical implication.

major comments (3)
  1. [Section 3.2, Theorem 3.3 and the paragraph following Eq. (4)] Eq. (4) is a one-sided bound: it is equivalent to liminf_n -log(1 - MIS_n)/(n(δ_2^n)^2) ≥ 1/(2σ^2), which upper-bounds 1 - MIS_n by e^{-n r_Q(1+o(1))}. The proof of Proposition C.3 uses Lemma B.1 only in the direction ∆_n ≤ ... ≤ 2(1 - p_1^n), with no reverse inequality. Consequently the statements that the MIS is 'of order 1 - e^{-n r_Q(1+o(1))}' and that 'r_Q ≥ r_R implies A_Q produces more secure parameters than A_R' do not follow: the true exponent could be strictly larger than r_Q, and the ordering of two true exponents can be reversed relative to the ordering of the lower bounds. The same one-sided issue appears in Section 3.1, where the text concludes an 'order 1 - e^{-n C_P(1+o(1))}' behavior from the upper bound in Eq. (3). I ask the authors either to prove the matching lower bound on 1 - MIS_n (equality of the moderate-deviation rate, together with a comparison of ∆_n with 1 - p_1^n) or to explicitly reframe r_Q as a heuristic indicator that is validated empirically rather than as a quantity whose ordering is theoretically guaranteed to match the MIS ordering.
  2. [Section 4.1, Algorithm 1] The estimator replaces the supremum over the entire codebook Θ̄ and the global minima m_θ̄_1, m_θ̄_2 by quantities computed on the K quantized checkpoints visited along one training trajectory. Since the theorem's σ^2 is a maximum over all k ≤ K_n while Algorithm 1 takes a maximum over k ≤ K, the estimator can only underestimate the true denominator and therefore overestimate r_Q; if the true near-minimal-loss quantizers are never visited, the bias can change the ranking. The only support currently offered for this trajectory-coverage assumption is the empirical histogram in Figure 10, which shows that the argmax of Λ_{k,k} occurs at small k on four datasets. Please state the coverage hypothesis explicitly as an assumption of the estimator, and add a small-scale experiment with exhaustive or randomized codebook enumeration to quantify the resulting bias and to check whether the ranking is stable under this bias.
  3. [Section 5.1.2, Figure 2] The empirical validation compares r_Q with the 'baseline estimation of the MIS', which is itself an approximate discriminator trained on a specific feature set; this is an internal baseline, not a gold standard for the true MIS. The average Spearman correlation of about 0.86 therefore demonstrates consistency with one baseline rather than that r_Q ranks quantizers by the true MIS. If the theoretical ranking claim is replaced by a heuristic claim, this validation is a reasonable starting point, but the paper should then report sensitivity of the rankings to discriminator architecture and hyperparameters, and should soften claims such as 'demonstrating that r_Q reliably ranks quantization methods by their privacy levels' in the last sentence of Section 5.1.2.
minor comments (5)
  1. [Section 3.1, Eq. (3)] The displayed theorem uses 'lim' with a one-sided inequality, but the proof of Theorem C.1 establishes at most a limsup statement; please use limsup or liminf notation consistently in theorems and in the text.
  2. [Section 4.1, Algorithm 1] The symbol r_Q is used for both the intermediate quantity 1/2 (max ...)^{-1} in line 17 and the returned value r_Q (mval[2] - mval[1])^2 in line 18; renaming one of the two would prevent confusion.
  3. [Appendix E.1 and Appendix A.2] There are typos ('distribtuions', 'traing') and a missing mathematical symbol in the statement of H1.1, where the loss ℓ should appear in the expectation; a careful proofreading pass is needed.
  4. [Figure 1 and Figure 2] The captions are not fully self-explanatory: the meaning of 'sp=0.83', 'kmodes = 6', and the correlations computed over which subsets of runs should be stated clearly in the captions or in the text.
  5. [Section 6, Limitations] The limitation paragraph is honest and useful, but it should also mention that Algorithm 1's trajectory-coverage assumption, and not just the fixed-model versus distribution-level scope, is a restriction on the practical interpretation of r_Q.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular reduction found; the main gap is a one-sided bound over-read as an exact rate, which is a soundness issue rather than circularity.

full rationale

The derivation chain is not circular. The MIS-as-total-variation identity (Theorem A.2) is imported from the authors' prior work, but it is a general, parameter-free identity for any algorithm and dataset distribution; it does not assume the quantization rate or the target ranking, so invoking it is normal mathematical dependency rather than a self-referential load-bearing citation. The proofs of Theorems 3.1 and 3.3 (Appendix C) are self-contained reductions of 1-MIS to probabilities that the empirical-average loss of a non-optimal quantized codebook exceeds the best codebook, followed by large/moderate-deviation estimates; no fitted constant is renamed as a prediction. Algorithm 1 directly plugs validation-loss gaps and variances into the theoretical expression for r_Q; its accuracy rests on the heuristic trajectory-coverage assumption (Figure 10), which is a robustness concern but not a circular definition. The validation against the discriminator-based MIS baseline is an internal consistency check between two estimators of the same quantity, not a reduction of r_Q to the baseline. The serious weakness is logical, not circular: Theorem 3.3 proves only liminf -log(1-MIS_n)/(n(δ_2^n)^2) ≥ 1/(2σ²), while the text in Section 3.2 converts this one-sided bound into an exact order 1-e^{-n r_Q(1+o(1))} and a ranking rule; that inference would require a matching moderate-deviation lower bound, which is not supplied. This over-reading is a soundness/correctness issue and should be scored there, not as circularity.

Assumptions & free parameters 0 free parameters · 6 assumptions · 1 invented entities

The ledger is dominated by technical moment and limit assumptions inherited from the MIS framework, plus one ad hoc trajectory-coverage assumption that carries the practical estimator. No free parameters are fitted to force the central result; the unproven parts are the one-sided bound (soundness) and the coverage assumption (reproducibility of rankings).

assumptions (6)
  • standard math MIS_n(P,A) = 1 - ||P(θ,z1) - P_θ ⊗ P||_TV (Theorem A.2, Aubinais et al. 2023)
    Used as the starting identity for both theorems, accepted from prior work without proof in this paper (Appendix A.1).
  • domain assumption Training data are i.i.d. from P and the algorithm's output depends on data only through the empirical distribution (Section 2.1)
    Needed for p_n^1 to be expressed as a probability of a sum condition and for the large and moderate deviation arguments.
  • domain assumption H1.1-H1.2: exponential moments of the renormalized loss gaps and strictly positive minimal loss gap δ2 (Appendix A.2)
    Required for Theorem C.1; if the moment generating function is infinite in every direction the rate constant degenerates.
  • domain assumption H2.1-H2.4: limits of variances and ratios exist, uniform exponential moments, finitely many asymptotically near-optimal quantizers, and tightness in l2 (Appendix A.3)
    Needed for the Hilbert-space central limit and moderate deviation step in Proposition C.3; H2.3 is strong and may fail for overparameterized models.
  • ad hoc to paper The training trajectory of A visits or approximates the lowest-loss quantized models, so the empirical max over k≤K in Algorithm 1 estimates the true max over the codebook (Section 4.1, Figure 10)
    This is the key feasibility assumption for the estimator; only empirical evidence is provided, no worst-case guarantee.
  • domain assumption The discriminator g_ψ trained in Section 4.2 approximates the Neyman-Pearson likelihood-ratio test and hence the supremum over MIAs
    Used to justify the baseline MIS estimate; in practice the neural net is not permutation invariant to weight order, which the authors acknowledge.
invented entities (1)
  • MIS rate indicator r_Q
    purpose: Rank quantizers by asymptotic membership-inference security without running expensive attacks
    r_Q is estimated from validation losses along one training trajectory; the paper validates it only against its own discriminator-based MIS estimate, with no out-of-paper falsifiable prediction, and the ranking implication is not entailed by the one-sided theorem.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Membership Inference Risks in Quantized Models: A Theoretical and Empirical Study." pith.science (2026). https://pith.science/paper/TWYNWEAK

@misc{pith2026250206567,
  author       = {Pith},
  title        = {Pith review of: Membership Inference Risks in Quantized Models: A Theoretical and Empirical Study},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TWYNWEAK}},
  note         = {Machine review of arXiv:2502.06567}
}
read the original abstract

Quantizing machine learning models has demonstrated its effectiveness in lowering memory and inference costs while maintaining performance levels comparable to those of the original models. In this work, we investigate the impact of quantization procedures on privacy in data-driven models, focusing on their vulnerability to membership inference attacks. Membership Inference Security (MIS) has recently been proposed to characterize the privacy of machine learning models against the most powerful (and possibly unknown) attacks. However, quantifying MIS appears to be computationally very difficult. In this paper, we propose a new MIS indicator for post-training quantization procedures of machine learning models that minimizes an empirical loss. This new indicator is a byproduct of a theoretical asymptotic analysis of the MIS in this context. We also present a methodology for empirically estimating our MIS indicator. Using synthetic datasets and real-world data (in the context of drug discovery), we demonstrate the effectiveness of our approach in assessing and ranking the MIS of different quantizers.

Figures

Figures reproduced from arXiv: 2502.06567 by the authors.

Figure 1
Figure 1. Stability of Privacy rankings. To obtain reliable estimates of r n Q, we average its value over multiple runs krun (number of classifiers trained). The central plot illustrates how the rankings of quantizers, based on r n Q, evolve with the number of runs. Each column of pie charts represents the proportion of quantizers predicted at each rank (across 100 different subsets of krun runs) with connecting lines showing… view at source ↗
Figure 2
Figure 2. Relationship between r n Q and the MIS. Each sub-plot displays the estimated values of r n Q and the MIS for each quantizer under varying experimental configurations, with their corresponding Spearman correlation (ρsp). The strong correlations confirm that our method enables the comparison of different quantization techniques’ security. estimation of the MIS. To perform this comparison, we rely on synthetic experime… view at source ↗
Figure 3
Figure 3. Impact of quantization on classification tasks. Evolution of the privacy of each downstream model r n Q along with relative performances of the quantized models compared to the original on classification task. 1.58b 50% 1.58b 33% 1.58b 90% Sign 2 bits 3 bits 4 bits 5 bits r n 0 1 r val R2 Lipophilicity 1.58b 90% 1.58b 50% 1.58b 33% Sign 2 bits 4 bits 3 bits 5 bits FreeSolv Caco2 Solubility PPBR VDss LD50 [PITH_FULL… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Impact of quantization on regression tasks. Evolution of the privacy of each downstream model r n Q along with relative performances of the quantized models compared to the original on regression task. 5.2.2. PRIVACY-PERFORMANCE TRADE-OFF Classification tasks [PITH_FU…
Figure 5
Figure 5. Figure 5: Illustration of the quantization functions used on the interval [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]
Figure 6
Figure 6. Figure 6: Evolution of r n Q with the quantized model’s performance as the ratio of the original model’s training accuracy (line) for various quantizers in our synthetic setup. In this section, we provide additional details on the synthetic experiments conducted to evaluate the …
Figure 7
Figure 7. Figure 7: Correlation between the rankings obtained with the baseline MIS method (resp. [PITH_FULL_IMAGE:figures/full_fig_p021_7.png]
Figure 8
Figure 8. Figure 8: Visualization of the synthetic datasets used in the experiments, through a PCA projection in dimension 2 (the [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]
Figure 9
Figure 9. Figure 9: Evolution of the privacy of each downstream model [PITH_FULL_IMAGE:figures/full_fig_p024_9.png]
Figure 10
Figure 10. Figure 10: Evolution of Λk,k with the k, where the indexes are sorted with decreasing values of average loss, and the histogram of the index of the maximum value of Λk,k for each quantizer. 25 [PITH_FULL_IMAGE:figures/full_fig_p025_10.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Bits and Memories: Measuring Verbatim Extraction Across LLM Quantization

    cs.LG 2026-07 conditional novelty 7.0 of 10

    Quantizing LLMs selectively forgets memorized text faster than capability, but 1B-scale 4-bit models still extract ~72% of memorized sequences, so quantization is not a privacy defense.

Reference graph

Works this paper leans on

57 extracted references · 28 canonical work pages · cited by 1 Pith paper

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Chemberta-2: Towards chemical foundation models, 2022

    Ahmad, W., Simon, E., Chithrananda, S., Grand, G., and Ramsundar, B. Chemberta-2: Towards chemical foundation models, 2022

    work page 2022

  3. [3]

    and Gin \'e , E

    Araujo, A. and Gin \'e , E. The central limit theorem for real and banach valued random variables. (No Title), 1980

    work page 1980

  4. [4]

    Fundamental limits of membership inference attacks on machine learning models

    Aubinais, E., Gassiat, E., and Piantanida, P. Fundamental limits of membership inference attacks on machine learning models. arXiv preprint arXiv:2310.13786, 2023

    work page arXiv 2023

  5. [5]

    Hardware-aware dnn compression via diverse pruning and mixed-precision quantization

    Balaskas, K., Karatzas, A., Sad, C., Siozios, K., Anagnostopoulos, I., Zervakis, G., and Henkel, J. Hardware-aware dnn compression via diverse pruning and mixed-precision quantization. IEEE Transactions on Emerging Topics in Computing, 12 0 (4): 0 1079--1092, 2024

    work page 2024

  6. [6]

    Scalable methods for 8-bit training of neural networks

    Banner, R., Hubara, I., Hoffer, E., and Soudry, D. Scalable methods for 8-bit training of neural networks. Advances in neural information processing systems, 31, 2018

    work page 2018

  7. [7]

    Estimating or propagating gradients through stochastic neurons for conditional computation

    Bengio, Y., L \'e onard, N., and Courville, A. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432, 2013

    arXiv 2013

  8. [8]

    and Hutter, M

    Carbon-Mangels, M. and Hutter, M. C. Selecting Relevant Descriptors for Classification by Bayesian Estimates : A Comparison with Decision Trees and Support Vector Machines Approaches for Disparate Data Sets . Molecular Informatics, 30 0 (10): 0 885--895, 2011. ISSN 1868-1751. doi:10.1002/minf.201100069. URL https://onlinelibrary.wiley.com/doi/abs/10.1002/...

    doi:10.1002/minf.201100069 2011

Show all 57 references
  1. [9]

    Membership inference attacks from first principles

    Carlini, N., Chien, S., Nasr, M., Song, S., Terzis, A., and Tramer, F. Membership inference attacks from first principles. In 2022 IEEE Symposium on Security and Privacy (SP), pp.\ 1897--1914. IEEE, 2022

    2022

  2. [10]

    Extracting training data from diffusion models

    Carlini, N., Hayes, J., Nasr, M., Jagielski, M., Sehwag, V., Tramer, F., Balle, B., Ippolito, D., and Wallace, E. Extracting training data from diffusion models. In 32nd USENIX Security Symposium (USENIX Security 23), pp.\ 5253--5270, 2023

    2023

  3. [11]

    Privacy-aware compression for federated data analysis

    Chaudhuri, K., Guo, C., and Rabbat, M. Privacy-aware compression for federated data analysis. In Cussens, J. and Zhang, K. (eds.), Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence, volume 180 of Proceedings of Machine Learning Research, pp....

    2022

  4. [12]

    Moderate deviations and associated laplace approximations for sums of independent random vectors

    De Acosta, A. Moderate deviations and associated laplace approximations for sums of independent random vectors. Transactions of the American Mathematical Society, 329 0 (1): 0 357--375, 1992

    1992

  5. [13]

    Bounding information leakage in machine learning

    Del Grosso, G., Pichler, G., Palamidessi, C., and Piantanida, P. Bounding information leakage in machine learning. Neurocomputing, 534: 0 1--17, 2023

    2023

  6. [14]

    Large deviations techniques and applications

    Dembo, A. Large deviations techniques and applications. Springer, 2009

    2009

  7. [15]

    and The PyTorch Lightning team

    Falcon, W. and The PyTorch Lightning team . PyTorch Lightning , March 2019. URL https://github.com/Lightning-AI/lightning

    2019

  8. [16]

    and Lao, Y

    Famili, A. and Lao, Y. Deep neural network quantization framework for effective defense against membership inference attacks. Sensors, 23 0 (18), 2023. ISSN 1424-8220. doi:10.3390/s23187722. URL https://www.mdpi.com/1424-8220/23/18/7722

    2023 doi

  9. [17]

    M., Madhukar, N

    Gayvert, K. M., Madhukar, N. S., and Elemento, O. A Data - Driven Approach to Predicting Successes and Failures of Clinical Trials . Cell Chemical Biology, 23 0 (10): 0 1294--1301, October 2016. ISSN 2451-9448. doi:10.1016/j.chembiol.2016.07.023

    2016 doi

  10. [18]

    and Gray, R

    Gersho, A. and Gray, R. M. Vector Quantization and Signal Compression . Kluwer Academic Publishers, 1992. ISBN 9780792391814. URL http://www.worldcat.org/isbn/9780792391814

    1992

  11. [19]

    W., and Keutzer, K

    Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M. W., and Keutzer, K. A survey of quantization methods for efficient neural network inference. In Low-Power Computer Vision, pp.\ 291--326. Chapman and Hall/CRC, 2022

    2022

  12. [20]

    A survey of low-bit large language models: Basics, systems, and algorithms

    Gong, R., Ding, Y., Wang, Z., Lv, C., Zheng, X., Du, J., Qin, H., Guo, J., Magno, M., and Liu, X. A survey of low-bit large language models: Basics, systems, and algorithms. arXiv preprint arXiv:2409.16694, 2024

    2024

  13. [21]

    K., Thompson, P., Ambite, J

    Gupta, U., Stripelis, D., Lam, P. K., Thompson, P., Ambite, J. L., and Ver Steeg, G. Membership inference attacks on deep regression models for neuroimaging. In Medical Imaging with Deep Learning, pp.\ 228--251. PMLR, 2021

    2021

  14. [22]

    and Tsaftaris, S

    Hartley, J. and Tsaftaris, S. A. Measuring unintended memorisation of unique private features in neural networks. arXiv preprint arXiv:2202.08099, 2022

    2022 arXiv

  15. [23]

    Hayes, J., Melis, L., Danezis, G., and De Cristofaro, E. L. Membership inference attacks against generative models; 2018. URL https://api. semanticscholar. org/CorpusID, 202588705

    2018

  16. [24]

    A., Welbl, J., Clark, A., Hennigan, T., Noland, E., Millican, K., van den Driessche, G., Damoc, B., Guy, A., Osindero, S., Simonyan, K., Elsen, E., Rae, J

    Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., de Las Casas, D., Hendricks, L. A., Welbl, J., Clark, A., Hennigan, T., Noland, E., Millican, K., van den Driessche, G., Damoc, B., Guy, A., Osindero, S., Simonyan, K., Elsen, E., Rae, J. W., Vin...

    2022 arXiv

  17. [25]

    S., and Zhang, X

    Hu, H., Salcic, Z., Sun, L., Dobbie, G., Yu, P. S., and Zhang, X. Membership inference attacks on machine learning: A survey. ACM Computing Surveys (CSUR), 54 0 (11s): 0 1--37, 2022

    2022

  18. [26]

    W., Xiao, C., Sun, J., and Zitnik, M

    Huang, K., Fu, T., Gao, W., Zhao, Y., Roohani, Y., Leskovec, J., Coley, C. W., Xiao, C., Sun, J., and Zitnik, M. Therapeutics data commons: Machine learning datasets and tasks for drug discovery and development. Proceedings of Neural Information Processing Systems, NeurIPS Dat...

    2021

  19. [27]

    Quantization and training of neural networks for efficient integer-arithmetic-only inference

    Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., and Kalenichenko, D. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 2...

    2018

  20. [28]

    B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D

    Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. Scaling laws for neural language models, 2020. URL https://arxiv.org/abs/2001.08361

    2020 arXiv

  21. [29]

    Towards Model Quantization on the Resilience Against Membership Inference Attacks

    Kowalski, C., Famili, A., and Lao, Y. Towards Model Quantization on the Resilience Against Membership Inference Attacks . In 2022 IEEE International Conference on Image Processing ( ICIP ) , pp.\ 3646--3650, October 2022. doi:10.1109/ICIP46576.2022.9897681. URL https://ieeexpl...

    2022

  22. [30]

    Computer- Aided Prediction of Rodent Carcinogenicity by PASS and CISOC - PSCT

    Lagunin, A., Filimonov, D., Zakharov, A., Xie, W., Huang, Y., Zhu, F., Shen, T., Yao, J., and Poroikov, V. Computer- Aided Prediction of Rodent Carcinogenicity by PASS and CISOC - PSCT . QSAR & Combinatorial Science, 28 0 (8): 0 806--810, 2009. ISSN 1611-0218. doi:10.1002/qsar...

    2009 doi

  23. [31]

    Awq: Activation-aware weight quantization for on-device llm compression and acceleration

    Lin, J., Tang, J., Tang, H., Yang, S., Chen, W.-M., Wang, W.-C., Xiao, G., Dang, X., Gan, C., and Han, S. Awq: Activation-aware weight quantization for on-device llm compression and acceleration. Proceedings of Machine Learning and Systems, 6: 0 87--100, 2024

    2024

  24. [32]

    Pre-training molecular graph representation with 3d geometry

    Liu, S., Wang, H., Liu, W., Lasenby, J., Guo, H., and Tang, J. Pre-training molecular graph representation with 3d geometry. In International Conference on Learning Representations, 2022 a . URL https://openreview.net/forum?id=xQUe1pOKPam

    2022

  25. [33]

    \ ML-Doctor \ : Holistic risk assessment of inference attacks against machine learning models

    Liu, Y., Wen, R., He, X., Salem, A., Zhang, Z., Backes, M., De Cristofaro, E., Fritz, M., and Zhang, Y. \ ML-Doctor \ : Holistic risk assessment of inference attacks against machine learning models. In 31st USENIX Security Symposium (USENIX Security 22), pp.\ 4525--4542, 2022 b

    2022

  26. [34]

    Fbi-llm: Scaling up fully binarized llms from scratch via autoregressive distillation

    Ma, L., Sun, M., and Shen, Z. Fbi-llm: Scaling up fully binarized llms from scratch via autoregressive distillation. arXiv preprint arXiv:2407.07093, 2024 a

    2024 arXiv

  27. [35]

    The era of 1-bit llms: All large language models are in 1.58 bits, 2024 b

    Ma, S., Wang, H., Ma, L., Wang, L., Wang, W., Huang, S., Dong, L., Wang, R., Xue, J., and Wei, F. The era of 1-bit llms: All large language models are in 1.58 bits, 2024 b . URL https://arxiv.org/abs/2402.17764

    2024 arXiv

  28. [36]

    Murakonda, S. K. and Shokri, R. Ml privacy meter: Aiding regulatory compliance by quantifying the privacy risks of machine learning. arXiv preprint arXiv:2007.09339, 2020

    2007 arXiv

  29. [37]

    v., Blankevoort, T., and Welling, M

    Nagel, M., Baalen, M. v., Blankevoort, T., and Welling, M. Data-free quantization through weight equalization and bias correction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 1325--1334, 2019

    2019

  30. [38]

    A., Bondarenko, Y., Van Baalen, M., and Blankevoort, T

    Nagel, M., Fournarakis, M., Amjad, R. A., Bondarenko, Y., Van Baalen, M., and Blankevoort, T. A white paper on neural network quantization. arXiv preprint arXiv:2106.08295, 2021

    2021 arXiv

  31. [39]

    Overcoming oscillations in quantization-aware training

    Nagel, M., Fournarakis, M., Bondarenko, Y., and Blankevoort, T. Overcoming oscillations in quantization-aware training. In International Conference on Machine Learning, pp.\ 16318--16330. PMLR, 2022

    2022

  32. [40]

    Push quantization-aware training toward full precision performances via consistency regularization

    Pang, J., Cai, T., Zhang, B., Wu, J., and Tao, Y. Push quantization-aware training toward full precision performances via consistency regularization. arXiv preprint arXiv:2402.13497, 2024

    2024 arXiv

  33. [41]

    White-box vs black-box: Bayes optimal strategies for membership inference

    Sablayrolles, A., Douze, M., Schmid, C., Ollivier, Y., and J \'e gou, H. White-box vs black-box: Bayes optimal strategies for membership inference. In International Conference on Machine Learning, pp.\ 5558--5567. PMLR, 2019

    2019

  34. [42]

    Membership inference attacks against machine learning models

    Shokri, R., Stronati, M., Song, C., and Shmatikov, V. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pp.\ 3--18. IEEE, 2017

    2017

  35. [43]

    Validating adme qsar models using marketed drugs

    Siramshetty, V., Williams, J., Nguyen, D.-T., Neyra, J., Southall, N., Mathé, E., Xu, X., and Shah, P. Validating adme qsar models using marketed drugs. SLAS DISCOVERY: Advancing the Science of Drug Discovery, 26 0 (10): 0 1326--1336, 2021. doi:10.1177/24725552211017520. URL h...

    2021 doi

  36. [44]

    and Raghunathan, A

    Song, C. and Raghunathan, A. Information leakage in embedding models. In Proceedings of the 2020 ACM SIGSAC conference on computer and communications security, pp.\ 377--390, 2020

    2020

  37. [45]

    Machine learning models that remember too much

    Song, C., Ristenpart, T., and Shmatikov, V. Machine learning models that remember too much. In Proceedings of the 2017 ACM SIGSAC Conference on computer and communications security, pp.\ 587--601, 2017

    2017

  38. [46]

    Beyond memorization: Violating privacy via inference with large language models

    Staab, R., Vero, M., Balunovi \'c , M., and Vechev, M. Beyond memorization: Violating privacy via inference with large language models. arXiv preprint arXiv:2310.07298, 2023

    2023 arXiv

  39. [47]

    3d infomax improves gnns for molecular property prediction

    Stärk, H., Beaini, D., Corso, G., Tossou, P., Dallago, C., Günnemann, S., and Liò, P. 3d infomax improves gnns for molecular property prediction. arXiv preprint arXiv:2110.04126, 2021

    2021 arXiv

  40. [48]

    and Piantanida, P

    Villard, J. and Piantanida, P. Secure multiterminal source coding with side information at the eavesdropper. IEEE Transactions on Information Theory, 59 0 (6): 0 3668--3692, 2013. doi:10.1109/TIT.2013.2245394

    2013

  41. [49]

    Wang, H., Li, W., Jin, X., Cho, K., Ji, H., Han, J., and Burke, M. D. Chemical-reaction-aware molecule representation learning. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=6sh3pIzKS-

    2022

  42. [50]

    Bitnet: Scaling 1-bit transformers for large language models

    Wang, H., Ma, S., Dong, L., Huang, S., Wang, H., Ma, L., Yang, F., Wang, R., Wu, Y., and Wei, F. Bitnet: Scaling 1-bit transformers for large language models. arXiv preprint arXiv:2310.11453, 2023

    2023 arXiv

  43. [51]

    A survey on responsible llms: Inherent risk, malicious use, and mitigation strategy

    Wang, H., Fu, W., Tang, Y., Chen, Z., Huang, Y., Piao, J., Gao, C., Xu, F., Jiang, T., and Li, Y. A survey on responsible llms: Inherent risk, malicious use, and mitigation strategy. arXiv preprint arXiv:2501.09431, 2025

    2025 arXiv

  44. [52]

    Ladder: Enabling efficient \ Low-Precision \ deep learning computing through hardware-aware tensor transformation

    Wang, L., Ma, L., Cao, S., Zhang, Q., Xue, J., Shi, Y., Zheng, N., Miao, Z., Yang, F., Cao, T., et al. Ladder: Enabling efficient \ Low-Precision \ deep learning computing through hardware-aware tensor transformation. In 18th USENIX Symposium on Operating Systems Design and Im...

    2024

  45. [53]

    Killing two birds with one stone: Quantization achieves privacy in distributed learning

    Yan, G., Li, T., Wu, K., and Song, L. Killing two birds with one stone: Quantization achieves privacy in distributed learning. Digital Signal Processing, 146: 0 104353, 2024. ISSN 1051-2004. doi:https://doi.org/10.1016/j.dsp.2023.104353. URL https://www.sciencedirect.com/scien...

    2024

  46. [54]

    Randomized quantization is all you need for differential privacy in federated learning, 2023

    Youn, Y., Hu, Z., Ziani, J., and Abernethy, J. Randomized quantization is all you need for differential privacy in federated learning, 2023. URL https://arxiv.org/abs/2306.11913

    2023 arXiv

  47. [55]

    Vit-1.58 b: Mobile vision transformers in the 1-bit era

    Yuan, Z., Zhou, R., Wang, H., He, L., Ye, Y., and Sun, L. Vit-1.58 b: Mobile vision transformers in the 1-bit era. arXiv preprint arXiv:2406.18051, 2024

    2024 arXiv

  48. [56]

    Abq-llm: Arbitrary-bit quantized inference acceleration for large language models

    Zeng, C., Liu, S., Xie, Y., Liu, H., Wang, X., Wei, M., Yang, S., Chen, F., and Mei, X. Abq-llm: Arbitrary-bit quantized inference acceleration for large language models. arXiv preprint arXiv:2408.08554, 2024

    2024 arXiv

  49. [57]

    A survey on model compression for large language models

    Zhu, X., Li, J., Liu, Y., Ma, C., and Wang, W. A survey on model compression for large language models. Transactions of the Association for Computational Linguistics, 12: 0 1556--1577, 2024

    2024

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.