Pith. sign in

REVIEW 3 major objections 7 minor 1 cited by

NestQuant: Nested Lattice Quantization for Matrix Products and LLMs

T0 review · 3 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read NestQuant quantizes weights, activations, and KV cache together on a nested Gosset lattice, halving the perplexity gap to the full-precision Llama-3-8B at 4 bits.

desk verdict Strong empirical results for runtime lattice quantization, with a real but fixable gap in the QA-LDLQ justification. read the letter →

arxiv 2502.09720 v3 pith:DVA3XILX submitted 2025-02-13 cs.LG cs.AIcs.ITmath.IT

Semyon Savkin , Eitan Porat , Or Ordentlich , Yury Polyanskiy This is my paper · ORCID
classification cs.LGcs.AIcs.ITmath.IT
keywords post-trainingquantizationnestedlatticeGossetVoronoicodesmatrixmultiplicationLLMinferenceKVcacheactivation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes NestQuant, a post-training quantization scheme that replaces per-number rounding with vector quantization onto a nested Gosset lattice. The central claim is that one quantizer, applied uniformly to weights, activations, and the KV cache, can serve as a drop-in replacement for any matrix multiplication in an LLM. At 4 bits on Llama-3-8B it reports 6.6 wikitext2 perplexity, against 7.3 for SpinQuant and OstQuant and 8.2 for QuaRot, with the unquantized model at 6.14; the authors describe this as more than a 55% reduction in the perplexity gap. A reader should care because the method is a general matrix-product primitive, motivated by an information-theoretic lower bound, and the paper reports consistent gains across Llama models from 1B to 70B, with full 4-bit quantization often matching or beating prior methods that leave the KV cache at 16 bits.

What carries the argument

The load-bearing object is the Voronoi code of the Gosset lattice E8: the codebook $C = \Lambda \cap qV_{\Lambda}$ for an integer nesting ratio $q = 2^R$, where encoding sends the coordinates of the nearest lattice point modulo $q$ and decoding returns the minimum-energy point of that coset. This shaping wastes far fewer bit patterns on rare vectors than the cubic cells of uniform quantization, while still admitting a fast nearest-neighbor oracle. Around it sit three supporting mechanisms: a small union of scaled codebooks $\beta_t C$ chosen by dynamic programming to control overload error; a random Hadamard rotation that makes high-dimensional blocks approximately Gaussian; and QA-LDLQ, which replaces the weight matrix by $\tilde{W} = W H(H + \epsilon^2 I)^{-1}$ before quantization to compensate for activation noise modeled as independent zero-mean Gaussian noise with covariance $\epsilon^2 I$.

What would settle it

Measure the empirical conditional mean and covariance of the activation quantization error $Z = \hat{X} - X$ on the first value-projection layer of Llama-3-70B using a handful of wikitext2 sequences; if $E[Z\mid X]$ is not near zero or the covariance is far from $\epsilon^2 I$, the QA-LDLQ update is not the true minimizer and the reported 70B gains should be rechecked with the measured covariance.

Watch

Extended reading notes

Core claim

NestQuant's core claim is that nested-lattice vector quantization, not scalar rounding, is the right primitive for low-precision matrix multiplication in LLMs. The paper builds an 8-dimensional Voronoi code from the Gosset lattice E8, encodes each length-8 block by its nearest lattice point modulo an integer nesting ratio q, and decodes to the minimum-energy representative of the coset. A Hadamard rotation is applied first so weight and activation blocks behave like near-Gaussian vectors, and a union of a few scaled copies of the codebook absorbs rare large vectors. With this codebook, NestQuant quantizes weights, keys, values, and activations all to 4 bits and reports 6.6 wikitext2 perplexity on Llama-3-8B (full precision 6.14), and 3.61 on Llama-3-70B at W4A4KV4, versus 4.01 for OstQuant and 6.66 for QuaRot. It also reports synthetic Gaussian matrix-product error close to the information-theoretic lower bound and well below uniform quantization.

Load-bearing premise

The load-bearing premise is that activation quantization error is zero-mean, independent, isotropic Gaussian noise; if that model is wrong, the QA-LDLQ correction that makes the largest-model results work may be suboptimal.

Editorial extensions

If this is right

  • At 4 bits, quantizing weights, activations, and KV cache together with NestQuant closes more than half of the wikitext2 perplexity gap to the unquantized Llama-3-8B compared with SpinQuant, OstQuant, and QuaRot.
  • Across Llama-2 and Llama-3 models from 1B to 70B, 4-bit NestQuant in full quantization is reported to match or beat prior methods that quantize weights and activations but keep the KV cache at 16 bits, except on Llama-2-7B.
  • Weight-only 4-bit NestQuant also improves perplexity over GPTQ, LLM-QAT, and SpinQuant on Llama-3-8B, reaching 6.3 versus 7.2, 7.7, and 6.5.
  • Because the quantizer is a generic matrix-multiplication replacement, its benefit is not tied to a specific LLM component; any attention or MLP product can use it.
  • On synthetic Gaussian matrices, NestQuant's product error tracks the information-theoretic lower bound for rate-R matrix multiplication more closely than uniform quantization, suggesting the gains are structural rather than model-specific.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because NestQuant deliberately skips learned rotations and fine-tuning, composing its lattice codebook with optimized rotations is a natural next step that the paper's numbers suggest would push gains further.
  • QA-LDLQ's Gaussian noise model is testable: plugging the empirically measured covariance of activation quantization error into $\tilde{W} = W H(H+J)^{-1}$, per layer, would show whether the 70B results depend on the isotropic-noise assumption.
  • The multi-scale beta union is a portable idea; any lattice or trellis codebook with a fast decoder could use the same dynamic-programming selection of scaling coefficients.
  • Using zstd-compressed scaling coefficients means the reported bitrate is content-dependent; a common benchmark that fixes both rate and hardware kernel would make comparisons with scalar quantizers cleaner.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper introduces NestQuant, a post-training quantization method that replaces scalar uniform quantizers with a nested-lattice (Gosset E8) vector quantizer based on Voronoi codes, applied to weights, activations, and KV cache of LLMs. The method combines Hadamard rotations, L2 normalization, multiple scaling coefficients, and a new weight-correction rule (QA-LDLQ) that accounts for activation quantization noise. The authors report perplexity results on Llama-2 and Llama-3 models up to 70B, claiming 4-bit end-to-end quantization with perplexity close to the unquantized model and superior to SpinQuant, QuaRot, OstQuant, and DuQuant.

Significance. If the empirical claims are correct, NestQuant is a significant advance: it provides a practical vector quantizer whose synthetic matrix-multiplication performance is close to the information-theoretic lower bound of Eq. (1), and it achieves the strongest reported W4A4KV4 perplexity on Llama-3-70B. The paper is also valuable for its honest bit accounting (including zstd compression of scaling coefficients), its detailed pseudocode for E8 encoding/decoding and the CUDA kernel description, and its explicit attempt to connect nested-lattice theory to LLM quantization. The main risk to the headline claims is the untested noise model underlying QA-LDLQ, which is load-bearing for the 70B results.

major comments (3)
  1. [4.5, Lemma 4.2, Eq. (7), Appendix B] QA-LDLQ is load-bearing for the Llama-3-70B results: Section 4.5 states that standard LDLQ gives infinite perplexity at W4A4, and Table 2 reports NestQuant's 70B numbers after applying QA-LDLQ. However, the derivation of the corrected weight Wtilde = W H (H+J)^{-1} assumes the activation quantization error Z is zero-mean and independent of X, with covariance J = epsilon^2 I. The paper's own Appendix B says 'We model quantization noise as a random Gaussian vector with mean 0 and covariance matrix J = ε²I.' NestQuant's encoder is a lattice quantizer without subtractive dither, so Z = Q(X) - X is generally correlated with X at finite rate and dimension. Then Eq. (7) omits the cross term (W-U)E[XZ^T]U^T + ..., and Wtilde is no longer the minimizer of E[||delta(U)||^2]. The authors report no measurements of E[XZ^T] or of the resulting suboptimality for the problematic layers (e.g., the value projection of layer 0 of Llama-3-70B). Since the 70B superiority claim depends on this correction, the paper should either justify the independence assumption empirically for rotated activations at 4 bits or modify QA-LDLQ to account for nonzero correlation, and quantify the sensitivity of the Table 2 70B perplexities to the choice of noise model.
  2. [5.2, Appendix G/H.1] The 'uniform superiority' claim is partly based on hyperparameters selected on the evaluation set, and the reported score differences are sometimes small. In Section 5.2, q=14 and k=4 are chosen after inspecting validation perplexity (Table 3 and Figure 8), and Appendix G uses the same wikitext2 validation split for the reported perplexities; this creates a risk of selection bias for the headline numbers. Additionally, all results appear to be single runs, so margins like 5.53 vs 5.56 (Llama-2-7B, W4A4KV16, Table 2) or 6.30 vs 6.31 (weight-only) are within typical run-to-run noise. The authors should provide repeated runs with standard errors (or at least state the number of runs), and ideally a nested validation split for hyperparameter selection, before the abstract-level 'uniform superiority' claim can be taken at face value.
  3. [Appendix B, Figure 6] The procedure for choosing the QA-LDLQ noise parameter epsilon is not specified. Appendix B says epsilon^2 'depends on the quantization rate and the statistics of X' and Figure 6 shows a tradeoff between output fidelity and noise amplification, but no exact selection rule or pseudocode is given. Because the Llama-3-70B W4A4 result in Table 2 is obtained with this correction, the paper should state how epsilon is computed for each layer (e.g., from the calibration set, with a formula or grid-search criterion), so that the result is reproducible.
minor comments (7)
  1. [Section 4.5 footnote] The footnote says 'produces ∞ perplexity' while Appendix B says 'ppl ∼ 10^4'; please use one consistent value.
  2. [Appendix D.1 proof] In case 2 of the proof, 'for u, v∈ D8' should read 'for u, w∈ D8'; the current notation defines v twice and obscures the argument.
  3. [Appendices F and G] There are repeated typos such as 'overload'/'overflow' and 'perplxity'; please proofread the appendices.
  4. [Tables 1 and 2] Table 1 reports 6.6 for Llama-3-8B W+KV+A while Table 2 reports 6.63 for the same configuration; please clarify the rounding convention.
  5. [Abstract] The phrase 'drop-in replacement for any matrix multiplication' overstates the practical requirements: the method needs calibration data to estimate H, a DP step to choose betas, and Hadamard rotations; please qualify the claim.
  6. [Appendix F and Algorithm 3] The dynamic programming optimizes the FIRST-β strategy while Algorithm 3 encodes with OPT-β (trying all betas and picking the best); the paper should explain why this mismatch does not affect the reported bit rates or perplexities.
  7. [Appendix G] The calibration set for running DP on activations is described as '6 full-length sequences'; please report the corresponding number of tokens, as this is small relative to typical calibration sets (e.g., 128 segments).

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: empirical claims are benchmarked against external baselines; the only self-citation, the information-theoretic lower bound of Eq. (1), is motivational and not load-bearing.

full rationale

The paper's central results are perplexity and benchmark numbers for quantized Llama models compared against external methods (SpinQuant, QuaRot, OstQuant, QuIP#, DuQuant) and the unquantized model. None of these numbers is produced by the theory; they come from running the proposed quantizer. The QA-LDLQ correction (Section 4.5, Lemma 4.2) is derived algebraically under an explicitly stated stochastic model of the activation quantization error Z (zero mean, independent of X, covariance J = eps^2 I). That assumption may be violated in practice, which is a correctness risk, but it is not circular: the derivation does not assume the target perplexity. The beta scaling coefficients are fitted to calibration data by dynamic programming (Appendix F), a standard calibration fit, not a prediction. The only self-citation is Eq. (1), the matrix-multiplication quantization lower bound from the authors' prior work [8]; it motivates the use of nested lattices and serves as a benchmark in Figure 3, but the paper explicitly resorts to a low-dimensional Gosset lattice because the asymptotically optimal lattices in [8] lack efficient decoding. Thus the reported improvements do not reduce to the cited theorem. No step of the derivation is equivalent to its inputs by construction.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical entities. The free parameters are conventional PTQ hyperparameters, but the beta scheduling and epsilon noise model are tuned to the benchmark. The central geometric assumption is Gaussianity of rotated 8-vectors, inherited from lattice quantization theory.

free parameters (5)
  • nesting ratio q = 14 (4-bit setting); also 7, 8, 10, 12 across experiments
    Integer lattice nesting ratio sets the bit rate; q=14 selected for 4-bit comparisons.
  • number of scaling coefficients k = 4
    Chosen by ablation in H.1: k=3 suboptimal, k=4,5,8 comparable; k=4 is fastest.
  • scaling coefficients beta_1..beta_k = Per tensor type and layer; initial set [3.5, 4.5, 6.0, 14.5, 25.0]/q
    Selected by dynamic programming on calibration data (Appendix F); margins 3.0/q (weights) and 4.0/q (others) added to avoid overload.
  • QA-LDLQ noise parameter epsilon = Not fixed; swept logarithmically from 1e-5 to 1
    Controls bias-variance tradeoff in the weight correction; choice illustrated in Figure 6.
  • Rotation construction for non-power-of-2 dimensions = H1 x H2 (hardcoded Hadamard) chosen after comparisons
    Ablation in Table 7 shows this rotation beats Fourier and orthogonal x Hadamard alternatives.
assumptions (4)
  • domain assumption After random orthogonal rotation, 8-dimensional chunks of weights, activations, and KV cache are approximately iid Gaussian.
    Motivates Gaussian-tailored lattice codebooks; Stam's theorem gives O(d^2/n) total variation, cited in Section 3.
  • ad hoc to paper Activation quantization error Z is zero-mean, independent of X, with covariance J = epsilon^2 I.
    Used in QA-LDLQ derivation (Appendix B and Lemma 4.2); not justified by data and is load-bearing for 70B results.
  • standard math The information-theoretic lower bound for Gaussian matrix multiplication from Ordentlich-Polyanskiy [8] is the correct benchmark.
    Used as the synthetic baseline in Figure 3; it is self-cited but a published theoretical result.
  • domain assumption Beta coefficients tuned on 6 calibration sequences generalize to the evaluation distribution.
    Section G relies on small calibration set plus manual margins; no evidence of robustness across seeds.

how reviews work

0 comments
Cite this review

Pith. "Pith review of NestQuant: Nested Lattice Quantization for Matrix Products and LLMs." pith.science (2026). https://pith.science/paper/DVA3XILX

@misc{pith2026250209720,
  author       = {Pith},
  title        = {Pith review of: NestQuant: Nested Lattice Quantization for Matrix Products and LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DVA3XILX}},
  note         = {Machine review of arXiv:2502.09720}
}
read the original abstract

Post-training quantization (PTQ) has emerged as a critical technique for efficient deployment of large language models (LLMs). This work proposes NestQuant, a novel PTQ scheme for weights and activations that is based on self-similar nested lattices. Recent works have mathematically shown such quantizers to be information-theoretically optimal for low-precision matrix multiplication. We implement a practical low-complexity version of NestQuant based on Gosset lattice, making it a drop-in quantizer for any matrix multiplication step (e.g., in self-attention, MLP etc). For example, NestQuant quantizes weights, KV-cache, and activations of Llama-3-8B to 4 bits, achieving perplexity of 6.6 on wikitext2. This represents more than 55% reduction in perplexity gap with respect to unquantized model (perplexity of 6.14) compared to state-of-the-art Metas SpinQuant (perplexity 7.3), OstQuant (7.3) and QuaRot (8.2). Comparisons on bigger models (up to 70B) and on various LLM evaluation benchmarks confirm uniform superiority of NestQuant.

Figures

Figures reproduced from arXiv: 2502.09720 by the authors.

Figure 1
Figure 1. Perplexity vs number of bits/entry in quantized Llama [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Demonstrating advantage of NestQuant in 2D. Typical weights and activations are vectors inside the black circle. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. RMSE for quantized matrix multiplication for iid [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: The quantization scheme of multi-head attention. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Complement Gaussian measure of a 8-dimensional cube (corresponding to shaping using an ℓ∞ ball), a Voronoi region of the Gosset lattice E8 (corresponding to shaping using Voronoi codes with base lattice E8), and a Euclidean ball (corresponding to shaping with a ball, w…
Figure 6
Figure 6. Figure 6: We run QA-LDLQ for value projection layer of the first transformer block of Llama-3-70B. We try different values [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Granular and overload error for standard Gaussian vectors, [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: The perplexity-bitrate scaling of NestQuant with different values of [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Price of metric universality in vector quantization is at most 0.11 bit

    cs.IT 2026-02 conditional novelty 6.0 of 10

    A universal vector-quantization codebook exists that is within 0.11 bit/coordinate of covariance-adaptive waterfilling simultaneously for all input covariances, for Gaussian weights.

Reference graph

Works this paper leans on

41 extracted references · 39 canonical work pages · cited by 1 Pith paper

  1. [1]

    Conway and N

    J. Conway and N. Sloane. A fast encoding method for lattice codes and quantizers. IEEE Transactions on Information Theory, 29(6):820–824, 1983

    work page 1983

  2. [2]

    Polyanskiy and Y

    Y . Polyanskiy and Y . Wu.Information Theory: From Coding to Learning. Cambridge University Press, 2024

    2024

  3. [3]

    R. Zamir. Lattice Coding for Signals and Networks: A Structured Coding Approach to Quantization, Modulation, and Multiuser Information Theory. Cambridge University Press, 2014

    work page 2014

  4. [4]

    A. J. Stam. Limit theorems for uniform distributions on spheres in high-dimensional euclidean spaces. Journal of Applied Probability, 19(1):221–228, 1982

    work page 1982

  5. [5]

    A. Gersho. Asymptotically optimal block quantization. IEEE Transactions on Information Theory, 25(4):373–380, 1979

    work page 1979

  6. [6]

    A. C. Hung and T. H. Meng. Multidimensional rotations for robust quantization of image data. IEEE Transactions on Image Processing, 7(1):1–12, 1998

    work page 1998

  7. [7]

    Hadad and U

    R. Hadad and U. Erez. Dithered quantization via orthogonal transformations. IEEE Transactions on Signal Processing, 64(22):5887–5900, 2016

    work page 2016

  8. [8]

    Ordentlich and Y

    O. Ordentlich and Y . Polyanskiy. Optimal quantization for matrix multiplication, 2024

    work page 2024

Show all 41 references
  1. [9]

    Zamir and M

    R. Zamir and M. Feder. On lattice quantization noise. IEEE Transactions on Information Theory, 42(4):1152–1159, 1996

    1996

  2. [10]

    M. S. Charikar. Similarity estimation techniques from rounding algorithms. In Proceedings of the 34th Annual ACM Symposium on Theory of Computing (STOC 2002), pages 380–388, 2002

    2002

  3. [11]

    Datar, N

    M. Datar, N. Immorlica, P. Indyk, and V . S. Mirrokni. Locality-sensitive hashing scheme based onp-stable distributions. In Proceedings of the 20th Annual Symposium on Computational Geometry (SCG 2004), pages 253–262, 2004

    2004

  4. [12]

    Andoni and P

    A. Andoni and P. Indyk. Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. Communi- cations of the ACM, 51(1):117–122, 2008

    2008

  5. [13]

    Dettmers, M

    T. Dettmers, M. Lewis, Y . Belkada, and L. Zettlemoyer. LLM.int8(): 8-bit matrix multiplication for transformers at scale. In Advances in Neural Information Processing Systems 35 (NeurIPS 2022), pages 29498–29512, 2022

    2022

  6. [14]

    G. Xiao, J. Lin, M. Seznec, H. Wu, J. Demouth, and S. Han. Smoothquant: Accurate and efficient post-training quantization for large language models. In Proceedings of the 40th International Conference on Machine Learning (ICML 2023) , volume 202 of Proceedings of Machine Learn...

    2023

  7. [15]

    Ashkboos, A

    S. Ashkboos, A. Mohtashami, M. L. Croci, B. Li, P. Cameron, M. Jaggi, D. Alistarh, T. Hoefler, and J. Hensman. Quarot: Outlier-free 4-bit inference in rotated llms. In Advances in Neural Information Processing Systems 38 (NeurIPS 2024), 2024

    2024

  8. [16]

    Z. Liu, C. Zhao, I. Fedorov, B. Soran, D. Choudhary, R. Krishnamoorthi, V . Chandra, Y . Tian, and T. Blankevoort. Spinquant: Llm quantization with learned rotations, 2024

    2024

  9. [17]

    J. Li, F. Li, and S. Todorovic. Efficient riemannian optimization on the stiefel manifold via the cayley transform. In International Conference on Learning Representations (ICLR 2020), 2020

    2020

  10. [18]

    Tseng, J

    A. Tseng, J. Chee, Q. Sun, V . Kuleshov, and C. De Sa. Quip#: Even better llm quantization with hadamard incoherence and lattice codebooks. In Proceedings of the 41st International Conference on Machine Learning (ICML 2024), volume 235 of Proceedings of Machine Learning Resear...

    2024

  11. [19]

    Juang and A

    B.-H. Juang and A. H. Gray. Multiple stage vector quantization for speech coding. In IEEE International Conference on Acoustics, Speech, and Signal Processing, 1982

    1982

  12. [20]

    Tseng, Q

    A. Tseng, Q. Sun, D. Hou, and C. De Sa. Qtip: Quantization with trellises and incoherence processing. In Advances in Neural Information Processing Systems 38 (NeurIPS 2024), 2024

    2024

  13. [21]

    J. Chee, Y . Cai, V . Kuleshov, and C. De Sa. Quip: 2-bit quantization of large language models with guarantees. InAdvances in Neural Information Processing Systems 37 (NeurIPS 2023), 2023

    2023

  14. [22]

    Frantar, S

    E. Frantar, S. P. Singh, and D. Alistarh. Optimal brain compression: A framework for accurate post-training quantization and pruning. In Advances in Neural Information Processing Systems 36 (NeurIPS 2022), 2022

    2022

  15. [23]

    Dinur, G

    I. Dinur, G. Kindler, R. Raz, and S. Safra. Approximating CVP to within almost-polynomial factors is np-hard. Combina- torica, 23(2):205–243, 2003

    2003

  16. [24]

    Agrell and B

    E. Agrell and B. Allen. On the best lattice quantizers. IEEE Transactions on Information Theory, 2023

    2023

  17. [25]

    nvcomp: Gpu-accelerated compression library, 2025

    NVIDIA Corporation. nvcomp: Gpu-accelerated compression library, 2025. Accessed 2025-01-30

    2025

  18. [26]

    Conway and N

    J. Conway and N. Sloane. Fast quantizing and decoding algorithms for lattice quantizers and codes. IEEE Transactions on Information Theory, 28(2):227–232, 1982. 12

    1982

  19. [27]

    H. Lin, H. Xu, Y . Wu, J. Cui, Y . Zhang, L. Mou, L. Song, Z. Sun, and Y . Wei. Duquant: Distributing outliers via dual transformation makes stronger quantized llms. In Advances in Neural Information Processing Systems 38 (NeurIPS 2024), 2024

    2024

  20. [28]

    The llama 3 herd of models, 2024

    Llama Team, AI @ Meta. The llama 3 herd of models, 2024

    2024

  21. [29]

    Clark, I

    P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge, 2018

    2018

  22. [30]

    Zellers, A

    R. Zellers, A. Holtzman, Y . Bisk, A. Farhadi, and Y . Choi. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019), pages 4791–4800, 2019

    2019

  23. [31]

    Y . Bisk, R. Zellers, R. Le Bras, J. Gao, and Y . Choi. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI 2020), pages 7432–7439, 2020

    2020

  24. [32]

    Sakaguchi, R

    K. Sakaguchi, R. Le Bras, C. Bhagavatula, and Y . Choi. Winogrande: An adversarial winograd schema challenge at scale. In Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI 2020), pages 8732–8740, 2020

    2020

  25. [33]

    X. Hu, Y . Cheng, D. Yang, Z. Xu, Z. Yuan, J. Yu, C. Xu, Z. Jiang, and S. Zhou. Ostquant: Refining large language model quantization with orthogonal and scaling transformations for better distribution fitting. In International Conference on Learning Representations (ICLR 2025), 2025

    2025

  26. [34]

    Frantar, S

    E. Frantar, S. Ashkboos, T. Hoefler, and D. Alistarh. Gptq: Accurate post-training quantization for generative pre-trained transformers. In Advances in Neural Information Processing Systems 37 (NeurIPS 2023), 2023

    2023

  27. [35]

    Merity, C

    S. Merity, C. Xiong, J. Bradbury, and R. Socher. Pointer sentinel mixture models. In International Conference on Learning Representations (ICLR 2017), 2017. 13 A Figures Figure 5: Complement Gaussian measure of a 8-dimensional cube (corresponding to shaping using an ℓ∞ ball), ...

    2017

  28. [36]

    The optimal coordinate to flip can be determined through dist, and the new value of flipped coordinate — through g

    In pseudocode, we storeP disti as ∆ We note that we should check the constraint that the sum of coordinates in D8 is even, and if it is not, “flip" one of the rounding directions. The optimal coordinate to flip can be determined through dist, and the new value of flipped coord...

  29. [37]

    Note that c1(x) = g(x) and c2(x) = g x − 1 2 + 1 2

    Then, the algorithm chooses the closest point among these candidates to x. Note that c1(x) = g(x) and c2(x) = g x − 1 2 + 1 2. We will prove that for u ∈ D8, g(x + u) = g(x) + u for any x ∈ R8. Now, let’s show the original lemma. Note that since we are choosing the closest can...

  30. [38]

    Then: c1(x + v) = g(x + v) = g(x) + v = c1(x) + v c2(x + v) = g x + v − 1 2 + 1 2 = g x − 1 2 + 1 2 + v = c2(x) + v

    v ∈ D8. Then: c1(x + v) = g(x + v) = g(x) + v = c1(x) + v c2(x + v) = g x + v − 1 2 + 1 2 = g x − 1 2 + 1 2 + v = c2(x) + v

  31. [39]

    Then, we say that v = u − 1 2 = w + 1 2 for u, v∈ D8. c1(x + v) = g(x + v) = g x + u − 1 2 = g x − 1 2 + u = g x − 1 2 + 1 2 + v = c2(x) + v c2(x + v) = g x − 1 2 + v + 1 2 = g(x + w) + 1 2 = g(x) + w + 1 2 = g(x) + v = c1(x) + v 16 Thus, in both cases the condition on candida...

  32. [40]

    First-β: Use the smallest β, which does not result in an overflow error

  33. [41]

    Opt-β: Try all the values of β, and choose the one that has the smallest reconstruction MSE. K 2 4 6 8 10 OPT-β 0.0878 0.0795 0.0708 0.0669 0.0646 FIRST-β 0.0878 0.0798 0.0712 0.0676 0.0656 Table 5: Mean RMSE for reconstructed iid standard Gaussian 8-vectors, q = 16, k betas a...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.