Pith. sign in

REVIEW 3 major objections 7 minor 23 references

NeUQI: Near-Optimal Uniform Quantization Parameter Initialization for Low-Bit LLMs

T0 review · 3 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Deriving a near-optimal zero-point for each scale makes low-bit LLM quantization consistently better.

desk verdict Solid PTQ initialization paper with a genuine algorithmic contribution, weakened by an unproved localization assumption and some baseline fairness issues. read the letter →

arxiv 2505.17595 v4 pith:L3ZDW7NC submitted 2025-05-23 cs.LG cs.CL

Li Lin , Xinyu Hu , Xiaojun Wan This is my paper · ORCID
classification cs.LGcs.CL
keywords post-trainingquantizationuniformlow-bitlargelanguagemodelsparameterinitializationzero-pointoptimizationscalesearchHessiandiagonalapproximation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Quantizing a model to very few bits requires choosing two numbers for each weight block: a scale, the grid spacing, and a zero-point, the grid's offset. The paper argues that the usual Min-Max formula picks these numbers badly at 2-3 bits, and that this initialization, not the quantization procedure itself, is what limits low-bit LLMs. NeUQI derives a near-optimal zero-point for any given scale, reducing the joint two-variable search to a one-variable search over the scale, and solves that search efficiently. In experiments on LLaMA and Qwen families at 2-4 bits, this starting point consistently outperforms existing post-training quantization baselines, and it is strong enough that a light distillation step beats much heavier fine-tuning methods.

What carries the argument

The load-bearing object is the piecewise-quadratic zero-point loss $L(z)=\sum_i h_i (x_i + z - \mathrm{clip}(\lfloor x_i+z\rceil,0,2^k-1))^2$, where $x_i$ is a weight divided by the scale and $h_i$ its Hessian diagonal weight. Because each term changes at transition points spaced exactly 1 apart, sweeping all transition points in sorted order and updating a running quadratic in $O(n\log n)$ time yields the exact zero-point optimum for a fixed scale. A simplified two-transition-point surrogate produces $z_S$, and the coarse-to-fine scale search reduces the number of zero-point solves to $O(\sqrt{T})$.

What would settle it

Compute the exact global minimizer of the zero-point loss in Eq. 7 by a full transition-point sweep for all rows of a real 2-bit quantized LLaMA layer, and compare each to the returned $z_S$; if any exact minimizer lies more than 1 away, the restricted search has found a suboptimal zero-point and the near-optimality claim is falsified for realistic data.

Watch

Extended reading notes

Core claim

NeUQI's central claim is that a much better uniform-quantization initialization is available almost for free once the zero-point is treated as a floating-point variable rather than an integer fixed by the Min-Max formula. For a fixed scale, the Hessian-weighted quantization loss becomes a piecewise quadratic function of the zero-point, and the paper gives an $O(n\log n)$ sweep over its transition points that finds its exact global minimum; a two-transition-point approximation gives a fast starting point $z_S$, after which the exact search restricted to $[z_S-1, z_S+1]$ is claimed to be near-optimal. With that zero-point solver, only the scale remains, and a coarse-to-fine grid search over it costs $O(\sqrt{T})$ zero-point evaluations. The paper reports that this recipe beats GPTQ, GPTAQ, MagR, and search-based LeanQuant across LLaMA 2/3 and Qwen 2.5 at 2-4 bits, and that replacing only the initialization inside the fine-tuning method EfficientQAT improves it further.

Load-bearing premise

The load-bearing premise is that, for realistic weight vectors and Hessian weights, the exact best zero-point always falls within one unit of the fast approximate zero-point $z_S$, so a search restricted to $[z_S-1,z_S+1]$ cannot miss it.

Editorial extensions

If this is right

  • At 2-bit channel-wise quantization, NeUQI lowers perplexity and raises average zero-shot accuracy on every LLaMA and Qwen model tested, with the largest gains on 70B-scale models.
  • Relaxing the integer zero-point constraint costs less than 0.01 average bit-width in the 2-bit channel-wise setting, and NeUQI still beats integer-constrained methods running at a slightly higher average bit-width.
  • NeUQI composes with Hadamard-transform-based quantization pipelines, improving W2A16, W2A4, and W4A4 results over the same pipelines with other initializations.
  • With about 0.5M tokens of lightweight distillation on LLaMA 2 7B at 2 bits, NeUQI surpasses PV-tuning, which uses roughly 1B tokens on a per-token basis, and also improves the strong fine-tuning method EfficientQAT when only its initialization is replaced.
  • Swapping only the initialization inside EfficientQAT improves its 2-bit results on LLaMA 2 7B and LLaMA 3 8B, and freezing the NeUQI-provided scale in the first stage helps further.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: The same transition-point sweep works for any per-sample convex quadratic of the quantizer parameter, so a similar near-optimal initialization could be derived for clipping thresholds, per-group scales, or activation quantizers.
  • Inference: The $[z_S-1,z_S+1]$ window is a one-unit guarantee that is asserted rather than proved; if weighted outlier distributions ever push the exact optimum further away, widening the window adaptively would be a cheap safety valve.
  • Inference: The results suggest the bottleneck at very low bit-widths is not optimization capacity but the quality of the starting parameters, which would shift future work toward better priors and initialization-first recipes.
  • Inference: Making floating-point zero-points a first-class citizen in inference kernels would remove the main deployment obstacle; if that happens, the integer-constrained formulations used by most libraries may become an unnecessary handicap.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The manuscript proposes NeUQI, a post-training uniform quantization parameter initialization method for large language models. It formulates the quantization loss under a diagonal Hessian approximation, derives a near-optimal zero-point for a fixed scale by exact minimization of a piecewise quadratic function via sorted transition points, and reduces the joint scale/zero-point optimization to a one-dimensional scale search with a coarse-to-fine grid. The authors argue that the conventional Min-Max initialization imposes two constraints (extreme-value-determined parameters and an integer zero-point) and show empirically that relaxing them improves perplexity and zero-shot accuracy across LLaMA and Qwen families at 2-4 bits. They also demonstrate gains when NeUQI is used as an initialization for distillation and quantization-aware fine-tuning.

Significance. If the claims hold, NeUQI is a valuable practical contribution: it is a fast, well-specified initialization scheme with O(n log n) per-scale zero-point computation, and the experiments show consistent and often large improvements over strong PTQ baselines at 2-3 bits, with additional benefits as a warm start for fine-tuning. The paper provides detailed pseudo-code for the optimization algorithms and a runtime analysis, and the experimental coverage (three model families, sizes 7B-72B, multiple bit-widths and group sizes) is extensive. The main caveat is that the theoretical 'near-optimal' guarantee is asserted rather than proven; the central claim is therefore empirical, and the empirical evaluation has some comparability issues.

major comments (3)
  1. [§4.2.1, Eq. (8) and Algorithm 4] The paper restricts the exact zero-point search to [zS−1, zS+1] based on the unproved assumption that the true minimizer of Eq. (7) lies within this interval. This is load-bearing: the scale-only loss of Section 4.2.2 is evaluated with this restricted zero-point, so any violation would make the reported losses suboptimal and directly undermine the 'near-optimal' part of the central claim. Table 1 reports only aggregate relative loss for LLaMA-2-7B at 2 bits, which is not sufficient to validate the bound across scales, bit-widths, or model families. Please provide a proof of the localization bound (or a counterexample analysis), or present a validation across the full grid of scales and bit-widths, or weaken the claim to 'approximately optimal under the localization assumption'.
  2. [§5.2, Tables 2 and 11] MagR is evaluated without the coordinate descent iterations used in the original MagR paper. Since MagR's published results are obtained with those iterations, this modification makes the comparison potentially misleading: the table may understate MagR's performance, and it is unclear whether 'fairness' is achieved by disabling a component of the baseline. Please report MagR with its standard configuration, or provide a clear justification for the modified setting and show that the conclusions are unchanged.
  3. [§5.1, Table 4] Several baseline numbers, including LeanQuant† in Table 4, are taken from the original papers rather than computed in the authors' evaluation pipeline. Differences in hardware, calibration data, or evaluation setups can affect perplexity and accuracy, so the claim of 'consistently outperforms existing methods' is not supported on equal footing for all baselines. Please either run all baselines under the same setting or explicitly quantify the expected sensitivity to these differences.
minor comments (7)
  1. [§3.1, Eq. (2)] The formulation's relaxation of the integer zero-point constraint is central to the method, but the hardware support discussion is relegated to Appendix E; a brief pointer in the main text would help readers assess deployability.
  2. [Table 2] The MagR row for Qwen 2.5 7B contains entries like '23.1242.09' and '37.7660.9951.30' that are missing separators and are difficult to read; please format them consistently.
  3. [§5.4] The text refers to 'Min-Max+1', but the method is defined as 'Min-Max+' in Appendix A; please use a consistent name.
  4. [§4.2.1] The phrase 'sharing NeUQI’s loss function 5' should read 'loss function (5)' to avoid ambiguity with a numbered section.
  5. [Table 4] LeanQuant† has missing values (—) for LLaMA 3 8B on Wiki2 and C4; please indicate whether these are unavailable or infeasible.
  6. [§4.2.2, Eq. (9)] The search space is restricted to scales no larger than the Min-Max scale, but no justification is given for this upper bound; a sentence explaining why the optimal scale cannot exceed this value would be useful, especially for heavy-tailed weight distributions.
  7. [Appendix C, Algorithm 4] The comment 'Since(⌈zS −1 +x i − 1/2⌉+ 2) +1/2 −x i ≥ zS + 1' is difficult to parse; consider reformatting it as a clearer derivation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: NeUQI's scale/zero-point search optimizes its own proxy loss, while the headline claims are validated on external perplexity/accuracy benchmarks; the unproved radius-1 localization in Section 4.2.1 is an unproved assumption, not a circular reduction.

full rationale

The paper's derivation chain is self-contained rather than circular. For a fixed scale, NeUQI defines the zero-point problem as minimizing L(z)=Σ h_i (x_i+z-clip(floor(x_i+z),...))^2 (Eqs. 6-7), computes an O(n log n) optimum of the flattened surrogate Eq. 8, then restricts the exact search to [zS−1,zS+1] (Section 4.2.1). The step 'We then assume that the true optimum lies in the neighborhood of zS' is an unproved localization assumption, and if violated the claimed near-optimality for a fixed scale would fail. That is a correctness or robustness gap, not circularity: the surrogate is not fitted to the true minimizer, nor is the exact loss defined in terms of zS. The scale-only optimization (Eq. 9) and coarse-to-fine search are likewise evaluated on the same proxy loss only for runtime/loss ablations (Table 1); the paper's central empirical claims are measured on WikiText-2/C4 perplexity and five zero-shot accuracy benchmarks, which are external to the quantities optimized. The method does not fit any parameter to those benchmarks and then report them as predictions. There are no load-bearing self-citations: the cited related work (GPTQ, LeanQuant, MagR, etc.) is used for baselines, not to justify NeUQI's derivation. 'Near-optimal' is relative to the paper's own quantization loss, but that is the standard objective being optimized rather than a renamed input; the external task results carry the independent content. Score 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The method contributes an algorithm, not new physical entities. It relies on several standard approximations (diagonal Hessian, proxy MSE) and one unproved assumption about the location of the exact optimum. The two hand-set grid sizes are free parameters.

free parameters (2)
  • T (scale search grid size) = 2048
    Number of candidate scales in the full search; set by hand to balance runtime and accuracy. The paper does not ablate T.
  • Tc (coarse grid size) = 64
    Coarse search grid size; set by hand to O(sqrt(T)). Chosen to make the coarse-to-fine search efficient with minimal loss increase.
assumptions (4)
  • domain assumption Diagonal Hessian approximation: cross-weight interactions are negligible, so the loss decouples per element.
    Adopted from prior work (LeCun et al., GPTQ), stated in Section 4.2; standard in PTQ but not exact.
  • ad hoc to paper The optimal zero-point of the exact loss (Eq. 7) lies within [zS-1, zS+1] of the approximate solution zS.
    Stated as an assumption in Section 4.2.1 without proof; the empirical Table 1 supports it only on LLaMA-2 7B under 2-bit quantization.
  • domain assumption The per-weight weighted MSE proxy loss correlates with final model quality.
    Used throughout Section 4; the paper's gains on downstream benchmarks provide indirect support, but it is a heuristic shared with prior PTQ methods.
  • domain assumption Uniform quantization with floating-point zero-points is supported by available inference kernels (e.g., BitBLAS in vLLM).
    Appendix E cites BitBLAS support; if this support is unavailable on target hardware, the practical benefit of relaxing the integer constraint is reduced.

how reviews work

0 comments
Cite this review

Pith. "Pith review of NeUQI: Near-Optimal Uniform Quantization Parameter Initialization for Low-Bit LLMs." pith.science (2026). https://pith.science/paper/L3ZDW7NC

@misc{pith2026250517595,
  author       = {Pith},
  title        = {Pith review of: NeUQI: Near-Optimal Uniform Quantization Parameter Initialization for Low-Bit LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/L3ZDW7NC}},
  note         = {Machine review of arXiv:2505.17595}
}
read the original abstract

Large language models (LLMs) achieve impressive performance across domains but face significant challenges when deployed on consumer-grade GPUs or personal devices such as laptops, due to high memory consumption and inference costs. Post-training quantization (PTQ) of LLMs offers a promising solution that reduces their memory footprint and decoding latency. In practice, PTQ with uniform quantization representation is favored due to its efficiency and ease of deployment, as uniform quantization is widely supported by mainstream hardware and software libraries. Recent studies on low-bit uniform quantization have led to noticeable improvements in post-quantization model performance; however, they mainly focus on quantization methodologies, while the initialization of quantization parameters remains underexplored and still relies on the conventional Min-Max formula. In this work, we identify the limitations of the Min-Max formula, move beyond its constraints, and propose NeUQI, a method that efficiently determines near-optimal initialization for uniform quantization. Our NeUQI simplifies the joint optimization of the scale and zero-point by deriving the zero-point for a given scale, thereby reducing the problem to a scale-only optimization. Benefiting from the improved quantization parameters, our NeUQI consistently outperforms existing methods in the experiments with the LLaMA and Qwen families on various settings and tasks. Furthermore, when combined with a lightweight distillation strategy, NeUQI even achieves superior performance to PV-tuning, a considerably more resource-intensive method.

Figures

Figures reproduced from arXiv: 2505.17595 by the authors.

Figure 1
Figure 1. Perplexity on C4 (left) and average accuracy across five common benchmarks (right) with Qwen 2.5 family, plotted against the base-2 logarithm (log2 ) of model memory usage on the x-axis. Gray arrows indicate the direction of better performance, corresponding to lower perplexity and higher accuracy with smaller model memory usage. The results include non-quantized models (BF16) and quantized models at 3-bit using GPT… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 4 canonical work pages

  1. [2]

    Implementation Details D.1

    +1 2 −x i ≥z S + 1, there are at most two transition points in the interval[zS −1, , zS + 1).} end for SortTby transition pointt LI(z)← P i hi xi +z−clip(⌊x i +z S −1⌉,0,2 k −1) 2 (tfirst, δfirst)←first element ofT z′ ←arg minz∈[zS −1, tfirst] LI(z) (z∗,L ∗)←(z ′,L I(z′)) foreach(t, δ(z))inTdo LI(z)← LI(z) +δ(z) Let next transition point bet ′ (orz S + 1i...

    work page 2024

  2. [5]

    Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al

    URL https://zenodo.org/records/12608602. Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al. The llama 3 herd of models,

    arXiv

  3. [6]

    Hu, X., Cheng, Y ., Yang, D., Xu, Z., Yuan, Z., Yu, J., Xu, C., Jiang, Z., and Zhou, S

    URLhttps://arxiv.org/abs/2407.21783. Hu, X., Cheng, Y ., Yang, D., Xu, Z., Yuan, Z., Yu, J., Xu, C., Jiang, Z., and Zhou, S. Ostquant: Refining large language model quantization with orthogonal and scaling transformations for better distribution fitting,

    arXiv

  4. [7]

    Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., and Kalenichenko, D

    URL https://arxiv.org/abs/2501.13987. Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., and Kalenichenko, D. Quan- tization and training of neural networks for efficient integer-arithmetic-only inference,

    arXiv

  5. [9]

    Kwon, W., Li, Z., Zhuang, S., Sheng, Y ., Zheng, L., Yu, C

    URLhttps: //arxiv.org/abs/1806.08342. Kwon, W., Li, Z., Zhuang, S., Sheng, Y ., Zheng, L., Yu, C. H., Gonzalez, J. E., Zhang, H., and Stoica, I. Efficient memory management for large language model serving with pagedattention. InProceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles,

    arXiv

  6. [10]

    S., and Solla, S

    LeCun, Y ., Denker, J. S., and Solla, S. A. Opti- mal brain damage. In Touretzky, D. S. (ed.),Ad- vances in Neural Information Processing Systems 2, [NIPS Conference, Denver, Colorado, USA, Novem- ber 27-30, 1989], pp. 598–605. Morgan Kaufmann,

    work page 1989

  7. [12]

    Mqbench: Towards reproducible and deployable model quantization benchmark.arXiv preprint arXiv:2111.03759,

    Li, Y ., Shen, M., Ma, J., Ren, Y ., Zhao, M., Zhang, Q., Gong, R., Yu, F., and Yan, J. Mqbench: Towards reproducible and deployable model quantization benchmark.arXiv preprint arXiv:2111.03759,

    arXiv

  8. [13]

    Lin, H., Xu, H., Wu, Y ., Cui, J., Zhang, Y ., Mou, L., Song, L., Sun, Z., and Wei, Y

    URL https://arxiv.org/abs/ 2504.02692. Lin, H., Xu, H., Wu, Y ., Cui, J., Zhang, Y ., Mou, L., Song, L., Sun, Z., and Wei, Y . Duquant: Distributing outliers via dual transformation makes stronger quantized llms. Advances in Neural Information Processing Systems, 37: 87766–87800, 2024a. Lin, J., Tang, J., Tang, H., Yang, S., Chen, W.-M., Wang, W.-C., Xiao...

    arXiv 2024

Show all 23 references
  1. [14]

    Malinovskii, V ., Mazur, D., Ilin, I., Kuznedelev, D., Burlachenko, K., Yi, K., Alistarh, D., and Richtarik, P

    URL https://arxiv.org/abs/ 2405.16406. Malinovskii, V ., Mazur, D., Ilin, I., Kuznedelev, D., Burlachenko, K., Yi, K., Alistarh, D., and Richtarik, P. Pv-tuning: Beyond straight-through estimation for ex- treme llm compression.Advances in Neural Information Processing Systems,...

    arXiv

  2. [15]

    Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y ., Li, W., and Liu, P

    URL https://arxiv.org/abs/2303.08774. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y ., Li, W., and Liu, P. J. Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of machine learning research, 21 (140):1–67,

    arXiv

  3. [16]

    Touvron, H., Martin, L., Stone, K., Albert, P., Alma- hairi, A., Babaei, Y ., Bashlykov, N., Batra, S., Bhar- gava, P., Bhosale, S., et al

    URL https://arxiv.org/ abs/2308.13137. Touvron, H., Martin, L., Stone, K., Albert, P., Alma- hairi, A., Babaei, Y ., Bashlykov, N., Batra, S., Bhar- gava, P., Bhosale, S., et al. Llama 2: Open founda- tion and fine-tuned chat models,

    arXiv

  4. [17]

    Xiao, G., Lin, J., Seznec, M., Wu, H., Demouth, J., and Han, S

    URL https: //arxiv.org/abs/2307.09288. Xiao, G., Lin, J., Seznec, M., Wu, H., Demouth, J., and Han, S. Smoothquant: Accurate and efficient post-training quantization for large language models. InInternational Conference on Machine Learning, pp. 38087–38099. PMLR,

    arXiv

  5. [18]

    Zellers, R., Holtzman, A., Bisk, Y ., Farhadi, A., and Choi, Y

    URL https://arxiv.org/ abs/2412.15115. Zellers, R., Holtzman, A., Bisk, Y ., Farhadi, A., and Choi, Y . HellaSwag: Can a machine really finish your sen- tence? In Korhonen, A., Traum, D., and M `arquez, L. (eds.),Proceedings of the 57th Annual Meeting of the Association for Co...

    arXiv

  6. [20]

    org/abs/2406.00800

    URL https://arxiv. org/abs/2406.00800. Zhang, T. and Shrivastava, A. Leanquant: Accurate and scalable large language model quantization with loss- error-aware grid. InThe Thirteenth International Confer- ence on Learning Representations,

    arXiv

  7. [128]

    Size Bits Group Method Wiki2↓C4↓ ArcC↑ArcE↑HellaS↑PiQA↑WinoG↑Acc↑ 7B BF16 - - 6.39 10.02 48.29 80.56 60.00 78.67 72.69 68.04 4 128 GPTQ 6.61 10.19 48.21 80.35 59.2479.0571.35 67.64 NeUQI 6.55 10.17 48.63 80.98 59.36 78.67 73.95 68.32 - GPTQ 7.06 10.61 46.59 79.25 58.08 78.56 6...

  8. [1989]

    URL http://papers.nips.cc/paper/ 250-optimal-brain-damage. Li, Y . and Panda, P. Tesseraq: Ultra low-bit llm post-training quantization with block reconstruction.arXiv preprint arXiv:2410.19103,

    arXiv

  9. [2017]

    Krishnamoorthi, R

    URL https: //arxiv.org/abs/1712.05877. Krishnamoorthi, R. Quantizing deep convolutional networks for efficient inference: A whitepaper,

    arXiv

  10. [2018]

    Dettmers, T., Lewis, M., Belkada, Y ., and Zettlemoyer, L

    URL https://arxiv.org/abs/ 1803.05457. Dettmers, T., Lewis, M., Belkada, Y ., and Zettlemoyer, L. Gpt3. int8 (): 8-bit matrix multiplication for transform- ers at scale.Advances in neural information processing systems, 35:30318–30332,

    arXiv

  11. [2019]

    doi: 10.18653/v1/P19-1472

    Association for Compu- tational Linguistics. doi: 10.18653/v1/P19-1472. URL https://aclanthology.org/P19-1472/. Zhang, A., Wang, N., Deng, Y ., Li, X., Yang, Z., and Yin, P. Magr: Weight magnitude reduction for enhancing post- training quantization,

    doi

  12. [2023]

    org/abs/2210.17323

    URL https://arxiv. org/abs/2210.17323. Gao, L., Tow, J., Abbasi, B., Biderman, S., Black, S., DiPofi, A., Foster, C., Golding, L., Hsu, J., Le Noac’h, A., Li, H., McDonell, K., Muennighoff, N., Ociepa, C., Phang, J., Reynolds, L., Schoelkopf, H., Skowron, A., Sutawika, L., Tan...

    arXiv

  13. [2024]

    Ashkboos, S., Mohtashami, A., Croci, M., Li, B., Cameron, P., Jaggi, M., Alistarh, D., Hoefler, T., and Hensman, J

    URLhttps://arxiv.org/abs/2403.06082. Ashkboos, S., Mohtashami, A., Croci, M., Li, B., Cameron, P., Jaggi, M., Alistarh, D., Hoefler, T., and Hensman, J. Quarot: Outlier-free 4-bit inference in rotated llms. Advances in Neural Information Processing Systems, 37: 100213–100240,

    arXiv

  14. [2025]

    Frantar, E., Ashkboos, S., Hoefler, T., and Alistarh, D

    URL https://arxiv.org/abs/2312.07950. Frantar, E., Ashkboos, S., Hoefler, T., and Alistarh, D. Gptq: Accurate post-training quantization for generative pre- trained transformers,

    arXiv

  15. [2048]

    In contrast, Int-Search employs a grid search over candidate scale values, with the zero-point constrained to be a k-bit unsigned integer

    denotes the number of grid points andn the number of weights. In contrast, Int-Search employs a grid search over candidate scale values, with the zero-point constrained to be a k-bit unsigned integer. A simple implementation of this scale-based search has a time complexity of ...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.