Pith. sign in

REVIEW 3 major objections 5 minor 49 references

Exploring Model Invariance with Discrete Search for Ultra-Low-Bit Quantization

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proposes InvarExplore, a discrete hill-climbing search over permutation, scaling, and rotation of a Transformer's weights that improves 2-bit quantization of OPT models from 1.3B to 13B on top of existing quantizers.

desk verdict The activation-matching term in Algorithm 1 is constant, so the paper's core mechanism doesn't work as printed; the idea is still worth refereeing. read the letter →

arxiv 2502.06844 v1 pith:LWI7GMDR submitted 2025-02-06 cs.LG cs.AIcs.CL

Yuqiao Wen , Yanshuai Cao , Lili Mou This is my paper · ORCID
classification cs.LGcs.AIcs.CL
keywords post-trainingquantizationultra-low-bitmodelinvariancepermutationdiscretehill-climbingsearchweight-onlylargelanguagemodelsactivationmatching
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that permutation, scaling, and rotation of a trained Transformer's weights—each leaving the unquantized model's behavior intact or nearly intact—can be explored together to make 2-bit quantization markedly less damaging. It introduces a discrete hill-climbing search for this, since permutation is non-differentiable and previous gradient-based methods cannot easily reach it. The central claim is that the search is an add-on: applied after GPTQ, AWQ, or OmniQuant, it lowers perplexity and raises reasoning accuracy consistently for OPT models from 1.3B to 13B parameters. A sympathetic reader would care because 2-bit quantization is where existing methods degrade sharply, and the method needs only forward passes on a short calibration set.

What carries the argument

The central object is the PSR transform acting on a Transformer feed-forward block $z = W_{\mathrm{down}} f(W_{\mathrm{up}} x + b_{\mathrm{up}}) + b_{\mathrm{down}}$. Replacing $(W_{\mathrm{up}}, b_{\mathrm{up}}, W_{\mathrm{down}})$ by $(PSRW_{\mathrm{up}}, PSRb_{\mathrm{up}}, W_{\mathrm{down}}R^{\top}S^{-1}P^{\top})$ leaves the unquantized output unchanged when the symmetry is exact (permutation; positive scaling for ReLU-type activations) and nearly unchanged for small rotations, which are built as a block-diagonal matrix of 2D rotations parameterized by angles $\phi_1,\dots,\phi_{d/2}$. The search is a hill-climbing loop that at each step shuffles 10% of a sampled layer's neurons and takes Gaussian random-walk steps in scale ($\sigma_s=10^{-2}$) and rotation angle ($\sigma_r=10^{-5}$), then accepts the move only if the quantized model's cross-entropy plus $\alpha$ times the activation MSE on the calibration batch improves. This machinery is what makes permutation searchable at all, and it is what lets the three invariance types reinforce each other.

What would settle it

Run InvarExplore's rotation search on a pretrained model with GELU or SiLU activations at 2-bit and measure the unquantized cross-entropy before and after accepted moves; if the unquantized loss drifts by more than about 0.01% or the perplexity gain over AWQ disappears, the approximate-rotation premise, and with it the claimed generality of the PSR search, is falsified.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a trained Transformer has many exactly or nearly equivalent parameterizations—permuting neurons, rescaling features, and applying small rotations between the up and down projections of a feed-forward block—and some of these quantize far better than the original weights. Because round-to-nearest quantization depends on the min and max of each weight group, an invariant transform that reduces outlier spread lowers quantization error without changing the unquantized function. InvarExplore's hill-climbing search over the combined PSR transform, guided by cross-entropy plus activation MSE on 512-token sequences, is the means to find those parameterizations; the paper reports, for example, that on the 13B OPT model it cuts AWQ's WikiText-2 perplexity from 35.89 to 26.26 at 2-bit/group-128 and raises average six-task reasoning accuracy from 51.85% to 55.13%. Permutation, although a special case of orthogonal transformation, creates symmetric local optima that gradient-based rotation learners cannot escape, and because permutation is non-differentiable the paper argues a discrete search is required.

Load-bearing premise

The load-bearing premise is that small rotations are close enough to exact invariance: the paper checks this once, seeing a 0.001% cross-entropy change on a 13B OPT model, and the rotation part of the search—and with it part of the reported synergy—collapses if that approximation fails under quantization or on other architectures.

Editorial extensions

If this is right

  • On WikiText-2 at 2-bit/group-128, applying InvarExplore to AWQ lowers the 13B OPT perplexity from 35.89 to 26.26 and the C4 perplexity from 37.36 to 27.00.
  • The six-task reasoning average for the same setting rises from 51.85% to 55.13% on the 13B model, and consistent gains appear across OPT 1.3B, 2.7B, 6.7B, and 13B.
  • The 13B ablation shows each transformation alone beats the AWQ baseline on most tasks, and the combined PSR search beats each single transformation, supporting the synergy claim.
  • InvarExplore can be run with only forward passes on a short calibration set (32 sequences of 512 tokens), and even with activation matching disabled it still beats AWQ with no extra memory cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the same PSR search should transfer to leaky ReLU models, since the scaling symmetry is exact for any piecewise-linear activation with a fixed slope; the 0.001% rotation check would need to be rerun there.
  • A testable extension the paper does not run is wrapping the same forward-only hill-climbing search around non-integer quantizers such as vector quantization; the black-box objective should carry over, but this is conjecture.
  • An unstated corollary of the paper's Table 4 trend is that activation matching on more layers should keep improving final quantization quality when GPU memory permits, effectively trading search-time memory for accuracy.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes INVAREXPLORE, a framework for improving ultra-low-bit post-training quantization of LLMs by searching over permutation, scaling, and rotation transformations of Transformer feed-forward blocks. These transformations preserve (or approximately preserve) the unquantized model's function, while changing the rounding behavior under integer quantization. The authors present an activation-guided discrete hill-climbing search, apply it on top of GPTQ, AWQ, and OmniQuant for OPT models from 1.3B to 13B at 2-bit quantization, and report perplexity on WikiText-2 and C4 plus average accuracy across six reasoning tasks. They also provide ablations for individual transformations, bit widths, activation-matching layers, and calibration-set size.

Significance. If the empirical claims hold after a corrected algorithm is supplied, the contribution is useful: a model-agnostic add-on to existing PTQ methods, and one of the few attempts to optimize permutation invariance through discrete search rather than gradient-based methods that cannot easily handle permutation symmetry. The paper is clearly written, positions itself well against prior work, and openly acknowledges that rotation invariance is approximate. However, the central 'activation-guided' mechanism is not implemented as written in Algorithm 1, no code or random-seed details are provided, and the reported gains over the strongest baseline, OmniQuant, are often small or accompanied by task-level regressions. These issues prevent the current version from being reproducible and make the strength of the central claim difficult to assess.

major comments (3)
  1. [Section 3.2, Algorithm 1, and Table 4] As written, Algorithm 1 does not implement an activation-guided search. Lines 1–2 compute H = M(X, θ0) and H0 = M(X, quant(θ0)) once, before the search loop; every candidate θ′ is then evaluated at line 16 with L′ = LKL(X, quant(θ′)) + α MSE(H, H0). Because H and H0 are fixed, the activation-matching term is constant and cannot influence acceptance. The search as pseudocoded therefore reduces to calibration cross-entropy alone, and the result in Table 4 that matching more layers improves performance cannot follow from the stated algorithm. In addition, the text says that only 10% of the neurons in a layer are changed per update, but lines 12–14 sample full new permutation, scaling, and rotation vectors for the chosen layer. Please rewrite the pseudocode so that the activation-matching term is computed against the candidate quantized model, e.g., MSE(M(X, θ0), M(X, quant(θ′))) over the matched layers, make the subset update explicit, and release code or precise pseudocode that matches the experiments.
  2. [Section 4, Tables 1 and 5] The paper claims 'consistent' improvement over OmniQuant, but the point estimates show several ties and regressions. For OPT-2.7B, the average reasoning accuracy is unchanged at 50.62, with BoolQ dropping from 56.91 to 56.09; for OPT-13B, BoolQ drops from 66.09 to 65.78, PIQA from 73.07 to 72.96, and WinoGrande from 62.12 to 61.80. The gains over OmniQuant in Table 1 are often small, e.g., WikiText-2 perplexity 12.94 vs. 12.73 at 13B. Because the search involves random shuffles and Gaussian random walks, single point estimates are insufficient to support a claim of consistent improvement. Please report means and standard deviations over multiple random seeds, and where possible paired tests across tasks; the aggregate win/loss count in Appendix A is not a substitute for per-setting variance.
  3. [Section 3.2, Eqs. (16)–(17), and Table 2] The paper acknowledges that the rotation transform is only approximately invariant, but the supporting evidence is a single FP16 pilot measurement on 13B OPT (WikiText-2 cross-entropy 2.31528 vs. 2.31525). The approximation must hold after quantization and for every layer where rotations are applied, yet Table 2 shows rotation-only is a substantial contributor (PPL 30.21 vs. AWQ 35.89 on WikiText-2). Please add a per-layer or per-block analysis of the invariance violation, or quantify how much the rotation search changes the unquantized model's outputs across model sizes. Without this, the reader cannot determine whether the rotation component is exploiting the claimed invariance or merely fitting the calibration set by altering the model's behavior.
minor comments (5)
  1. [Section 3.2, after Eq. (9)] The sentence 'Here, Eqn. (13) follows because permutation...' appears to be a cross-reference error; the justification concerns the permutation invariance derivation in Eqns. (8)–(9), not the later scaling equation (13).
  2. [Section 3.2, Eqns. (10)–(22)] The transformed parameters reuse the same symbols Wup, bup, and Wdown as the original parameters. Using tildes or primes would avoid ambiguity when reading Eqns. (21)–(22).
  3. [Section 3.2, Eq. (23) and Algorithm 1] The main text defines the loss using CE, while Algorithm 1 writes LKL without defining it. Please align the notation and define LKL explicitly.
  4. [Section 4.1] The hyperparameters α, σs, and σr are said to be obtained by a grid search on the calibration set. Please specify the grid and report sensitivity to these values; otherwise the method appears to have several free parameters tuned on the same objective used for search.
  5. [Section 4.2, Figure 1] The optimization curves in Figure 1 do not indicate whether they are single runs or averaged over seeds, and the reported acceptance ratio appears to be a single-trajectory quantity. Adding seed variance would strengthen the analysis.

Circularity Check

1 steps flagged · score 6.0 of 10

The activation-matching term in Algorithm 1 is constant by construction, so the paper's central claim of an activation-guided search, and the Table 4 layer-matching analysis, are not supported by the printed algorithm; no other circularity was found.

  1. other [Section 3.2, Eq. (23) and Algorithm 1 (lines 1-3, 16)]
    "Let H and H0 be the activations of the quantized model and the original model, respectively. Our loss is L(X, quant(θ)) = CE(X, quant(θ)) + α MSE(H, H0). Algorithm 1: 1 H ← M(X, θ0); 2 H0 ← M(X, quant(θ0)); ... 16 L′ ← LKL(X, quant(θ′)) + α MSE(H, H0)."

    Lines 1-2 fix H and H0 from the initial model and its quantized version before the loop; line 16 reuses them for every candidate θ′, so MSE(H, H0) is a constant that never depends on θ′ and cannot influence acceptance. As printed, the objective reduces to LKL(X, quant(θ′)) plus a constant, i.e., to cross-entropy-only hill climbing. Therefore the paper's central 'activation-guided discrete search' claim and the Table 4 claim that matching more layers improves performance do not follow from the stated algorithm; the activation-guidance mechanism is vacuous by construction.

full rationale

The derivation of the invariance transformations is mathematically straightforward and not circular: Eqs. (8)-(15) verify exact invariance for permutation and scaling, and Eqs. (16)-(17) explicitly acknowledge that rotation is approximate, supported by an empirical check. No load-bearing self-citation was found; the only same-author citation (Lin et al., 2024a) is background on a manual zigzag permutation and is not used to justify a result. Hyperparameters are tuned on the calibration set and performance is reported on held-out test sets, which is fitting rather than circular. The one serious issue is the Algorithm 1 / Eq. (23) mismatch described above: because H and H0 are fixed, the activation-matching term is constant by construction, so the central activation-guidance claim and Table 4 reduce to a cross-entropy-only search as written. This is a partial by-construction vacuity of a central mechanism, hence score 6; the invariance framework and main empirical comparisons retain independent content, and if the algorithm were corrected to compare candidate activations, the circularity or vacuity would disappear.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or architectural entities. It relies on known linear-algebra invariances, one approximate invariance (rotation), and several hyperparameters tuned on the calibration set. The rotation approximate invariance is the most fragile assumption.

free parameters (5)
  • alpha (balancing hyperparameter in Eq. 23) = Set so cross-entropy is 10x more important than activation MSE at the start
    Chosen by grid search on the calibration set during a pilot study; affects the search objective.
  • sigma_s (scaling random-walk std) = 1e-2
    Chosen by grid search on calibration loss; controls step size for scaling proposals.
  • sigma_r (rotation random-walk std) = 1e-5
    Chosen by grid search on calibration loss; kept small because large rotations cause high variance.
  • step_size (fraction of neurons changed per update) = 10%
    Selected based on experience as a balance between acceptance rate and progress; not rigorously tuned.
  • number of activation-matching layers = 10 (out of 40 for 13B OPT)
    Restricted by GPU memory; ablation shows more layers help, so this is a resource-driven choice.
assumptions (4)
  • standard math Permutation invariance: f(xP) = f(x)P for elementwise activations
    Used in Eq. (8)-(9) to show permutation is an exact invariance for ReLU networks.
  • domain assumption Scaling invariance: f(sx) = s f(x) for ReLU/LeakyReLU
    Used in Eq. (12)-(13); holds only for positive-homogeneous activations like ReLU, which is present in OPT.
  • ad hoc to paper Rotation approximate invariance: small rotations change the model output negligibly
    Invoked in Eq. (16)-(19); the paper supports this with a single empirical loss difference of 0.001%, not a derivation or bound.
  • domain assumption Calibration set is representative of the test distribution
    The search optimizes on 32 sequences of 512 tokens from the Pile and assumes this transfers to WikiText-2, C4, and reasoning tasks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Exploring Model Invariance with Discrete Search for Ultra-Low-Bit Quantization." pith.science (2026). https://pith.science/paper/LWI7GMDR

@misc{pith2026250206844,
  author       = {Pith},
  title        = {Pith review of: Exploring Model Invariance with Discrete Search for Ultra-Low-Bit Quantization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LWI7GMDR}},
  note         = {Machine review of arXiv:2502.06844}
}
read the original abstract

Large language models have been increasing in size due to their success in a wide range of applications. This calls for a pressing need to reduce memory usage to make them more accessible. Post-training quantization is a popular technique which uses fewer bits (e.g., 4--8 bits) to represent the model without retraining it. However, it remains a challenging task to perform quantization in an ultra-low-bit setup (e.g., 2 bits). In this paper, we propose InvarExplore, a unified framework that systematically explores different model invariance at the same time, allowing us to take advantage of the synergy between each type of invariance. Importantly, InvarExplore features a discrete search algorithm that enables us to explore permutation invariance, which is under-studied as it cannot be optimized with gradient-based methods. Results show that InvarExplore is compatible with existing state-of-the-art methods, achieving an add-on performance improvement over strong competing methods.

Figures

Figures reproduced from arXiv: 2502.06844 by the authors.

Figure 1
Figure 1. Optimization curves for the 13B OPT model across different numbers of calibration sequences: (a) [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 32 canonical work pages

  1. [1]

    A. Agarap. Deep learning using rectified linear units (ReLU) . arXiv preprint arXiv:1803.08375, 2018. URL https://arxiv.org/abs/1803.08375

    arXiv 2018

  2. [2]

    Ashkboos, A

    S. Ashkboos, A. Mohtashami, M. L. Croci, B. Li, M. Jaggi, D. Alistarh, T. Hoefler, and J. Hensman. QuaRot : Outlier-free 4-bit inference in rotated LLMs . In Advances in Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=dfqsW38v1X

    work page 2024

  3. [3]

    Bengio, N

    Y. Bengio, N. L \'e onard, and A. Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432, 2013. URL https://arxiv.org/abs/1308.3432

    arXiv 2013

  4. [4]

    Y. Bisk, R. Zellers, J. Gao, Y. Choi, et al. PIQA : Reasoning about physical commonsense in natural language. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 7432--7439, 2020. URL https://ojs.aaai.org/index.php/AAAI/article/view/6239

    work page 2020

  5. [5]

    Clark, K

    C. Clark, K. Lee, M.-W. Chang, T. Kwiatkowski, M. Collins, and K. Toutanova. B ool Q : Exploring the surprising difficulty of natural yes/no questions. In Proceedings of the Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies , pages 2924--2936, 2019. URL https://aclanthology.org/N19-1300/

    work page 2019

  6. [6]

    Clark, I

    P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord. Think you have solved question answering? T ry ARC , the AI 2 reasoning challenge. arXiv preprint arXiv:1803.05457, 2018. URL https://arxiv.org/abs/1803.05457

    arXiv 2018

  7. [7]

    Courbariaux, Y

    M. Courbariaux, Y. Bengio, and J.-P. David. Binary C onnect: Training deep neural networks with binary weights during propagations. In Advances in Neural Information Processing Systems, 2015. URL https://proceedings.neurips.cc/paper_files/paper/2015/file/3e15cc11f979ed25912dff5b0669f2cd-Paper.pdf

    work page 2015

  8. [8]

    Dettmers, M

    T. Dettmers, M. Lewis, Y. Belkada, and L. Zettlemoyer. GPT3 .int8(): 8-bit matrix multiplication for transformers at scale. In Advances in Neural Information Processing Systems, pages 30318--30332, 2022. URL https://openreview.net/forum?id=dXiGWqBoxaD

    work page 2022

Show all 49 references
  1. [9]

    Dodge, M

    J. Dodge, M. Sap, A. Marasovi \'c , W. Agnew, G. Ilharco, D. Groeneveld, M. Mitchell, and M. Gardner. Documenting large webtext corpora: A case study on the Colossal Clean Crawled Corpus . In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pa...

    2021

  2. [10]

    Dubey, A

    A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Yang, A. Fan, et al. The L lama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. URL https://arxiv.org/abs/2407.21783

    2024 arXiv

  3. [11]

    Egiazarian, A

    V. Egiazarian, A. Panferov, D. Kuznedelev, E. Frantar, A. Babenko, and D. Alistarh. Extreme compression of large language models via additive quantization. In Proceedings of the International Conference on Machine Learning, 2024. URL https://proceedings.mlr.press/v235/egiazari...

    2024

  4. [12]

    Frantar and D

    E. Frantar and D. Alistarh. Optimal brain compression: A framework for accurate post-training quantization and pruning. In Advances in Neural Information Processing Systems, pages 4475--4488, 2022. URL https://openreview.net/forum?id=ksVGCOlOEba

    2022

  5. [13]

    Frantar, S

    E. Frantar, S. Ashkboos, T. Hoefler, and D. Alistarh. GPTQ : Accurate post-training quantization for generative pre-trained transformers. In International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=tcbBPnfwxS

    2023

  6. [14]

    L. Gao, S. Biderman, S. Black, L. Golding, T. Hoppe, C. Foster, J. Phang, H. He, A. Thite, N. Nabeshima, et al. The P ile: An 800 GB dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027, 2020. URL https://arxiv.org/abs/2101.00027

    2020 arXiv

  7. [15]

    Goldberg

    D. Goldberg. What every computer scientist should know about floating-point arithmetic. ACM Computing Surveys, 23 0 (1): 0 5–48, 1991. URL https://doi.org/10.1145/103162.103163

    1991

  8. [16]

    Y. Gong, L. Liu, M. Yang, and L. Bourdev. Compressing deep convolutional networks using vector quantization. In International Conference on Learning Representations, 2015. URL https://arxiv.org/abs/1412.6115

    2015 arXiv

  9. [17]

    R. Gray. Vector quantization. IEEE ASSP Magazine, 1 0 (2): 0 4--29, 1984. URL https://ieeexplore.ieee.org/document/1162229

    1984

  10. [18]

    S. Han, H. Mao, and W. J. Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. In International Conference on Learning Representations, 2016. URL https://arxiv.org/abs/1510.00149

    2016 arXiv

  11. [19]

    Hassibi, D

    B. Hassibi, D. Stork, and G. Wolff. Optimal brain surgeon: Extensions and performance comparisons. In Advances in Neural Information Processing Systems, 1993. URL https://proceedings.neurips.cc/paper_files/paper/1993/file/b056eb1587586b71e2da9acfe4fbd19e-Paper.pdf

    1993

  12. [20]

    E. J. Hu, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen, et al. LoRA : Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=nZeVKeeFYf9

    2022

  13. [21]

    IEEE standard for floating-point arithmetic

    IEEE Computer Society . IEEE standard for floating-point arithmetic. IEEE Std 754-2008, pages 1--70, 2008. URL https://ieeexplore.ieee.org/document/4610935

    2008

  14. [22]

    Jacob, S

    B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, and D. Kalenichenko. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2704--2...

    2018

  15. [23]

    X. Jiao, Y. Yin, L. Shang, X. Jiang, X. Chen, L. Li, F. Wang, and Q. Liu. T iny BERT : Distilling BERT for natural language understanding. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4163--4174, 2020. URL https://aclanthology.org/2020.findin...

    2020

  16. [24]

    K\" o ster, T

    U. K\" o ster, T. Webb, X. Wang, M. Nassar, A. K. Bansal, W. Constable, O. Elibol, S. Gray, S. Hall, L. Hornof, A. Khosrowshahi, C. Kloss, R. J. Pai, and N. Rao. Flexpoint: An adaptive numerical format for efficient training of deep neural networks. In Advances in Neural Infor...

    2017

  17. [25]

    Krishnamoorthi

    R. Krishnamoorthi. Quantizing deep convolutional networks for efficient inference: A whitepaper. arXiv preprint arXiv:1806.08342, 2018. URL https://arxiv.org/abs/1806.08342

    2018 arXiv

  18. [26]

    S. Li, X. Ning, L. Wang, T. Liu, X. Shi, S. Yan, G. Dai, H. Yang, and Y. Wang. Evaluating quantized large language models. In Proceedings of the International Conference on Machine Learning, pages 28480--28524, 2024. URL https://proceedings.mlr.press/v235/li24bb.html

    2024

  19. [27]

    H. Lin, H. Xu, Y. Wu, J. Cui, Y. Zhang, L. Mou, L. Song, Z. Sun, and Y. Wei. DuQuant : Distributing outliers via dual transformation makes stronger quantized LLMs . In Advances in Neural Information Processing Systems, 2024 a . URL https://openreview.net/forum?id=mp8u2Pcmqz

    2024

  20. [28]

    J. Lin, J. Tang, H. Tang, S. Yang, W.-M. Chen, W.-C. Wang, G. Xiao, X. Dang, C. Gan, and S. Han. AWQ : Activation-aware weight quantization for on-device LLM compression and acceleration. In Proceedings of Machine Learning and Systems, pages 87--100, 2024 b . URL https://arxiv...

    2024 arXiv

  21. [29]

    Z. Lin, M. Courbariaux, R. Memisevic, and Y. Bengio. Neural networks with few multiplications. In International Conference on Learning Representations, 2016. URL https://arxiv.org/abs/1510.03009

    2016 arXiv

  22. [30]

    F. Liu, W. Zhao, Z. He, Y. Wang, Z. Wang, C. Dai, X. Liang, and L. Jiang. Improving neural network efficiency via post-training quantization with adaptive floating-point. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5281--5290, 2021. URL ht...

    2021

  23. [31]

    Z. Liu, C. Zhao, I. Fedorov, B. Soran, D. Choudhary, R. Krishnamoorthi, V. Chandra, Y. Tian, and T. Blankevoort. SpinQuant : LLM quantization with learned rotations. arXiv preprint arXiv:2405.16406, 2024. URL https://arxiv.org/abs/2405.16406

    2024 arXiv

  24. [32]

    Merity, C

    S. Merity, C. Xiong, J. Bradbury, and R. Socher. Pointer sentinel mixture models. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=Byj72udxe

    2022

  25. [33]

    Micikevicius, S

    P. Micikevicius, S. Narang, J. Alben, G. Diamos, E. Elsen, D. Garcia, B. Ginsburg, M. Houston, O. Kuchaiev, G. Venkatesh, et al. Mixed precision training. In International Conference on Learning Representations, 2018. URL https://openreview.net/forum?id=r1gs9JgRZ

    2018

  26. [34]

    Introducing C hat GPT , 2022

    OpenAI. Introducing C hat GPT , 2022. URL https://openai.com/index/chatgpt/

    2022

  27. [35]

    Sakaguchi, R

    K. Sakaguchi, R. Le Bras, C. Bhagavatula, and Y. Choi. Wino G rande: An adversarial Winograd Schema Challenge at scale. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 8732--8740, 2020. URL https://ojs.aaai.org/index.php/AAAI/article/view/6399

    2020

  28. [36]

    W. Shao, M. Chen, Z. Zhang, P. Xu, L. Zhao, Z. Li, K. Zhang, P. Gao, Y. Qiao, and P. Luo. OmniQuant : Omnidirectionally calibrated quantization for large language models. In International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=8Wuvhh0LYW

    2024

  29. [37]

    J. Su, M. Ahmed, Y. Lu, S. Pan, W. Bo, and Y. Liu. RoFormer : Enhanced transformer with rotary position embedding. Neurocomputing, 568: 0 127063, 2024. URL https://www.sciencedirect.com/science/article/pii/S0925231223011864

    2024

  30. [38]

    S. Sun, Y. Cheng, Z. Gan, and J. Liu. Patient knowledge distillation for BERT model compression. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing, pages 4323--4332, 2019 a...

    2019

  31. [39]

    X. Sun, J. Choi, C.-Y. Chen, N. Wang, S. Venkataramani, V. V. Srinivasan, X. Cui, W. Zhang, and K. Gopalakrishnan. Hybrid 8-bit floating point ( HFP8 ) training and inference for deep neural networks. In Advances in Neural Information Processing Systems, 2019 b . URL https://p...

    2019

  32. [40]

    X. Sun, N. Wang, C.-Y. Chen, J. Ni, A. Agrawal, X. Cui, S. Venkataramani, K. El Maghraoui, V. V. Srinivasan, and K. Gopalakrishnan. Ultra-low precision 4-bit training of deep neural networks. In Advances in Neural Information Processing Systems, pages 1796--1807, 2020. URL htt...

    2020

  33. [41]

    Tseng, J

    A. Tseng, J. Chee, Q. Sun, V. Kuleshov, and C. De Sa. QuIP \#: Even better LLM quantization with H adamard incoherence and lattice codebooks. In Proceedings of the International Conference on Machine Learning, 2024. URL https://proceedings.mlr.press/v235/tseng24a.html

    2024

  34. [42]

    N. Wang, J. Choi, D. Brand, C.-Y. Chen, and K. Gopalakrishnan. Training deep neural networks with 8-bit floating point numbers. In Advances in Neural Information Processing Systems, 2018. URL https://proceedings.neurips.cc/paper_files/paper/2018/file/335d3d1cd7ef05ec77714a2151...

    2018

  35. [43]

    X. Wei, Y. Zhang, Y. Li, X. Zhang, R. Gong, J. Guo, and X. Liu. Outlier S uppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 1...

    2023

  36. [44]

    H. Wu, P. Judd, X. Zhang, M. Isaev, and P. Micikevicius. Integer quantization for deep learning inference: Principles and empirical evaluation. arXiv preprint arXiv:2004.09602, 2020. URL https://arxiv.org/abs/2004.09602

    2004 arXiv

  37. [45]

    G. Xiao, J. Lin, M. Seznec, H. Wu, J. Demouth, and S. Han. S mooth Q uant: Accurate and efficient post-training quantization for large language models. In Proceedings of the International Conference on Machine Learning, pages 38087--38099, 2023. URL https://proceedings.mlr.pre...

    2023

  38. [46]

    Z. Yuan, L. Niu, J. Liu, W. Liu, X. Wang, Y. Shang, G. Sun, Q. Wu, J. Wu, and B. Wu. RPTQ : Reorder-based post-training quantization for large language models. arXiv preprint arXiv:2304.01089, 2023. URL https://arxiv.org/abs/2304.01089

    2023 arXiv

  39. [47]

    Zellers, A

    R. Zellers, A. Holtzman, Y. Bisk, A. Farhadi, and Y. Choi. H ella S wag: Can a machine really finish your sentence? In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 4791--4800, 2019. URL https://aclanthology.org/P19-1472/

    2019

  40. [48]

    Zhang, S

    S. Zhang, S. Roller, N. Goyal, M. Artetxe, M. Chen, S. Chen, C. Dewan, M. Diab, X. Li, X. V. Lin, et al. OPT : Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068, 2022. URL https://arxiv.org/abs/2205.01068

    2022 arXiv

  41. [49]

    S. Zhou, Y. Wu, Z. Ni, X. Zhou, H. Wen, and Y. Zou. Do R e F a- N et: Training low bitwidth convolutional neural networks with low bitwidth gradients. arXiv preprint arXiv:1606.06160, 2016. URL https://arxiv.org/abs/1606.06160

    2016 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.