Pith. sign in

REVIEW 18 cited by

QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2402.04396 v2 pith:XAEMDIAU submitted 2024-02-06 cs.LG cs.AIcs.CL

Albert Tseng , Jerry Chee , Qingyao Sun , Volodymyr Kuleshov , Christopher De Sa This is my paper · ORCID
classification cs.LGcs.AIcs.CL
keywords quipquantizationachievesbettercodebookshadamardincoherenceintroduce
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
abstract

Post-training quantization (PTQ) reduces the memory footprint of LLMs by quantizing their weights to low-precision. In this work, we introduce QuIP#, a weight-only PTQ method that achieves state-of-the-art results in extreme compression regimes ($\le$ 4 bits per weight) using three novel techniques. First, QuIP# improves QuIP's (Chee et al., 2023) incoherence processing by using the randomized Hadamard transform, which is faster and has better theoretical properties. Second, QuIP# uses vector quantization to take advantage of the ball-shaped sub-Gaussian distribution that incoherent weights possess: specifically, we introduce a set of hardware-efficient codebooks based on the highly symmetric $E_8$ lattice, which achieves the optimal 8-dimension unit ball packing. Third, QuIP# uses fine-tuning to improve fidelity to the original model. Our experiments show that QuIP# outperforms existing PTQ methods, enables new behaviors in PTQ scaling, and supports fast inference. Our code can be found at https://github.com/Cornell-RelaxML/quip-sharp.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 18 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FPTQuant: Function-Preserving Transforms for LLM Quantization

    cs.LG 2025-06 conditional novelty 7.0 of 10

    FPTQuant introduces function-preserving transforms that make transformer activations amenable to static 4-bit quantization with minimal inference overhead.

  2. Leech Lattice Vector Quantization for Efficient LLM Compression

    cs.LG 2026-03 conditional novelty 6.5 of 10

    Leech-lattice vector quantization with codebook-free indexing and shape-gain search reaches state-of-the-art ~2-bit post-training LLM compression and reduces reliance on Hadamard rotations.

  3. TaskPress: Query-Agnostic KV Cache Compression via Task-Guided Pruning

    cs.AI 2026-08 conditional novelty 6.0 of 10

    A task guide acts as a meta-query to compress the KV cache once per task, so one compressed memory serves many future questions.

  4. GyRot: Leveraging Hidden Synergy between Rotation and Fine-grained Group Quantization for Low-bit LLM Inference

    cs.AR 2026-07 conditional novelty 6.0 of 10

    Rotation and fine-grained group quantization can work together if rotation spans several quantization groups and outlier channels are permuted onto harmonic Hadamard rows, enabling 4-bit LLM inference with integer-onl...

  5. A Motion-Aware Vector Quantization Framework with Centroid Reuse for Efficient VLA Inference

    cs.AI 2026-07 conditional novelty 6.0 of 10

    Motion-aware vector quantization plus centroid-reuse GEMM and a custom accelerator yield multi-fold VLA speedups over GPU and prior accelerators with ~2.5% average success-rate drop.

  6. Reliability Scaling Laws for Quantized Large Language Models

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Reliability of quantized LLMs peaks nonlinearly at 4-bit precision under fixed total model bits, while accuracy scales monotonically, and quantization can improve robustness to natural perturbations.

  7. FAIR-Calib: Frontier-Aware Instability-Reweighted Calibration for Post-Training Quantization of Diffusion Large Language Models

    cs.LG 2026-06 unverdicted novelty 6.0 of 10

    FAIR-Calib is a frontier-aware instability-reweighted calibration framework for PTQ of dLLMs that minimizes reweighted hidden-state MSE to reduce frontier decision flips.

  8. Price of metric universality in vector quantization is at most 0.11 bit

    cs.IT 2026-02 conditional novelty 6.0 of 10

    A universal vector-quantization codebook exists that is within 0.11 bit/coordinate of covariance-adaptive waterfilling simultaneously for all input covariances, for Gaussian weights.

  9. SALAAD: Sparse And Low-Rank Adaptation via ADMM for Large Language Model Inference

    cs.LG 2026-02 conditional novelty 6.0 of 10

    SALAAD uses an ADMM-style optimizer and an adaptive controller to force LLM weights into sparse-plus-low-rank form during pretraining, so one checkpoint can be elastically deployed at many parameter budgets.

  10. From Segments to Scenes: Temporal Understanding for Agentic Autonomous Driving via Vision-Language Models

    cs.CV 2025-12 conditional novelty 6.0 of 10

    TAD, a 5,861-question benchmark, shows VLMs score far below humans on temporal understanding of driving videos, and an ego-trajectory text summary (TCogMap) substantially boosts their scores.

  11. CCQ: Convolutional Code for Extreme Low-bit Quantization in LLMs

    cs.LG 2025-07 conditional novelty 6.0 of 10

    CCQ compresses LLMs to 2.0-2.75 bits per weight using convolutional codes and bit-shift decoding, shrinking 671B-parameter models to under 200GB.

  12. PCDVQ: Enhancing Vector Quantization for Large Language Models via Polar Coordinate Decoupling

    cs.LG 2025-06 conditional novelty 6.0 of 10

    PCDVQ compresses LLM weights to 2 bits by quantizing vector directions and magnitudes separately with distribution-matched codebooks, reporting modest zero-shot accuracy gains over prior vector quantization baselines.

  13. BASE-Q: Bias and Asymmetric Scaling Enhanced Rotational Quantization for Large Language Models

    cs.LG 2025-05 conditional novelty 6.0 of 10

    BASE-Q combines bias correction and asymmetric scaling under fixed rotations to improve 4-bit weight-activation quantization, narrowing the accuracy gap to full precision by up to 50.5% over prior rotation-based methods.

  14. RoSTE: An Efficient Quantization-Aware Supervised Fine-Tuning Approach for Large Language Models

    cs.LG 2025-02 conditional novelty 6.0 of 10

    RoSTE couples quantization-aware supervised fine-tuning with per-layer Hadamard rotation selection, reducing quantization outliers and improving 4-bit quantized LLM accuracy over SFT-then-PTQ baselines.

  15. Break Through the Compression Bottleneck: From Theory to Practice

    cs.CL 2026-05 reject novelty 5.0 of 10

    The paper asserts a first proof that low-rank decomposition and quantization are non-orthogonal tools for LLM compression, recommends low-rank-first ordering, and adds a diagonal scaling fix (DAM) that reduces the com...

  16. LCD: Advancing Extreme Low-Bit Clustering for Large Language Models via Knowledge Distillation

    cs.LG 2025-05 reject novelty 5.0 of 10

    LCD clusters LLM weights into tiny codebooks under a Hessian-guided objective and uses lookup-table inference to reach 2-3 bits, with reported speedups up to 6.2x.

  17. Cross-Layer Error Compensation and Finite-Sample Feature-Statistics Matching for Extreme Low-Bit Quantization of Large Language Models

    cs.NE 2026-07 conditional novelty 4.0 of 10

    A whole-network final-feature and statistics matching objective improves 1.125-bit and 4.125-bit LLM quantization over layer-local and distillation baselines, but its cross-layer mechanism reduces exactly to final-fea...

  18. Rethinking the Outlier Distribution in Large Language Models: An In-depth Study

    cs.CL 2025-05 reject novelty 4.0 of 10

    The paper claims that most massive activations in LLMs are residual-propagated 'fake' outliers that can be removed without accuracy loss.

Pith tools