REVIEW 14 cited by
Extreme Compression of Large Language Models via Additive Quantization
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The emergence of accurate open large language models (LLMs) has led to a race towards performant quantization techniques which can enable their execution on end-user devices. In this paper, we revisit the problem of "extreme" LLM compression-defined as targeting extremely low bit counts, such as 2 to 3 bits per parameter-from the point of view of classic methods in Multi-Codebook Quantization (MCQ). Our algorithm, called AQLM, generalizes the classic Additive Quantization (AQ) approach for information retrieval to advance the state-of-the-art in LLM compression, via two innovations: 1) learned additive quantization of weight matrices in input-adaptive fashion, and 2) joint optimization of codebook parameters across each transformer blocks. Broadly, AQLM is the first scheme that is Pareto optimal in terms of accuracy-vs-model-size when compressing to less than 3 bits per parameter, and significantly improves upon all known schemes in the extreme compression (2bit) regime. In addition, AQLM is practical: we provide fast GPU and CPU implementations of AQLM for token generation, which enable us to match or outperform optimized FP16 implementations for speed, while executing in a much smaller memory footprint.
Forward citations
Cited by 14 Pith papers
-
FPTQuant: Function-Preserving Transforms for LLM Quantization
FPTQuant introduces function-preserving transforms that make transformer activations amenable to static 4-bit quantization with minimal inference overhead.
-
A Motion-Aware Vector Quantization Framework with Centroid Reuse for Efficient VLA Inference
Motion-aware vector quantization plus centroid-reuse GEMM and a custom accelerator yield multi-fold VLA speedups over GPU and prior accelerators with ~2.5% average success-rate drop.
-
Reliability Scaling Laws for Quantized Large Language Models
Reliability of quantized LLMs peaks nonlinearly at 4-bit precision under fixed total model bits, while accuracy scales monotonically, and quantization can improve robustness to natural perturbations.
-
Breaking the Rounding Trap: Securing LLMs against Quantization-Conditioned Backdoors
QuantGuard uses differentiable rounding control to break quantization-boundary backdoors in LLMs, reducing post-quantization attack success to clean-model levels across six models and INT8/FP4/NF4.
-
Smaller Models, Unexpected Costs: Trade-offs in LLM Quantization for Automated Program Repair
Empirical evaluation of 13 quantization configurations on 6 LLMs for APR shows reduced memory (up to 85%) but increased inference time/energy, different repaired problem sets with little overlap, and 48% of configs st...
-
SALAAD: Sparse And Low-Rank Adaptation via ADMM for Large Language Model Inference
SALAAD uses an ADMM-style optimizer and an adaptive controller to force LLM weights into sparse-plus-low-rank form during pretraining, so one checkpoint can be elastically deployed at many parameter budgets.
-
From Segments to Scenes: Temporal Understanding for Agentic Autonomous Driving via Vision-Language Models
TAD, a 5,861-question benchmark, shows VLMs score far below humans on temporal understanding of driving videos, and an ego-trajectory text summary (TCogMap) substantially boosts their scores.
-
CCQ: Convolutional Code for Extreme Low-bit Quantization in LLMs
CCQ compresses LLMs to 2.0-2.75 bits per weight using convolutional codes and bit-shift decoding, shrinking 671B-parameter models to under 200GB.
-
Q-resafe: Assessing Safety Risks and Quantization-aware Safety Patching for Quantized Large Language Models
Q-resafe restores much of the safety lost in quantized LLMs by distilling the original model's responses through DPO while selectively updating only safety-critical weights.
-
PCDVQ: Enhancing Vector Quantization for Large Language Models via Polar Coordinate Decoupling
PCDVQ compresses LLM weights to 2 bits by quantizing vector directions and magnitudes separately with distribution-matched codebooks, reporting modest zero-shot accuracy gains over prior vector quantization baselines.
-
NSNQuant: A Double Normalization Approach for Calibration-Free Low-Bit Vector Quantization of KV Cache
NSNQuant applies a Normalize-Shift-Normalize transform plus a Hadamard rotation to make KV cache channels match a standard normal distribution, so one codebook trained on random noise can quantize them without calibration.
-
RoSTE: An Efficient Quantization-Aware Supervised Fine-Tuning Approach for Large Language Models
RoSTE couples quantization-aware supervised fine-tuning with per-layer Hadamard rotation selection, reducing quantization outliers and improving 4-bit quantized LLM accuracy over SFT-then-PTQ baselines.
-
Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMs
The study introduces TruthfulnessEval and reports that 4-bit quantization preserves simple true/false accuracy, but explicit 'lie' prompts make quantized and full-precision LLMs output falsehoods even when internal pr...
-
The Hitchhiker's Guide to Agentic AI: From Foundations to Systems
A survey-style reference book mapping the full agentic-AI stack from transformer internals to production deployment, with no new research result.
Discussion (0). Continue with ORCID to comment.