REVIEW 16 cited by
BiLLM: Pushing the Limit of Post-Training Quantization for LLMs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Pretrained large language models (LLMs) exhibit exceptional general language processing capabilities but come with significant demands on memory and computational resources. As a powerful compression technology, binarization can extremely reduce model weights to a mere 1 bit, lowering the expensive computation and memory requirements. However, existing quantization techniques fall short of maintaining LLM performance under ultra-low bit-widths. In response to this challenge, we present BiLLM, a groundbreaking 1-bit post-training quantization scheme tailored for pretrained LLMs. Based on the weight distribution of LLMs, BiLLM first identifies and structurally selects salient weights, and minimizes the compression loss through an effective binary residual approximation strategy. Moreover, considering the bell-shaped distribution of the non-salient weights, we propose an optimal splitting search to group and binarize them accurately. BiLLM achieving for the first time high-accuracy inference (e.g. 8.41 perplexity on LLaMA2-70B) with only 1.08-bit weights across various LLMs families and evaluation metrics, outperforms SOTA quantization methods of LLM by significant margins. Moreover, BiLLM enables the binarization process of the LLM with 7 billion weights within 0.5 hours on a single GPU, demonstrating satisfactory time efficiency. Our code is available at https://github.com/Aaronhuang-778/BiLLM.
Forward citations
Cited by 16 Pith papers
-
Event-Priori-Based Vision-Language Model for Efficient Visual Understanding
EP-VLM uses event-camera motion data to sparsify image patches before a vision-language model processes them, cutting FLOPs by about half with a small accuracy drop.
-
QoS-Efficient Serving of Multiple Mixture-of-Expert LLMs Using Partial Runtime Reconfiguration
A single-GPU serving system for multiple fine-tuned MoE LLMs achieves near-single-model throughput by sharing similar experts and reconfiguring non-expert layers at runtime.
-
LightNobel: Improving Sequence Length Limitation in Protein Structure Prediction Model via Adaptive Activation Quantization
LightNobel combines token-wise adaptive activation quantization with a custom accelerator to run protein structure prediction on sequences up to about 10,000 amino acids within 80GB memory.
-
FBQuant: FeedBack Quantization for Large Language Models
FBQuant redefines sub-branch compensation as Q(W - Sigma) + Sigma, bounding per-weight reconstruction error by half the quantizer step and improving 3-bit LLM accuracy.
-
BiDM: Pushing the Limit of Quantization for Diffusion Models
BiDM achieves the first fully binarized (W1A1) diffusion model, with FID 22.74 on LSUN-Bedrooms, by adding timestep-aware binary structures and patch-based distillation.
-
Efficient LLM Inference using Dynamic Input Pruning and Cache-Aware Masking
Dynamic Input Pruning with cache-aware masking cuts simulated DRAM footprint and raises token throughput on SwiGLU LLMs at small perplexity cost.
-
Low-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokens
The paper derives a scaling law for quantization-induced loss increase as a function of model size, training tokens, and bit width, and uses it to argue that low-bit quantization will hurt future fully trained LLMs.
-
LUQ: Layerwise Ultra-Low Bit Quantization for Multimodal Large Language Models
LUQ assigns ultra-low bit widths to transformer layers whose output activations have low entropy, cutting multimodal LLM memory by up to 40 percent with modest benchmark loss.
-
Compress Any Segment Anything Model (SAM)
Birkhoff compresses 18 SAM variants to about one-fifth their size with less than 1% accuracy loss, data-free, using a trajectory-based codebook and a fused GPU operator.
-
BiVM: Accurate Binarized Neural Network for Efficient Video Matting
BiVM is a 1-bit binarized video matting network that beats prior binarized methods on accuracy and efficiency, with 11.82 MAD on VideoMatte240K versus 28.49 for ReActNet-binarized RVM.
-
BBAL: A Bidirectional Block Floating Point-Based Quantisation Accelerator for Large Language Models
A flag-bit block floating point format reduces quantization error versus BFP and enables a hardware accelerator for LLM linear and nonlinear layers.
-
Cross-Layer Error Compensation and Finite-Sample Feature-Statistics Matching for Extreme Low-Bit Quantization of Large Language Models
A whole-network final-feature and statistics matching objective improves 1.125-bit and 4.125-bit LLM quantization over layer-local and distillation baselines, but its cross-layer mechanism reduces exactly to final-fea...
-
BAQ: Efficient Bit Allocation Quantization for Large Language Models
BAQ assigns per-column bitwidths to LLM weights by solving a convex quantization-loss problem, and reports large 2-bit perplexity gains over GPTQ.
-
An Empirical Study of Qwen3 Quantization
Qwen3 stays accurate down to 4-bit weights but degrades sharply at 3 bits and below, and appears more quantization-sensitive than LLaMA3.
-
Efficient Strategy for Improving Large Language Model (LLM) Capabilities
Proposes a combined data-selection, training-strategy, and architecture-adjustment approach to improve LLM capability under resource constraints.
-
Resource-Efficient Language Models: Quantization for Fast and Accessible Inference
A survey of post-training quantization techniques for large language models, covering schemes, granularities, and popular methods, with no new experimental results.
Discussion (0). Continue with ORCID to comment.