Pith. sign in

REVIEW 11 cited by

OneBit: Towards Extremely Low-bit Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2402.11295 v6 pith:V3E74MD6 submitted 2024-02-17 cs.CL

Yuzhuang Xu , Xu Han , Zonghan Yang , Shuo Wang , Qingfu Zhu , Zhiyuan Liu , Weidong Liu , Wanxiang Che This is my paper · ORCID
classification cs.CL
keywords llmsmodelsbit-widthextremelymatricesonebitperformanceweight
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Model quantification uses low bit-width values to represent the weight matrices of existing models to be quantized, which is a promising approach to reduce both storage and computational overheads of deploying highly anticipated LLMs. However, current quantization methods suffer severe performance degradation when the bit-width is extremely reduced, and thus focus on utilizing 4-bit or 8-bit values to quantize models. This paper boldly quantizes the weight matrices of LLMs to 1-bit, paving the way for the extremely low bit-width deployment of LLMs. For this target, we introduce a 1-bit model compressing framework named OneBit, including a novel 1-bit parameter representation method to better quantize LLMs as well as an effective parameter initialization method based on matrix decomposition to improve the convergence speed of the quantization framework. Sufficient experimental results indicate that OneBit achieves good performance (at least 81% of the non-quantized performance on LLaMA models) with robust training processes when only using 1-bit weight matrices.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Anda: Unlocking Efficient LLM Inference with a Variable-Length Grouped Activation Data Format

    cs.AR 2024-11 conditional novelty 7.0 of 10

    Anda stores LLM activations in a group-shared-exponent format with per-module mantissa widths and uses bit-serial hardware that skips unused bits, reporting roughly 2.4x speedup, 4.0x area efficiency, and 3.1x energy ...

  2. From Sweep to Seam: Interleaved Cross-Block Post-Training Quantization

    cs.AI 2026-08 conditional novelty 6.0 of 10

    Interleaving cross-block quantization so seam pairs are refined twice reduces perplexity and prevents divergence in low-bit LLM compression, with a conditional bound showing seams multiply an error contraction factor.

  3. LiquidGEMM: Hardware-Efficient W4A8 GEMM Kernel for High-Performance LLM Serving

    cs.DC 2025-09 conditional novelty 6.0 of 10

    A W4A8 GEMM kernel using shift-based overflow-safe dequantization and a hardware-scheduled pipeline reports up to 2.9x speedup over prior W4A8 kernels.

  4. GeLaCo: An Evolutionary Approach to Layer Compression

    cs.CL 2025-07 conditional novelty 6.0 of 10

    Evolutionary search over layer-merging configurations, scored by module-wise activation similarity, yields competitive LLM compression and the first size-quality Pareto fronts.

  5. CCQ: Convolutional Code for Extreme Low-bit Quantization in LLMs

    cs.LG 2025-07 conditional novelty 6.0 of 10

    CCQ compresses LLMs to 2.0-2.75 bits per weight using convolutional codes and bit-shift decoding, shrinking 671B-parameter models to under 200GB.

  6. Q-resafe: Assessing Safety Risks and Quantization-aware Safety Patching for Quantized Large Language Models

    cs.LG 2025-06 conditional novelty 6.0 of 10

    Q-resafe restores much of the safety lost in quantized LLMs by distilling the original model's responses through DPO while selectively updating only safety-critical weights.

  7. RoSTE: An Efficient Quantization-Aware Supervised Fine-Tuning Approach for Large Language Models

    cs.LG 2025-02 conditional novelty 6.0 of 10

    RoSTE couples quantization-aware supervised fine-tuning with per-layer Hadamard rotation selection, reducing quantization outliers and improving 4-bit quantized LLM accuracy over SFT-then-PTQ baselines.

  8. Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMs

    cs.AI 2025-08 conditional novelty 5.0 of 10

    The study introduces TruthfulnessEval and reports that 4-bit quantization preserves simple true/false accuracy, but explicit 'lie' prompts make quantized and full-precision LLMs output falsehoods even when internal pr...

  9. APT-LLM: Exploiting Arbitrary-Precision Tensor Core Computing for LLM Acceleration

    cs.LG 2025-08 conditional novelty 5.0 of 10

    A bit-sliced Tensor Core scheme with a bipolar integer encoding and adaptive kernel autotuning reports 1.65-3.99x speedups for ultra-low-bit GPTQ-quantized LLM inference on three NVIDIA GPUs.

  10. Cross-Layer Error Compensation and Finite-Sample Feature-Statistics Matching for Extreme Low-Bit Quantization of Large Language Models

    cs.NE 2026-07 conditional novelty 4.0 of 10

    A whole-network final-feature and statistics matching objective improves 1.125-bit and 4.125-bit LLM quantization over layer-local and distillation baselines, but its cross-layer mechanism reduces exactly to final-fea...

  11. Hamming Attention Distillation: Binarizing Keys and Queries for Efficient Long-Context Transformers

    cs.LG 2025-02 conditional novelty 4.0 of 10

    Hamming Attention Distillation binarizes transformer keys and queries to +1/-1 and prunes attention to the top N links, reporting single-point accuracy losses and large simulated hardware savings.

Pith tools