Pith. sign in

REVIEW 13 cited by

Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2304.09145 v3 pith:WUAL7SQN submitted 2023-04-18 cs.CL

Xiuying Wei , Yunchen Zhang , Yuhang Li , Xiangguo Zhang , Ruihao Gong , Jinyang Guo , Xianglong Liu This is my paper · ORCID
classification cs.CL
keywords quantizationmodelschannel-wiseoutlierscalingshiftinglanguagesuppression
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Post-training quantization~(PTQ) of transformer language models faces significant challenges due to the existence of detrimental outliers in activations. We observe that these outliers are concentrated in specific channels and are asymmetric across channels. To address this issue, we propose the Outlier Suppression+~(OS+) framework, which contains the channel-wise shifting for asymmetry and channel-wise scaling for concentration. We show that these operations can be seamlessly migrated into subsequent modules while maintaining equivalence. Second, we propose a fast and stable scheme to calculate effective shifting and scaling values. The channel-wise shifting aligns the center of each channel for removal of outlier asymmetry. The channel-wise scaling quantitatively evaluates changes brought by migration and quantization for better quantization burden balance. We validate our OS+ under both standard and fine-grained quantization settings with models including BERT, OPT, BLOOM, BLOOMZ, and LLaMA. Comprehensive results across various tasks demonstrate the superiority of our approach. Especially, with standard quantization, OS+ can achieve near-floating-point performance on both small models and large language models on 8-bit and 6-bit. Besides, we establish a new state-of-the-art for 4-bit BERT with 15.5\% improvement. Our code is available at \url{https://github.com/ModelTC/Outlier_Suppression_Plus}.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 13 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FPTQuant: Function-Preserving Transforms for LLM Quantization

    cs.LG 2025-06 conditional novelty 7.0 of 10

    FPTQuant introduces function-preserving transforms that make transformer activations amenable to static 4-bit quantization with minimal inference overhead.

  2. DECA: A Near-Core LLM Decompression Accelerator Grounded on a 3D Roofline Model

    cs.AR 2025-05 conditional novelty 7.0 of 10

    A near-core decompression accelerator plus a 3D roofline model speeds up compressed LLM matrix multiplication by up to 4x in simulation.

  3. MXSens: Sensitivity-Aware Mixed-Precision Quantization for Efficient LLM Inference

    cs.LG 2026-07 conditional novelty 6.0 of 10

    MXSens allocates 8-bit precision to the 32 most sensitive columns per layer, 6-bit to moderately sensitive columns, and 4-bit elsewhere in MXINT, improving WikiText-2 perplexity over prior 4-bit LLM quantization methods.

  4. LiquidGEMM: Hardware-Efficient W4A8 GEMM Kernel for High-Performance LLM Serving

    cs.DC 2025-09 conditional novelty 6.0 of 10

    A W4A8 GEMM kernel using shift-based overflow-safe dequantization and a hardware-scheduled pipeline reports up to 2.9x speedup over prior W4A8 kernels.

  5. DMQ: Dissecting Outliers of Diffusion Models for Post-Training Quantization

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A post-training quantization method that combines learned channel scaling and power-of-two scaling keeps diffusion image quality high at 4-bit weight, 6-bit activation precision.

  6. Q-resafe: Assessing Safety Risks and Quantization-aware Safety Patching for Quantized Large Language Models

    cs.LG 2025-06 conditional novelty 6.0 of 10

    Q-resafe restores much of the safety lost in quantized LLMs by distilling the original model's responses through DPO while selectively updating only safety-critical weights.

  7. BASE-Q: Bias and Asymmetric Scaling Enhanced Rotational Quantization for Large Language Models

    cs.LG 2025-05 conditional novelty 6.0 of 10

    BASE-Q combines bias correction and asymmetric scaling under fixed rotations to improve 4-bit weight-activation quantization, narrowing the accuracy gap to full precision by up to 50.5% over prior rotation-based methods.

  8. Efficient Reasoning on the Edge

    cs.LG 2026-03 accept novelty 5.5 of 10

    LoRA adapters, budget-forced GRPO, dynamic switching, parallel verification and FPTQuant enable practical chain-of-thought reasoning on quantized Qwen2.5-7B for edge devices.

  9. PoTPTQ: A Two-step Power-of-Two Post-training for LLMs

    cs.CL 2025-07 conditional novelty 5.0 of 10

    A two-step power-of-two quantization method improves low-bit LLaMA perplexity and dequantization speed, but reporting inconsistencies and missing code temper the result.

  10. Quaff: Quantized Parameter-Efficient Fine-Tuning under Outlier Spatial Stability Hypothesis

    cs.LG 2025-05 conditional novelty 5.0 of 10

    Quaff shows that activation outlier channels keep their spatial positions during LLM fine-tuning, and exploits this stability to cut fine-tuning memory and latency with INT8 quantization while matching or beating full...

  11. Rethinking the Outlier Distribution in Large Language Models: An In-depth Study

    cs.CL 2025-05 reject novelty 4.0 of 10

    The paper claims that most massive activations in LLMs are residual-propagated 'fake' outliers that can be removed without accuracy loss.

  12. NQKV: A KV Cache Quantization Scheme Based on Normal Distribution Characteristics

    cs.LG 2025-05 conditional novelty 4.0 of 10

    NQKV is a block-wise 4-bit KV cache quantizer that uses a normal-distribution lookup table to let OPT models run with larger batches or longer contexts at negligible measured accuracy cost.

  13. KerZOO: Kernel Function Informed Zeroth-Order Optimization for Accurate and Accelerated LLM Fine-Tuning

    cs.LG 2025-05 reject novelty 3.0 of 10

    Kernel functions with vanishing third-moment conditions reduce the leading bias term in zeroth-order gradient estimates, yielding faster LLM fine-tuning than MeZO and HiZOO on several classification and generation benchmarks.

Pith tools