Pith. sign in

REVIEW 24 cited by

A Survey of Quantization Methods for Efficient Neural Network Inference

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2103.13630 v3 pith:7RPBKFQU submitted 2021-03-25 cs.CV

Amir Gholami , Sehoon Kim , Zhen Dong , Zhewei Yao , Michael W. Mahoney , Kurt Keutzer This is my paper
classification cs.CV
keywords computationsneuralproblemquantizationefficientnetworknumericalresearch
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

As soon as abstract mathematical computations were adapted to computation on digital computers, the problem of efficient representation, manipulation, and communication of the numerical values in those computations arose. Strongly related to the problem of numerical representation is the problem of quantization: in what manner should a set of continuous real-valued numbers be distributed over a fixed discrete set of numbers to minimize the number of bits required and also to maximize the accuracy of the attendant computations? This perennial problem of quantization is particularly relevant whenever memory and/or computational resources are severely restricted, and it has come to the forefront in recent years due to the remarkable performance of Neural Network models in computer vision, natural language processing, and related areas. Moving from floating-point representations to low-precision fixed integer values represented in four bits or less holds the potential to reduce the memory footprint and latency by a factor of 16x; and, in fact, reductions of 4x to 8x are often realized in practice in these applications. Thus, it is not surprising that quantization has emerged recently as an important and very active sub-area of research in the efficient implementation of computations associated with Neural Networks. In this article, we survey approaches to the problem of quantizing the numerical values in deep Neural Network computations, covering the advantages/disadvantages of current methods. With this survey and its organization, we hope to have presented a useful snapshot of the current research in quantization for Neural Networks and to have given an intelligent organization to ease the evaluation of future research in this area.

Discussion (0). Sign in to comment.

Forward citations

Cited by 24 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 39 citations worldwide. Full citation record

  1. When Bits Break Recourse: Counterfactual-Faithful Quantization

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    Quantization can preserve accuracy while invalidating algorithmic recourse; CFQ trains the quantized model at teacher recourse points and preserves counterfactual validity and recourse cost.

  2. No Certificate, No Categorical Speech Act: A Brouwerian Assertibility Constraint for Public Reason

    cs.CY 2026-03 unverdicted novelty 7.0 of 10

    An AI may assert or deny high-stakes claims only when it can exhibit a publicly contestable certificate; otherwise it is obligated to return Undetermined.

  3. When Do Neural Networks Learn World Models?

    cs.LG 2025-02 conditional novelty 7.0 of 10

    With Boolean variables, a low-degree bias, and a task distribution weighted toward simple functions of the latents, multi-task training provably recovers the latent world model up to permutations and negations.

  4. ExaGEMM: Exploration Framework for CPU-Driven ML Inference via Associative In-Register Computing for Low-Bit GEMM

    cs.AR 2026-07 conditional novelty 6.0 of 10

    ExaGEMM co-explores lightweight CPU ISA support and lookup-table GEMM kernels for 1/2/4-bit ML inference, prunes ~99% of candidates, and reports up to 13.3x simulated speedups over software.

  5. Lyapunov-Guided Training for Hardware-Safe Neural Networks Under Fixed-Point Arithmetic

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Monotone Lyapunov projection of layerwise hidden-state energy suppresses two's-complement overflow under wrapping fixed-point QAT/PTQ, recovering 86.55% MNIST accuracy where unconstrained models collapse to chance.

  6. MoBiQuant: Mixture-of-Bits Quantization for Token-Adaptive Any-Precision LLM

    cs.LG 2026-02 conditional novelty 6.0 of 10

    MoBiQuant assigns each token its own quantization bit-width via learned routing over recursive residual bit slices, enabling elastic 2–6-bit LLM inference that matches static PTQ accuracy.

  7. You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations

    cs.CL 2025-11 reject novelty 6.0 of 10

    TAQ estimates per-layer importance from hidden representations and output sensitivity on task calibration data to allocate mixed precision in a training-free PTQ setting, outperforming task-agnostic baselines on accur...

  8. SQAP-VLA: A Synergistic Quantization-Aware Pruning Framework for High-Performance Vision-Language-Action Models

    cs.CV 2025-09 conditional novelty 6.0 of 10

    A training-free framework jointly quantizes a VLA model to 4 bits and prunes visual tokens, recovering or exceeding full-precision success rates at 1.93x speedup.

  9. Hardware Acceleration of Kolmogorov-Arnold Network (KAN) in Large-Scale Systems

    cs.AR 2025-09 conditional novelty 6.0 of 10

    A co-design of quantization, lookup-table sharing, and analog in-memory circuits lets large KAN recommendation models (39-63MB) scale with 28-41x area growth for 500K-807Kx parameter growth, with 0.11-0.23% accuracy l...

  10. Forget the Data and Fine-Tuning! Just Fold the Network to Compress

    cs.LG 2025-02 conditional novelty 6.0 of 10

    Model folding compresses a network by k-means clustering similar neurons across adjacent layers and repairing activation statistics without data (Fold-AR, Fold-DIR), surpassing prior data-free methods at high sparsity.

  11. Setup Once, Secure Always: A Single-Setup Secure Federated Learning Aggregation Protocol with Forward and Backward Secrecy for Dynamic Users

    cs.CR 2025-02 conditional novelty 6.0 of 10

    A single-setup secure aggregation protocol for federated learning that achieves forward and backward secrecy, dynamic user participation, and dropout tolerance using fresh random masks with a cyclic key-negation trick.

  12. Lightweight Image Classification of Raptor Species for Edge Devices: Rare-Species Dataset Expansion via Video Frame Extraction, Knowledge Distillation, and TensorRT Deployment

    cs.CV 2026-07 conditional novelty 5.0 of 10

    A three-student ensemble distilled from DINOv2-L reaches 0.935 macro recall on six raptor species with ~1/9 the teacher's parameters and runs at 313 images/s on an Orin Nano, with gains driven by rare-species data exp...

  13. PicoSAM3: Real-Time In-Sensor Region-of-Interest Segmentation

    cs.CV 2026-03 conditional novelty 5.0 of 10

    A 1.3M-parameter CNN with ROI-implicit prompting and SAM3 distillation reaches ~65% mIoU on COCO/LVIS and 11.82 ms INT8 inference fully in-sensor on the Sony IMX500.

  14. LCS: An AI-based Low-Complexity Scaler for Power-Efficient Super-Resolution of Game Content

    cs.CV 2025-07 conditional novelty 5.0 of 10

    An efficient neural upscaler trained on natively rendered game image pairs beats traditional upscalers on the LPIPS and NIQE perceptual metrics, at a fraction of the parameter count.

  15. BitTTS: Highly Compact Text-to-Speech Using 1.58-bit Quantization and Weight Indexing

    eess.AS 2025-06 conditional novelty 5.0 of 10

    Quantization-aware training with ternary weights plus base-3 weight indexing reduces a JETS/HiFi-GAN TTS model from 25.66 MB to 4.39 MB while keeping naturalness MOS around 3.1 to 3.3.

  16. Large Language Model Meets Constraint Propagation

    cs.CL 2025-05 conditional novelty 5.0 of 10

    Using BERT-like predictions to preview future positions reduces autoregressive LLM calls and increases the count of feasible solutions inside the GenCP constrained text generation framework, on the tasks tested.

  17. Is (Selective) Round-To-Nearest Quantization All You Need?

    cs.LG 2025-05 conditional novelty 5.0 of 10

    Selective round-to-nearest quantization, combined with Marlin-based kernels, matches or beats GPTQ, AWQ, and BitsAndBytes on accuracy and speed in most tested settings.

  18. MixFrag: Fragility-Guided Mixed-Precision Post-Training Quantization for Vision Transformers

    cs.CV 2026-07 conditional novelty 4.5 of 10

    KL-isolation fragility plus MCKP bit allocation yields mixed-precision ViT PTQ that lags recent ImageNet PTQ but reports large COCO AP gains at MP3/MP3.

  19. QScheduler: Adaptive Gradient Sampling for Zeroth-Order On-Device Training on INT8 NPUs

    cs.LG 2026-07 conditional novelty 4.0 of 10

    An adaptive sample-count scheduler for zeroth-order optimization matches fixed-q baselines on EuroSAT and STL-10 while avoiding q hyperparameter sweeps.

  20. Performance Analysis of Post-Training Quantization for CNN-based Conjunctival Pallor Anemia Detection

    eess.IV 2025-07 conditional novelty 4.0 of 10

    A MobileNet classifier reaches about 93% accuracy on the CP-AnemiC conjunctival pallor dataset, and FP16 post-training quantization preserves accuracy while INT8 and INT4 degrade it sharply.

  21. Design of an Edge-based Portable EHR System for Anemia Screening in Remote Health Applications

    cs.ET 2025-07 conditional novelty 4.0 of 10

    A compact edge EHR with a random forest model estimates hemoglobin from fingernail images with RMSE 1.969 g/dL and 79.2% sensitivity on a public 250-image dataset.

  22. TD-MPC-Opt: Distilling Model-Based Multi-Task Reinforcement Learning Agents

    cs.LG 2025-07 conditional novelty 4.0 of 10

    Reward-level distillation and FP16 quantization compress a 317M-parameter TD-MPC2 agent to 1M parameters, reaching 28.45 normalized score on MT30, though most of the gap over the original 18.93 comes from a longer tra...

  23. Can Post-Training Quantization Benefit from an Additional QLoRA Integration?

    cs.CL 2025-02 conditional novelty 4.0 of 10

    Adding QLoRA on top of a 4-bit post-training quantized LLM improves task accuracy over quantization alone, and can match or beat full 16-bit fine-tuning on some tasks.

  24. QPART: Adaptive Model Quantization and Dynamic Workload Balancing for Accuracy-aware Edge Inference

    cs.DC 2025-06 conditional novelty 3.0 of 10

    QPART chooses, for each edge inference request, a layer split point and per-layer bit widths that minimize time, energy, and server cost subject to an accuracy budget, cutting communication payload by over 80% with me...

Pith tools