REVIEW 3 cited by
PARQ: Piecewise-Affine Regularized Quantization
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We develop a principled method for quantization-aware training (QAT) of large-scale machine learning models. Specifically, we show that convex, piecewise-affine regularization (PAR) can effectively induce the model parameters to cluster towards discrete values. We minimize PAR-regularized loss functions using an aggregate proximal stochastic gradient method (AProx) and prove that it has last-iterate convergence. Our approach provides an interpretation of the straight-through estimator (STE), a widely used heuristic for QAT, as the asymptotic form of PARQ. We conduct experiments to demonstrate that PARQ obtains competitive performance on convolution- and transformer-based vision tasks.
Forward citations
Cited by 3 Pith papers
-
Beyond Discreteness: Sample Complexity Analysis of Straight-Through Estimator for 1-bit Quantization
For a two-layer binary network with Gaussian inputs, O(n^2) samples guarantee ergodic convergence of STE training and O(n^4) guarantee that iterates revisit the optimal weights, even under label noise.
-
CAGE: Curvature-Aware Gradient Estimation For Accurate Quantization-Aware Training
CAGE, a curvature-aware correction that adds the quantization error to the gradient, reduces loss in low-bit quantization-aware training, letting 3-bit CAGE-trained models match 4-bit baseline-trained models.
-
TorchAO: PyTorch-Native Training-to-Serving Model Optimization
TorchAO delivers a PyTorch-native, end-to-end workflow for FP8 training, QAT, PTQ, and sparsity, with benchmarked speedups and production use in quantized Llama releases.
Discussion (0). Sign in to comment.