Pith. sign in

REVIEW 4 major objections 4 minor 53 references

Low-Dimensional High-Leverage Subspace Optimization: Beyond Full-Parameter Coupled Training for Neural Network Quantization

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Retuning only the normalization layer's per-channel scale and shift, all other weights frozen, restores collapsed 4-bit MobileNetV2 from 0.33% to 66.11% top-1 on ImageNet, beating a reconstruction-based quantizer.

desk verdict A clearly-written paper with a striking empirical claim about quantization-friendly BN affine tuning; the theory is neat but mismatched to the implementation, and the evaluations are single-run, so the central claim needs more evidence before I'd trust it. read the letter →

arxiv 2608.03919 v1 pith:3Q7EBGTC submitted 2026-08-04 cs.CV

Peng Xia , Junbiao Pang , Zheng Huang This is my paper · ORCID
classification cs.CV
keywords normalizationaffineparameterslow-bitquantizationpost-trainingquantization-awaretrainingsubspaceoptimizationbatchRMSNormknowledgedistillation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most low-bit quantization pipelines adapt every parameter at once, coupling weight, quantizer, and normalization updates into one optimization. This paper argues that the standard pipeline overlooks a decisive lever: the per-channel scale and shift of normalization layers. The proposed method, NAP, freezes all convolutional and linear weights and tunes only these affine parameters, on the same fake-quantized graph used at inference. On ImageNet MobileNetV2, the collapsed 4-bit baseline jumps from 0.33% to 66.11% top-1 accuracy (beating the 60.71% of a reconstruction-based quantizer) while updating about 1.4% of the trainable parameters. If the claim is right, a large share of quantization robustness lives in a subspace the field has been treating as an afterthought, and it can be bought almost for free.

What carries the argument

The central object is the normalization affine subspace Φ = {γ_l, β_l}: each channel of a normalized activation is rescaled and shifted by a_{l,c} = γ_{l,c}·h_{l,c} + β_{l,c}, so one or two scalars broadcast control over a whole channel across all spatial positions or tokens. Around it sits a three-way decomposition — backbone weights W define features, quantization parameters Ω define the discretization grid, and Φ regulates the channel-wise responses the quantizer sees — with NAP freezing W and updating only Φ, always under the same fake-quantized graph used for evaluation. The compensating identity is Eq. (13), floor(a_c) = α_c·a_c + δ_c + e_c: given per-channel affine error (α_c, δ_c), c

What would settle it

Disable the affine parameters entirely (fix γ=1, β=0 in every batch-norm layer) and rerun the NAP pipeline at W4A4 on ImageNet MobileNetV2; if accuracy still approaches 66%, the affine response subspace is not the causal mechanism. Complementarily, switch activation quantization from per-tensor to per-channel: the theorem predicts NAP's gain should grow if channel-wise affine error dominates, while flat or lower accuracy would refute the cancellation story.

Watch

Extended reading notes

Core claim

The paper's central claim: normalization affine parameters — γ and β in batch norm, γ alone in RMSNorm — form a low-dimensional response-control subspace with outsized leverage over quantization robustness. On a saturated QAT checkpoint, tuning only these 34.2K parameters lifts CIFAR-100 MobileNetV2 W4A4 from 68.21% to 69.83% while extra full QAT barely moves it. On ImageNet, RTN collapses MobileNetV2 to 0.33% at W4A4; NAP alone reaches 66.11%, beating QDrop's 60.71% without updating any conv or linear weight. The paper's local model: if a quantized pre-BN channel equals its affine map plus a residual, Eqs. (14)-(15) cancel the affine part exactly, leaving only nonlinear rounding and clippin

Load-bearing premise

The load-bearing premise is that each channel's quantization error decomposes into a channel-wise scale-and-offset term that batch-norm γ and β can cancel, but the implemented activation quantizer is per-tensor (one scale for all channels), so the theorem is a conditional existence result whose experimental setting only approximately matches it — and the headline ImageNet gain additionally assumes 20 training epochs over the full training set.

Editorial extensions

If this is right

  • Reconstruction-based PTQ gains a near-free preconditioner: on ImageNet W3A4, NAP+QDrop reaches 53.45% vs 51.75% for QDrop alone; on Cityscapes W3A4, 61.36% vs 58.21% mIoU.
  • Saturated QAT checkpoints are not locally optimal along the affine direction — NAP improves five of the tested W4A4 checkpoints (e.g., OOQ-Freeze 70.02→70.68) — so standard full-parameter training under-optimizes normalization responses.
  • Response adaptation and quantization-grid adaptation are complementary subspaces: alternating NAP-QAT on Qwen2.5-3B at W4A4 gives 166.65 perplexity on OSCAR vs 483.57 for AWQ-style scaling and 233.27 for grid-only tuning.
  • NAP adds no inference branch, keeps batch-norm folding intact, and tunes only ~1.4% of the parameters, at measured cost 1.0× vs 1.83× for full tuning.
  • The paper's own boundaries: compensation is backend-specific (mismatched graph costs 17.39 accuracy points), needs broad tuning-data coverage (10K ImageNet images give 3.88% vs 66.11% for the full set), and can conflict with specialized QAT checkpoints (W2A4 OOQ-Freeze drops 67.42→62.81).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The per-channel cancellation theorem is proved under a per-channel error model, but the ImageNet activation quantizer is per-tensor; the theorem is thus an existence proof for channel-wise controllers, and its fit to the per-tensor experiments is an empirical bridge the paper does not fully justify. Switching activation quantization to per-channel would test whether the affine mechanism actually c
  • The sharp data-dependence cliff (3.88% at 10K images) suggests NAP's bottleneck is distribution coverage, not parameter count; synthetic or distilled calibration data mimicking the full training set is a direct test of whether the subspace claim survives without the full dataset.
  • The OOQ-Freeze failure suggests a cheap selection rule for post-QAT NAP: measure the alignment between the channel-wise affine component of the quantization error and the normalization Jacobian's column space, and skip the affine stage when alignment is low.
  • The broadcast-control argument should extend to any per-channel affine structure inside a quantized graph — LayerNorm weights, or an inserted per-channel scaling before the quantizer; NAP's distinctive move is reusing parameters already present rather than adding new ones.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes Normalization Affine Preconditioning (NAP), which freezes backbone weights and tunes only normalization affine parameters (BN γ/β or RMSNorm γ) under the target fake-quantization graph. The central claim is that these parameters form a low-dimensional, high-leverage response-control subspace that can recover collapsed low-bit accuracy. Experiments on MobileNetV2/ImageNet report W4A4 accuracy rising from 0.33% (RTN) to 66.11% with NAP alone, outperforming QDrop; further experiments on CIFAR-100 QAT checkpoints, Cityscapes U-Net, and Qwen2.5-3B language modeling show improvements. The paper also provides a theoretical analysis (Eqs. 13–16) claiming that BN affine parameters cancel the channel-wise affine component of quantization distortion, with nonlinear residuals forming an irreducible boundary. Ablations examine parameter-subset comparisons, knowledge distillation, backend alignment, and data dependence.

Significance. If the empirical results hold, the observation that a tiny parameter subset (≈34K parameters) can rescue extremely low-bit models is practically valuable and conceptually interesting, especially because NAP adds no inference branch and can precede or follow existing PTQ/QAT methods. The paper's strengths include controlled equal-size parameter-subset comparisons (Table 5), a matched-versus-mismatched backend ablation (Table 6), honest reporting of failure cases (e.g., W2A4 OOQ-Freeze degradation in Table 2, small-data collapse in Table 7), and consistent results across classification, segmentation, and language modeling. However, the paper does not ship code, does not report error bars or multiple seeds, and its main theoretical justification does not match the implemented quantizer granularity. The empirical evidence is suggestive but currently incomplete; the theoretical explanation is not established.

major comments (4)
  1. [Method, 'Why Is the Affine Subspace High-Leverage?' Eqs. (13)–(16); Experimental Protocol] The cancellation derivation assumes a per-channel affine decomposition Q(a_c)=α_c a_c+δ_c+e_c with channel-wise constants α_c, δ_c. However, the Experimental Protocol states that activations are quantized asymmetrically and per-tensor. A per-tensor quantizer uses one scale and zero-point shared by all channels, so the quantization error of channel c cannot be written with independent channel-wise affine constants. Consequently, Eqs. (14)–(16) do not apply to the implemented graph, and the claim that BN affine parameters 'fully cancel the channel-wise affine component of quantization distortion' is unsupported as stated. The 'irreducible error boundary' is likewise a restatement of the assumption that e_c lies outside the affine subspace, not an independently derived bound. This is load-bearing because it is the paper's stated theoretical confirmation of the method's principle. Please eit
  2. [Algorithm 1 vs 'CNN optimization' in Experimental Protocol] Algorithm 1 fixes the quantization parameters Ω0 after calibration, but the protocol says 'observers are recalibrated after each tuning epoch.' These statements conflict. If quantization scales are updated during NAP, the comparison is not purely affine-parameter adaptation; quantizer-grid recalibration may by itself contribute to the observed gains. This confounds the attribution of improvements to the normalization affine subspace. Please clarify which procedure was actually used. If recalibration occurs, report an ablation with fixed Ω0 and another with recalibrated Ω, and state how observers are updated relative to the frozen BN running statistics.
  3. [Table 1 vs Table 7 and Experimental Protocol] The headline ImageNet comparison is not matched in data budget. QDrop uses 20K reconstruction iterations on a calibration set, while NAP is tuned for 20 epochs on the full ImageNet training set. Table 7 shows that NAP with 10K tuning images reaches only 3.88% W4A4, versus 66.11% with full data. Thus the comparison to QDrop conflates method with data access, and the paper's own ablation shows the current ImageNet procedure is not a calibration-only or few-shot PTQ method. Please report results under equal data budgets—for example, NAP with QDrop's calibration data, and QDrop with full-data tuning—or reposition NAP explicitly as a full-training-data post-processing/pre-conditioning method.
  4. [Experimental Protocol: Hardware, software, and runs] Every table entry is obtained from a single run with random seed 42. Given randomness in initialization, data order, stochastic QAT, and reconstruction procedures, the main numerical claims (e.g., 66.11 vs 60.71 in Table 1) cannot be assessed for statistical significance. The paper states that code will be released upon publication, but for a paper whose evidence is primarily empirical, the absence of code or repeated-seed results is a major limitation. Please provide mean ± std over at least three seeds for the key tables, or release code as part of the revision.
minor comments (4)
  1. [Throughout] The provided text has many missing spaces (e.g., 'networks,rooted', 'thea', 'Wealso') suggesting source corruption or broken line-breaking. Please fix the formatting in the camera-ready version.
  2. [Related Work and Table 4] Norm Tweaking is identified as the closest prior work for RMSNorm-based LLM quantization, but it is not compared in Table 4. A direct comparison with Norm Tweaking under the same protocol would strengthen the positioning and clarify the claimed distinction.
  3. [Figure 1] The 'matched extra QAT' and 'quantizer-only tuning' baselines are not described in detail. Please specify the number of parameters updated, hyperparameters, and number of epochs for these branches so that the diagnostic is reproducible.
  4. [Table 7] The W3A3 rows show 0.09–0.11 for 5K/10K samples versus 40.21 for full data; this complete collapse is informative but should be discussed explicitly. Is it due to clipping failure, optimization instability, or something else?

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: NAP's empirical claims are self-contained, and the affine-cancellation analysis is a conditional algebraic identity, not a circular fit.

full rationale

The paper's central empirical claim is that fine-tuning only normalization affine parameters under the target fake-quantized graph recovers collapsed low-bit accuracy. This claim is established by direct experiments against external baselines (QDrop, LSQ+, OOQ, StableQAT) and by controlled ablations (Table 5 matched parameter-subspace controls; Table 6 backend-mismatch ablation). No parameter is fitted to a subset and then renamed as a prediction; the ImageNet result is honestly disclosed as requiring 20 epochs on the full training set (Table 7), which is a data-efficiency limitation, not a circular step. The theoretical section (Eqs. 13-16) is a conditional statement: if a pre-BN activation's quantized form decomposes into a channel-wise affine component plus a residual e_c, then choosing gamma* and beta* as in Eqs. (14)-(15) removes the affine component and leaves a scaled residual. That is a valid deduction from the stated assumptions, not a circular reduction. The phrase "irreducible error boundary" is best read as irreducibility within the affine correction subspace, and the paper itself qualifies the result as "under this local model." The residual e_c is defined as the non-affine component, but the algebra still does the work of showing that no affine choice removes it. The per-tensor versus per-channel mismatch is a real validity concern: the theorem assumes channel-wise affine constants, while the MQBench backend quantizes activations per-tensor, so the proof does not directly apply to the experimental graph. However, that flaw concerns explanatory scope and correctness, not circularity under the criteria used here. There are no load-bearing self-citations, no imported uniqueness theorems, and no known result merely renamed. The paper explicitly credits Norm Tweaking as prior normalization-only work and distinguishes its own contribution. Therefore no circular step is present.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central results rest on a small set of hand-set hyperparameters (lambda_KD, lambda_CS, T, learning rates, epoch counts) and on the assumed per-channel affine decomposition of quantization error. The paper invents no new physical or architectural entities; the 'response-control subspace' is a conceptual label for existing normalization parameters. The main axiom burden is the local affine-error model, which is not derived from the actual per-tensor quantizer.

free parameters (6)
  • lambda_KD (distillation weight) = 1.0
    Hand-set weight in the NAP objective (Eq. 6); not derived from theory. Accuracy is reported at this value only.
  • lambda_CS (channel-scale regularizer) = 1e-4
    Hand-set regularizer coefficient (Eq. 7) that discourages dominant gamma channels; introduced ad hoc to stabilize affine tuning.
  • Distillation temperature T = 4 (CNNs), 1 (LLMs)
    Hand-set temperature in the KL distillation term (Eq. 6); chosen by convention, no sensitivity analysis.
  • Learning rate = 5e-4 (ImageNet/LLM), 1e-4 (CIFAR post-QAT)
    Hand-selected AdamW learning rates; results would shift with different values.
  • Tuning epochs/steps = 20 epochs (standalone NAP), 10 epochs (pre-QDrop), 200 steps (LLM)
    Stopping criteria chosen by hand; the ImageNet result depends on full-data 20-epoch tuning.
  • AWQ-style alpha = 0.5
    Borrowed from AWQ for equalization in LLM experiments; used as-is.
assumptions (4)
  • domain assumption Per-channel affine decomposition of quantization error (Eq. 13: floor(a_c) = alpha_c * a_c + delta_c + e_c)
    The entire cancellation proof (Eqs. 14-16) assumes quantization error is channel-wise affine plus residual; the actual per-tensor activation quantizer does not guarantee this decomposition, making the theorem conditional.
  • domain assumption BN running statistics are frozen and remain valid under affine adaptation
    NAP fixes running mean and variance and only changes gamma and beta; this assumes the normalized distribution statistics do not need to adapt, a choice not justified in the paper.
  • domain assumption The local projection view (Eqs. 17-18) approximates the distillation objective as a linear system with Jacobian J_q
    Used to claim NAP corrects the expressible component of mismatch; ignores curvature and nonlinearity beyond first order.
  • ad hoc to paper Channel-scale regularizer LCS (Eq. 7) is benign and not responsible for the gains
    Added to prevent a few channels from dominating; no ablation isolates its effect, so the central results include this extra regularization term.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Low-Dimensional High-Leverage Subspace Optimization: Beyond Full-Parameter Coupled Training for Neural Network Quantization." pith.science (2026). https://pith.science/paper/3Q7EBGTC

@misc{pith2026260803919,
  author       = {Pith},
  title        = {Pith review of: Low-Dimensional High-Leverage Subspace Optimization: Beyond Full-Parameter Coupled Training for Neural Network Quantization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3Q7EBGTC}},
  note         = {Machine review of arXiv:2608.03919}
}
read the original abstract

Low-bit quantization suffers severe accuracy degradation on compact networks, rooted in the dominant full-parameter coupled training paradigm that ignores parameter subspace heterogeneity. Their limited feature redundancy leaves little room to absorb quantization errors. Conventional pipelines adopt monolithic optimization: PTQ reconstructs fixed pretrained models without improving inherent quantization friendliness; QAT updates all parameters jointly, suffering from gradient coupling between backbone weights and calibration parameters. In this paper, we identify normalization affine parameters as a low-dimensional high-leverage subspace dominating quantization robustness, and propose Normalization Affine Preconditioning (NAP) for targeted subspace optimization. For PTQ, NAP freezes backbone weights and fine-tunes only affine parameters under the target fake-quantization graph on full-precision models, proactively boosting quantization friendliness before downstream reconstruction. For QAT, we introduce an alternating QAT-NAP schema that decouples feature learning and numerical calibration, breaking the performance ceiling of saturated joint training. Theoretical analysis confirms BN affine parameters fully cancel the channel-wise affine component of quantization distortion, while nonlinear rounding and clipping residuals form the irreducible error boundary; distillation-guided NAP acts as directional flatness optimization, projecting teacher-student logit mismatch onto the restricted subspace. Experiments on ImageNet and CIFAR-100 show NAP recovers severely collapsed low-bit quantization, consistently boosts reconstruction-based PTQ, and outperforms saturated full-parameter QAT with negligible tuning cost. This work reveals the principle of targeted low-dimensional subspace optimization, offering a new perspective beyond full-parameter coupled training for efficient deep learning.

Figures

Figures reproduced from arXiv: 2608.03919 by the authors.

Figure 1
Figure 1. Diagnostic study after saturated LSQ+ QAT. All adaptation branches start from the same LSQ+ QAT checkpoint on CIFAR-100 MobileNetV2. The FP32 accuracy of the pretrained model is 71.3%. 2021a), randomly dropping activation quantization during optimization (Wei et al. 2022), or compensating predic￾tion differences (Liu et al. 2023a). These techniques have substantially improved moderate-bit quantization, but ex￾treme … view at source ↗
Figure 2
Figure 2. Target-aligned normalization affine pre￾conditioning. The full-precision teacher is fixed, while the fake-quantized student updates only normalization affine pa￾rameters. The adapted channel responses reduce the scale and offset mismatch presented to the target quantization backend. Low-Dimensional Adaptation Subspaces Parameter-efficient adaptation suggests that small parame￾ter subsets can exert disproportionately… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 42 canonical work pages

  1. [1]

    Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    Quantization and training of neural networks for efficient integer-arithmetic-only inference , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

  2. [2]

    Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , year=

    Data-Free Quantization Through Weight Equalization and Bias Correction , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , year=

    work page

  3. [3]

    and Keutzer, Kurt , booktitle=

    Dong, Zhen and Yao, Zhewei and Gholami, Amir and Mahoney, Michael W. and Keutzer, Kurt , booktitle=

    work page

  4. [4]

    International Conference on Learning Representations (ICLR) , year=

    Learned Step Size Quantization , author=. International Conference on Learning Representations (ICLR) , year=

  5. [5]

    Bhalgat, Yash and Lee, Jinwon and Nagel, Markus and Blankevoort, Tijmen and Kwak, Nojun , booktitle=

    work page

  6. [6]

    Proceedings of the 37th International Conference on Machine Learning (ICML) , series=

    Up or Down? Adaptive Rounding for Post-Training Quantization , author=. Proceedings of the 37th International Conference on Machine Learning (ICML) , series=

    work page

  7. [7]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year=

    ZeroQ: A Novel Zero Shot Quantization Framework , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year=

    work page

  8. [8]

    and Keutzer, Kurt , booktitle=

    Dong, Zhen and Yao, Zhewei and Cai, Yaohui and Arfeen, Daiyaan and Gholami, Amir and Mahoney, Michael W. and Keutzer, Kurt , booktitle=

    work page

Show all 53 references
  1. [9]

    Li, Yuhang and Gong, Ruihao and Tan, Xu and Yang, Yang and Hu, Peng and Zhang, Qi and Yu, Fengwei and Wang, Wei and Gu, Shi , booktitle=

  2. [10]

    Li, Yuhang and Shen, Mingzhu and Ma, Jian and Ren, Yan and Zhao, Mingxin and Zhang, Qi and Gong, Ruihao and Yu, Fengwei and Yan, Junjie , booktitle=

  3. [11]

    Wei, Xiuying and Gong, Ruihao and Li, Yuhang and Liu, Xianglong and Yu, Fengwei , booktitle=

  4. [12]

    Yuan, Zhihang and Xue, Chenhao and Chen, Yiqi and Wu, Qiang and Sun, Guangyu , booktitle=

  5. [13]

    Lin, Yang and Zhang, Tianyu and Sun, Peiqin and Li, Zheng and Zhou, Shuchang , booktitle=

  6. [14]

    Frantar, Elias and Ashkboos, Saleh and Hoefler, Torsten and Alistarh, Dan , booktitle=

  7. [15]

    Proceedings of the 40th International Conference on Machine Learning (ICML) , series=

    SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models , author=. Proceedings of the 40th International Conference on Machine Learning (ICML) , series=

  8. [16]

    Liu, Jiawei and Niu, Lin and Yuan, Zhihang and Yang, Dawei and Wang, Xinggang and Liu, Wenyu , booktitle=

  9. [17]

    Shin, Juncheol and So, Junhyuk and Park, Sein and Kang, Seungyeop and Yoo, Se Jung and Park, Eunhyeok , booktitle=

  10. [18]

    Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , year=

    Towards Accurate Post-Training Quantization for Vision Transformer , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , year=

  11. [19]

    Li, Zhikai and Xiao, Junrui and Yang, Lianwei and Gu, Qingyi , booktitle=

  12. [20]

    Yuan, Zhihang and Niu, Lin and Liu, Jiawei and Liu, Wenyu and Wang, Xinggang and Shang, Yuzhang and Sun, Guangyu and Wu, Qiang and Wu, Jiaxiang and Wu, Bingzhe , journal=

  13. [21]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP) , year=

    Outlier Suppression+: Accurate Quantization of Large Language Models by Equivalent and Optimal Shifting and Scaling , author=. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP) , year=

    2023

  14. [22]

    Dettmers, Tim and Svirschevski, Ruslan and Egiazarian, Vage and Kuznedelev, Denis and Frantar, Elias and Ashkboos, Saleh and Borzunov, Alexander and Hoefler, Torsten and Alistarh, Dan , journal=

  15. [23]

    Lin, Ji and Tang, Jiaming and Tang, Haotian and Yang, Shang and Chen, Wei-Ming and Wang, Wei-Chen and Xiao, Guangxuan and Dang, Xingyu and Gan, Chuang and Han, Song , booktitle=

  16. [24]

    International Conference on Learning Representations (ICLR) , year=

    OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models , author=. International Conference on Learning Representations (ICLR) , year=

  17. [25]

    Proceedings of the 41st International Conference on Machine Learning (ICML) , year=

    Outlier-Aware Slicing for Post-Training Quantization in Vision Transformer , author=. Proceedings of the 41st International Conference on Machine Learning (ICML) , year=

  18. [26]

    Lee, Jung Hyun and Kim, Jeonghoon and Yang, June Yong and Kwon, Se Jung and Yang, Eunho and Yoo, Kang Min and Lee, Dongsoo , journal=

  19. [27]

    2025 , publisher =

    Sun, Yuxuan and Liu, Ruikang and Bai, Haoli and Bao, Han and Zhao, Kang and Li, Yuening and Hu, Jiaxin and Yu, Xianzhi and Hou, Lu and Yuan, Chun and Jiang, Xin and Liu, Wulong and Yao, Jun , booktitle =. 2025 , publisher =

    2025

  20. [28]

    Jiang, Runqing and Zhang, Ye and Wang, Longguang and Yu, Pengpeng and Guo, Yulan , booktitle=

  21. [29]

    Wu, Zhenhua and others , booktitle=

  22. [30]

    Choi, Jungwook and Wang, Zhuo and Venkataramani, Swagath and Chuang, Pierce I-Jen and Srinivasan, Vijayalakshmi and Gopalakrishnan, Kailash , booktitle=

  23. [31]

    Zhou, Shuchang and Ni, Zekun and Zhou, Xinyu and Wen, He and Wu, Yuxin and Zou, Yuheng , booktitle=

  24. [32]

    Proceedings of the 32nd International Conference on Machine Learning (ICML) , year=

    Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , author=. Proceedings of the 32nd International Conference on Machine Learning (ICML) , year=

  25. [33]

    arXiv preprint arXiv:1607.06450 , year=

    Layer Normalization , author=. arXiv preprint arXiv:1607.06450 , year=

    arXiv

  26. [34]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

    MobileNetV2: Inverted Residuals and Linear Bottlenecks , author =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

  27. [35]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages =

    Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks , author =. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages =

  28. [36]

    Proceedings of the 39th International Conference on Machine Learning , series =

    Overcoming Oscillations in Quantization-Aware Training , author =. Proceedings of the 39th International Conference on Machine Learning , series =. 2022 , publisher =

    2022

  29. [37]

    Chen, Tianyi and Chen, Sihan and Qu, Xiaoyi and Zhao, Dan and Yan, Ruomei and Ko, Jongwoo and Liang, Luming and Cameron, Pashmina , journal =

  30. [38]

    Advances in Neural Information Processing Systems , volume =

    Root Mean Square Layer Normalization , author =. Advances in Neural Information Processing Systems , volume =

  31. [39]

    arXiv preprint arXiv:1503.02531 , year =

    Distilling the Knowledge in a Neural Network , author =. arXiv preprint arXiv:1503.02531 , year =

    arXiv

  32. [40]

    and Li, Bo and Cameron, Pashmina and Jaggi, Martin and Alistarh, Dan and Hoefler, Torsten and Hensman, James , booktitle =

    Ashkboos, Saleh and Mohtashami, Amirkeivan and Croci, Maximilian L. and Li, Bo and Cameron, Pashmina and Jaggi, Martin and Alistarh, Dan and Hoefler, Torsten and Hensman, James , booktitle =

  33. [41]

    Liu, Zechun and Zhao, Changsheng and Fedorov, Igor and Soran, Bilge and Choudhary, Dhruv and Krishnamoorthi, Raghuraman and Chandra, Vikas and Tian, Yuandong and Blankevoort, Tijmen , booktitle =

  34. [42]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume =

    Norm Tweaking: High-Performance Low-Bit Quantization of Large Language Models , author =. Proceedings of the AAAI Conference on Artificial Intelligence , volume =. 2024 , doi =

    2024

  35. [43]

    Liu, Zechun and Oguz, Barlas and Zhao, Changsheng and Chang, Ernie and Stock, Pierre and Mehdad, Yashar and Shi, Yangyang and Krishnamoorthi, Raghuraman and Chandra, Vikas , journal =

  36. [44]

    2025 , publisher =

    Chen, Mengzhao and Shao, Wenqi and Xu, Peng and Wang, Jiahao and Gao, Peng and Zhang, Kaipeng and Luo, Ping , booktitle =. 2025 , publisher =

    2025

  37. [45]

    arXiv preprint arXiv:2412.15115 , year =

    arXiv

  38. [46]

    Deng, Jia and Dong, Wei and Socher, Richard and Li, Li-Jia and Li, Kai and Fei-Fei, Li , booktitle =

  39. [47]

    Learning Multiple Layers of Features from Tiny Images , author =

  40. [48]

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages =

    The Cityscapes Dataset for Semantic Urban Scene Understanding , author =. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages =

  41. [49]

    2015 , publisher =

    Ronneberger, Olaf and Fischer, Philipp and Brox, Thomas , booktitle =. 2015 , publisher =

    2015

  42. [50]

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages =

    Deep Residual Learning for Image Recognition , author =. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages =

  43. [51]

    Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages =

    A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages , author =. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages =. 2020 , publisher =

    2020

  44. [52]

    Journal of Machine Learning Research , volume =

    Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer , author =. Journal of Machine Learning Research , volume =

  45. [53]

    arXiv preprint arXiv:1609.07843 , year =

    Pointer Sentinel Mixture Models , author =. arXiv preprint arXiv:1609.07843 , year =

    arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.