Pith. sign in

REVIEW 4 major objections 5 minor 46 references

Enhancing Quantization-Aware Training on Edge Devices via Relative Entropy Coreset Selection and Cascaded Layer Correction

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read QuaRC claims that relative-entropy coreset selection plus cascaded layer correction lets 1% data QAT beat state-of-the-art coreset methods.

desk verdict The ImageNet headline is carried by CLC, not the RES coreset; a useful empirical paper that needs a decomposed ablation and error bars before its central claim is supported. read the letter →

arxiv 2507.17768 v1 pith:TVPOFJ56 submitted 2025-07-17 cs.LG cs.CV

Yujia Tong , Jingling Yuan , Chuang Hu This is my paper · ORCID
classification cs.LGcs.CV
keywords quantization-awaretrainingcoresetselectionrelativeentropylow-bitquantizationedgecomputingknowledgedistillationlayer-wisecorrectiondata-efficient
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

QuaRC is a framework for making quantization-aware training (QAT) practical on edge devices when only a small fraction of the data can be used. It argues that the best coreset for retraining a quantized model is not the most diverse or hardest samples, but the samples whose output distributions diverge most between the full-precision and quantized models, measured by relative entropy. On top of this selection, it adds a Cascaded Layer Correction loss that aligns intermediate layer outputs of the quantized model with the full-precision model, reducing error accumulation that ordinary knowledge distillation misses on small data. If correct, low-bit quantized models can be retrained on-device with 10% or less of the data while approaching full-data accuracy.

What carries the argument

The two load-bearing mechanisms are the Relative Entropy Score (RES) and Cascaded Layer Correction (CLC). RES is the KL divergence $d_{\mathrm{RES}} = \sum_m p_Q^{(m)}(w_q,x_q)\log\left(p_Q^{(m)}(w_q,x_q)/p_F^{(m)}(w_r,x_r)\right)$, computed per sample between quantized and full-precision output logits, used to rank and select the coreset. CLC adds the loss $\mathcal{L}_{\mathrm{CLC}} = \sum_c p^i_Q(w_q,x_q)\log\left(p^i_Q(w_q,x_q)/p^i_F(w_r,x_r)\right)$, aligning chosen intermediate-layer distributions of the quantized model to the full-precision model, with total loss $\mathcal{L}_{\mathrm{TOTAL}} = \mathcal{L}_{\mathrm{KD}} + \beta \mathcal{L}_{\mathrm{CLC}}$.

What would settle it

Run RES-guided coreset selection on a new architecture or dataset where the ranking of per-sample relative entropies is measured against final accuracy over many seeds; if the Spearman correlation is not consistently positive, or if coresets chosen by low relative entropy match or beat high-entropy coresets, the central claim fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that quantization error, not generic data difficulty, should drive coreset selection for QAT. It defines the Relative Entropy Score (RES) as the KL divergence between the output probability distributions of the full-precision and quantized models for each sample, and selects the top S% of samples by this score combined with gradient-based scores. It reports that this selection alone improves 2-bit MobileNetV2 on 1% CIFAR-100 from 46.84% to 48.77% Top-1 accuracy over the ACS baseline, and that the Cascaded Layer Correction strategy, which minimizes KL divergence between intermediate-layer outputs, contributes the larger gain (to 55.25%), with the full QuaRC reaching 56.36%. On ImageNet-1K with 2-bit ResNet-18 and a 1% subset, QuaRC achieves 46.34% Top-1, a 5.72% absolute improvement over ACS. The paper also reports a Spearman correlation of 0.867 between the mean relative entropy of coresets and the resulting Top-1 accuracy as evidence for the selection principle.

Load-bearing premise

The selection rule assumes that samples whose outputs differ most between the quantized and full-precision models are the best ones to train on, an assumption supported mainly by a single Spearman correlation of 0.867 computed over 10 coresets.

Editorial extensions

If this is right

  • With 10% or less of the training data, QuaRC can cut QAT training time while keeping accuracy close to full-data training; on CIFAR-100, the gain over ACS grows as the subset shrinks.
  • The Cascaded Layer Correction loss improves accuracy for every coreset selection method tested, including random sampling, so intermediate-layer alignment addresses a source of error that is independent of which samples are chosen.
  • On the UAV crack-detection case study, coreset QAT finishes in 2.55 minutes versus 7.02x longer for full-data QAT, while losing less than 0.5% Top-1 accuracy, making on-device retraining practical.
  • RES and CLC are complementary: together they outperform either alone at every bitwidth tested (2/3/4-bit).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the relative-entropy ranking is a stable property of the quantization process, RES could be computed once per calibration set and reused across bitwidths, avoiding repeated full-dataset forward passes; the paper does not test this.
  • The CLC idea generalizes beyond classifiers to any hierarchical network, including Transformers, but the paper only evaluates convolutional classifiers, so that extension is speculative.
  • A testable extension is to compare RES against uncertainty-based or loss-based selection to see whether relative entropy captures quantization-specific signal or just general sample difficulty.
  • Because RES depends on the current quantized model, its benefit may shrink as the model improves during training; periodic reselection every R epochs is assumed but the selection interval is not ablated.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes QuaRC, a two-phase framework for quantization-aware training (QAT) on edge devices with small coresets. In the selection phase, QuaRC introduces a Relative Entropy Score (RES) that ranks training samples by the KL divergence between the outputs of the full-precision and quantized models. In the training phase, it adds a Cascaded Layer Correction (CLC) loss that aligns intermediate-layer outputs of the quantized student with those of the full-precision teacher. Experiments on CIFAR-100 and ImageNet-1K with MobileNetV2 and ResNet-18 across 2/3/4-bit settings report consistent gains over a coreset-based baseline, with a headline result of 5.72% Top-1 improvement over ACS on a 1% ImageNet-1K subset for 2-bit ResNet-18. The paper also gives a complexity analysis, a convergence proof for SGD on a fixed coreset, an ablation study, and a UAV crack-detection case study.

Significance. If the reported gains hold, QuaRC is practically valuable: it targets a real bottleneck (QAT on edge devices with scarce data) and combines a quantization-error-aware selection criterion with a layer-wise correction loss. The CLC component is shown to be consistently beneficial across coreset selection methods (Table V), and the efficiency-accuracy analysis (Fig. 4) supports the practical motivation. The authors provide a clear ablation table on CIFAR-100 and a real-world case study. However, the central RES contribution is not isolated on the flagship ImageNet benchmark, the supporting correlation analysis is statistically weak, the convergence proof does not cover the actual algorithm, and all experimental results lack variance estimates, so the strength of the central claim exceeds what the current evidence supports.

major comments (4)
  1. [§V-B and §V-C (Table II vs. Table III)] The flagship ImageNet claim (5.72% Top-1 improvement over ACS at 1% subset, 2w2a ResNet-18) is never decomposed. In the CIFAR-100 ablation (Table III), CLC alone contributes +8.41 Top-1 over ACS, while RES alone contributes +1.93, and RES on top of CLC contributes +1.11. Because the ImageNet setup also uses a stronger ResNet-101 teacher, the reported 5.72% gain cannot be attributed to the relative-entropy coreset selection. Please provide an ImageNet ablation (ACS, +RES, +CLC, +RES+CLC) with the same teacher and training schedule, or explicitly report the RES-only and CLC-only contributions on this benchmark.
  2. [§III-B, Fig. 2(a)] The core premise of RES is validated with a Spearman correlation computed over only 10 coresets (rho=0.867, p=0.00117), all from one architecture (MobileNetV2), one bitwidth (2w32a), and one dataset fraction (1% of CIFAR-100), with no repeated runs or confidence intervals. With n=10, this correlation estimate is highly unstable, and the later ablations show that RES's incremental accuracy gain is small. To make the relative-entropy selection rule credible, report correlations over more coresets (e.g., 30 or more) and across at least one additional architecture/bitwidth, or provide bootstrap confidence intervals.
  3. [§IV-B, Theorem 1] The convergence analysis is for vanilla SGD on a fixed coreset DS with a Lipschitz-smooth, bounded-gradient loss; it does not model the actual QuaRC pipeline. In QuaRC, the coreset is re-selected every R epochs so DS changes over time, the training loss includes L_KD and beta*L_CLC, and gradients are computed through the straight-through estimator with quantization error. Consequently, Theorem 1 does not establish convergence of QuaRC, only a known property of SGD for a generic smooth loss. The authors should either state this limitation explicitly or provide an analysis that accounts for the changing coreset and the quantization/staleness error.
  4. [§V-A, Tables I-II] All reported accuracies are single-run numbers with no error bars or standard deviations. Given the extremely small subset fractions (1%), run-to-run variation from coreset selection and training initialization can be substantial. For the headline 5.72% ImageNet improvement, without variance estimates it is not possible to assess whether the difference from ACS is significant. Please report mean and standard deviation over at least three seeds for the main comparisons, or justify why a single seed is sufficient.
minor comments (5)
  1. [§V-C, Table IV] The text states that the dRES-only row achieves 55.42% Top-1, but the table reports 55.41%; please reconcile.
  2. [§IV-A, Eq. (10)] The efficiency inequality is stated as holding when S is sufficiently small, but the second term O(2*F*N*T/R) also depends on the selection interval R. For small R (e.g., R=1), the inequality may not hold. Please state the explicit condition on S and R, or include R in the comparison.
  3. [§III-C, Eq. (8)] The notation in L_CLC is confusing: the summation index is c (layers) while the probabilities use subscript i (pi_Q, pi_F). Please use a consistent layer index and define pi_Q/pi_F as the intermediate outputs at layer c.
  4. [§V-A and Algorithm 1] The hyperparameters beta and R are set manually for each dataset (beta=1e5, R=50 for CIFAR-100; beta=3e3, R=10 for ImageNet) with no sensitivity analysis. Please include a brief study of how performance varies with beta and R, since the claimed robustness depends on these choices.
  5. [Algorithm 1] The input list includes 'Initial coreset DS(t)' but t is the loop variable and DS(t) is defined during the loop; please rename this to DS(0) or clarify.

Circularity Check

1 steps flagged · score 1.0 of 10

Minor self-consistency in the CLC KL-divergence evaluation; the core QuaRC derivation is otherwise self-contained.

  1. self definitional [Section III-C Eq. (8); Section V-C3 Fig. 3]
    "To minimize the difference between the intermediate layer outputs of the full-precision and quantized models, we propose a Cascaded Layer Correction (CLC) training strategy... reducing the difference in the output of the intermediate layer between the full-precision model and the quantized model is equivalent to optimizing the following loss function in training: LCLC = ... In the case of 2-bit weight quantization, the baseline quantized model exhibits a KL divergence of 2.4e-5... After applying the CLC strategy, the KL divergence is reduced to 1.0e-5."

    The KL divergence reported in Fig. 3 as evidence that CLC 'effectively mitigates quantization errors' is the same quantity minimized by LCLC in Eq. (8). Showing that the minimized objective decreased after training is a self-consistency check, not an independent verification of quantization-error reduction. The accuracy improvements in Tables I, III, and V are external evidence and are not logically forced, so this circularity is confined to the KL-based visualization argument and does not drive the main result.

full rationale

QuaRC's central claims are empirical: RES selects samples by the KL divergence between quantized and full-precision outputs, and the reported accuracy gains are measured against external baselines on held-out test sets. No parameter is fitted to a subset and then renamed as a prediction; RES is a fixed, pre-training heuristic, and the ImageNet/CIFAR results come from standard QAT training. The only step that approaches circularity is the CLC visualization: Fig. 3 reports that the intermediate-layer KL divergence drops after CLC training, but that KL divergence is exactly the LCLC objective in Eq. (8), so the decrease is a self-consistency check rather than an independent proof of error reduction. The accuracy tables provide independent support, so this does not compromise the main comparison. The skeptic's concern that the ImageNet gain is not decomposed between RES and CLC is an attribution/ablation gap, not circularity. No load-bearing self-citation was found; the paper builds on external works such as ACS and LSQ+ for implementation details.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The method rests on standard QAT background, a selection heuristic validated by a weak correlation, and layer-alignment assumptions; no new physical entities or fitted constants beyond hyperparameters.

free parameters (3)
  • beta (CLC loss weight) = 1e5 on CIFAR-100, 3e3 on ImageNet-1K
    Set by hand per dataset; controls strength of intermediate-layer alignment in Eq. 9. No sensitivity analysis is provided.
  • R (coreset selection interval) = 50 on CIFAR-100, 10 on ImageNet-1K
    Chosen following ACS [16]; not tuned or ablated, though it affects how often expensive RES computation is repeated.
  • Selected intermediate layers for CLC = Not specified
    Eq. 8 sums over 'c' but the paper never states which layers are corrected or how many; this choice materially changes the training objective.
assumptions (3)
  • standard math Loss function is Lipschitz-smooth and gradients are bounded on the coreset (Assumptions 1 and 2, Section IV-B).
    Used in the convergence proof; standard but not verified for quantized networks.
  • domain assumption The full-precision teacher and quantized student have aligned intermediate layer structures so that KL divergence between layer outputs is meaningful.
    Assumed in Section III-C and acknowledged as a limitation in Section VII-B.
  • domain assumption Relative entropy between output logits is a valid proxy for quantization error severity per sample.
    Motivation in Section III-B backed only by a 10-point correlation, not a proof.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Quantization-Aware Training on Edge Devices via Relative Entropy Coreset Selection and Cascaded Layer Correction." pith.science (2026). https://pith.science/paper/TVPOFJ56

@misc{pith2026250717768,
  author       = {Pith},
  title        = {Pith review of: Enhancing Quantization-Aware Training on Edge Devices via Relative Entropy Coreset Selection and Cascaded Layer Correction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TVPOFJ56}},
  note         = {Machine review of arXiv:2507.17768}
}
read the original abstract

With the development of mobile and edge computing, the demand for low-bit quantized models on edge devices is increasing to achieve efficient deployment. To enhance the performance, it is often necessary to retrain the quantized models using edge data. However, due to privacy concerns, certain sensitive data can only be processed on edge devices. Therefore, employing Quantization-Aware Training (QAT) on edge devices has become an effective solution. Nevertheless, traditional QAT relies on the complete dataset for training, which incurs a huge computational cost. Coreset selection techniques can mitigate this issue by training on the most representative subsets. However, existing methods struggle to eliminate quantization errors in the model when using small-scale datasets (e.g., only 10% of the data), leading to significant performance degradation. To address these issues, we propose QuaRC, a QAT framework with coresets on edge devices, which consists of two main phases: In the coreset selection phase, QuaRC introduces the ``Relative Entropy Score" to identify the subsets that most effectively capture the model's quantization errors. During the training phase, QuaRC employs the Cascaded Layer Correction strategy to align the intermediate layer outputs of the quantized model with those of the full-precision model, thereby effectively reducing the quantization errors in the intermediate layers. Experimental results demonstrate the effectiveness of our approach. For instance, when quantizing ResNet-18 to 2-bit using a 1% data subset, QuaRC achieves a 5.72% improvement in Top-1 accuracy on the ImageNet-1K dataset compared to state-of-the-art techniques.

Figures

Figures reproduced from arXiv: 2507.17768 by the authors.

Figure 1
Figure 1. The overview of QuaRC. B. Coreset Selection for QAT Coreset selection techniques can enhance the efficiency of QAT by identifying the most representative subsets for training. However, most existing coreset selection methods are designed for full-precision models and do not account for the unique characteristics of quantized models. For instance, classical methods such as Moderate [25], Contextual Diversity (CD) [26… view at source ↗
Figure 2
Figure 2. (a) We select coresets with different mean values of [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Comparison of the selected inter-layer output distributions between the full-precision MobileNetV2 model and the [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: (a) The efficiency of different methods. (b) Balance [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: UAV prototype. ... ... ... 300 images ... coreset(10%) Select By RES 13:00 pm to 14:00 pm ℒ𝐶𝐿𝐶 ℒ𝐾𝐷 Quantized MobileNetV2 (To be trained) FP MobileNetV2 (trained) 3000 images [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 7
Figure 7. Figure 7: Analysis of Top-1 Accuracy and Training Time. [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 31 canonical work pages

  1. [1]

    Remote sensing image scene classifi- cation: Benchmark and state of the art,

    G. Cheng, J. Han, and X. Lu, “Remote sensing image scene classifi- cation: Benchmark and state of the art,” Proceedings of the IEEE , vol. 105, no. 10, pp. 1865–1883, 2017

    2017

  2. [2]

    Image based techniques for crack detection, classification and quantification in asphalt pavement: a review,

    H. Zakeri, F. M. Nejad, and A. Fahimifar, “Image based techniques for crack detection, classification and quantification in asphalt pavement: a review,” Archives of Computational Methods in Engineering , vol. 24, pp. 935–977, 2017

    work page 2017

  3. [3]

    Efficient acceleration of deep learning inference on resource-constrained edge devices: A review,

    M. M. H. Shuvo, S. K. Islam, J. Cheng, and B. I. Morshed, “Efficient acceleration of deep learning inference on resource-constrained edge devices: A review,” Proceedings of the IEEE , vol. 111, no. 1, pp. 42– 91, 2022

    work page 2022

  4. [4]

    A survey of deep learning on mobile devices: Applications, optimizations, challenges, and research opportunities,

    T. Zhao, Y . Xie, Y . Wang, J. Cheng, X. Guo, B. Hu, and Y . Chen, “A survey of deep learning on mobile devices: Applications, optimizations, challenges, and research opportunities,” Proceedings of the IEEE , vol. 110, no. 3, pp. 334–354, 2022

    work page 2022

  5. [5]

    Mobilenets: Efficient convo- lutional neural networks for mobile vision applications,

    A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convo- lutional neural networks for mobile vision applications,” arXiv preprint arXiv:1704.04861, 2017

    arXiv 2017

  6. [6]

    Energy efficient federated learning over heterogeneous mobile devices via joint design of weight quantization and wireless transmission,

    R. Chen, L. Li, K. Xue, C. Zhang, M. Pan, and Y . Fang, “Energy efficient federated learning over heterogeneous mobile devices via joint design of weight quantization and wireless transmission,” IEEE Transactions on Mobile Computing (TMC) , vol. 22, no. 12, pp. 7451–7465, 2022

    work page 2022

  7. [7]

    Bi-deepvit: Binarized transformer for efficient sensor-based human activity recognition,

    F. Luo, A. Li, S. Khan, K. Wu, and L. Wang, “Bi-deepvit: Binarized transformer for efficient sensor-based human activity recognition,” IEEE Transactions on Mobile Computing (TMC) , 2025

    work page 2025

  8. [8]

    Post-training piecewise linear quantization for deep neural networks,

    J. Fang, A. Shafiee, H. Abdel-Aziz, D. Thorsley, G. Georgiadis, and J. H. Hassoun, “Post-training piecewise linear quantization for deep neural networks,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16 . Springer, 2020, pp. 69–86

    work page 2020

Show all 46 references
  1. [9]

    Towards accurate post- training network quantization via bit-split and stitching,

    P. Wang, Q. Chen, X. He, and J. Cheng, “Towards accurate post- training network quantization via bit-split and stitching,” in International Conference on Machine Learning . PMLR, 2020, pp. 9847–9856

    2020

  2. [10]

    Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients,

    S. Zhou, Y . Wu, Z. Ni, X. Zhou, H. Wen, and Y . Zou, “Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients,” arXiv preprint arXiv:1606.06160 , 2016

    2016 arXiv

  3. [11]

    Learned step size quantization,

    S. K. Esser, J. L. McKinstry, D. Bablani, R. Appuswamy, and D. S. Modha, “Learned step size quantization,” in International Conference on Learning Representations , 2020

    2020

  4. [12]

    Herding dynamical weights to learn,

    M. Welling, “Herding dynamical weights to learn,” in Proceedings of the 26th annual international conference on machine learning , 2009, pp. 1121–1128

    2009

  5. [13]

    Deep learning on a data diet: Finding important examples early in training,

    M. Paul, S. Ganguli, and G. K. Dziugaite, “Deep learning on a data diet: Finding important examples early in training,” Advances in neural information processing systems , vol. 34, pp. 20 596–20 607, 2021

    2021

  6. [14]

    Grad- match: Gradient matching based data subset selection for efficient deep model training,

    K. Killamsetty, S. Durga, G. Ramakrishnan, A. De, and R. Iyer, “Grad- match: Gradient matching based data subset selection for efficient deep model training,” in International Conference on Machine Learning . PMLR, 2021, pp. 5464–5474

    2021

  7. [15]

    Beyond neural scaling laws: beating power law scaling via data pruning,

    B. Sorscher, R. Geirhos, S. Shekhar, S. Ganguli, and A. Morcos, “Beyond neural scaling laws: beating power law scaling via data pruning,” Advances in Neural Information Processing Systems , vol. 35, pp. 19 523–19 536, 2022

    2022

  8. [16]

    Robust and efficient quantization-aware training via coreset selection,

    X. Huang, Z. Liu, S.-Y . Liu, and K.-T. Cheng, “Robust and efficient quantization-aware training via coreset selection,” Transactions on Ma- chine Learning Research , 2024. 13

    2024

  9. [17]

    Q-vit: Accurate and fully quantized low-bit vision transformer,

    Y . Li, S. Xu, B. Zhang, X. Cao, P. Gao, and G. Guo, “Q-vit: Accurate and fully quantized low-bit vision transformer,” Advances in neural information processing systems , vol. 35, pp. 34 451–34 463, 2022

    2022

  10. [18]

    Over- coming oscillations in quantization-aware training,

    M. Nagel, M. Fournarakis, Y . Bondarenko, and T. Blankevoort, “Over- coming oscillations in quantization-aware training,” in International Conference on Machine Learning . PMLR, 2022, pp. 16 318–16 330

    2022

  11. [19]

    Estimating or propagating gradients through stochastic neurons for conditional computation,

    Y . Bengio, N. L ´eonard, and A. Courville, “Estimating or propagating gradients through stochastic neurons for conditional computation,” arXiv preprint arXiv:1308.3432, 2013

    2013 arXiv

  12. [20]

    Crossvit: Cross-attention multi- scale vision transformer for image classification,

    C.-F. R. Chen, Q. Fan, and R. Panda, “Crossvit: Cross-attention multi- scale vision transformer for image classification,” in Proceedings of the IEEE/CVF international conference on computer vision , 2021, pp. 357– 366

    2021

  13. [21]

    Apprentice: Using knowledge distillation techniques to improve low-precision network accuracy,

    A. Mishra and D. Marr, “Apprentice: Using knowledge distillation techniques to improve low-precision network accuracy,” in International Conference on Learning Representations , 2018. [Online]. Available: https://openreview.net/forum?id=B1ae1lZRb

    2018

  14. [22]

    Sdq: Stochastic differentiable quantization with mixed precision,

    X. Huang, Z. Shen, S. Li, Z. Liu, H. Xianghong, J. Wicaksana, E. Xing, and K.-T. Cheng, “Sdq: Stochastic differentiable quantization with mixed precision,” in International Conference on Machine Learning . PMLR, 2022, pp. 9295–9309

    2022

  15. [23]

    Apprentice: Using knowledge distillation techniques to improve low-precision network accuracy,

    A. Mishra and D. Marr, “Apprentice: Using knowledge distillation techniques to improve low-precision network accuracy,” arXiv preprint arXiv:1711.05852, 2017

    2017 arXiv

  16. [24]

    Oscillation-free quantization for low-bit vision transformers,

    S.-Y . Liu, Z. Liu, and K.-T. Cheng, “Oscillation-free quantization for low-bit vision transformers,” in International Conference on Machine Learning. PMLR, 2023, pp. 21 813–21 824

    2023

  17. [25]

    Moderate coreset: A universal method of data selection for real-world data- efficient deep learning,

    X. Xia, J. Liu, J. Yu, X. Shen, B. Han, and T. Liu, “Moderate coreset: A universal method of data selection for real-world data- efficient deep learning,” in The Eleventh International Conference on Learning Representations , 2023. [Online]. Available: https: //openreview.net/...

    2023

  18. [26]

    Contextual diversity for active learning,

    S. Agarwal, H. Arora, S. Anand, and C. Arora, “Contextual diversity for active learning,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVI

    2020

  19. [27]

    Springer, 2020, pp. 137–153

    2020

  20. [28]

    An empirical study of example forgetting during deep neural network learning,

    M. Toneva, A. Sordoni, R. T. des Combes, A. Trischler, Y . Bengio, and G. J. Gordon, “An empirical study of example forgetting during deep neural network learning,” in International Conference on Learning Representations , 2019. [Online]. Available: https: //openreview.net/for...

    2019

  21. [29]

    Hard sample matters a lot in zero-shot quantization,

    H. Li, X. Wu, F. Lv, D. Liao, T. H. Li, Y . Zhang, B. Han, and M. Tan, “Hard sample matters a lot in zero-shot quantization,” in Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition , 2023, pp. 24 417–24 426

    2023

  22. [30]

    Pytorch dis- tributed: Experiences on accelerating data parallel training,

    S. Li, Y . Zhao, R. Varma, O. Salpekar, P. Noordhuis, T. Li, A. Paszke, J. Smith, B. Vaughan, P. Damania, and S. Chintala, “Pytorch dis- tributed: Experiences on accelerating data parallel training,” CoRR, vol. abs/2006.15704, 2020

    2006 arXiv

  23. [31]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky, G. Hinton et al. , “Learning multiple layers of features from tiny images,” 2009

    2009

  24. [32]

    Imagenet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition . Ieee, 2009, pp. 248–255

    2009

  25. [33]

    Pytorch: An imperative style, high-performance deep learning library,

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al. , “Pytorch: An imperative style, high-performance deep learning library,” Advances in neural information processing systems , vol. 32, 2019

    2019

  26. [34]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778

    2016

  27. [35]

    Lsq+: Improving low-bit quantization through learnable offsets and better initialization,

    Y . Bhalgat, J. Lee, M. Nagel, T. Blankevoort, and N. Kwak, “Lsq+: Improving low-bit quantization through learnable offsets and better initialization,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops , 2020, pp. 696–697

    2020

  28. [36]

    Communication-efficient satellite-ground federated learning through progressive weight quantization,

    C. Yang, J. Yuan, Y . Wu, Q. Sun, A. Zhou, S. Wang, and M. Xu, “Communication-efficient satellite-ground federated learning through progressive weight quantization,” IEEE Transactions on Mobile Com- puting (TMC), vol. 23, no. 9, pp. 8999–9011, 2024

    2024

  29. [37]

    Binarized neural network for edge intelligence of sensor-based human activity recognition,

    F. Luo, S. Khan, Y . Huang, and K. Wu, “Binarized neural network for edge intelligence of sensor-based human activity recognition,” IEEE transactions on mobile computing (TMC), vol. 22, no. 3, pp. 1356–1368, 2021

    2021

  30. [38]

    Adversarial active learning for deep networks: a margin based approach,

    M. Ducoffe and F. Precioso, “Adversarial active learning for deep networks: a margin based approach,” arXiv preprint arXiv:1802.09841 , 2018

    2018 arXiv

  31. [39]

    Active learning by acquiring contrastive examples,

    K. Margatina, G. Vernikos, L. Barrault, and N. Aletras, “Active learning by acquiring contrastive examples,” in Proceedings of the 2021 Confer- ence on Empirical Methods in Natural Language Processing , 2021, pp. 650–663

    2021

  32. [40]

    Coresets for data- efficient training of machine learning models,

    B. Mirzasoleiman, J. Bilmes, and J. Leskovec, “Coresets for data- efficient training of machine learning models,” in International Con- ference on Machine Learning . PMLR, 2020, pp. 6950–6960

    2020

  33. [41]

    Object detection with deep learning: A review,

    Z.-Q. Zhao, P. Zheng, S.-t. Xu, and X. Wu, “Object detection with deep learning: A review,” IEEE transactions on neural networks and learning systems, vol. 30, no. 11, pp. 3212–3232, 2019

    2019

  34. [42]

    Object detection in 20 years: A survey,

    Z. Zou, K. Chen, Z. Shi, Y . Guo, and J. Ye, “Object detection in 20 years: A survey,” Proceedings of the IEEE, vol. 111, no. 3, pp. 257–276, 2023

    2023

  35. [43]

    Review the state-of- the-art technologies of semantic segmentation based on deep learning,

    Y . Mo, Y . Wu, X. Yang, F. Liu, and Y . Liao, “Review the state-of- the-art technologies of semantic segmentation based on deep learning,” Neurocomputing, vol. 493, pp. 626–646, 2022

    2022

  36. [44]

    Semantic segmentation using vision transformers: A survey,

    H. Thisanke, C. Deshan, K. Chamith, S. Seneviratne, R. Vidanaarachchi, and D. Herath, “Semantic segmentation using vision transformers: A survey,” Engineering Applications of Artificial Intelligence , vol. 126, p. 106669, 2023

    2023

  37. [45]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Pro- ceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technolo...

    2019

  38. [46]

    Language mod- els are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., “Language mod- els are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020

    1901

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.