Pith. sign in

REVIEW 5 major objections 6 minor 46 references

DFQ-ViT: Data-Free Quantization for Vision Transformers without Fine-tuning

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Data-free ViT quantization can match real-data calibration accuracy without fine-tuning.

desk verdict Useful empirical pipeline for data-free ViT quantization; the theory section does not prove what it claims, so treat the experiments as the main contribution. read the letter →

arxiv 2507.14481 v1 pith:CYRLE2FV submitted 2025-07-19 cs.CV cs.AI

Yujia Tong , Jingling Yuan , Tian Zhang , Jianquan Liu , Chuang Hu This is my paper · ORCID
classification cs.CVcs.AI
keywords data-freequantizationvisiontransformerpost-trainingsyntheticsamplesynthesiscurriculumlearningactivationcorrectionmatrixlow-bitimageclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DFQ-ViT is a pipeline for quantizing vision transformers when the original training data is unavailable and fine-tuning is not allowed. The paper claims that two mechanisms, synthesizing calibration images from easy (large-crop, global) to hard (small-crop, local) and additively correcting intermediate activations with an Activation Correction Matrix, let a quantized model match or beat models calibrated with real images. On ImageNet with 3-bit weights and 8-bit activations, DeiT-T reaches 27.97% top-1 accuracy, 4.29 points above the previous data-free method PSAQ-ViT, and on several W4/A8 settings DFQ-ViT exceeds real-data calibration. If true, data-free post-training quantization becomes a practical route to deploying ViTs on resource-limited devices without retraining costs.

What carries the argument

Two mechanisms carry the pipeline. The Easy-to-Hard (E2H) synthesis schedule uses RandomResizedCrop with a cosine-decreasing lower crop bound $\delta_t = \delta_l + (\delta_u-\delta_l)(1+\cos(\pi t/T))/2$, so early iterations see large crops that capture global contours and later iterations focus on small crops that add local texture; Theorem 1 claims this order lowers the final synthesis loss relative to a fixed crop, provided Assumptions 1-4 hold. The Activation Correction Matrix (ACM), $\mathrm{Matrix}^i_{ACM} = \frac{1}{N}\sum_{n=1}^N [p^i_T(w_r,x) - p^i_Q(w_q,x)]$, is a per-block additive bias computed once on synthetic samples and added to quantized intermediate activations every $\gamma$ layers during inference, aligning them with the full-precision model and stopping quantization-error accumulation.

What would settle it

On a fixed ViT (say DeiT-T), run both E2H and fixed-crop synthesis from the same noise initialization, recording $\sum_{t=0}^{T-1}\|\nabla_x L(x_t^{E2H};\delta_t)\|^2$ and the corresponding fixed-crop sum. If the E2H sum fails to exceed the fixed-crop sum by $K G \Delta/(\eta-\beta\eta^2/2)$, then the cumulative-gradient assumption behind Theorem 1 is violated and the claimed theoretical basis for E2H is falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that data-free quantization of vision transformers does not need fine-tuning or real data to be competitive: by generating synthetic calibration samples in an easy-to-hard order, large crops first to establish global structure and then progressively smaller crops to add local detail, and by adding a per-layer Activation Correction Matrix that shifts quantized intermediate activations toward their full-precision values, DFQ-ViT closes most of the gap to data-dependent post-training quantization. The paper reports, for example, 27.97% top-1 accuracy for DeiT-T at W3/A8, a 4.29-point gain over PSAQ-ViT, and 65.84% at W4/A8, above the 65.38% real-data-calibrated baseline. The authors also claim a theoretical guarantee, Theorem 1, that the E2H schedule yields lower final synthesis loss than a fixed-crop schedule under smoothness, bounded-gradient, and cumulative-gradient assumptions.

Load-bearing premise

The whole theoretical argument rests on the assumption that the easy-to-hard schedule accumulates at least as much squared gradient strength as a fixed-crop schedule, plus a fixed margin set by the crop range; if a real vision transformer does not satisfy that inequality, the proof that E2H is better collapses.

Editorial extensions

If this is right

  • Quantized ViTs can be deployed in privacy-sensitive settings where the original training set cannot be touched, because calibration runs entirely on synthetic images.
  • At several W4/A8 settings the no-data pipeline matches or exceeds calibration on real ImageNet images, so data-free need not imply a quality tax at moderate bit-widths.
  • The ACM adds at most 0.02% of the model's parameter count and no inference latency, so the accuracy recovery is nearly free at deployment.
  • E2H adds no time over PSAQ-ViT in sample generation (125 seconds on DeiT-T), so the reported gains do not come from extra synthesis compute.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not test is whether the E2H crop schedule transfers to other data-free objectives or to detection and segmentation ViTs, where multi-scale structure matters in different ways.
  • Because ACM is a mean activation shift estimated from only 16 synthetic images, its correction may partly encode synthesis artifacts; using more diverse calibration samples could change the measured margin over real-data calibration.
  • An apples-to-apples comparison against PSAQ-ViT V2 under matched compute would separate the value of the no-fine-tuning constraint from the value of E2H and ACM themselves; the paper excludes V2 because it requires retraining.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes DFQ-ViT, a data-free quantization pipeline for Vision Transformers that combines an Easy-to-Hard (E2H) sample synthesis strategy with an Activation Correction Matrix (ACM). The E2H strategy progressively shrinks the crop size during synthetic-image generation, motivated by curriculum learning, while ACM corrects intermediate activations of the quantized model during inference. The authors report experiments on ImageNet-1k, CIFAR-10, and CIFAR-100 for DeiT-T/S/B and Swin-T/S, showing that DFQ-ViT often outperforms PSAQ-ViT and sometimes matches or exceeds calibration with real data. They also provide a theoretical analysis claiming to prove the superiority of E2H over fixed-crop synthesis.

Significance. If the empirical results are taken at face value, the pipeline is practically useful: it requires no fine-tuning, adds negligible parameters through ACM, and maintains roughly the same generation and inference cost as PSAQ-ViT while improving accuracy on most tested settings. The paper is less convincing on the theory side. The claimed proof of E2H superiority rests on Assumption 4, which essentially assumes the superiority in cumulative gradient norm, and Lemma 2 uses an invalid mean-value-type inequality. The empirical ablations in Table 3 provide some support for E2H and ACM individually, but the hyperparameter choices in Table 4 are tuned on the same evaluation benchmark used for final numbers, which weakens the comparison. Overall, the empirical contribution is valuable but the theoretical claim needs substantial revision or removal.

major comments (5)
  1. [Section 3.3.2, Eq. (15)] Assumption 4 is not a regularity condition on the loss but a trajectory-dependent inequality that already contains the superiority of E2H. It compares cumulative squared gradient norms along the E2H and Fixed trajectories, so it cannot be verified from Assumptions 1–3 alone. Theorem 1 then substitutes Eq. (15) directly into Eq. (26), making the proof circular with respect to the claimed result. The manuscript should either prove Eq. (15) from more primitive conditions or explicitly label the E2H superiority as an unverified assumption.
  2. [Section 3.3.3, Eq. (21) in Lemma 2] The inequality in Eq. (21) does not follow from Assumption 1. Assumption 1 bounds the difference of gradients with respect to x at two values of δ, but Eq. (21) bounds the difference of loss values with respect to δ. Applying the mean value theorem in δ would require a bound on ∂L/∂δ, not on ∇_x L. The factor ||x_{t+1}-x_t|| is also dimensionally incompatible with a Lipschitz bound in δ. Lemma 2 is therefore unproved as stated.
  3. [Section 3.3.3, Eq. (28)] The direction of the final inequality is reversed. From Eq. (27), define B = L0 - (η - βη^2/2) Σ ||∇L_fixed||². Lemma 1 gives L(x_T^Fixed;δ_T) ≤ B, so B ≥ L(x_T^Fixed;δ_T). Therefore subtracting the positive term K G Δ(1 - η) gives B - K G Δ(1 - η) ≥ L(x_T^Fixed;δ_T) - K G Δ(1 - η), not the ≤ direction written in Eq. (28). In addition, the conclusion that the final term is negative requires η < 1, while the theorem only assumes η ≤ 2/β. Thus Eq. (29) does not follow even if Lemma 2 and Assumption 4 are accepted.
  4. [Section 3.3.3, Theorem 1 statement] The Fixed strategy is defined with δ_t = δ_max for all t, but the theorem compares L(x_T^E2H;δ_T) with L(x_T^Fixed;δ_T), evaluating the Fixed trajectory at δ_T even though that trajectory never uses δ_T. Since no monotonicity of L in δ is assumed, L(x_T^Fixed;δ_T) is unrelated to the losses obtained along the Fixed trajectory. The theorem should compare both strategies at the same δ value actually used by the Fixed strategy, or add an explicit monotonicity assumption on L with respect to δ.
  5. [Section 4.1 and Table 4] The default number of calibration samples (16) and the number of iterations per sample (500) are selected using the same ImageNet-1k evaluation benchmark on which the final accuracy numbers are reported. The paper does not describe a separate validation split or a fixed protocol for choosing these hyperparameters. This creates a risk of selection bias in the headline comparisons and should be disclosed, ideally by adding a validation split or by fixing the settings a priori.
minor comments (6)
  1. [Abstract and Section 4.2] The claim of being '4.29% higher than the state-of-the-art' should be qualified as applying to DeiT-T at W3/A8; on Swin-T at W3/A8, DFQ-ViT is 0.84% lower than PSAQ-ViT in Table 2.
  2. [Section 3.4, Eq. (31)] It is unclear whether the synthetic samples used to compute the ACM are the same as those used for calibration, and whether the matrix is computed for every layer or only every γ-th layer; please state this more precisely.
  3. [Section 3.4, Eqs. (32)–(33)] The notation x_{i-γ} is used both as an input activation and as the output of the previous correction step; the recurrence should be defined explicitly to avoid confusion.
  4. [Table 3] The caption and text should state that the 'Baseline' row corresponds to PSAQ-ViT without E2H and ACM, and that the '+ACM' row uses PSAQ-ViT sample generation; otherwise the additive decomposition of the gains is ambiguous.
  5. [Section 4.6, Table 5] The entry 'Deit-B' should be 'DeiT-B', and the identical ACM parameter counts for Swin-T and Swin-S should be explained, since the two models have different layer configurations.
  6. [Related Work and Section 4.1] The exclusion of PSAQ-ViT V2 should be justified in more detail, and the claim that there is no prior work on data-free quantization for ViTs without fine-tuning should be qualified in light of recent PTQ methods for ViTs that also avoid retraining.

Circularity Check

1 steps flagged · score 6.0 of 10

Theorem 1's proof of E2H superiority is conditional on Assumption 4, which already asserts the gradient-accumulation advantage that drives the conclusion; the empirical results remain independently grounded.

  1. self definitional [Section 3.3.2, Assumption 4 (Eq. 15), used in Theorem 1 (Eq. 22)]
    "Assumption 4 (Cumulative Gradient). For any T > 0, the cumulative gradient of E2H satisfies: sum_{t=0}^{T-1} ||∇_x L(x_t^E2H; δ_t)||^2 >= sum_{t=0}^{T-1} ||∇_x L(x_t^Fixed; δ_max)||^2 + K G Δ / (η - βη^2/2) (15) where Δ = δ_max − δ_min, η is learning rate. ... Theorem 1 (Superiority of E2H Strategy). Under Assumptions 1-4 ... The E2H strategy achieves better final performance: L(x_T^E2H; δ_T) ≤ L(x_T^Fixed; δ_T) (22)"

    Assumption 4 is not a benign regularity condition; it is a direct comparison between the two optimization trajectories that already states the E2H cumulative-gradient advantage. In the proof, Eq. (15) is substituted into the summed E2H bound at Eq. (26), and the additive constant K G Δ / (η − βη²/2) is exactly sized to cancel the E2H penalty KηGΔ at Eqs. (27)-(28). The conclusion L(x_T^E2H; δ_T) ≤ L(x_T^Fixed; δ_T) therefore follows by algebraic restatement of the assumption, not from Assumptions 1-3. The paper's contribution claim 'we theoretically prove the effectiveness of the E2H strategy' thus reduces, for its theoretical component, to an assumption that already encodes the superiority result.

full rationale

The only load-bearing circular step is the theoretical proof of E2H superiority: Assumption 4 (Eq. 15) asserts the exact cumulative-gradient advantage that Theorem 1 (Eq. 22) then recovers as a final-loss inequality. This is not a case of a harmless unproven condition; it is the desired result baked into the assumptions. The proof also contains a separate logical gap at Eq. (28) when it invokes η < 1 despite the theorem only assuming η ≤ 2/β, but that is a correctness issue rather than circularity. The ACM component is explicitly inspired by the external bias-correction work of Nagel et al. (2019), and the empirical comparisons against PSAQ-ViT, real-data calibration, and Gaussian noise are independent benchmarks with their own reported numbers. No self-citation chain is load-bearing, and the empirical claims do not reduce to fitted inputs. Overall, the circularity is confined to the theoretical derivation, so a partial score of 6 is appropriate.

Assumptions & free parameters 6 free parameters · 4 assumptions · 1 invented entities

The pipeline depends on hand-tuned synthesis hyperparameters, an assumed gradient-advantage inequality in the proof, and a calibrated activation correction matrix computed from synthetic samples; none of these are derived from first principles or shipped as code.

free parameters (6)
  • crop bounds delta_l and delta_u = 0.08 and 1.0
    Set manually in Section 4.1; they define the difficulty range of the E2H curriculum and directly control synthetic sample quality.
  • loss weights alpha and beta = alpha=1.0, beta=0.05
    Chosen by hand in Section 4.1 for the synthesis loss Eq. (8); these balance the patch-similarity, one-hot, and total-variance terms.
  • number of calibration samples = 16
    Selected in Section 4.3 (Table 4) based on ImageNet validation performance; the paper notes that 4 samples are too few and 24 are too noisy.
  • number of synthesis iterations per sample = 500
    Selected in Section 4.3 (Table 4) as the best trade-off; 100 iterations underfit and 1000 overfit.
  • ACM insertion interval gamma = not reported
    Defined in Eqs. (32)-(33) as the frequency with which the correction matrix is added, but no value or layer choice is given anywhere in the paper.
  • synthesis learning rate = 0.25 (DeiT), 0.2 (Swin)
    Reported in Section 4.1; chosen without ablations, and it controls the quality of synthetic images.
assumptions (4)
  • standard math Assumption 1: Lipschitz smoothness of the synthesis loss gradient with respect to the crop scale delta (Eq. 12).
    A regularity condition used in Lemma 2; standard in optimization analysis but not verified for the actual loss Eq. (8).
  • standard math Assumption 2: beta-smoothness of the synthesis loss in the image x (Eq. 13).
    Standard descent lemma assumption; not verified for the adversarial synthesis loss, which includes one-hot cross-entropy and total variation.
  • standard math Assumption 3: bounded gradient norm G for all x and delta (Eq. 14).
    Standard bounded-gradient assumption; the actual loss gradients are not shown to be bounded.
  • ad hoc to paper Assumption 4: the cumulative squared gradient norm of E2H exceeds that of the fixed strategy by K G Delta / (eta - beta eta^2/2) (Eq. 15).
    This is the load-bearing premise of Theorem 1; it asserts the superiority mechanism of E2H and is never derived from Assumptions 1-3. It is effectively the conclusion of the theorem restated as an inequality over gradients.
invented entities (1)
  • Activation Correction Matrix (ACM)
    purpose: A per-layer bias vector computed as the mean difference between full-precision and quantized intermediate activations over the 16 synthetic calibration samples, added every gamma layers during inference to reduce quantization error accumulation (Eqs. 31-33).
    The paper introduces ACM specifically for this pipeline; its effectiveness is only shown in the internal ablation (Table 3), and no external prediction or release verifies it independently.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DFQ-ViT: Data-Free Quantization for Vision Transformers without Fine-tuning." pith.science (2026). https://pith.science/paper/CYRLE2FV

@misc{pith2026250714481,
  author       = {Pith},
  title        = {Pith review of: DFQ-ViT: Data-Free Quantization for Vision Transformers without Fine-tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CYRLE2FV}},
  note         = {Machine review of arXiv:2507.14481}
}
read the original abstract

Data-Free Quantization (DFQ) enables the quantization of Vision Transformers (ViTs) without requiring access to data, allowing for the deployment of ViTs on devices with limited resources. In DFQ, the quantization model must be calibrated using synthetic samples, making the quality of these synthetic samples crucial. Existing methods fail to fully capture and balance the global and local features within the samples, resulting in limited synthetic data quality. Moreover, we have found that during inference, there is a significant difference in the distributions of intermediate layer activations between the quantized and full-precision models. These issues lead to a severe performance degradation of the quantized model. To address these problems, we propose a pipeline for Data-Free Quantization for Vision Transformers (DFQ-ViT). Specifically, we synthesize samples in order of increasing difficulty, effectively enhancing the quality of synthetic data. During the calibration and inference stage, we introduce the activation correction matrix for the quantized model to align the intermediate layer activations with those of the full-precision model. Extensive experiments demonstrate that DFQ-ViT achieves remarkable superiority over existing DFQ methods and its performance is on par with models quantized through real data. For example, the performance of DeiT-T with 3-bit weights quantization is 4.29% higher than the state-of-the-art. Our method eliminates the need for fine-tuning, which not only reduces computational overhead but also lowers the deployment barriers for edge devices. This characteristic aligns with the principles of Green Learning by improving energy efficiency and facilitating real-world applications in resource-constrained environments.

Figures

Figures reproduced from arXiv: 2507.14481 by the authors.

Figure 1
Figure 1. Comparison of sample synthesis strategies. (a) Existing methods use fixed maximum crop size which primarily captures [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The overview of DFQ-ViT. During the sample generation phase, we consider the difficulty and order of synthetic samples, [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Top-1 accuracy of various ViT models under W4/A8 quantization settings after data-free quantization on the CIFAR-10 and [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Visualization of synthetic samples: The first column is Gaussian Noise. For columns from the second onward, the first row [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]
Figure 5
Figure 5. Figure 5: Visualizations of 100, 200, 300, 500 steps (salamander) [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 29 canonical work pages

  1. [1]

    Ron Banner, Yury Nahshan, and Daniel Soudry. 2019. Post training 4-bit quantization of convolutional networks for rapid-deployment. Advances in Neural Information Processing Systems 32 (2019)

    work page 2019

  2. [2]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901

    2020

  3. [3]

    Yaohui Cai, Zhewei Yao, Zhen Dong, Amir Gholami, Michael W Mahoney, and Kurt Keutzer. 2020. Zeroq: A novel zero shot quantization framework. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 13169–13178

    work page 2020

  4. [4]

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. 2020. End-to-end object detection with transformers. In European conference on computer vision . Springer, 213–229

    work page 2020

  5. [5]

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In2009 IEEE conference on computer vision and pattern recognition . Ieee, 248–255

    2009

  6. [6]

    Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale.Advances in neural information processing systems 35 (2022), 30318–30332

    work page 2022

  7. [7]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers) . 4171–4186

    2019

  8. [8]

    Wei Dong, Xing Zhang, Bihui Chen, Dawei Yan, Zhijun Lin, Qingsen Yan, Peng Wang, and Yang Yang. 2024. Low-rank rescaled vision transformer fine-tuning: A residual design approach. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 16101–16110

    work page 2024

Show all 46 references
  1. [9]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, G Heigold, S Gelly, et al. 2020. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In International Confere...

    2020

  2. [10]

    Steven K Esser, Jeffrey L McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dharmendra S Modha. 2019. Learned step size quantization. arXiv preprint arXiv:1902.08153 (2019)

    2019 arXiv

  3. [11]

    Elias Frantar and Dan Alistarh. 2023. Sparsegpt: Massive language models can be accurately pruned in one-shot. In International Conference on Machine Learning. PMLR, 10323–10337

    2023

  4. [12]

    Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W Mahoney, and Kurt Keutzer. 2022. A survey of quantization methods for efficient neural network inference. In Low-Power Computer Vision. Chapman and Hall/CRC, 291–326

    2022

  5. [13]

    Alex Graves, Marc G Bellemare, Jacob Menick, Remi Munos, and Koray Kavukcuoglu. 2017. Automated curriculum learning for neural networks. In international conference on machine learning . Pmlr, 1311–1320

    2017

  6. [14]

    Guy Hacohen and Daphna Weinshall. 2019. On the power of curriculum learning in training deep networks. In International conference on machine learning. PMLR, 2535–2544

    2019

  7. [15]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition . 770–778

    2016

  8. [16]

    Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh, Michael W Mahoney, Sophia Shao, Kurt Keutzer, and Amir Gholami. 2024. Kvquant: Towards 10 million context length llm inference with kv cache quantization. Advances in Neural Information Processing Systems 37 (2024), 1270–1303

    2024

  9. [17]

    Raghuraman Krishnamoorthi. 2018. Quantizing deep convolutional networks for efficient inference: A whitepaper. arXiv preprint arXiv:1806.08342 (2018)

    2018 arXiv

  10. [18]

    Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning multiple layers of features from tiny images. (2009)

    2009

  11. [19]

    Changhun Lee, Jungyu Jin, Taesu Kim, Hyungjun Kim, and Eunhyeok Park. 2024. Owq: Outlier-aware weight quantization for efficient fine-tuning and inference of large language models. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 38. 13355–13364

    2024

  12. [20]

    Fengfu Li, Bin Liu, Xiaoxing Wang, Bo Zhang, and Junchi Yan. 2016. Ternary weight networks. arXiv preprint arXiv:1605.04711 (2016)

    2016 arXiv

  13. [21]

    Ming Li, Lichang Chen, Jiuhai Chen, Shwai He, Jiuxiang Gu, and Tianyi Zhou. 2024. Selective reflection-tuning: Student-selected data recycling for llm instruction-tuning. In Findings of the Association for Computational Linguistics ACL 2024 . 16189–16211

    2024

  14. [22]

    Yanjing Li, Sheng Xu, Baochang Zhang, Xianbin Cao, Peng Gao, and Guodong Guo. 2022. Q-vit: Accurate and fully quantized low-bit vision transformer. Advances in neural information processing systems 35 (2022), 34451–34463

    2022

  15. [23]

    Zhikai Li, Mengjuan Chen, Junrui Xiao, and Qingyi Gu. 2023. PSAQ-ViT v2: Toward accurate and general data-free quantization for vision transformers. IEEE Transactions on Neural Networks and Learning Systems (2023)

    2023

  16. [24]

    Zhikai Li, Liping Ma, Mengjuan Chen, Junrui Xiao, and Qingyi Gu. 2022. Patch similarity aware data-free quantization for vision transformers. In European conference on computer vision . Springer, 154–170

    2022

  17. [25]

    Yang Lin, Tianyu Zhang, Peiqin Sun, Zheng Li, and Shuchang Zhou. 2021. Fq-vit: Post-training quantization for fully quantized vision transformer. arXiv preprint arXiv:2111.13824 (2021)

    2021 arXiv

  18. [26]

    Xiangbin Liu, Jiesheng He, Liping Song, Shuai Liu, and Gautam Srivastava. 2021. Medical image classification based on an adaptive size deep learning model. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 17, 3s (2021), 1–18

    2021

  19. [27]

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision . 10012–10022. Manuscript submi...

    2021

  20. [28]

    Zhenhua Liu, Yunhe Wang, Kai Han, Wei Zhang, Siwei Ma, and Wen Gao. 2021. Post-training quantization for vision transformer.Advances in Neural Information Processing Systems 34 (2021), 28092–28103

    2021

  21. [29]

    Markus Nagel, Mart van Baalen, Tijmen Blankevoort, and Max Welling. 2019. Data-free quantization through weight equalization and bias correction. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 1325–1334

    2019

  22. [30]

    Biao Qian, Yang Wang, Richang Hong, and Meng Wang. 2023. Rethinking data-free quantization as a zero-sum game. In Proceedings of the AAAI conference on artificial intelligence , Vol. 37. 9489–9497

    2023

  23. [31]

    Rajarshi Saha, Varun Srivastava, and Mert Pilanci. 2023. Matrix compression via randomized low rank and low precision factorization. Advances in Neural Information Processing Systems 36 (2023)

    2023

  24. [32]

    Min Tan, Fu Yuan, Jun Yu, Guijun Wang, and Xiaoling Gu. 2022. Fine-grained image classification via multi-scale selective hierarchical biquadratic pooling. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 18, 1s (2022), 1–23

    2022

  25. [33]

    Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. 2021. Training data-efficient image transformers & distillation through attention. In International conference on machine learning . PMLR, 10347–10357

    2021

  26. [34]

    Xin Wang, Yudong Chen, and Wenwu Zhu. 2021. A survey on curriculum learning. IEEE transactions on pattern analysis and machine intelligence 44, 9 (2021), 4555–4576

    2021

  27. [35]

    Kan Wu, Jinnian Zhang, Houwen Peng, Mengchen Liu, Bin Xiao, Jianlong Fu, and Lu Yuan. 2022. Tinyvit: Fast pretraining distillation for small vision transformers. In European conference on computer vision . Springer, 68–85

    2022

  28. [36]

    Shoukai Xu, Haokun Li, Bohan Zhuang, Jing Liu, Jiezhang Cao, Chuangrun Liang, and Mingkui Tan. 2020. Generative low-bitwidth data free quantization. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XII 16 . Springer, 1–17

    2020

  29. [37]

    Xin Xu, Shiqin Wang, Zheng Wang, Xiaolong Zhang, and Ruimin Hu. 2021. Exploring image enhancement for salient object detection in low light images. ACM transactions on multimedia computing, communications, and applications (TOMM) 17, 1s (2021), 1–19

    2021

  30. [38]

    Huanrui Yang, Hongxu Yin, Maying Shen, Pavlo Molchanov, Hai Li, and Jan Kautz. 2023. Global Vision Transformer Pruning With Hessian-Aware Saliency. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 18547–18557

    2023

  31. [39]

    Zhendong Yang, Zhe Li, Ailing Zeng, Zexian Li, Chun Yuan, and Yu Li. 2024. ViTKD: Feature-based knowledge distillation for vision transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 1379–1388

    2024

  32. [40]

    Zeyuan Yin and Zhiqiang Shen. 2024. Dataset Distillation via Curriculum Data Synthesis in Large Data Era. Transactions on Machine Learning Research (2024)

    2024

  33. [41]

    Hao Yu and Jianxin Wu. 2023. A unified pruning framework for vision transformers. Science China Information Sciences 66, 7 (2023), 179101

    2023

  34. [42]

    Zhihang Yuan, Chenhao Xue, Yiqi Chen, Qiang Wu, and Guangyu Sun. 2021. PTQ4ViT: Post-training quantization framework for vision transformers with twin uniform quantization. arXiv preprint arXiv:2111.12293 (2021)

    2021 arXiv

  35. [43]

    Xiangguo Zhang, Haotong Qin, Yifu Ding, Ruihao Gong, Qinghua Yan, Renshuai Tao, Yuhang Li, Fengwei Yu, and Xianglong Liu. 2021. Diversifying sample generation for accurate data-free quantization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognit...

    2021

  36. [44]

    Yunshan Zhong, Mingbao Lin, Gongrui Nan, Jianzhuang Liu, Baochang Zhang, Yonghong Tian, and Rongrong Ji. 2022. Intraq: Learning synthetic images with intra-class heterogeneity for zero-shot network quantization. In Proceedings of the IEEE/CVF Conference on Computer Vision and ...

    2022

  37. [45]

    Shuchang Zhou, Yuxin Wu, Zekun Ni, Xinyu Zhou, He Wen, and Yuheng Zou. 2016. Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients. arXiv preprint arXiv:1606.06160 (2016)

    2016 arXiv

  38. [46]

    Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. 2020. Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159 (2020). Manuscript submitted to ACM

    2020 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.