Pith. sign in

REVIEW 3 major objections 5 minor 70 references

MPQ-DMv2: Flexible Residual Mixed Precision Quantization for Low-Bit Diffusion Models with Temporal Distillation

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a residual binary branch, temporal distillation, and SVD-based LoRA initialization make 2-4 bit quantized diffusion models approach full-precision quality.

desk verdict Good empirical extension with three plausible components, but the SOTA claims and the OOLRI theory are both overstated as written; worth sending to review after a careful revision. read the letter →

arxiv 2507.04290 v1 pith:WZSDVECG submitted 2025-07-06 cs.CV

Weilun Feng , Chuanguang Yang , Haotong Qin , Yuqi Li , Xiangqi Li , Zhulin An , Libo Huang , Boyu Diao
show 5 more authors
Fuzhen Zhuang Michele Magno Yongjun Xu Yingli Tian Tingwen Huang
This is my paper
classification cs.CV
keywords diffusionmodelquantizationmixed-precisionresiduallow-bittemporalrelationdistillationlow-rankinitializationimagegenerationStable
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that extremely low-bit (2-4 bit) quantization of diffusion models can be made practical, with quality close to the full-precision model, by fixing three recognized bottlenecks: uniform quantizers mishandle outlier-heavy weight distributions, per-timestep activation quantization ignores the denoising trajectory, and LoRA fine-tuning starts from zero. It proposes three coordinated mechanisms: a flexible residual quantizer with a binary branch, a memory-based temporal relation distillation, and a quantization-error-based SVD initialization of LoRA, and reports consistent gains over prior methods on LDM, Stable Diffusion, and DiT-XL at several bit widths. If the claims hold, low-bit diffusion inference becomes usable on resource-constrained devices, with roughly 10-16x smaller models and 2-3x speedups.

What carries the argument

The load-bearing object is the residual quantization decomposition $\hat{Q}_n(x)=Q_{n-1}(x)+Q^{\mathrm{res}}_1(x-Q_{n-1}(x))$, where the residual branch is a binary quantizer $\Delta_{\mathrm{res}}\operatorname{sign}(r)$. This turns a fixed uniform quantizer into a piecewise nonuniform one with an adaptively learned step; channels share the base quantizer, so mixed precision $(n-1,n,n+1)$ is implemented as bit-masked tensor additions. The optimization machinery is the reduction of quantization-aware LoRA initialization to the low-rank approximation $\|E-L_1L_2\|_F^2$, whose optimum is the truncated SVD of the quantization residual $E=W-Q(W)$, plus the temporal distillation loss $L_{\mathrm{MTRD}}=\mathrm{KL}(r^{\mathrm{FP}}_t\parallel r^Q_t)$ over a memory queue of full-precision features.

What would settle it

Compute, on real LDM weights, the Frobenius norm of the difference between $Q(W+L_1L_2)$ and $Q(W)+L_1L_2$ for the small low-rank updates actually used in fine-tuning; if that difference is not much smaller than the quantization residual $E=W-Q(W)$, the linearization behind the SVD initialization is invalid. A second, cleaner test is to replace the SVD initialization with a random low-rank initialization and hold every other component fixed; if the FID gap at W2A4 largely disappears, the theoretical story is not bearing the empirical weight.

Watch

Extended reading notes

Core claim

The central claim is that the failure of existing diffusion-model quantization at 2-4 bits comes from three addressable causes, and that jointly fixing them yields state-of-the-art results. FZ-RMQ decomposes each channel's quantization into a shared (n-1)-bit base quantizer plus a binary residual quantizer, so the effective step size becomes piecewise nonuniform and can represent salient outliers without wasting bits on the dense core. MTRD keeps an online FIFO queue of full-precision features from all timesteps, builds a reference similarity matrix, and distills the quantized model's relational distribution to the full-precision one via KL divergence in $L_{\mathrm{total}}=L_{\mathrm{align}}+\alpha L_{\mathrm{MTRD}}$. OOLRI models the initialization problem as minimizing $\|E-L_1L_2\|_F^2$ with $E=W-Q(W)$, claims to prove this objective is convex and 2-smooth, and initializes $L_1,L_2$ from the top-$r$ singular vectors of $E$. The paper reports, among other results, ImageNet LDM-4 FID dropping from 36.59 to 32.55 at W2A4, LSUN-Churches FID from 21.83 to 15.50 at W2A4, and Stable Diffusion CLIP score from 26.96 to 28.28 at W3A4.

Load-bearing premise

The load-bearing premise is that quantization acts roughly linearly on the small LoRA correction, so quantizing the adjusted weight equals quantizing the original weight and then adding the correction; because quantization rounds and clips values, this linearity is exactly where the argument could break.

Editorial extensions

If this is right

  • At 2-4 bit settings the quantized model fits in roughly 1/10 to 1/16 of the full-precision memory footprint; the paper reports 2.04x to 3.35x runtime speedups on LDM-4.
  • The residual binary branch lets the same average bit budget represent outlier-heavy channels better than uniform mixed precision, so quality no longer collapses when a few channels dominate the quantization range.
  • Temporal relation distillation makes the quantized denoising trajectory behave more like the full-precision trajectory, reducing error accumulation over sampling steps.
  • Quantization-error-based LoRA initialization lowers channel-wise weight error before fine-tuning begins and shortens the calibration-time optimization needed to recover quality.
  • The framework transfers across U-Net latent diffusion, Stable Diffusion text-to-image, and Transformer DiT-XL architectures, with the largest reported gains at the most extreme bit widths.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the tested diffusion settings, the binary residual branch should transfer to any outlier-heavy weight tensor; uniform quantizers in large-language-model or video-model compression face the same sparse-outlier problem, and the added cost is one binary mask and tensor addition.
  • The temporal-relation distillation loss is not tied to quantization; the same online FIFO queue and KL alignment could supervise step-distilled few-step diffusion models, where trajectory consistency is the core training signal.
  • OOLRI's benefit is explicitly testable at larger ranks: if truncated SVD initialization only speeds convergence, then with enough fine-tuning iterations random initialization should catch up, which would separate optimization-speed value from a genuinely better optimum.
  • A fair external reproduction would check whether FZ-RMQ and OOLRI alone, without MTRD, already account for the full W2A4 gap to MPQ-DM on LDM-4; the paper's ablation suggests most of that gap comes from the first two components.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes MPQ-DMv2, a mixed-precision quantization framework for low-bit diffusion models. It introduces three components: Flexible Z-Order Residual Mixed Quantization (FZ-RMQ), which adds a binary residual quantizer to a uniform base quantizer to better handle outliers; Memory-based Temporal Relation Distillation (MTRD), which aligns quantized and full-precision features through a temporal memory queue and KL-based relational distillation; and Object-Oriented Low-Rank Initialization (OOLRI), which initializes LoRA adapters by truncated SVD of the quantization residual. The method is evaluated on LDM-4/LDM-8, Stable Diffusion v1.4, and DiT-XL/2 across 2–4 bit weight and activation settings, with FID/IS/sFID/Precision and CLIP score metrics, plus ablation and efficiency studies. The manuscript claims state-of-the-art results by a large margin over prior methods such as EfficientDM, MPQ-DM, PTQ-D, TFMQ, QuEST, and PTQ4DiT.

Significance. If the empirical claims held as stated, MPQ-DMv2 would be a practically valuable contribution: it reports roughly 10–16x model size reduction and 2–3x inference speedup with low-bit quantization of diffusion models, and it extends a previously published framework (MPQ-DM) across U-Net and Transformer architectures. The manuscript has genuine strengths: broad experiments across different model families, comparison against several external baselines rather than only the authors' own methods, a systematic ablation of the three proposed components, and efficiency measurements for both calibration and inference. However, the central 'great margin over SOTA' claim is not consistently supported by the paper's own tables, and one of the three proposed components rests on a theoretical derivation that is not valid as written; these issues are load-bearing for the manuscript's headline contribution and must be corrected.

major comments (3)
  1. [§V-D, Table I] The text in Section V-D states that 'our MPQ-DMv2 achieved FID surpassing FP method for the first time' under W2A6 on ImageNet LDM-4, but Table I reports FP FID 11.28, base MPQ-DMv2 FID 13.91, and MPQ-DM+ FID 11.00; only the MPQ-DMv2+ variant (FID 10.96) surpasses FP and MPQ-DM+. Similarly, at W2A4, base MPQ-DMv2 FID 32.55 is worse than MPQ-DM+ FID 27.11, and the gain over MPQ-DM+ for MPQ-DMv2+ is only 0.13 FID. The abstract's 'surpasses current SOTA methods by a great margin' is therefore not supported for the base method in the most difficult ImageNet settings, and the text conflates the '+' variant with the base method. The authors should either compare the base method against MPQ-DM+ explicitly and revise the headline claim, or demonstrate the claimed margin for MPQ-DMv2+ separately without attributing it to the base variant.
  2. [§IV-C2, Eq. (23)] The derivation of OOLRI relies on the first-order Taylor expansion Q(W + L1L2) ≈ Q(W) + L1L2. This expansion is not justified because the quantizer Q involves rounding and clipping and is piecewise constant; its derivative is zero almost everywhere and undefined at discontinuities. Consequently, the reduction of Eq. (22) to the low-rank approximation problem Eq. (25) does not follow, and Theorem IV.1 only establishes convexity and smoothness of the surrogate f(X) = ||E − X||_F^2, not convergence or optimality for the actual QA-LoRA objective in Eq. (22). The theoretical support for OOLRI as currently stated is therefore invalid; the authors should remove or substantially reframe the theoretical claims, for example by presenting OOLRI as a heuristic initialization and supporting it only through the ablation evidence.
  3. [§V-F, Table V] The ablation study does not isolate the contribution of OOLRI. Starting from the '+MTRD (KL)' row, the next row '+OOLRI (MPQ-DMv2)' simultaneously adds OOLRI and reports the final full method, so the FID improvement from 33.02 to 32.55 cannot be attributed solely to OOLRI. An additional row that adds OOLRI to the FZ-RMQ baseline, or to the '+MTRD (KL)' configuration without changing any other component, is needed to support the claim that OOLRI is independently effective.
minor comments (5)
  1. [Fig. 2] The framework name is written 'MPQ-DM2' in the Figure 2 caption, while the rest of the manuscript uses 'MPQ-DMv2'.
  2. [§V-D, DiT experiment] The text says 'ImageNet 256×266 and 512×512'; the first resolution should be 256×256.
  3. [§V-D, DiT paragraph] The word 'prensent' should be 'present'.
  4. [§V-A, Implementation details] The hyperparameter notation for the memory queue is inconsistent: the text states 'we set L = 20 k and k = 1024 for ImageNet dataset' while Fig. 8 and the surrounding text refer to queue size L and sample size k; the units (number of feature vectors versus memory in MB) should be stated more clearly to avoid confusion.
  5. [§IV-A4, Eq. (9)] The optimization strategy selection in Eq. (9) is described as involving a search, but the overhead of this additional search and whether it is performed on calibration data or training data is not specified; a short clarification would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claims are benchmarked against external methods, and the self-citations to prior MPQ-DM and relational distillation work are not load-bearing; the OOLRI Taylor-expansion gap and the W2A6 text/table inconsistency are soundness and correctness issues, not circularity.

full rationale

MPQ-DMv2's headline result is an empirical comparison against external baselines (EfficientDM, PTQ-D, TFMQ, QuEST, PTQ4DiT, HAWQ-V3), so the central claim is not derived from its own outputs by construction. FZ-RMQ selects bit-widths and optimization modes by minimizing activation-aware output error (Eq. 10); MTRD adds a KL relational distillation loss (Eqs. 15-19); OOLRI initializes LoRA from the truncated SVD of the quantization residual E = W - Q(W) (Eqs. 24-28). None of these equations defines the evaluated metric (FID/IS/CLIP) as its own input, and no fitted parameter is renamed as a prediction. Self-citations to MPQ-DM [33] and relational distillation [44] supply the baseline framework and distillation inspiration, but the paper's claims are not justified solely by those citations, and both cited works are externally published prior results. The OOLRI derivation does contain a genuine soundness gap: Eq. (23) approximates Q(W+L1L2) by Q(W)+L1L2 although Q is piecewise constant, and Theorem IV.1 proves convexity/smoothness of the surrogate objective only, not convergence of a concrete algorithm on the actual quantization loss; this is an unsupported-assumption issue, not a circular reduction. Likewise, the Sec. V-D statement that MPQ-DMv2 achieved FID surpassing FP under W2A6 conflicts with Table I, where base MPQ-DMv2 has FID 13.91 versus FP 11.28; that is an empirical inconsistency outside circularity scoring. Overall, the derivation chain is self-contained against external benchmarks, and no circular step was found.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central framework rests on modeling choices: calibration-data output discrepancy as a proxy for final quality, approximate linearity of the quantizer under low-rank updates, and stability of cached temporal features. The OOLRI theory is the weakest of these because the Taylor expansion is not justified for rounding quantizers. The method introduces algorithmic components but no new physical or ontological entities.

free parameters (6)
  • Distillation weight alpha = 1.0
    Selected by ablation on LDM-4 ImageNet W2A4 (Table VI); all tested alpha values improve over no distillation, with the best FID at alpha = 1.0.
  • Memory queue length L = 20,000 for ImageNet; 4,000 for LSUN/COCO
    Ablated in Fig. 8a; performance saturates around L = 20k, so larger queues are not used despite added memory.
  • Reference sampling size k = 1024 for ImageNet; 204 for LSUN/COCO
    Ablated in Fig. 8b; chosen at the saturation point of the performance-memory tradeoff.
  • Channel group size g = C/10 channels per group
    Empirically set in Sec. IV-A5 to balance search cost and granularity; no dedicated ablation is reported.
  • Extra 2-bit channel allocation for MPQ-DMv2+ = 10% of channels
    Follows the MPQ-DM implementation, adds about 0.6% model size, and is used for the '+' variants in Tables I, II, and IV.
  • Per-channel bit allocation {c_i} = n-1, n, or n+1 bits per channel
    Determined by optimizing Eq. (10) on calibration data; this searched configuration is central to FZ-RMQ and is not derived from an independent principle.
assumptions (4)
  • standard math Eckart-Young-Mirsky theorem gives the optimal rank-r approximation in Frobenius norm.
    Invoked in Sec. IV-C4 to justify the truncated SVD initialization of the LoRA matrices.
  • domain assumption The quantizer Q is locally smooth enough for the first-order expansion Q(W+L1L2) is approximately Q(W)+L1L2.
    Used in Eq. (23) of Sec. IV-C2; questionable because rounding and clipping make the derivative zero or undefined almost everywhere.
  • domain assumption Minimizing output discrepancy on calibration data (Eq. 10) selects bit allocations and optimization modes that generalize to the full denoising trajectory.
    FZ-RMQ search uses calibration activations only; no transfer guarantee is provided for unseen prompts or longer sampling trajectories.
  • domain assumption Cached full-precision features in FIFO queues remain representative anchors for temporal relation distillation throughout training.
    MTRD in Sec. IV-B2 assumes the queue preserves the temporal structure of the denoising process as weights are updated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MPQ-DMv2: Flexible Residual Mixed Precision Quantization for Low-Bit Diffusion Models with Temporal Distillation." pith.science (2026). https://pith.science/paper/WZSDVECG

@misc{pith2026250704290,
  author       = {Pith},
  title        = {Pith review of: MPQ-DMv2: Flexible Residual Mixed Precision Quantization for Low-Bit Diffusion Models with Temporal Distillation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WZSDVECG}},
  note         = {Machine review of arXiv:2507.04290}
}
read the original abstract

Diffusion models have demonstrated remarkable performance on vision generation tasks. However, the high computational complexity hinders its wide application on edge devices. Quantization has emerged as a promising technique for inference acceleration and memory reduction. However, existing quantization methods do not generalize well under extremely low-bit (2-4 bit) quantization. Directly applying these methods will cause severe performance degradation. We identify that the existing quantization framework suffers from the outlier-unfriendly quantizer design, suboptimal initialization, and optimization strategy. We present MPQ-DMv2, an improved \textbf{M}ixed \textbf{P}recision \textbf{Q}uantization framework for extremely low-bit \textbf{D}iffusion \textbf{M}odels. For the quantization perspective, the imbalanced distribution caused by salient outliers is quantization-unfriendly for uniform quantizer. We propose \textit{Flexible Z-Order Residual Mixed Quantization} that utilizes an efficient binary residual branch for flexible quant steps to handle salient error. For the optimization framework, we theoretically analyzed the convergence and optimality of the LoRA module and propose \textit{Object-Oriented Low-Rank Initialization} to use prior quantization error for informative initialization. We then propose \textit{Memory-based Temporal Relation Distillation} to construct an online time-aware pixel queue for long-term denoising temporal information distillation, which ensures the overall temporal consistency between quantized and full-precision model. Comprehensive experiments on various generation tasks show that our MPQ-DMv2 surpasses current SOTA methods by a great margin on different architectures, especially under extremely low-bit widths.

Figures

Figures reproduced from arXiv: 2507.04290 by the authors.

Figure 1
Figure 1. The FID score for LDM-8 LSUN-Churches model under different quantization settings, lower FID indicates better performance. WxAy denotes x-bit weight and y-bit activation quantization, e.g., W2A4 denotes 2-bit weight and 4-bit activation quantization. Our MPQ-DMv2 surpasses current quantization methods by a great margin. I. INTRODUCTION Diffusion Models (DMs) [1], [2] have recently emerged as a powerful generative pa… view at source ↗
Figure 2
Figure 2. Overview of proposed MPQ-DM2 framework. The framework consists of Flexible Z-Order Residual Mixed Quantization to use efficient binary branch for flexible quantizer design, Memory-based Temporal Relation Distillation for denoising temporal consistency distillation, and Object-Oriented Low￾Rank Initialization to use prior quantization error for informative initialization. successes, quantizing diffusion models remain… view at source ↗
Figure 3
Figure 3. Comparison of 3-bit quantization step size distribution on LDM [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Cosine similarity heatmaps across timesteps in the denoising process [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 4
Figure 4. Figure 4: With MTRD, the quantized correlation heatmaps are [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Channel-wise weight error distributions under W3A6 quantization [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Visual comparison of different methods under W3A4 quantization setting on ImageNet 256 [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Visual comparison of different methods under W4A6 quantization [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Ablation study on memory-based online queue hyper-parameters. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

70 extracted references · 34 canonical work pages

  1. [1]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems , vol. 33, pp. 6840– 6851, 2020

    2020

  2. [2]

    Diffusion models beat gans on image synthesis,

    P. Dhariwal and A. Nichol, “Diffusion models beat gans on image synthesis,” Advances in neural information processing systems , vol. 34, pp. 8780–8794, 2021

    2021

  3. [3]

    High- resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High- resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10 684–10 695

    2022

  4. [4]

    Text- guided mask-free local image retouching,

    Z. Liu, F. Zhang, J. He, J. Wang, Z. Wang, and L. Cheng, “Text- guided mask-free local image retouching,” in 2023 IEEE International Conference on Multimedia and Expo (ICME) . IEEE, 2023, pp. 2783– 2788

    work page 2023

  5. [5]

    Vidm: Video implicit diffusion models,

    K. Mei and V . Patel, “Vidm: Video implicit diffusion models,” in Proceedings of the AAAI conference on artificial intelligence , vol. 37, no. 8, 2023, pp. 9117–9125

    work page 2023

  6. [6]

    Cogvideo: Large- scale pretraining for text-to-video generation via transformers,

    W. Hong, M. Ding, W. Zheng, X. Liu, and J. Tang, “Cogvideo: Large- scale pretraining for text-to-video generation via transformers,” arXiv preprint arXiv:2205.15868, 2022

    arXiv 2022

  7. [7]

    Sora: A review on background, technology, limitations, and opportunities of large vision models,

    Y . Liu, K. Zhang, Y . Li, Z. Yan, C. Gao, R. Chen, Z. Yuan, Y . Huang, H. Sun, J. Gao et al. , “Sora: A review on background, technology, limitations, and opportunities of large vision models,” arXiv preprint arXiv:2402.17177, 2024

    arXiv 2024

  8. [8]

    Diffusion-based layer-wise semantic reconstruction for unsupervised out-of-distribution detection,

    Y . Yang, D. Cheng, C. Fang, Y . Wang, C. Jiao, L. Cheng, and N. Wang, “Diffusion-based layer-wise semantic reconstruction for unsupervised out-of-distribution detection,” 2024

    work page 2024

Show all 70 references
  1. [9]

    Osdface: One-step diffusion model for face restoration,

    J. Wang, J. Gong, L. Zhang, Z. Chen, X. Liu, H. Gu, Y . Liu, Y . Zhang, and X. Yang, “Osdface: One-step diffusion model for face restoration,” in Proceedings of the Computer Vision and Pattern Recognition Confer- ence, 2025, pp. 12 626–12 636

    2025

  2. [10]

    A survey on audio diffusion models: Text to speech synthesis and enhancement in generative ai,

    C. Zhang, C. Zhang, S. Zheng, M. Zhang, M. Qamar, S.-H. Bae, and I. S. Kweon, “A survey on audio diffusion models: Text to speech synthesis and enhancement in generative ai,” arXiv preprint arXiv:2303.13336 , 2023

    2023 arXiv

  3. [11]

    Diffusion models in vision: A survey,

    F.-A. Croitoru, V . Hondru, R. T. Ionescu, and M. Shah, “Diffusion models in vision: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 45, no. 9, pp. 10 850–10 869, 2023

    2023

  4. [12]

    A resource-aware workload scheduling method for unbalanced gemms on gpus,

    H. Liu, B. Diao, W. Chen, and Y . Xu, “A resource-aware workload scheduling method for unbalanced gemms on gpus,” The Computer Journal , p. bxae110, 10 2024. [Online]. Available: https://doi.org/10.1093/comjnl/bxae110

    2024 doi

  5. [13]

    Sketch-fusion: A gradient compression method with multi-layer fusion for communication- efficient distributed training,

    L. Dai, L. Gong, Z. An, Y . Xu, and B. Diao, “Sketch-fusion: A gradient compression method with multi-layer fusion for communication- efficient distributed training,” Journal of Parallel and Distributed Computing, vol. 185, p. 104811, 2024. [Online]. Available: https: //www.sc...

    2024

  6. [14]

    Foundation models and intelligent decision-making: Progress, challenges, and perspectives,

    J. Huang, Y . Xu, Q. Wang, Q. C. Wang, X. Liang, F. Wang, Z. Zhang, W. Wei, B. Zhang, L. Huang et al., “Foundation models and intelligent decision-making: Progress, challenges, and perspectives,” The Innova- tion, 2025

    2025

  7. [15]

    A survey of quantization methods for efficient neural network infer- ence,

    A. Gholami, S. Kim, Z. Dong, Z. Yao, M. W. Mahoney, and K. Keutzer, “A survey of quantization methods for efficient neural network infer- ence,” in Low-Power Computer Vision. Chapman and Hall/CRC, 2022, pp. 291–326

    2022

  8. [16]

    Compression of convolutional neural networks: A short survey,

    R. Pilipovi ´c, P. Buli´c, and V . Risojevi´c, “Compression of convolutional neural networks: A short survey,” in 2018 17th International Symposium INFOTEH-JAHORINA (INFOTEH). IEEE, 2018, pp. 1–6

    2018

  9. [17]

    Reg-ptq: Regression- specialized post-training quantization for fully quantized object detec- tor,

    Y . Ding, W. Feng, C. Chen, J. Guo, and X. Liu, “Reg-ptq: Regression- specialized post-training quantization for fully quantized object detec- tor,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 16 174–16 184

    2024

  10. [18]

    Scp: A structure combination pruning method via structured sparse for deep convolutional neural net- works,

    Q. Chen, B. Diao, Y . Yang, and Y . Xu, “Scp: A structure combination pruning method via structured sparse for deep convolutional neural net- works,” in International Conference on Pattern Recognition . Springer, 2024, pp. 238–253

    2024

  11. [19]

    A survey of techniques for optimizing transformer inference,

    K. T. Chitty-Venkata, S. Mittal, M. Emani, V . Vishwanath, and A. K. Somani, “A survey of techniques for optimizing transformer inference,” Journal of Systems Architecture , p. 102990, 2023

    2023

  12. [20]

    Q-mamba: Towards more efficient mamba models via post-training quantization,

    C. Tianqi, Y . Chen, W. Xu, Z. Zhu, P. Wang, and J. Cheng, “Q-mamba: Towards more efficient mamba models via post-training quantization,” 2025

    2025

  13. [21]

    Q-mamba: On first exploration of vision mamba for image quality assessment,

    F. Guan, X. Li, Z. Yu, Y . Lu, and Z. Chen, “Q-mamba: On first exploration of vision mamba for image quality assessment,” arXiv preprint arXiv:2406.09546, 2024

    2024 arXiv

  14. [22]

    Learned step size quantization,

    S. K. Esser, J. L. McKinstry, D. Bablani, R. Appuswamy, and D. S. Modha, “Learned step size quantization,” arXiv preprint arXiv:1902.08153, 2019

    1902 arXiv

  15. [23]

    Quantization and training of neural networks for efficient integer-arithmetic-only inference,

    B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, and D. Kalenichenko, “Quantization and training of neural networks for efficient integer-arithmetic-only inference,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 2...

    2018

  16. [24]

    Quantizing deep convolutional networks for efficient inference: A whitepaper. arxiv 2018,

    R. Krishnamoorthi, “Quantizing deep convolutional networks for efficient inference: A whitepaper. arxiv 2018,” arXiv preprint arXiv:1806.08342, 1806

    2018 arXiv

  17. [25]

    Binarydm: Towards accurate binarization of diffusion model,

    X. Zheng, H. Qin, X. Ma, M. Zhang, H. Hao, J. Wang, Z. Zhao, J. Guo, and X. Liu, “Binarydm: Towards accurate binarization of diffusion model,” arXiv preprint arXiv:2404.05662 , 2024

    2024 arXiv

  18. [26]

    Q-dm: An efficient low-bit quantized diffusion model,

    Y . Li, S. Xu, X. Cao, X. Sun, and B. Zhang, “Q-dm: An efficient low-bit quantized diffusion model,” Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  19. [27]

    Bidm: Pushing the limit of quantization for diffusion models,

    X. Zheng, X. Liu, Y . Bian, X. Ma, Y . Zhang, J. Wang, J. Guo, and H. Qin, “Bidm: Pushing the limit of quantization for diffusion models,” arXiv preprint arXiv:2412.05926 , 2024

    2024 arXiv

  20. [28]

    Improving post training neural quantization: Layer-wise calibration and integer programming,

    I. Hubara, Y . Nahshan, Y . Hanani, R. Banner, and D. Soudry, “Improving post training neural quantization: Layer-wise calibration and integer programming,” arXiv preprint arXiv:2006.10518 , 2020

    2006 arXiv

  21. [29]

    Towards accurate post- training network quantization via bit-split and stitching,

    P. Wang, Q. Chen, X. He, and J. Cheng, “Towards accurate post- training network quantization via bit-split and stitching,” in International Conference on Machine Learning . PMLR, 2020, pp. 9847–9856

    2020

  22. [30]

    Qdrop: Randomly dropping quantization for extremely low-bit post-training quantization,

    X. Wei, R. Gong, Y . Li, X. Liu, and F. Yu, “Qdrop: Randomly dropping quantization for extremely low-bit post-training quantization,” arXiv preprint arXiv:2203.05740, 2022

    2022 arXiv

  23. [31]

    Pd-quant: Post-training quantization based on prediction difference metric,

    J. Liu, L. Niu, Z. Yuan, D. Yang, X. Wang, and W. Liu, “Pd-quant: Post-training quantization based on prediction difference metric,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 24 427–24 437

    2023

  24. [32]

    Efficientdm: Effi- cient quantization-aware fine-tuning of low-bit diffusion models,

    Y . He, J. Liu, W. Wu, H. Zhou, and B. Zhuang, “Efficientdm: Effi- cient quantization-aware fine-tuning of low-bit diffusion models,” arXiv preprint arXiv:2310.03270, 2023. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 15

    2023 arXiv

  25. [33]

    Mpq-dm: Mixed precision quantization for extremely low bit diffusion models,

    W. Feng, H. Qin, C. Yang, Z. An, L. Huang, B. Diao, F. Wang, R. Tao, Y . Xu, and M. Magno, “Mpq-dm: Mixed precision quantization for extremely low bit diffusion models,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 39, no. 16, 2025, pp. 16 595– 16 603

    2025

  26. [34]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685 , 2021

    2021 arXiv

  27. [35]

    Denoising diffusion implicit models,

    J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” arXiv preprint arXiv:2010.02502 , 2020

    2010 arXiv

  28. [36]

    Scalable diffusion models with transformers,

    W. Peebles and S. Xie, “Scalable diffusion models with transformers,” in Proceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 4195–4205

    2023

  29. [37]

    Multi-party collaborative attention control for image customization,

    H. Yang, C. Yang, Q. Wang, Z. An, W. Feng, L. Huang, and Y . Xu, “Multi-party collaborative attention control for image customization,” in Proceedings of the Computer Vision and Pattern Recognition Confer- ence, 2025, pp. 7942–7951

    2025

  30. [38]

    One-step effective diffusion network for real-world image super-resolution,

    R. Wu, L. Sun, Z. Ma, and L. Zhang, “One-step effective diffusion network for real-world image super-resolution,” Advances in Neural Information Processing Systems , vol. 37, pp. 92 529–92 553, 2024

    2024

  31. [39]

    Pseudo numerical methods for diffusion models on manifolds,

    L. Liu, Y . Ren, Z. Lin, and Z. Zhao, “Pseudo numerical methods for diffusion models on manifolds,” arXiv preprint arXiv:2202.09778, 2022

    2022 arXiv

  32. [40]

    Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps,

    C. Lu, Y . Zhou, F. Bao, J. Chen, C. Li, and J. Zhu, “Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps,” Advances in Neural Information Processing Systems , vol. 35, pp. 5775–5787, 2022

    2022

  33. [41]

    Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models,

    ——, “Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models,” arXiv preprint arXiv:2211.01095 , 2022

    2022 arXiv

  34. [42]

    Progressive distillation for fast sampling of diffusion models,

    T. Salimans and J. Ho, “Progressive distillation for fast sampling of diffusion models,” arXiv preprint arXiv:2202.00512 , 2022

    2022 arXiv

  35. [43]

    Consistency models,

    Y . Song, P. Dhariwal, M. Chen, and I. Sutskever, “Consistency models,” 2023

    2023

  36. [44]

    Relational diffusion distillation for efficient image generation,

    W. Feng, C. Yang, Z. An, L. Huang, B. Diao, F. Wang, and Y . Xu, “Relational diffusion distillation for efficient image generation,” in Proceedings of the 32nd ACM International Conference on Multimedia , 2024, pp. 205–213

    2024

  37. [45]

    Q-vdit: Towards accurate quantization and distillation of video-generation diffusion transformers,

    W. Feng, C. Yang, H. Qin, X. Li, Y . Wang, Z. An, L. Huang, B. Diao, Z. Zhao, Y . Xu et al. , “Q-vdit: Towards accurate quantization and distillation of video-generation diffusion transformers,” arXiv preprint arXiv:2505.22167, 2025

    2025 arXiv

  38. [46]

    Diverse sample generation: Pushing the limit of generative data-free quantization,

    H. Qin, Y . Ding, X. Zhang, J. Wang, X. Liu, and J. Lu, “Diverse sample generation: Pushing the limit of generative data-free quantization,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 45, no. 10, pp. 11 689–11 706, 2023

    2023

  39. [47]

    Pushing the limit of post-training quantization,

    R. Gong, X. Liu, Y . Li, Y . Fan, X. Wei, and J. Guo, “Pushing the limit of post-training quantization,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2025

    2025

  40. [48]

    Hawq: Hessian aware quantization of neural networks with mixed-precision,

    Z. Dong, Z. Yao, A. Gholami, M. W. Mahoney, and K. Keutzer, “Hawq: Hessian aware quantization of neural networks with mixed-precision,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 293–302

    2019

  41. [49]

    Smoothquant: Accurate and efficient post-training quantization for large language models,

    G. Xiao, J. Lin, M. Seznec, H. Wu, J. Demouth, and S. Han, “Smoothquant: Accurate and efficient post-training quantization for large language models,” in International Conference on Machine Learning . PMLR, 2023, pp. 38 087–38 099

    2023

  42. [50]

    Q-diffusion: Quantizing diffusion models,

    X. Li, Y . Liu, L. Lian, H. Yang, Z. Dong, D. Kang, S. Zhang, and K. Keutzer, “Q-diffusion: Quantizing diffusion models,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 17 535–17 545

    2023

  43. [51]

    Post-training quantiza- tion on diffusion models,

    Y . Shang, Z. Yuan, B. Xie, B. Wu, and Y . Yan, “Post-training quantiza- tion on diffusion models,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2023, pp. 1972–1981

    2023

  44. [52]

    Ptqd: Accurate post-training quantization for diffusion models,

    Y . He, L. Liu, J. Liu, W. Wu, H. Zhou, and B. Zhuang, “Ptqd: Accurate post-training quantization for diffusion models,” Advances in Neural Information Processing Systems , vol. 36, 2024

    2024

  45. [53]

    Tfmq-dm: Temporal feature maintenance quantization for diffusion models,

    Y . Huang, R. Gong, J. Liu, T. Chen, and X. Liu, “Tfmq-dm: Temporal feature maintenance quantization for diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, 2024, pp. 7362–7371

    2024

  46. [54]

    Towards accu- rate post-training quantization for diffusion models,

    C. Wang, Z. Wang, X. Xu, Y . Tang, J. Zhou, and J. Lu, “Towards accu- rate post-training quantization for diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 16 026–16 035

    2024

  47. [55]

    Ptq4dit: Post-training quantization for diffusion transformers,

    J. Wu, H. Wang, Y . Shang, M. Shah, and Y . Yan, “Ptq4dit: Post-training quantization for diffusion transformers,” arXiv preprint arXiv:2405.16005, 2024

    2024 arXiv

  48. [56]

    Quest: Low-bit diffusion model quantization via efficient selective finetuning,

    H. Wang, Y . Shang, Z. Yuan, J. Wu, and Y . Yan, “Quest: Low-bit diffusion model quantization via efficient selective finetuning,” arXiv preprint arXiv:2402.03666, 2024

    2024 arXiv

  49. [57]

    Ompq: Orthogonal mixed precision quantization,

    Y . Ma, T. Jin, X. Zheng, Y . Wang, H. Li, Y . Wu, G. Jiang, W. Zhang, and R. Ji, “Ompq: Orthogonal mixed precision quantization,” in Proceedings of the AAAI conference on artificial intelligence , vol. 37, no. 7, 2023, pp. 9029–9037

    2023

  50. [58]

    dabnn: A super fast inference framework for binary neural networks on arm devices,

    J. Zhang, Y . Pan, T. Yao, H. Zhao, and T. Mei, “dabnn: A super fast inference framework for binary neural networks on arm devices,” in Proceedings of the 27th ACM international conference on multimedia , 2019, pp. 2272–2275

    2019

  51. [59]

    Online knowledge distillation via mutual contrastive learning for visual recogni- tion,

    C. Yang, Z. An, H. Zhou, F. Zhuang, Y . Xu, and Q. Zhang, “Online knowledge distillation via mutual contrastive learning for visual recogni- tion,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 45, no. 8, pp. 10 212–10 227, 2023

    2023

  52. [60]

    Self-supervised visual feature learning with deep neural networks: A survey,

    L. Jing and Y . Tian, “Self-supervised visual feature learning with deep neural networks: A survey,” IEEE transactions on pattern analysis and machine intelligence, vol. 43, no. 11, pp. 4037–4058, 2020

    2020

  53. [61]

    A generalization of the eckart-young-mirsky matrix approximation theorem,

    G. H. Golub, A. Hoffman, and G. W. Stewart, “A generalization of the eckart-young-mirsky matrix approximation theorem,”Linear Algebra and its applications , vol. 88, pp. 317–327, 1987

    1987

  54. [62]

    Hawq-v3: Dyadic neural net- work quantization,

    Z. Yao, Z. Dong, Z. Zheng, A. Gholami, J. Yu, E. Tan, L. Wang, Q. Huang, Y . Wang, M. Mahoney et al., “Hawq-v3: Dyadic neural net- work quantization,” in International Conference on Machine Learning . PMLR, 2021, pp. 11 875–11 886

    2021

  55. [63]

    Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop,

    F. Yu, A. Seff, Y . Zhang, S. Song, T. Funkhouser, and J. Xiao, “Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop,” arXiv preprint arXiv:1506.03365 , 2015

    2015 arXiv

  56. [64]

    Imagenet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition . Ieee, 2009, pp. 248–255

    2009

  57. [65]

    Microsoft coco: Common objects in context,

    T.-Y . Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll ´ar, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part v 13 . Springer,...

    2014

  58. [66]

    Improved techniques for training gans,

    T. Salimans, I. Goodfellow, W. Zaremba, V . Cheung, A. Radford, and X. Chen, “Improved techniques for training gans,” Advances in neural information processing systems , vol. 29, 2016

    2016

  59. [67]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium,

    M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “Gans trained by a two time-scale update rule converge to a local nash equilibrium,” Advances in neural information processing systems , vol. 30, 2017

    2017

  60. [68]

    Generating images with sparse representations,

    C. Nash, J. Menick, S. Dieleman, and P. W. Battaglia, “Generating images with sparse representations,” arXiv preprint arXiv:2103.03841 , 2021

    2021 arXiv

  61. [69]

    Clipscore: A reference-free evaluation metric for image captioning,

    J. Hessel, A. Holtzman, M. Forbes, R. L. Bras, and Y . Choi, “Clipscore: A reference-free evaluation metric for image captioning,” arXiv preprint arXiv:2104.08718, 2021

    2021 arXiv

  62. [70]

    Cross- image relational knowledge distillation for semantic segmentation,

    C. Yang, H. Zhou, Z. An, X. Jiang, Y . Xu, and Q. Zhang, “Cross- image relational knowledge distillation for semantic segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 12 319–12 328

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.