Pith. sign in

REVIEW 6 major objections 5 minor 3 cited by

FPSAttention: Training-Aware FP8 and Sparsity Co-Design for Fast Video Diffusion

T0 review · 6 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read FP8 quantization and sparsity co-designed to speed video diffusion 4.96x

desk verdict Real systems contribution undermined by an overclaimed quality claim and sloppy quality tables. read the letter →

arxiv 2506.04648 v2 pith:5S2ZWVWM submitted 2025-06-05 cs.CV

Akide Liu , Zeyu Zhang , Zhexin Li , Xuehai Bai , Yizeng Han , Jiasheng Tang , Yuanjie Xing , Jichao Wu
show 7 more authors
Mingyang Yang Weihua Chen Jiahao He Yuanyu He Fan Wang Gholamreza Haffari Bohan Zhuang
This is my paper · ORCID
classification cs.CV
keywords FP8quantizationsparseattentionvideodiffusionquantization-awaretrainingtile-wisegranularitydenoisingstepscheduleFlashtransformeracceleration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FPSAttention claims that FP8 quantization and structured sparsity can be combined without quality loss, provided they are co-designed on the same 3D tile grid and trained jointly rather than applied naively at inference time. The paper argues that sparsity and quantization errors interact badly when applied separately: sparsity keeps high-magnitude attention scores, precisely where quantization error is largest. By unifying the granularity of both techniques, adapting it across denoising steps, and using a hardware-friendly fused kernel, the method reports a 7.09x attention kernel speedup and a 4.96x end-to-end speedup for 720p video generation on a 14B model, with VBench quality matching or slightly exceeding the BF16 baseline. If correct, this makes high-resolution video diffusion substantially faster on current GPUs.

What carries the argument

The load-bearing object is the 3D tile, a contiguous block of spatiotemporal tokens that serves simultaneously as the unit of FP8 quantization (one scale factor per tile) and the unit of block sparsity (the local window neighborhood). This single granularity ensures the sparse pattern and the quantization error land on the same hardware compute tiles, enabling a fused kernel built on FlashAttention-style execution with Hopper-architecture optimizations. The denoising step-aware schedule S(t) = [g(t), W(t)] then varies tile size and sparsity window across denoising phases, and quantization-aware training lets the model adapt to these approximation errors.

What would settle it

Run the same step-aware schedule without recalibration on a prompt distribution very different from the calibration prompts, or on a fine-tuned or longer-video variant, and check whether VBench quality drops measurably below the BF16 baseline while the speedup persists; if it does, the transferability of the schedule is falsified. A direct sweep over alpha1 and alpha2 on a held-out prompt set that finds no parameter choice preserving quality would also falsify the claim.

Watch

Extended reading notes

Core claim

The central claim is that jointly optimized FP8 quantization and block sparsity can accelerate 3D attention in video diffusion transformers while preserving generation quality. The method quantizes Q and K per 3D tile, V per channel, and P with a fixed scalar, then applies a tile-wise sparse attention pattern derived from Sliding Tile Attention. A denoising step-aware schedule switches between coarse-and-sparse, fine-and-dense, and intermediate regimes across early, middle, and late steps, and the entire framework is trained with quantization-aware training. On Wan2.1-14B at 720p, the authors report a 7.09x kernel speedup for attention and a 4.96x end-to-end speedup over the BF16 baseline while matching or slightly improving VBench scores.

Load-bearing premise

The schedule in Equation (6) assumes a fixed three-regime sensitivity pattern (early tolerant, middle sensitive, late intermediate) that was measured on the same model and prompt distribution; if this pattern does not transfer to other prompts, resolutions, or architectures, the quality-preserving speedup claim could fail even though the kernel speedup would remain.

Editorial extensions

If this is right

  • Attention kernel time drops by 7.09x and end-to-end generation time by 4.96x at 720p on the 14B model, making multi-hour video generation on a single GPU considerably faster.
  • Joint training recovers the 21% quality drop seen with a naive training-free combination of quantization and sparsity, and on the 1.3B model the total VBench score actually improves from 0.8019 to 0.8160.
  • Early and late denoising steps tolerate coarser tiles and sparser windows; only the middle steps need fine precision and dense attention, so compute can be distributed unevenly across the diffusion trajectory.
  • The method is orthogonal to step-distillation techniques, so further acceleration can be stacked on top of the reported speedups.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The three-regime schedule was calibrated on the same model and prompt distribution; if the sensitivity curve shifts for other architectures or longer videos, the schedule would need recalibration, and while the kernel speedup would survive, the quality-preservation claim might not.
  • A testable extension is to drive g(t) and W(t) from the measured cosine-similarity or MSE curves per sample rather than a fixed piecewise schedule, allowing the compression to adapt to content.
  • Because the tile granularity is shared between quantization and sparsity, the work suggests a broader design rule: hardware-efficient attention compression should be expressed on the exact tile that the kernel executes, which could extend to mixed-precision tiles such as some tiles in FP8 and others in INT8.
  • The observed quality improvement over the BF16 baseline may partly reflect evaluation settings (the paper notes VBench scores vary with prompt extension and CFG scale); the more robust claim is quality parity plus a large speedup.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 5 minor

Summary. FPSAttention proposes a training-aware co-design of FP8 quantization and structured tile sparsity for 3D attention in video diffusion transformers. The method introduces a unified 3D tile-wise granularity for quantization and sparsity, a denoising step-aware schedule (Eq. 6) that varies granularity and sparsity window size across early, middle, and late denoising regimes, and a fused FlashAttention/Triton kernel for Hopper GPUs. Experiments on Wan2.1 1.3B and 14B report a 7.09x attention-kernel speedup and a 4.96x end-to-end speedup at 720p, with VBench, PSNR, SSIM, and LPIPS quality results reported at 480p and a headline claim that generation quality is preserved.

Significance. If the central claim held, the paper would make a practically significant contribution: it would demonstrate that a large video diffusion transformer can be trained to tolerate aggressive joint FP8 and structured-sparsity approximations of attention, yielding large wall-clock speedups on Hopper hardware. The paper's strengths are the clear problem formulation, the hardware-oriented kernel description, the ablation quantifying the failure of a naive training-free combination (Table 5), and the honest limitations statement in Appendix G. However, the current evidence does not substantiate the headline 'without sacrificing generation quality' at 720p, and several reported quality tables are internally inconsistent. The kernel-speedup contribution is plausible and useful; the quality-preservation claim needs substantial rework.

major comments (6)
  1. [Abstract; Table 2 footnote; Table 7] The abstract's headline claim 'at 720p resolution—without sacrificing generation quality' is not supported by the reported experiments. Table 2's footnote states that quality and efficiency evaluation is based on 480p videos and that the double-dagger column indicates speedups via 720p with longer sequence length; no quality metric (VBench, PSNR, SSIM, LPIPS) is reported for 720p. Since attention error and tile-wise approximation depend on sequence length, the 480p quality results do not by themselves establish quality preservation at 720p. Please either report 720p quality numbers or restrict the quality claim to 480p.
  2. [Appendix D, Tables 6 and 7] The quality tables contain internal inconsistencies that make the 480p evidence unreliable as reported. The Wan 13B baseline in Table 7 has Quality Score 0.6577, while the Wan 1.3B baseline in Table 6 is 0.8332; the same Table 7 baseline reports Appearance Style 0.2164 and Temporal Style 0.2408, values that are implausible for a 14B model and resemble copy-paste artifacts. In Table 7 the Ours-Q+S Quality Score (0.7103) equals its Imaging Quality, and the baseline Quality Score equals the STA Imaging Quality from Table 2. These entries need to be corrected and the raw evaluation values verified.
  3. [Section 4.2, Tables 3 and 4; Table 2] The ablation tables contain exact duplicated metric values: Table 3's tile size (12,16,16) and Table 4's window (6,6,1) both report PSNR 20.45621, SSIM 0.78932, and LPIPS 0.15743, and Table 3's largest tile (24,32,32) reports exactly the PSNR/SSIM/LPIPS of the 'Ours Quant' row in Table 2 (20.99712/0.79820/0.15114). If these are the same default configuration, the default settings should be stated explicitly; otherwise the duplicated numbers undermine the ablation evidence.
  4. [Section 3.3, Eq. (6); Appendix F, Table 8] The proposed denoising step-aware schedule is not reproducible because none of its hyperparameters are reported. Eq. (6) introduces alpha1, alpha2, g_coarse, g_fine, g_intermediate, W_sparse, W_dense, and W_medium_density, but no values for these quantities appear in the main text, Algorithm 1, or the hyperparameter table in Appendix F. Please report the exact schedule used for the reported results, including the actual tile sizes and window sizes in each regime.
  5. [Section 3.3, Figure 5; Appendix G] The three-regime schedule is motivated and tuned from observations on the same model family and prompt distribution used for evaluation (Figure 5 says 'All measurements from inference are with identical prompts'), and the paper's own limitations note that the hyperparameters may need re-optimization for other architectures. As a result, the quality-preservation claim is at risk of being benchmark-specific. A concrete test would be to evaluate the fixed schedule on held-out prompts, other resolutions, and at least one other architecture (e.g., HunyuanVideo), or to provide a sensitivity analysis showing that performance is stable to reasonable variations in alpha1, alpha2, and the granularities.
  6. [Section 4.1, Table 5] The statement that FPSAttention 'achieves superior performance across all quality metrics' and the claim 'without sacrificing generation quality' are contradicted by the paper's own Table 5: compared with the baseline, FPSAttention lowers Subject Consistency (0.9338 vs 0.9457), Background Consistency (0.9156 vs 0.9503), and Temporal Flickering (0.9336 vs 0.9844) on Wan 1.3B. The aggregate Total Score increases, but the dimension-level trade-offs should be acknowledged and discussed rather than described as lossless.
minor comments (5)
  1. [Throughout; Appendix B, Table 7, Table 8] The model size is reported inconsistently: the main text and Table 1 say Wan2.1-14B, while Appendix B, Table 7, and Table 8 use '13B' (and 'WanX21FPS-13B'); please unify the naming.
  2. [Table 2 footnote] The footnote is self-contradictory: it says 'The quality and efficiency evaluation is based on 480p videos' and then says the double-dagger column indicates 'speedups via 720p'; please clarify which columns and numbers correspond to 480p and which to 720p.
  3. [Section 4.1, Evaluation protocol] No protocol is given for computing PSNR/SSIM/LPIPS (which frames, color space, number of sampled videos, and how reference videos are selected); please specify so the numbers can be reproduced.
  4. [Appendix A, Algorithm 1] Algorithm 1 sets score_mod to the identity in line 28, but Section 3.2 states that the attention matrix P is quantized with the SageAttention2 fixed scalar 1/448; the algorithm should include the P-quantization step or explain why it is omitted.
  5. [Figure 5] Figure 5 uses a non-uniform set of denoising steps (140, 666, 841, 922, 969, 999); please justify the choice or use uniform sampling so the three-regime claim is easier to assess.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's central claims are empirical measurements, not derivations, and no load-bearing argument reduces to its own inputs.

full rationale

The paper's core claims are empirical: measured kernel and end-to-end speedups (Table 1, Table 2) and evaluated generation quality on VBench with PSNR/SSIM/LPIPS (Table 2, Tables 6-7). These are benchmark measurements, not quantities derived from fitting the same data. The denoising step-aware schedule in Equation (6) is motivated by the observed error patterns in Figure 5 and then used during training and inference; this is hyperparameter selection based on empirical measurement, not a fitted parameter renamed as a prediction, and the quality result is not forced by construction. The tile-wise granularity is justified by hardware alignment with FlexAttention/FlashAttention and by comparison with per-token, per-channel, and per-group alternatives, which are independent design choices. I checked the load-bearing citations: STA, SageAttention, SageAttention2, FlexAttention, and related baselines are external prior work, not self-citations by the present authors, and no 'uniqueness theorem' is imported from the authors' own prior work. The abstract's '720p resolution—without sacrificing generation quality' claim is only partially supported because Table 2's footnote states quality evaluation is based on 480p videos while the 720p speedup is reported separately, and the appendix tables contain apparent reporting inconsistencies (e.g., baseline Quality Score 0.8332 in Table 6 versus 0.6577 in Table 7, and baseline Appearance/Temporal Style values that look like copy-paste artifacts). Those are correctness and evidence concerns about extrapolating 480p quality to 720p, not circularity: the claim is under-supported rather than definitionally equivalent to its inputs. Under the stated rules, unsupported extrapolation and internally inconsistent tables are not circularity patterns, so the appropriate circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

FPSAttention introduces no new physical or mathematical entities. Its load-bearing elements are the fitted schedule hyperparameters (transition points, tile sizes, window sizes) and the empirical assumption that diffusion models can be trained to tolerate structured compression.

free parameters (3)
  • alpha1, alpha2 = not reported (transition points t1 = alpha1*D, t2 = alpha2*D)
    Transition points for the three-regime denoising schedule (Equation 6), selected via ablations on VBench.
  • g_coarse, g_fine, g_intermediate = e.g., tile sizes (24,32,32), (6,8,8), and intermediate
    Quantization tile sizes per denoising regime, chosen from Table 3 ablations.
  • W_sparse, W_dense, W_medium_density = e.g., window sizes (6,6,1), dense, medium
    Sparsity window sizes per denoising regime, chosen from Table 4 ablations.
assumptions (4)
  • domain assumption Diffusion models can tolerate and even correct tile-wise quantization and sparsity errors when trained with awareness of the approximations (Section 3.3, paragraph 1, and Figure 5).
    Load-bearing for the QAT approach: if the model cannot learn to compensate, quality would degrade.
  • domain assumption The sensitivity of the model to quantization and sparsity errors follows a consistent three-regime pattern across denoising steps (early tolerant, mid sensitive, late intermediate), as measured in Figure 5.
    Justifies the step-aware schedule (Equation 6); if the pattern does not hold for other prompts/models, the schedule is arbitrary.
  • domain assumption Tile-wise 3D sparsity and FP8 quantization align with GPU Tensor Core execution patterns and FlashAttention/FlexAttention, so theoretical FLOP reductions translate to wall-clock speedups (Section 3.4).
    The speedup claims rest on this hardware assumption; no released kernel to verify.
  • domain assumption VBench and PSNR/SSIM/LPIPS are valid proxies for video generation quality.
    The quality-preservation claim is evaluated through these metrics, not through user studies or downstream task performance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FPSAttention: Training-Aware FP8 and Sparsity Co-Design for Fast Video Diffusion." pith.science (2026). https://pith.science/paper/5S2ZWVWM

@misc{pith2026250604648,
  author       = {Pith},
  title        = {Pith review of: FPSAttention: Training-Aware FP8 and Sparsity Co-Design for Fast Video Diffusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5S2ZWVWM}},
  note         = {Machine review of arXiv:2506.04648}
}
read the original abstract

Diffusion generative models have become the standard for producing high-quality, coherent video content, yet their slow inference speeds and high computational demands hinder practical deployment. Although both quantization and sparsity can independently accelerate inference while maintaining generation quality, naively combining these techniques in existing training-free approaches leads to significant performance degradation due to the lack of joint optimization. We introduce FPSAttention, a novel training-aware co-design of FP8 quantization and sparsity for video generation, with a focus on the 3D bi-directional attention mechanism. Our approach features three key innovations: 1) A unified 3D tile-wise granularity that simultaneously supports both quantization and sparsity; 2) A denoising step-aware strategy that adapts to the noise schedule, addressing the strong correlation between quantization/sparsity errors and denoising steps; 3) A native, hardware-friendly kernel that leverages FlashAttention and is implemented with optimized Hopper architecture features for highly efficient execution. Trained on Wan2.1's 1.3B and 14B models and evaluated on the VBench benchmark, FPSAttention achieves a 7.09x kernel speedup for attention operations and a 4.96x end-to-end speedup for video generation compared to the BF16 baseline at 720p resolution-without sacrificing generation quality.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. BIFE: Better Interaction, Fewer Errors for Minute-Long Video Generation

    cs.CV 2025-11 conditional novelty 5.0 of 10

    BlockVid generates minute-long videos with a semantic sparse KV cache, Block Forcing training, and chunk-level noise scheduling, reporting large gains on its own LV-Bench and on VBench.

  2. PresentAgent: Multimodal Agent for Presentation Video Generation

    cs.CV 2025-07 reject novelty 5.0 of 10

    PresentAgent chains LLM segmentation, slide rendering, TTS, and ffmpeg to turn documents into narrated presentation videos, but the human-level claim rests on five documents and an unvalidated VLM judge.

  3. PAROAttention: Pattern-Aware ReOrdering for Efficient Sparse and Quantized Attention in Visual Generation Models

    cs.CV 2025-06 conditional novelty 4.0 of 10

    PAROAttention permutes tokens along frame, height, and width axes to make visual attention block-wise, enabling sparse and INT8/INT4 quantized attention with near-baseline generation quality.

Reference graph

Works this paper leans on

57 extracted references · 25 canonical work pages · cited by 3 Pith papers

  1. [1]

    Align your latents: High-resolution video synthesis with latent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22563–22575, 2023

    2023

  2. [2]

    Ditfastattn: Attention compression for diffusion transformer models

    Jintao Chen, Chendong Xiang, Haofeng Huang, Jia Wei, Haocheng Xi, Jun Zhu, and Jianfei Chen. Ditfastattn: Attention compression for diffusion transformer models. InAdvances in Neural Information Processing Systems (NeurIPS), 2024

    work page 2024

  3. [3]

    Mixdq: Memory- efficient few-step text-to-image diffusion models with metric-decoupled mixed precision quantization,

    T. Zhao, X. Ning, T. Fang, E. Liu, G. Huang, Z. Lin, S. Yan, G. Dai, and Y. Wang, “Mixdq: Memory- efficient few-step text-to-image diffusion models with metric-decoupled mixed precision quantization,” in European Conference on Computer Vision, 2024, pp. 285–302, Springer

    work page 2024

  4. [4]

    Q-dit: Accurate post-training quantization for diffusion transformers.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    Lei Chen, Yuan Meng, Chen Tang, Xinzhu Ma, Jingyan Jiang, Xin Wang, Zhi Wang, and Wenwu Zhu. Q-dit: Accurate post-training quantization for diffusion transformers.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    work page 2025

  5. [5]

    Flashattention-2: Faster attention with better parallelism and work partitioning.arXiv preprint arXiv:2307.08691, 2023

    Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning.arXiv preprint arXiv:2307.08691, 2023

    arXiv 2023

  6. [6]

    Flashattention: Fast and memory- efficient exact attention with io-awareness.Advances in neural information processing systems, 35:16344– 16359, 2022

    Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory- efficient exact attention with io-awareness.Advances in neural information processing systems, 35:16344– 16359, 2022

    2022

  7. [7]

    Diffusion models beat gans on image synthesis.Advances in neural information processing systems, 34:8780–8794, 2021

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis.Advances in neural information processing systems, 34:8780–8794, 2021

    2021

  8. [8]

    Dollar: Few-step video generation via distillation and latent reward optimization.arXiv preprint arXiv:2412.15689, 2024

    Zihan Ding, Chi Jin, Difan Liu, Haitian Zheng, Krishna Kumar Singh, Qiang Zhang, Yan Kang, Zhe Lin, and Yuchen Liu. Dollar: Few-step video generation via distillation and latent reward optimization.arXiv preprint arXiv:2412.15689, 2024

    arXiv 2024

Show all 57 references
  1. [9]

    Flex attention: A programming model for generating optimized attention kernels.Proceedings of Machine Learning and Systems, 2025

    Juechu Dong, Boyuan Feng, Driss Guessous, Yanbo Liang, and Horace He. Flex attention: A programming model for generating optimized attention kernels.Proceedings of Machine Learning and Systems, 2025

    2025

  2. [10]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. InICML, 2024

    2024

  3. [11]

    Ptqd: Accurate post-training quantization for diffusion models.Advances in neural information processing systems, 2023

    Yefei He, Luping Liu, Jing Liu, Weijia Wu, Hong Zhou, and Bohan Zhuang. Ptqd: Accurate post-training quantization for diffusion models.Advances in neural information processing systems, 2023

    2023

  4. [12]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. InAdvances in Neural Information Processing Systems, volume 33, pages 6840–6851, 2020

    2020

  5. [13]

    Image quality metrics: Psnr vs

    Alain Hore and Djemel Ziou. Image quality metrics: Psnr vs. ssim. In2010 20th international conference on pattern recognition, pages 2366–2369. IEEE, 2010

    2010

  6. [14]

    Tcaq-dm: Timestep- channel adaptive quantization for diffusion models

    Haocheng Huang, Jiaxin Chen, Jinyang Guo, Ruiyi Zhan, and Yunhong Wang. Tcaq-dm: Timestep- channel adaptive quantization for diffusion models. InProceedings of the AAAI Conference on Artificial Intelligence, number 16, pages 17404–17412, 2025

    2025

  7. [15]

    Tcaq-dm: Timestep- channel adaptive quantization for diffusion models

    Haocheng Huang, Jiaxin Chen, Jinyang Guo, Ruiyi Zhan, and Yunhong Wang. Tcaq-dm: Timestep- channel adaptive quantization for diffusion models. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 17404–17412, 2025

    2025

  8. [16]

    Tfmq-dm: Temporal feature maintenance quantization for diffusion models

    Yushi Huang, Ruihao Gong, Jing Liu, Tianlong Chen, and Xianglong Liu. Tfmq-dm: Temporal feature maintenance quantization for diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7362–7371, June 2024. 23

    2024

  9. [17]

    Vbench: Comprehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. InProceedings of the IEEE/CVF Conference on Computer Vision and Patt...

    2024

  10. [18]

    Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024

    2024 arXiv

  11. [19]

    Fp8 quantization: The power of the exponent.Advances in Neural Information Processing Systems, 35:14651–14662, 2022

    Andrey Kuzmin, Mart Van Baalen, Yuwei Ren, Markus Nagel, Jorn Peters, and Tijmen Blankevoort. Fp8 quantization: The power of the exponent.Advances in Neural Information Processing Systems, 35:14651–14662, 2022

    2022

  12. [20]

    Distrifusion: Distributed parallel inference for high-resolution diffusion models

    Muyang Li, Tianle Cai, Jiaxin Cao, Qinsheng Zhang, Han Cai, Junjie Bai, Yangqing Jia, Kai Li, and Song Han. Distrifusion: Distributed parallel inference for high-resolution diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p...

    2024

  13. [21]

    Svdquant: Absorbing outliers by low-rank components for 4-bit diffusion models

    Muyang Li, Yujun Lin, Zhekai Zhang, Tianle Cai, Xiuyu Li, Junxian Guo, Enze Xie, Chenlin Meng, Jun-Yan Zhu, and Song Han. Svdquant: Absorbing outliers by low-rank components for 4-bit diffusion models. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  14. [22]

    Cogvideox: Text-to-video diffusion models with an expert pipeline.arXiv preprint arXiv:2408.06072, 2024

    Zefan Li et al. Cogvideox: Text-to-video diffusion models with an expert pipeline.arXiv preprint arXiv:2408.06072, 2024

    2024 arXiv

  15. [23]

    Vdt: General- purpose video diffusion transformers via mask modeling.International Conference on Learning Represen- tations, 2023

    Haoyu Lu, Guoxing Yang, Nanyi Fei, Yuqi Huo, Zhiwu Lu, Ping Luo, and Mingyu Ding. Vdt: General- purpose video diffusion transformers via mask modeling.International Conference on Learning Represen- tations, 2023

    2023

  16. [24]

    Latte: Latent diffusion transformer for video generation.Transactions on Machine Learning Research, 2025

    Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Ziwei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation.Transactions on Machine Learning Research, 2025

    2025

  17. [25]

    Deepcache: Accelerating diffusion models for free

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. Deepcache: Accelerating diffusion models for free. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15762–15772, 2024

    2024

  18. [26]

    On the challenges and opportunities in generative ai.arXiv preprint arXiv:2403.00025, 2024

    Laura Manduchi, Kushagra Pandey, Clara Meister, Robert Bamler, Ryan Cotterell, Sina Däubener, Sophie Fellenz, Asja Fischer, Thomas Gärtner, Matthias Kirchler, et al. On the challenges and opportunities in generative ai.arXiv preprint arXiv:2403.00025, 2024

    2024 arXiv

  19. [27]

    Fp8 formats for deep learning

    Paulius Micikevicius, Dusan Stosic, Neil Burgess, Marius Cornea, Pradeep Dubey, Richard Grisenthwaite, Sangwon Ha, Alexander Heinecke, Patrick Judd, John Kamalu, et al. Fp8 formats for deep learning. arXiv preprint arXiv:2209.05433, 2022

    2022 arXiv

  20. [28]

    Joint pruning and channel-wise mixed-precision quantization for efficient deep neural networks.IEEE Transactions on Computers, 2024

    Beatrice Alessandra Motetti, Matteo Risso, Alessio Burrello, Enrico Macii, Massimo Poncino, and Daniele Jahier Pagliari. Joint pruning and channel-wise mixed-precision quantization for efficient deep neural networks.IEEE Transactions on Computers, 2024

    2024

  21. [29]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF international conference on computer vision, pages 4195–4205, 2023

    2023

  22. [30]

    Progressive distillation for fast sampling of diffusion models.arXiv preprint arXiv:2202.00512, 2022

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models.arXiv preprint arXiv:2202.00512, 2022

    2022 arXiv

  23. [31]

    Flashattention-3: Fast and accurate attention with asynchrony and low-precision, 2024

    Jay Shah, Ganesh Bikshandi, Ying Zhang, Vijay Thakkar, Pradeep Ramani, and Tri Dao. Flashattention-3: Fast and accurate attention with asynchrony and low-precision, 2024

    2024

  24. [32]

    Tr-dq: Time-rotation diffusion quantization.arXiv preprint arXiv:2503.06564, 2025

    Yihua Shao, Deyang Lin, Fanhu Zeng, Minxi Yan, Muyang Zhang, Siyu Chen, Yuxuan Fan, Ziyang Yan, Haozhe Wang, Jingcai Guo, et al. Tr-dq: Time-rotation diffusion quantization.arXiv preprint arXiv:2503.06564, 2025. 24

    2025 arXiv

  25. [33]

    Efficient post-training quantization with fp8 formats.Proceedings of Machine Learning and Systems, 6:483–498, 2024

    Haihao Shen, Naveen Mellempudi, Xin He, Qun Gao, Chang Wang, and Mengni Wang. Efficient post-training quantization with fp8 formats.Proceedings of Machine Learning and Systems, 6:483–498, 2024

    2024

  26. [34]

    Efficient diffusion models: A survey.arXiv preprint arXiv:2502.06805, 2025

    Hui Shen, Jingxuan Zhang, Boning Xiong, Rui Hu, Shoufa Chen, Zhongwei Wan, Xin Wang, Yu Zhang, Zixuan Gong, Guangyin Bao, et al. Efficient diffusion models: A survey.arXiv preprint arXiv:2502.06805, 2025

    2025 arXiv

  27. [35]

    Parallel sampling of diffusion models.Advances in Neural Information Processing Systems, 36:4263–4276, 2023

    Andy Shih, Suneel Belkhale, Stefano Ermon, Dorsa Sadigh, and Nima Anari. Parallel sampling of diffusion models.Advances in Neural Information Processing Systems, 36:4263–4276, 2023

    2023

  28. [36]

    Make-a-video: Text-to-video generation without text-video data.arXiv preprint arXiv:2209.14792, 2022

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data.arXiv preprint arXiv:2209.14792, 2022

    2022 arXiv

  29. [37]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017

    2017

  30. [38]

    Wan: Open and advanced large-scale video generative models, 2025

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, Pande...

    2025

  31. [39]

    Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, et al. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    2025 arXiv

  32. [40]

    Towards accurate post-training quantization for diffusion models

    Changyuan Wang, Ziwei Wang, Xiuwei Xu, Yansong Tang, Jie Zhou, and Jiwen Lu. Towards accurate post-training quantization for diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16026–16035, 2024

    2024

  33. [41]

    Controllable Generation in Diffusion Models: A Survey

    Wei Wang, Yifan Zhu, Yicong Li, and Xu Yan. Controllable Generation in Diffusion Models: A Survey. Journal of Computer Science and Technology, 39(3):601–626, 2024

    2024

  34. [42]

    Artificial-intelligence-generated content with diffusion models: A literature review.Mathematics, 12(7):977, 2024

    Xiaolong Wang, Zhijian He, and Xiaojiang Peng. Artificial-intelligence-generated content with diffusion models: A literature review.Mathematics, 12(7):977, 2024

    2024

  35. [43]

    Bovik, H.R

    Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility to structural similarity.IEEE Transactions on Image Processing, 13(4):600–612, 2004

    2004

  36. [44]

    Ptq4dit: Post-training quantization for diffusion transformers

    Junyi Wu, Haoxuan Wang, Yuzhang Shang, Mubarak Shah, and Yan Yan. Ptq4dit: Post-training quantization for diffusion transformers. InAdvances in neural information processing systems, 2024

    2024

  37. [45]

    Sparse videogen: Accelerating video diffusion transformers with spatial-temporal sparsity.International Conference on Machine Learning, 2025

    Haocheng Xi, Shuo Yang, Yilong Zhao, Chenfeng Xu, Muyang Li, Xiuyu Li, Yujun Lin, Han Cai, Jintao Zhang, Dacheng Li, et al. Sparse videogen: Accelerating video diffusion transformers with spatial-temporal sparsity.International Conference on Machine Learning, 2025

    2025

  38. [46]

    Jointsq: Joint sparsification-quantization for distributed learning

    Weiying Xie, Haowei Li, Jitao Ma, Yunsong Li, Jie Lei, Donglai Liu, and Leyuan Fang. Jointsq: Joint sparsification-quantization for distributed learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5778–5787, 2024

    2024

  39. [47]

    A survey on video diffusion models.ACM Computing Surveys, 57(2):1–42, 2024

    Zhen Xing, Qijun Feng, Haoran Chen, Qi Dai, Han Hu, Hang Xu, Zuxuan Wu, and Yu-Gang Jiang. A survey on video diffusion models.ACM Computing Surveys, 57(2):1–42, 2024

    2024

  40. [48]

    Hardware-friendly static quantization method for video diffusion transformers.arXiv preprint arXiv:2502.15077, 2025

    Sanghyun Yi, Qingfeng Liu, and Mostafa El-Khamy. Hardware-friendly static quantization method for video diffusion transformers.arXiv preprint arXiv:2502.15077, 2025. 25

    2025 arXiv

  41. [49]

    One-step diffusion with distribution matching distillation

    Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6613–6623, 2024

    2024

  42. [50]

    Native sparse attention: Hardware-aligned and natively trainable sparse attention.arXiv preprint arXiv:2502.11089, 2025

    Jingyang Yuan, Huazuo Gao, Damai Dai, Junyu Luo, Liang Zhao, Zhengyan Zhang, Zhenda Xie, YX Wei, Lean Wang, Zhiping Xiao, et al. Native sparse attention: Hardware-aligned and natively trainable sparse attention.arXiv preprint arXiv:2502.11089, 2025

    2025 arXiv

  43. [51]

    Sageattention2: Efficient attention with thorough outlier smoothing and per-thread int4 quantization

    Jintao Zhang, Haofeng Huang, Pengle Zhang, Jia Wei, Jun Zhu, and Jianfei Chen. Sageattention2: Efficient attention with thorough outlier smoothing and per-thread int4 quantization. InInternational Conference on Machine Learning (ICML), 2025

    2025

  44. [52]

    Sageattention: Accurate 8-bit attention for plug-and-play inference acceleration

    Jintao Zhang, Jia Wei, Pengle Zhang, Jun Zhu, and Jianfei Chen. Sageattention: Accurate 8-bit attention for plug-and-play inference acceleration. InInternational Conference on Learning Representations (ICLR), 2025

    2025

  45. [53]

    Spargeattn: Accurate sparse attention accelerating any model inference

    Jintao Zhang, Chendong Xiang, Haofeng Huang, Jia Wei, Haocheng Xi, Jun Zhu, and Jianfei Chen. Spargeattn: Accurate sparse attention accelerating any model inference. InInternational Conference on Machine Learning (ICML), 2025

    2025

  46. [54]

    Fast video generation with sliding tile attention.arXiv preprint arXiv:2502.04507, 2025

    Peiyuan Zhang, Yongqi Chen, Runlong Su, Hangliang Ding, Ion Stoica, Zhenghong Liu, and Hao Zhang. Fast video generation with sliding tile attention.arXiv preprint arXiv:2502.04507, 2025

    2025 arXiv

  47. [55]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 586–595, 2018

    2018

  48. [56]

    Vidit-q: Efficient and accurate quantization of diffusion transformers for image and video generation

    Tianchen Zhao, Tongcheng Fang, Haofeng Huang, Enshu Liu, Rui Wan, Widyadewi Soedarmadji, Shiyao Li, Zinan Lin, Guohao Dai, Shengen Yan, et al. Vidit-q: Efficient and accurate quantization of diffusion transformers for image and video generation. InInternational Conference on L...

    2025

  49. [57]

    Real-time video generation with pyramid attention broadcast

    Xuanlei Zhao, Xiaolong Jin, Kai Wang, and Yang You. Real-time video generation with pyramid attention broadcast. InInternational Conference on Learning Representations, 2025. 26

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.