Pith. sign in

REVIEW 3 major objections 6 minor 6 cited by

VMoBA: Mixture-of-Block Attention for Video Diffusion Models

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read VMoBA claims that replacing full attention with video-specific mixture-of-block attention speeds video-diffusion training by up to 1.48x while matching or beating generation quality.

desk verdict A real MoBA-for-video adaptation with measured training speedups, but the unexamined token-dropping in global selection and a thin quality lead should be chased before believing the headline claims. read the letter →

arxiv 2506.23858 v1 pith:5UWBZ4KN submitted 2025-06-30 cs.CV

Jianzong Wu , Liang Hou , Haotian Yang , Xin Tao , Ye Tian , Pengfei Wan , Di Zhang , Yunhai Tong This is my paper · ORCID
classification cs.CV
keywords videodiffusionmodelssparseattentionmixture-of-blockspatio-temporallocalitytrainingaccelerationlongsequencegenerationblockselection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to make video diffusion model training on long, high-resolution videos cheaper without sacrificing quality. It claims that the quadratic cost of full attention can be replaced by a sparse mixture-of-block attention adapted to video's 1D-2D-3D spatio-temporal structure. The reported result is a 2.92x reduction in FLOPs and up to 1.48x faster training on long sequences, with VBench scores comparable to or above full attention. The same mechanism also works in training-free inference on higher-resolution videos, giving up to 2.40x FLOPs and 1.35x latency savings.

What carries the argument

The central object is block-sparse attention built on block means: the key sequence is partitioned into non-overlapping blocks, each block is represented by its mean key vector, queries score these block means, and only the blocks with the highest query-key block similarities contribute their actual key-value pairs to the attention output. The three carrying mechanisms are the cyclical 1D-2D-3D layer-wise partition, which preserves spatio-temporal locality; global block selection, which pools scores across all queries in a head; and threshold-based selection, which fixes the number of blocks by cumulative normalized similarity. Together they reduce the complexity from quadratic to roughly $O(sd(s/s_b + k_{\text{avg}}s_b))$.

What would settle it

If a video attention map contains a block whose mean scores low but whose individual keys include some of the most-attended tokens in the sequence, VMoBA will drop those keys; one could measure this in a pretrained video diffusion model by comparing VMoBA's selected blocks with the token-level top-k of full attention, or by replacing each block mean with the maximum-scoring key inside the block and checking whether generation quality shifts.

Watch

Extended reading notes

Core claim

The central claim is that full self-attention in video diffusion transformers can be replaced during training by a block-sparse attention that exploits three properties observed in a pretrained video diffusion transformer: attention is local along one, two, or three dimensions depending on the layer; queries differ in how much total key similarity they carry; and heads differ in how concentrated their similarity distributions are. VMoBA responds with a layer-wise 1D-2D-3D block partition, global selection of the highest-scoring query-key blocks across an entire head, and a threshold that stops block accumulation once cumulative normalized similarity passes a hyperparameter. On spatially and temporally extended fine-tuning of the Wan 2.1 base model, the paper reports comparable or better VBench quality than full attention while cutting FLOPs by up to 2.92x and training time by up to 1.48x.

Load-bearing premise

The block-selection step assumes the mean of a key block is a good stand-in for every key inside it, so that ranking blocks by query-to-block-mean similarity preserves the query-key interactions full attention would keep.

Editorial extensions

If this is right

  • Training video diffusion models on longer sequences becomes materially cheaper: up to 2.92x fewer FLOPs and up to 1.48x less wall-clock training time with comparable or better quality.
  • The method transfers to training-free inference on high-resolution videos, achieving up to 2.40x FLOPs and 1.35x latency speedup over full attention.
  • The threshold hyperparameter directly trades compute for fidelity, so the same architecture can be tuned for speed or for closer approximation to full attention.
  • Pretraining-from-scratch experiments show validation loss close to full attention at longer sequence lengths, suggesting the sparse mechanism can serve as a training-time replacement rather than only a fine-tuning shortcut.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the block-mean selection rule implies VMoBA's advantage should grow with sequence length and with locally concentrated attention maps; diffuse, long-range attention would force the threshold to retain many blocks and erase the speedup.
  • Beyond the paper: a testable prediction is that VMoBA loses accuracy when key blocks are internally heterogeneous, because a block mean cannot represent a few highly attended tokens buried among many irrelevant ones; this could be probed by comparing selected blocks against the token-level top-k of full attention.
  • Beyond the paper: global selection couples all queries in a head, so one salient query can force extra blocks to be retained for the whole head; a per-query or per-region threshold may be a natural follow-up.
  • Beyond the paper: the fixed mod-3 1D-2D-3D schedule suggests the partition pattern itself could be learned or adapted per layer, which might improve both quality and efficiency beyond the reported settings.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes VMoBA, a mixture-of-block sparse attention mechanism for video diffusion models (VDMs), adapted from MoBA with three changes: a layer-wise recurrent 1D-2D-3D block partition, global block selection across query-key block similarities, and threshold-based selection of the number of attended blocks. The authors motivate these choices by analyzing attention patterns in a pre-trained Wan 2.1 model. They report training experiments on extended resolutions (93x576x1024 and 141x480x832) showing up to 2.92x FLOPs reduction and 1.48x wall-clock training speedup over full attention, with VBench mean scores comparable to or slightly better than full attention (68.34 vs 68.25 in one setting). They also report training-free inference results with speedups of 2.40x FLOPs and 1.35x latency at high resolution, plus ablations of partition strategy, selection strategy, threshold, and block count, and pre-training loss comparisons in the appendix.

Significance. If the results hold, VMoBA provides a practical drop-in sparse attention for training video diffusion models on longer sequences, with clear FLOPs and wall-clock speedups while maintaining generation quality. The paper's attention-pattern analysis, the 1D-2D-3D partition scheme, and the pre-training loss study over multiple model scales are useful contributions that go beyond a direct MoBA transplant. The work is also reproducible in principle since code is linked and the method is simple to implement on top of FlashAttention. However, the central causal claim - that the quality is preserved because of the block-selection design - is currently weakened by an unaddressed token-dropping phenomenon, by the mismatch between the proposed selection mechanism and the mechanism actually used in training-free experiments, and by the absence of uncertainty estimates on the headline quality numbers.

major comments (3)
  1. [Section 3.3, Eq. (3) and Section 3.4, Eq. (4)] The global TopkMask in Eq. (3) selects the top k entries from the full query-by-block similarity matrix, and Eq. (4) determines k as a number of global query-block pairs, not a per-query number of blocks. Nothing in the formulation enforces a lower bound per query row, so a query whose similarities all fall below the global cutoff receives an all-zero mask row; Step 3 then computes attention over zero blocks and, with the standard residual connection, the token is passed through unchanged. This is unacknowledged token dropping rather than a sparse approximation of full attention. The paper neither reports the frequency of such empty rows nor tests a variant that guarantees at least one block per query. Since the method is motivated by preserving the most important query-key interactions, this silent mechanism could be responsible for part of the quality and speedup effects in Table 2. The authors should measure the empty-row frequency during training and inference, describe how the FlashAttention kernel handles an empty row, and compare against a version with a per-query minimum-block constraint.
  2. [Appendix A, Table 4 and Section 4.2] The training-free results in Table 1 do not use the proposed global+threshold selection: Appendix A states that for those experiments VMoBA uses 'Local + TopK' query-key block selection, because the global+threshold strategy 'will cause vibration effects on the generated videos.' The main text and the abstract nevertheless present these results as evidence that VMoBA 'exhibits competitive performance in training-free inference,' without any caveat in Section 4.2 or the abstract. This is misleading: the training-free speedup and quality claims do not exercise the proposed selection mechanism, and the full method is implicitly acknowledged to be unstable in that setting. The paper should clearly distinguish 'VMoBA (full)' from 'VMoBA (local top-k)' throughout, and the abstract and Section 4.2 should either be reworded or the full method's training-free behavior should be reported with the proposed selection.
  3. [Table 2 and Section 4.2] The claim of 'comparable or even superior generation quality' rests on a VBench mean of 68.34 for VMoBA versus 68.25 for full attention at 93x576x1024, with mixed per-category results (e.g., Dynamic 56.91% vs 61.58% and ImageQual 67.45% vs 69.49% in favor of full attention). No standard errors, confidence intervals, or multiple seeds are reported for any of the quality numbers in Tables 1, 2, 3, or 5. A 0.09-point difference in the mean is within the range one would expect from sampling noise, so 'even superior' is not supported by the evidence. At minimum, the authors should report variance across seeds or sampling runs and replace the 'superior' language with 'comparable' unless a statistically significant advantage is demonstrated.
minor comments (6)
  1. [Section 3.3, Eq. (3)] The notation is inconsistent: the text and Eq. (3) call k the 'number of selected blocks,' but Eq. (4) defines k as a count of selected query-block similarity entries. This conflation is directly related to the empty-row issue and should be clarified.
  2. [Section 4.1] The sentence 'its token sparsity may vary across different settings (always smaller than tau)' is contradicted by Table 1, where the training-free VMoBA rows report sparsity 0.31 with tau set to 0.25. The contradiction disappears only after reading Appendix A, which explains that training-free experiments use local top-k; the main text should not assert a property that holds only for the global+threshold configuration.
  3. [Figure 2] The labels 'RankX' and the statement '4 blocks in this example' do not match the global pair-selection formulation: if selection operates on query-block pairs, the figure should show which query rows receive no selected blocks or explain why none do.
  4. [Appendix D, Table 5] The paper should state more prominently that the 1.48x training-time speedup applies only to the extended resolutions; Table 5 shows VMoBA is 1.00x at the original resolution and slower (0.86x) at short resolution, which is already acknowledged in the limitations but should be echoed in the abstract.
  5. [Equation (1)] The typesetting of Eq. (1) is broken: 'rearrange(K in (T H W))' is not a valid tensor expression, and the brace-enclosed cases are not formatted clearly. Please revise to standard tensor notation.
  6. [Table 4] There is a typo in the heading: 'Implementation detials' should be 'Implementation details.'

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: VMoBA's speedup and quality claims are measured externally, not defined into existence, though one training-free evaluation setting does not use the proposed selection mechanism.

full rationale

The core derivation chain is algorithmic rather than circular. Block partition is defined constructively in Eq. (1)-(2); the selection mask in Eq. (3) is built from query-to-key-block similarities; and the number of selected blocks k in Eq. (4) is the smallest k' whose cumulative normalized similarity reaches threshold τ. No predicted quantity is defined as the fitted parameter: τ is a hyperparameter selected by ablation (Table 3c), and the reported quality numbers are measured against the external VBench benchmark (68.34 vs. 68.25) and PSNR, not derived from the equations. The FLOPs and latency speedups are measured in Tables 1-2 and follow from the algorithm's complexity, not from a quantity that was fit to those same speedups. The motivating observations about 1D-2D-3D locality, query importance, and head concentration are made on the same Wan 2.1 model family used for evaluation, but this is domain motivation rather than equation-level circularity, and the final quality comparison is against full attention on held-out prompts. Self-citations to Wan, Koala-36M, and MoBA are used as base model, dataset, and baseline respectively; they are not invoked as authority to establish the central claim. One genuine scope limitation appears in Appendix A: the training-free experiments in Table 1 use Local+TopK selection rather than the proposed Global+Threshold mechanism because the latter causes 'vibration effects'; this weakens the claim that Table 1 validates the proposed selection, but it is a mismatch in experimental coverage, not a circular derivation. Overall, no load-bearing step reduces by construction to its own input.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on hand-set hyperparameters and on assumptions about attention locality and block-mean representativeness. No new physical entities are introduced. The threshold and block counts are empirical choices that affect the reported quality-efficiency tradeoff.

free parameters (3)
  • threshold tau = 0.25
    Controls the cumulative normalized similarity threshold for selecting key blocks per attention head. Chosen by ablation on the extended-resolution training runs in Table 3c, trading quality against training time.
  • block partition configuration = temporal block size 3; spatial block size 5x13; spatio-temporal blocks 8-48-72, varied by resolution
    Block sizes and block counts are hand-set per resolution in Appendix Table 4 and ablated in Table 3d. They directly affect both FLOPs and generation quality.
  • local top-k block counts for training-free inference = 2 | 6 | 18
    Used instead of global plus threshold selection in the training-free experiments, as stated in Appendix A. These values are manually chosen and are not part of the proposed algorithm's default mechanism.
assumptions (4)
  • domain assumption Spatio-temporal attention in video DiTs is sufficiently localized and stable that a fixed cyclic 1D-2D-3D block partition across layers captures the relevant structure.
    Invoked in Section 3.2, Eq. 1. If attention patterns shift during training or vary strongly with content, the fixed schedule would lose important interactions.
  • domain assumption The mean of key tokens within a block is a representative summary for block selection.
    Step 1 and Step 2 of Section 3.1 compute block means and select blocks based on query-to-mean similarity. High intra-block variance would make the selection miss important tokens.
  • domain assumption VBench subset scores and 2000-step fine-tuning on Koala-36M are adequate evidence for the generation-quality claim.
    Used in Section 4.1 and 4.2. No error bars or full training runs are provided, and quality differences near 0.1 points are treated as meaningful.
  • domain assumption Full attention can be approximated by selecting blocks according to cumulative normalized similarity with threshold tau.
    Section 3.4 defines the threshold rule; this is the algorithmic premise of the approximation and is not derived from first principles.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VMoBA: Mixture-of-Block Attention for Video Diffusion Models." pith.science (2026). https://pith.science/paper/5UWBZ4KN

@misc{pith2026250623858,
  author       = {Pith},
  title        = {Pith review of: VMoBA: Mixture-of-Block Attention for Video Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5UWBZ4KN}},
  note         = {Machine review of arXiv:2506.23858}
}
read the original abstract

The quadratic complexity of full attention mechanisms poses a significant bottleneck for Video Diffusion Models (VDMs) aiming to generate long-duration, high-resolution videos. While various sparse attention methods have been proposed, many are designed as training-free inference accelerators or do not optimally capture the unique spatio-temporal characteristics inherent in video data when trained natively. This paper introduces Video Mixture of Block Attention (VMoBA), a novel sparse attention mechanism specifically adapted for VDMs. Motivated by an in-depth analysis of attention patterns within pre-trained video transformers, which revealed strong spatio-temporal locality, varying query importance, and head-specific concentration levels, VMoBA enhances the original MoBA framework with three key modifications: (1) a layer-wise recurrent block partition scheme (1D-2D-3D) to dynamically adapt to diverse spatio-temporal attention patterns and improve efficiency; (2) global block selection to prioritize the most salient query-key block interactions across an entire attention head; and (3) threshold-based block selection to dynamically determine the number of attended blocks based on their cumulative similarity. Extensive experiments demonstrate that VMoBA significantly accelerates the training of VDMs on longer sequences, achieving 2.92x FLOPs and 1.48x latency speedup, while attaining comparable or even superior generation quality to full attention. Furthermore, VMoBA exhibits competitive performance in training-free inference, offering 2.40x FLOPs and 1.35x latency speedup for high-res video generation.

Figures

Figures reproduced from arXiv: 2506.23858 by the authors.

Figure 1
Figure 1. (a) VMoBA performs better than Full Attention while reducing training time. (b) VMoBA [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The overall pipeline of VMoBA. We first partition key blocks with Layer-wise Recurrent BLock Partition, then select the blocks using Global Block Selection and Threshold-based Block Selection. Finally, the attention is computed only with the selected blocks. partitioning, the mean of each key block is computed, resulting in a set of key blocks B. Step 2: Select Key Blocks. For each attention head, a similarity map i… view at source ↗
Figure 3
Figure 3. The query-key block attention map shows 1-2-3D distinct patterns. Video data inherently possesses strong spatio-temporal locality. Our analysis of attention maps from a full at￾tention DiT, Wan 2.1 1.3B [32], is illustrated in [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Summation of top 25% query-key simi￾larities. Different queries have varying importance. Full attention maps indicate that different query tokens receive varying degrees of attention, and their top similarity scores with key tokens can differ significantly. As shown in…
Figure 5
Figure 5. Figure 5: Sorted query-key block similarity and top 30%/50% cutoff lines. The right parts of the cutoff lines contain corresponding cumulative sum￾mations of query-key block similarity. The concentration level of similarity scores varies not only across queries but also across d…
Figure 6
Figure 6. Figure 6: Qualitative comparison of VMoBA and baseline methods. 4.3 Qualitative Results [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Validation loss comparison for pre-training between VMoBA and full attention. Blue and red lines are full attention and VMoBA, respectively. Deeper colors represent bigger model sizes. We conduct training-from-scratch experiments to evaluate the pre-training ability of…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Chimera: Designing and Chinchilla-Scaling Hybrid Visual Diffusion Transformers

    cs.CV 2026-07 conditional novelty 6.5 of 10

    A hybrid NoPE visual diffusion backbone with HeteroP scaling yields ~7× pretraining compute efficiency versus matched full attention and near-stable zero-shot 5s→30s video extrapolation.

  2. Rethinking Position Embedding as a Context Controller for Multi-Reference and Multi-Shot Video Generation

    cs.CV 2026-04 conditional novelty 6.0 of 10

    SideInfo-RoPE encodes reference-identity agreement as an extra rotary axis, disambiguating similar characters in multi-reference multi-shot video generation while keeping full semantic attention.

  3. SVG-EAR: Parameter-Free Linear Compensation for Sparse Video Generation via Error-aware Routing

    cs.CV 2026-03 conditional novelty 6.0 of 10

    Parameter-free centroid compensation plus error-aware block routing yields a better quality–density Pareto frontier for sparse attention in video DiTs than score-based sparsification.

  4. Light Forcing: Accelerating Autoregressive Video Diffusion via Sparse Attention

    cs.CV 2026-02 conditional novelty 6.0 of 10

    Sparse attention with chunk-aware sparsity growth and hierarchical frame/block selection accelerates autoregressive video diffusion at ~1.3x with VBench quality on par with dense attention.

  5. Trainable Log-linear Sparse Attention for Efficient Diffusion Transformers

    cs.CV 2025-12 conditional novelty 6.0 of 10

    LLSA uses hierarchical top-K selection plus coarse-token enrichment to make diffusion-transformer attention O(N log N), giving about 6x faster training at 256x256 pixel resolution with FID comparable to full attention.

  6. End-to-End Training for Autoregressive Video Diffusion via Self-Resampling

    cs.CV 2025-12 conditional novelty 6.0 of 10

    Resampling Forcing trains autoregressive video diffusion models on self-resampled degraded histories with a causal mask, achieving stable long-horizon generation without a teacher or discriminator.

Reference graph

Works this paper leans on

46 extracted references · 27 canonical work pages · cited by 6 Pith papers

  1. [1]

    Zeroscope, 2023

    Tim Brooks, Bill Peebles, Connor Holmes, Yufei Guo Will DePue, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Zeroscope, 2023. 2

    work page 2023

  2. [2]

    Sora: Creating video from text, 2024

    Tim Brooks, Bill Peebles, Connor Holmes, Yufei Guo Will DePue, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Sora: Creating video from text, 2024. 2

    work page 2024

  3. [3]

    Videocrafter1: Open diffusion models for high-quality video generation

    Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, et al. Videocrafter1: Open diffusion models for high-quality video generation. arXiv preprint arXiv:2310.19512, 2023. 2

    arXiv

  4. [4]

    Videocrafter2: Overcoming data limitations for high-quality video diffusion models

    Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models. arXiv preprint arXiv:2401.09047, 2024. 2

    arXiv 2024

  5. [5]

    δ-dit: A training-free acceleration method tailored for diffusion transformers

    Pengtao Chen, Mingzhu Shen, Peng Ye, Jianjian Cao, Chongjun Tu, Christos-Savvas Bouganis, Yiren Zhao, and Tao Chen. δ-dit: A training-free acceleration method tailored for diffusion transformers. CoRR,

    work page

  6. [6]

    Flashattention: Fast and memory- efficient exact attention with io-awareness

    Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory- efficient exact attention with io-awareness. NeurIPS, 2022. 4, 13

    work page 2022

  7. [7]

    Transformers are ssms: generalized models and efficient algorithms through structured state space duality

    Tri Dao and Albert Gu. Transformers are ssms: generalized models and efficient algorithms through structured state space duality. In ICML, 2024. 3

    work page 2024

  8. [8]

    A formal evaluation of psnr as quality measurement parameter for image segmentation algorithms

    Fernando A Fardo, Victor H Conforto, Francisco C de Oliveira, and Paulo S Rodrigues. A formal evaluation of psnr as quality measurement parameter for image segmentation algorithms. arXiv preprint arXiv:1605.07116, 2016. 6

    arXiv 2016

Show all 46 references
  1. [9]

    Mamba: Linear-time sequence modeling with selective state spaces

    Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752, 2023. 3

    2023 arXiv

  2. [10]

    Animatediff: Animate your personalized text-to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. In ICLR, 2024. 2

    2024

  3. [11]

    Vbench: Comprehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. In CVPR, 2024. 6

    2024

  4. [12]

    Hunyuanvideo: A systematic framework for large video generative models

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603, 2024. 2

    2024 arXiv

  5. [13]

    Distrifusion: Distributed parallel inference for high-resolution diffusion models

    Muyang Li, Tianle Cai, Jiaxin Cao, Qinsheng Zhang, Han Cai, Junjie Bai, Yangqing Jia, Kai Li, and Song Han. Distrifusion: Distributed parallel inference for high-resolution diffusion models. In CVPR, 2024. 7

    2024

  6. [14]

    Timestep embedding tells: It’s time to cache for video diffusion model

    Feng Liu, Shiwei Zhang, Xiaofeng Wang, Yujie Wei, Haonan Qiu, Yuzhong Zhao, Yingya Zhang, Qixiang Ye, and Fang Wan. Timestep embedding tells: It’s time to cache for video diffusion model. In CVPR, 2025. 3, 7

    2025

  7. [15]

    Pseudo numerical methods for diffusion models on manifolds

    Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds. In ICLR, 2022. 3

    2022

  8. [16]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Xingchao Liu, Chengyue Gong, et al. Flow straight and fast: Learning to generate and transfer data with rectified flow. In ICLR, 2023. 3

    2023

  9. [17]

    Vmamba: Visual state space model

    Yue Liu, Yunjie Tian, Yuzhong Zhao, Hongtian Yu, Lingxi Xie, Yaowei Wang, Qixiang Ye, Jianbin Jiao, and Yunfan Liu. Vmamba: Visual state space model. NeurIPS, 2024. 3

    2024

  10. [18]

    DPM-Solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. DPM-Solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. In NeurIPS, 2022. 3

    2022

  11. [19]

    Moba: Mixture of block attention for long-context llms

    Enzhe Lu, Zhejun Jiang, Jingyuan Liu, et al. Moba: Mixture of block attention for long-context llms. arXiv preprint arXiv:2502.13189, 2025. 2, 3, 7, 12, 13

    2025 arXiv

  12. [20]

    Latent consistency models: Synthesizing high-resolution images with few-step inference

    Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthesizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.04378, 2023. 3

    2023 arXiv

  13. [21]

    Fastercache: Training-free video diffusion model acceleration with high quality

    Zhengyao Lv, Chenyang Si, Junhao Song, Zhenyu Yang, Yu Qiao, Ziwei Liu, and Kwan-Yee K Wong. Fastercache: Training-free video diffusion model acceleration with high quality. arXiv preprint arXiv:2410.19355, 2024. 3, 7

    2024 arXiv

  14. [22]

    Block-attention for efficient prefilling

    Dongyang Ma, Yan Wang, and Tian Lan. Block-attention for efficient prefilling. In ICLR, 2024. 2, 3

    2024

  15. [23]

    DeepCache: Accelerating diffusion models for free

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. DeepCache: Accelerating diffusion models for free. arXiv preprint arXiv:2312.00858, 2023. 3

    2023 arXiv

  16. [24]

    Latte: Latent diffusion transformer for video generation

    Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Ziwei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation. arXiv preprint arXiv:2401.03048, 2024. 2 10

    2024 arXiv

  17. [25]

    Rwkv: Reinventing rnns for the transformer era

    Bo Peng, Eric Alcaide, Quentin Gregory Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Nguyen Chung, Leon Derczynski, et al. Rwkv: Reinventing rnns for the transformer era. In EMNLP, 2023. 3

    2023

  18. [26]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In CVPR, 2022. 2

    2022

  19. [27]

    Photorealistic text-to- image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to- image diffusion models with deep language understanding. NeurIPS, 2022. 2

    2022

  20. [28]

    Progressive distillation for fast sampling of diffusion models

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In ICLR,

  21. [29]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In ICLR, 2020. 3

    2020

  22. [30]

    Consistency models

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In ICML, 2023. 3

    2023

  23. [31]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. NeurIPS, 2017. 2

    2017

  24. [32]

    Wan: Open and advanced large-scale video generative models

    Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314, 2025. 2, 4, 7, 8, 13

    2025 arXiv

  25. [33]

    Lingen: Towards high-resolution minute-length text-to-video generation with linear computational complexity

    Hongjie Wang, Chih-Yao Ma, Yen-Cheng Liu, Ji Hou, Tao Xu, Jialiang Wang, Felix Juefei-Xu, Yaqiao Luo, Peizhao Zhang, Tingbo Hou, et al. Lingen: Towards high-resolution minute-length text-to-video generation with linear computational complexity. arXiv preprint arXiv:2412.09856, 2024. 3

    2024 arXiv

  26. [34]

    Modelscope text-to-video technical report

    Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. Modelscope text-to-video technical report. arXiv preprint arXiv:2308.06571, 2023. 2

    2023 arXiv

  27. [35]

    Koala-36m: A large-scale video dataset improving consistency between fine-grained conditions and video content

    Qiuheng Wang, Yukai Shi, Jiarong Ou, Rui Chen, Ke Lin, Jiahao Wang, Boyuan Jiang, Haotian Yang, Mingwu Zheng, Xin Tao, et al. Koala-36m: A large-scale video dataset improving consistency between fine-grained conditions and video content. arXiv preprint arXiv:2410.08260, 2024. 6

    arXiv

  28. [36]

    Lavie: High-quality video generation with cascaded latent diffusion models

    Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang, Yi Wang, Ceyuan Yang, Yinan He, Jiashuo Yu, Peiqing Yang, et al. Lavie: High-quality video generation with cascaded latent diffusion models. arXiv preprint arXiv:2309.15103, 2023. 2

    2023 arXiv

  29. [37]

    Sparse videogen: Accelerating video diffusion transformers with spatial-temporal sparsity

    Haocheng Xi, Shuo Yang, Yilong Zhao, Chenfeng Xu, Muyang Li, Xiuyu Li, Yujun Lin, Han Cai, Jintao Zhang, Dacheng Li, et al. Sparse videogen: Accelerating video diffusion transformers with spatial-temporal sparsity. arXiv preprint arXiv:2502.01776, 2025. 2, 3, 5, 7, 8, 12

    2025 arXiv

  30. [38]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. CoRR, 2024. 6

    2024

  31. [39]

    Improved distribution matching distillation for fast image synthesis

    Tianwei Yin, Michaël Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and William T Freeman. Improved distribution matching distillation for fast image synthesis. In NeurIPS, 2024. 3

    2024

  32. [40]

    Mambaout: Do we really need mamba for vision? CoRR, 2024

    Weihao Yu and Xinchao Wang. Mambaout: Do we really need mamba for vision? CoRR, 2024. 3

    2024

  33. [41]

    Native sparse attention: Hardware-aligned and natively trainable sparse attention

    Jingyang Yuan, Huazuo Gao, Damai Dai, et al. Native sparse attention: Hardware-aligned and natively trainable sparse attention. arXiv preprint arXiv:2502.11089, 2025. 2, 3

    2025 arXiv

  34. [42]

    Ditfastattn: Attention compression for diffusion transformer models

    Zhihang Yuan, Hanling Zhang, Lu Pu, Xuefei Ning, Linfeng Zhang, Tianchen Zhao, Shengen Yan, Guohao Dai, and Yu Wang. Ditfastattn: Attention compression for diffusion transformer models. NeurIPS, 2024. 2, 3, 7, 12

    2024

  35. [43]

    Spargeattn: Accurate sparse attention accelerating any model inference

    Jintao Zhang, Chendong Xiang, Haofeng Huang, Jia Wei, Haocheng Xi, Jun Zhu, and Jianfei Chen. Spargeattn: Accurate sparse attention accelerating any model inference. arXiv preprint arXiv:2502.18137,

  36. [44]

    Fast video generation with sliding tile attention

    Peiyuan Zhang, Yongqi Chen, Runlong Su, Hangliang Ding, Ion Stoica, Zhenghong Liu, and Hao Zhang. Fast video generation with sliding tile attention. arXiv preprint arXiv:2502.04507, 2025. 2, 3

    2025 arXiv

  37. [45]

    Real-time video generation with pyramid attention broadcast

    Xuanlei Zhao, Xiaolong Jin, Kai Wang, and Yang You. Real-time video generation with pyramid attention broadcast. CoRR, 2024. 3, 7 11 Appendix Table of contents. The supplementary includes the following sections: • Appendix A. Implementation details of the experiments. • Append...

    2024

  38. [512]

    The 1-2-3D block numbers of VMoBA are 10, 48, and 60

    for shorter length, and 46k (77 × 576 × 1024) for longer length. The 1-2-3D block numbers of VMoBA are 10, 48, and 60. The threshold τ is set to 0.25. The model configuration are shown in Table 6. We use the same architecture as Wan 2.1 [ 32]. For VMoBA models, we replace all ...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.