Pith. sign in

REVIEW 3 major objections 6 minor 65 references

AsymRnR: Video Diffusion Transformers Acceleration with Asymmetric Reduction and Restoration

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A training-free, model-agnostic method, AsymRnR, accelerates video diffusion transformers up to 1.3x by asymmetrically reducing redundant Q/K/V tokens, with negligible or even positive quality effects.

desk verdict Practical token-reduction speedup for video DiTs with a genuine asymmetric Q/KV twist, but the 'negligible degradation' headline is not supported by the paper's own per-dimension VBench numbers. read the letter →

arxiv 2412.11706 v3 pith:TJTYI7XZ submitted 2024-12-16 cs.CV

Wenhao Sun , Rong-Cheng Tu , Jingyi Liao , Zhao Jin , Dacheng Tao This is my paper · ORCID
classification cs.CV
keywords videodiffusiontransformerstokenreductionattentionaccelerationtraining-freeasymmetricmatchingcacheschedulingtext-to-videogeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the attention layers of video diffusion transformers are full of redundant tokens whose redundancy differs between query tokens and key/value tokens, between early and deep blocks, and between early and late denoising steps. Exploiting this, it proposes AsymRnR, which reduces only the redundant tokens, mainly key/value tokens in high-similarity blocks and timesteps, before self-attention and restores the dropped tokens afterward so the pretrained network is barely disturbed. If the claim holds, video DiTs like CogVideoX, Mochi-1, HunyuanVideo, and a 6-step distilled FastVideo can run 1.1x to 1.3x faster with VBench scores within roughly 0.01 of baseline and sometimes slightly better, without any training. This matters because video generation cost is the main barrier to practical use, and the method plugs into existing models.

What carries the argument

The load-bearing object is the asymmetric reduction-restoration operator AsymRnR, which applies bipartite soft matching separately to Q and K/V, discards the matched redundant tokens, runs self-attention on the shortened sequence, and then restores Q to its original length by replicating each reduced token according to its match. Supporting it are three mechanisms: a reduction schedule that thresholds a precomputed similarity map S(A,t,b) per feature type, denoising timestep, and block; a matching cache that reuses the bipartite matching result for s consecutive denoising steps, cutting matching cost by a factor of 1/s; and the negative Euclidean distance as the similarity metric, which is compatible with the KL-divergence estimator of Corollary 3.1.

What would settle it

Generate videos from prompts whose token-similarity statistics differ systematically from the precomputed schedule, such as fast camera motion or many small independently moving objects, and measure the VBench or LPIPS gap between AsymRnR and the unmodified baseline; if the gap substantially exceeds the roughly 0.01 the paper reports across such content, the context-agnostic assumption fails. A more direct check is to compute the actual KL divergence (using the paper's nearest-neighbor estimator) between original and reduced Q/K/V sequences on held-out prompts and verify that the schedule's reductions are the ones that minimize that divergence.

Watch

Extended reading notes

Core claim

The central claim is that matching-based token reduction can be made safe for video diffusion transformers by treating Q and K/V asymmetrically: queries act as the sensitive questioner and must be preserved or restored, while keys and values carry redundant information that can be reduced more aggressively. Combined with a reduction schedule that concentrates reduction in the blocks and timesteps where token similarity is high, and a matching cache that reuses the similarity matching across adjacent denoising steps, the method achieves 1.10x to 1.30x speedups on state-of-the-art video DiTs. The paper also grounds the design in a distributional argument: reduction should minimize the KL divergence between the reduced and original token distributions, and a nearest-neighbor Monte Carlo estimator justifies using (negative) Euclidean distance and direct token discarding rather than cosine similarity and mean-based merging.

Load-bearing premise

The whole scheme rests on two unproven premises: that minimizing KL divergence between the reduced and original token distributions is the right objective for preserving attention outputs, and that per-token similarity patterns are context-agnostic, so a schedule estimated on arbitrary prompts transfers to any new prompt.

Editorial extensions

If this is right

  • Applying AsymRnR to CogVideoX-2B, CogVideoX-5B, Mochi-1, HunyuanVideo, and the 6-step distilled FastVideo-Hunyuan yields 1.10x to 1.30x latency reductions with VBench scores within roughly 0.01 of baseline, and slightly higher than baseline on HunyuanVideo and Mochi-1.
  • AsymRnR is orthogonal to feature caching: stacked on top of PAB on HunyuanVideo it achieves a combined 1.71x speedup with negligible quality degradation.
  • AsymRnR is training-free and model-agnostic, and it also works on UNet-based video diffusion models, giving a 1.20x speedup on AnimateDiff with no perceptible quality loss.
  • Using negative Euclidean distance and direct token discarding instead of cosine similarity and mean-based merging improves both output quality and latency, with the quality benefit most visible when reducing value tokens.
  • Larger models tolerate the reduction better than smaller ones, and the method composes with step-distilled few-step samplers without further tuning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the context-agnostic similarity-pattern assumption holds, the same precomputed schedule could be reused across prompts with zero per-prompt profiling; a natural stress test is to deploy AsymRnR on out-of-distribution prompts such as fast camera motion or scenes with many small independently moving objects and check whether the claimed speed-quality tradeoff persists.
  • The KL-divergence framing suggests a stronger criterion than thresholding: one could directly minimize the estimated nearest-neighbor divergence per layer, yielding a content-adaptive per-token budget rather than fixed thresholds.
  • The Q-versus-K/V asymmetry may carry over to cross-attention and autoregressive attention in video models, where the questioner role also differs, though the paper only treats self-attention.
  • The matching cache exploits temporal smoothness of token matches; the same lazy-update idea could be applied to the reduction schedule itself, refreshing the schedule as the video content evolves rather than fixing it once.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes AsymRnR, a training-free token-reduction method for accelerating video diffusion transformers. It asymmetrically reduces query and key/value token sequences before self-attention and restores them afterward, schedules reductions adaptively across blocks and denoising steps based on precomputed similarity statistics, and introduces a matching cache that reuses bipartite matching results across timesteps. Experiments on CogVideoX-2B/5B, Mochi-1, HunyuanVideo, FastVideo-Hunyuan, and AnimateDiff report latency speedups of about 1.1x-1.3x with aggregate VBench score changes near 0.01 or better on several models, plus a combination with the PAB feature-caching method reaching 1.71x. The paper also presents a KL-divergence estimator (Corollary 3.1) as a theoretical motivation for Euclidean-distance matching.

Significance. If the empirical claims hold, AsymRnR is a practical and broadly applicable plug-in: it is training-free, combines with step distillation and feature caching, is evaluated on a standard benchmark with over 900 prompts, and comes with released code. The consistent speedups across five model families, the clear ablations of scheduling, matching cache, similarity metric, and partition stride, and the explicit compatibility discussion are strengths. However, the central claim of 'negligible degradation' is supported only at the aggregate VBench level; the paper's own per-dimension tables show substantial drops on semantic dimensions such as Multiple Objects and Color, and no uncertainty quantification is provided. The theoretical motivation is suggestive but is not a proof that KL-minimizing token reduction preserves attention outputs. Overall the contribution is valuable and the empirical core is defensible, but the quality claim and the generalization claim need to be substantially qualified and better supported.

major comments (3)
  1. [§4.2, Table 11; Abstract] The abstract's 'negligible degradation' is contradicted by the paper's own per-dimension VBench results. For CogVideoX-2B, the default AsymRnR configuration drops Multiple Objects from 0.6502 to 0.5480 (an absolute decrease of 0.102), Color from 0.8776 to 0.8252 (0.052), and Scene from 0.5378 to 0.5029 (0.035), while the aggregate VBench falls only from 0.8008 to 0.7917 (0.009). The OURS-FAST row shows a comparable Multiple Objects drop from 0.6502 to 0.5686. No error bars, multiple seeds, or significance tests are reported for VBench or latency in Section 4 or Appendix E, so it is impossible to judge whether these per-dimension degradations are reproducible or perceptually meaningful. Because 'negligible degradation' is a headline claim, the paper should either report uncertainty and a per-dimension analysis, or explicitly qualify the claim to something like 'small aggregate VBench change with some per-dimension degradation.'
  2. [§3.4, Eq. (5); Appendix B] The reduction schedule rests on the assumption that per-feature token similarity patterns are 'context-agnostic and can be considered an intrinsic property of the models.' Figure 4 asserts consistency across diverse contents, but no quantitative transfer evidence is given, and Appendix B shows that the similarity thresholds and reduction rates are tuned per model by visual inspection on a small number of prompts. If the similarity map is prompt-dependent, the precomputed schedule will reduce the wrong tokens for unseen prompts, and the claimed plug-and-play advantage disappears. Please provide evidence that a schedule estimated on one prompt set transfers to another, or at least a sensitivity analysis showing that VBench and latency are robust to the threshold choices.
  3. [§3.1, Corollary 3.1, Eq. (2); Appendix A] The theoretical foundation has several issues. Corollary 3.1 applies to covariance-stationary sequences sampled from two distributions P and P', but the reduction operates on a single deterministic token sequence, and the selected tokens depend on empirical distances computed from the same sequence, so the stationarity/independence condition is not satisfied. In addition, the additive term in Eq. (2), written 'log l/l′ − 1', does not match the estimator in Appendix Eq. (10); for the stated sample sizes the term should be log(l/(l′−1)), and as written it can even be negative. Finally, minimizing D_KL(P'||P) between token distributions is not shown to be equivalent to preserving the attention output softmax(QK^⊤)V; the paper motivates but does not prove this connection. Since the abstract advertises 'theoretical foundations,' this section should be corrected or explicitly labeled as heuristic motivation.
minor comments (6)
  1. [§3.5] The word 'metioned' should be 'mentioned'.
  2. [§4.2, 'Integration with Feature Caching'] The sentence 'the results are summarized in Table 1' should refer to Table 3, which reports the PAB combination.
  3. [§4.2, 'Integration with UNet-based video diffusion models'] The sentence 'The corresponding qualitative results are provided in Table 4' should say 'quantitative results' or point to the actual figure, since Table 4 contains numbers rather than qualitative comparisons.
  4. [Table 11] The header contains the typo 'MUTLIPLE OBJECTS'; it should be 'MULTIPLE OBJECTS'.
  5. [Figure 4] The caption states 'The shaded areas indicate the confidence interval,' but the text does not describe how the confidence interval is computed or over how many generations.
  6. [Section 5 (Limitation)] The limitation statement that 'visual discrepancies in the generated outputs' exist should be reconciled with the abstract's 'negligible degradation' wording, since the two statements are in tension.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: AsymRnR's quality claims are benchmarked against external VBench/LPIPS, and the reduction schedule is tuned by visual inspection rather than fitted to the reported metrics. Only minor non-load-bearing self-citations appear.

full rationale

The paper's central claim—training-free acceleration with negligible quality degradation—is supported by comparisons to external VBench and LPIPS scores on multiple video DiTs, not by feeding the method's own outputs back into the method. The reduction schedule in Section 3.4 (Eq. 5) uses per-model thresholds and rates tuned manually by visual inspection (Appendix B) to hit target speedups; the reported VBench numbers are not used to select these constants, so the quality results are not forced by construction. Corollary 3.1 in Section 3.1 is imported from Wang et al. (2009) and used as a motivation for matching-based reduction; it does not define the algorithm and is not a self-citation. The only author self-citations (Sun et al. 2024; Shuai et al. 2024; Tu et al. 2024) appear in survey-style sentences about the broader literature and do not carry the argument. The Limitation section (Section 5) honestly notes visual discrepancies and per-model hyperparameter sensitivity; those are correctness concerns (and the per-dimension VBench drops in Table 11 for CogVideoX-2B, e.g., Multiple Objects 0.6502 to 0.5480, deserve scrutiny), but they are not circularity. No equation in the paper is identical by construction to its own input, and no 'prediction' is a renamed fitted parameter. Score 2 reflects the presence of minor non-load-bearing self-citations and manual per-model tuning, not a circular derivation.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new physical entities are introduced. The matching cache is a software buffer, not a new theoretical object. The main cost of the method beyond the model itself is the per-model tuning of thresholds, rates, cache steps, and partition stride.

free parameters (3)
  • Similarity thresholds and reduction rates per feature per model = e.g. Q {0.6:0.4, 0.7:0.8}, V {0.8:0.3} for CogVideoX-2B (Table 8)
    Tuned manually by visual inspection in 10 iterations (Appendix B); directly control where and how many tokens are dropped.
  • Matching cache step s = s=5 for CogVideoX, s=3 for Mochi-1 and HunyuanVideo, disabled for FastVideo (Appendix C)
    Controls reuse of matching results across denoising steps; chosen to balance latency and quality (Table 6).
  • Partition stride = (2,2,2) for CogVideoX, (6,2,2) for Mochi-1, HunyuanVideo, and FastVideo (Table 9)
    Sets destination-token fraction rd in BSM; selected to trade off matching accuracy versus latency.
assumptions (4)
  • standard math Nearest-neighbor KL estimator (Wang et al. 2009) is consistent; Corollary 3.1 follows by covariance stationarity.
    Used in Section 3.1 to justify matching-based reduction as minimizing KL divergence between token distributions.
  • domain assumption Minimizing KL divergence between the reduced and original token distributions is the right objective for preserving pretrained attention outputs.
    Stated in Section 3.1 as 'To prevent the pretrained network from being affected by covariate shift'; not formally connected to softmax outputs.
  • domain assumption Similarity patterns across blocks and timesteps are model-specific but context-agnostic, so precomputed schedules transfer to new prompts.
    Section 3.4: 'such a pattern is model-specific but context-agnostic and can be considered an intrinsic property of the models.' This underpins the reduction scheduling.
  • domain assumption K and V share one reduction scheme because of one-to-one correspondence.
    Section 3.3: 'K and V must share the same reduction scheme due to their one-to-one correspondence.' This is a structural constraint, not an empirical finding.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AsymRnR: Video Diffusion Transformers Acceleration with Asymmetric Reduction and Restoration." pith.science (2026). https://pith.science/paper/TJTYI7XZ

@misc{pith2026241211706,
  author       = {Pith},
  title        = {Pith review of: AsymRnR: Video Diffusion Transformers Acceleration with Asymmetric Reduction and Restoration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TJTYI7XZ}},
  note         = {Machine review of arXiv:2412.11706}
}
read the original abstract

Diffusion Transformers (DiTs) have proven effective in generating high-quality videos but are hindered by high computational costs. Existing video DiT sampling acceleration methods often rely on costly fine-tuning or exhibit limited generalization capabilities. We propose Asymmetric Reduction and Restoration (AsymRnR), a training-free and model-agnostic method to accelerate video DiTs. It builds on the observation that redundancies of feature tokens in DiTs vary significantly across different model blocks, denoising steps, and feature types. Our AsymRnR asymmetrically reduces redundant tokens in the attention operation, achieving acceleration with negligible degradation in output quality and, in some cases, even improving it. We also tailored a reduction schedule to distribute the reduction across components adaptively. To further accelerate this process, we introduce a matching cache for more efficient reduction. Backed by theoretical foundations and extensive experimental validation, AsymRnR integrates into state-of-the-art video DiTs and offers substantial speedup.

Figures

Figures reproduced from arXiv: 2412.11706 by the authors.

Figure 1
Figure 1. Quality and speed comparison between baseline models, HunyuanVideo (Team, 2024c) and FastVideo-Hunyuan (Team, 2024a), with our AsymRnR. Our approach enables training-free, lossless acceleration for state-of-the-art video diffusion transformers. Abstract Diffusion Transformers (DiTs) have proven ef￾fective in generating high-quality videos but are hindered by high computational costs. Exist￾ing video DiT sampling acc… view at source ↗
Figure 2
Figure 2. Altering different components in video DiTs leads to varying degradation. Green blocks represent original attention blocks. Blue blocks represent attention blocks where 30% of the query tokens are randomly discarded, allowing only the remaining 70% to contribute to the output. Red blocks represent the same perturbation applied to key and value tokens. The comparison includes perturbing: (a) different features: Q or … view at source ↗
Figure 3
Figure 3. Overview of (a) symmetric and (b) asymmetric strategies. Both methods reduce the processing sequence length before self-attention to enhance efficiency and subsequently restore it to the original length for dense prediction. SymRnR performs reduction before mapping to Q, K, and V , whereas AsymRnR applies reduction afterward. This flexibility allows for the adaptive assignment of varying reduction rates to individua… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: CogVideoX (Yang et al., 2024) attention feature similarity distribution. The shaded areas indicate the confidence interval. Blocks are divided into four groups, each exhibiting distinct trends, with variations observed across different feature types. These patterns rem…
Figure 5
Figure 5. Figure 5: Heatmap of matching similarity at different denoising timesteps. The similarities across successive timesteps are nearly identical, but divergence increases with a larger step gap. eration R −1 A is modified correspondingly. This reduction scheduling is also asymmetric…
Figure 6
Figure 6. Figure 6: Qualitative comparison on CogVideoX-2B (Yang et al., 2024). ToMe (Bolya & Hoffman, 2023) exhibits blurriness (left) and pixelation (right), whereas our AsymRnR consistently performs well. The video examples are provided in the Supplementary Materials. avg. latency 348s…
Figure 7
Figure 7. Figure 7: Qualitative results on CogVideoX-5B (Yang et al., 2024), Mochi-1 (Team, 2024b), HunyuanVideo (Team, 2024c), and FastVideo-Hunyuan (Team, 2024a). ToMe (Bolya & Hoffman, 2023) is incompatible with these models; we present videos generated by the baseline models and our p…
Figure 8
Figure 8. Figure 8: Quantitative evaluation on HunyuanVideo. AsymRnR is compatible with the feature caching method PAB (Zhao et al., 2024), and together they achieve a 1.71× overall acceleration. Integration with Feature Caching. Our AsymRnR acceler￾ates sampling by reducing the computati…
Figure 9
Figure 9. Figure 9: Quality-latency trade-off for individual features. Uni￾formly reducing V shows superior quality, whereas reducing Q in isolation leads to a substantial quality decline [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: The distribution of feature Euclidean norms. The dashed line indicates the 95th percentile. Compared to input H, the value V norm distribution exhibits a longer tail, which can cause distortion when using cosine similarity for matching. AsymRnR and leave their integra…
Figure 11
Figure 11. Figure 11: Additional qualitative comparison on CogVideoX￾5B (Yang et al., 2024). (Huang et al., 2024), serving as an extended reference to Tables 1 and 2. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: Additional qualitative comparison on CogVideoX-2B (Yang et al., 2024). 16 [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]
Figure 14
Figure 14. Figure 14: Additional qualitative comparison on FastVideo￾Hunyuan (Team, 2024a) 17 [PITH_FULL_IMAGE:figures/full_fig_p017_14.png]
Figure 13
Figure 13. Figure 13: Additional qualitative comparison on Hunyuan￾Video (Team, 2024c) and Mochi-1 (Team, 2024b). avg. latency 159s avg. latency 128s FastVideo-Hunyuan +AsymRnR (original) (24% faster) [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

65 extracted references · 29 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    J., Kiros, J

    Ba, L. J., Kiros, J. R., and Hinton, G. E. Layer normalization. CoRR, abs/1607.06450, 2016

    arXiv 2016

  3. [3]

    ediff-i: Text-to-image diffusion models with an ensemble of expert denoisers

    Balaji, Y., Nah, S., Huang, X., Vahdat, A., Song, J., Kreis, K., Aittala, M., Aila, T., Laine, S., Catanzaro, B., Karras, T., and Liu, M. ediff-i: Text-to-image diffusion models with an ensemble of expert denoisers. CoRR, abs/2211.01324, 2022

    arXiv 2022

  4. [4]

    and Hoffman, J

    Bolya, D. and Hoffman, J. Token merging for fast stable diffusion. In CVPRW , pp.\ 4599--4603. IEEE , 2023

    work page 2023

  5. [5]

    Token merging: Your vit but faster

    Bolya, D., Fu, C., Dai, X., Zhang, P., Feichtenhofer, C., and Hoffman, J. Token merging: Your vit but faster. In ICLR . OpenReview.net, 2023

    work page 2023

  6. [6]

    \( \) -dit: A training-free acceleration method tailored for diffusion transformers

    Chen, P., Shen, M., Ye, P., Cao, J., Tu, C., Bouganis, C., Zhao, Y., and Chen, T. \( \) -dit: A training-free acceleration method tailored for diffusion transformers. CoRR, abs/2406.01125, 2024

    arXiv 2024

  7. [7]

    M., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarl \' o s, T., Hawkins, P., Davis, J

    Choromanski, K. M., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarl \' o s, T., Hawkins, P., Davis, J. Q., Mohiuddin, A., Kaiser, L., Belanger, D. B., Colwell, L. J., and Weller, A. Rethinking attention with performers. In ICLR . OpenReview.net, 2021

    work page 2021

  8. [8]

    M., and Jeni, L

    Choudhury, R., Zhu, G., Liu, S., Niinuma, K., Kitani, K. M., and Jeni, L. Don't look twice: Faster video transformers with run-length tokenization. CoRR, abs/2411.05222, 2024

    arXiv 2024

Show all 65 references
  1. [9]

    Vision transformers need registers

    Darcet, T., Oquab, M., Mairal, J., and Bojanowski, P. Vision transformers need registers. In ICLR . OpenReview.net, 2024

    2024

  2. [10]

    Attention is not all you need: Pure attention loses rank doubly exponentially with depth

    Dong, Y., Cordonnier, J., and Loukas, A. Attention is not all you need: Pure attention loses rank doubly exponentially with depth. CoRR, abs/2103.03404, 2021

    2021 arXiv

  3. [11]

    Scaling rectified flow transformers for high-resolution image synthesis

    Esser, P., Kulal, S., Blattmann, A., Entezari, R., M \" u ller, J., Saini, H., Levi, Y., Lorenz, D., Sauer, A., Boesel, F., Podell, D., Dockhorn, T., English, Z., and Rombach, R. Scaling rectified flow transformers for high-resolution image synthesis. In ICML . OpenReview.net, 2024

    2024

  4. [12]

    Animatediff: Animate your personalized text-to-image diffusion models without specific tuning

    Guo, Y., Yang, C., Rao, A., Liang, Z., Wang, Y., Qiao, Y., Agrawala, M., Lin, D., and Dai, B. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. In ICLR . OpenReview.net, 2024

    2024

  5. [13]

    Clockwork diffusion: Efficient generation with model-step distillation

    Habibian, A., Ghodrati, A., Fathima, N., Sauti \` e re, G., Garrepalli, R., Porikli, F., and Petersen, J. Clockwork diffusion: Efficient generation with model-step distillation. In CVPR , pp.\ 8352--8361. IEEE , 2024

    2024

  6. [14]

    Denoising diffusion probabilistic models

    Ho, J., Jain, A., and Abbeel, P. Denoising diffusion probabilistic models. In NeurIPS, 2020

    2020

  7. [15]

    J., Shen, Y., Wallis, P., Allen - Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W

    Hu, E. J., Shen, Y., Wallis, P., Allen - Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models. In ICLR . OpenReview.net, 2022

    2022

  8. [16]

    Vbench: Comprehensive benchmark suite for video generative models

    Huang, Z., He, Y., Yu, J., Zhang, F., Si, C., Jiang, Y., Zhang, Y., Wu, T., Jin, Q., Chanpaisit, N., Wang, Y., Chen, X., Wang, L., Lin, D., Qiao, Y., and Liu, Z. Vbench: Comprehensive benchmark suite for video generative models. In CVPR , pp.\ 21807--21818. IEEE , 2024

    2024

  9. [17]

    Seeing what you miss: Vision-language pre-training with semantic completion learning

    Ji, Y., Tu, R., Jiang, J., Kong, W., Cai, C., Zhao, W., Wang, H., Yang, Y., and Liu, W. Seeing what you miss: Vision-language pre-training with semantic completion learning. In CVPR , pp.\ 6789--6798. IEEE , 2023

    2023

  10. [18]

    M., and Habibian, A

    Kahatapitiya, K., Karjauv, A., Abati, D., Porikli, F., Asano, Y. M., and Habibian, A. Object-centric diffusion for efficient video editing. In ECCV , volume 15115 of Lecture Notes in Computer Science, pp.\ 91--108. Springer, 2024 a

    2024

  11. [19]

    S., and Xie, T

    Kahatapitiya, K., Liu, H., He, S., Liu, D., Jia, M., Ryoo, M. S., and Xie, T. Adaptive caching for faster video generation with diffusion transformers. CoRR, abs/2411.02397, 2024 b

    2024 arXiv

  12. [20]

    Kim, Y., Denton, C., Hoang, L., and Rush, A. M. Structured attention networks. In ICLR . OpenReview.net, 2017

    2017

  13. [21]

    Lookupvit: Compressing visual information to a limited number of tokens

    Koner, R., Jain, G., Jain, P., Tresp, V., and Paul, S. Lookupvit: Compressing visual information to a limited number of tokens. In ECCV , volume 15144 of Lecture Notes in Computer Science, pp.\ 322--337. Springer, 2024

    2024

  14. [22]

    Selective attention improves transformer

    Leviathan, Y., Kalman, M., and Matias, Y. Selective attention improves transformer. CoRR, abs/2410.02703, 2024

    2024 arXiv

  15. [23]

    Vidtome: Video token merging for zero-shot video editing

    Li, X., Ma, C., Yang, X., and Yang, M. Vidtome: Video token merging for zero-shot video editing. In CVPR , pp.\ 7486--7495. IEEE , 2024

    2024

  16. [24]

    Open-sora plan: Open-source large video generation model

    Lin, B., Ge, Y., Cheng, X., Li, Z., Zhu, B., Wang, S., He, X., Ye, Y., Yuan, S., Chen, L., et al. Open-sora plan: Open-source large video generation model. CoRR, abs/2412.00131, 2024

    2024 arXiv

  17. [25]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Liu, X., Gong, C., and Liu, Q. Flow straight and fast: Learning to generate and transfer data with rectified flow. In ICLR . OpenReview.net, 2023

    2023

  18. [26]

    Instaflow: One step is enough for high-quality diffusion-based text-to-image generation

    Liu, X., Zhang, X., Ma, J., Peng, J., and Liu, Q. Instaflow: One step is enough for high-quality diffusion-based text-to-image generation. In ICLR . OpenReview.net, 2024

    2024

  19. [27]

    Dpm-solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps

    Lu, C., Zhou, Y., Bao, F., Chen, J., Li, C., and Zhu, J. Dpm-solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps. In NeurIPS, 2022

    2022

  20. [28]

    Lcm-lora: A universal stable-diffusion acceleration module

    Luo, S., Tan, Y., Patil, S., Gu, D., von Platen, P., Passos, A., Huang, L., Li, J., and Zhao, H. Lcm-lora: A universal stable-diffusion acceleration module. CoRR, abs/2311.05556, 2023

    2023 arXiv

  21. [29]

    Deepcache: Accelerating diffusion models for free

    Ma, X., Fang, G., and Wang, X. Deepcache: Accelerating diffusion models for free. In CVPR , pp.\ 15762--15772. IEEE , 2024 a

    2024

  22. [30]

    Latte: Latent diffusion transformer for video generation

    Ma, X., Wang, Y., Jia, G., Chen, X., Liu, Z., Li, Y., Chen, C., and Qiao, Y. Latte: Latent diffusion transformer for video generation. CoRR, abs/2401.03048, 2024 b

    2024 arXiv

  23. [31]

    Multi-modal retrieval augmented multi-modal generation: A benchmark, evaluate metrics and strong baselines

    Ma, Z.-A., Lan, T., Tu, R.-C., Hu, Y., Huang, H., and Mao, X.-L. Multi-modal retrieval augmented multi-modal generation: A benchmark, evaluate metrics and strong baselines. CoRR, abs/2411.16365, 2024 c

    2024 arXiv

  24. [32]

    P., Ermon, S., Ho, J., and Salimans, T

    Meng, C., Rombach, R., Gao, R., Kingma, D. P., Ermon, S., Ho, J., and Salimans, T. On distillation of guided diffusion models. In CVPR , pp.\ 14297--14306. IEEE , 2023

    2023

  25. [33]

    and Xie, S

    Peebles, W. and Xie, S. Scalable diffusion models with transformers. In ICCV , pp.\ 4172--4182. IEEE , 2023

    2023

  26. [34]

    Rabe, M. N. and Staats, C. Self-attention does not need o(n\( ^ 2 \)) memory. CoRR, abs/2112.05682, 2021

    2021 arXiv

  27. [35]

    Dynamicvit: Efficient vision transformers with dynamic token sparsification

    Rao, Y., Zhao, W., Liu, B., Lu, J., Zhou, J., and Hsieh, C. Dynamicvit: Efficient vision transformers with dynamic token sparsification. In NeurIPS, pp.\ 13937--13949, 2021

    2021

  28. [36]

    High-resolution image synthesis with latent diffusion models

    Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In CVPR , pp.\ 10674--10685. IEEE , 2022

    2022

  29. [37]

    U-net: Convolutional networks for biomedical image segmentation

    Ronneberger, O., Fischer, P., and Brox, T. U-net: Convolutional networks for biomedical image segmentation. In MICCAI (3) , volume 9351 of Lecture Notes in Computer Science, pp.\ 234--241. Springer, 2015

    2015

  30. [38]

    and Ho, J

    Salimans, T. and Ho, J. Progressive distillation for fast sampling of diffusion models. In ICLR . OpenReview.net, 2022

    2022

  31. [39]

    Adversarial diffusion distillation

    Sauer, A., Lorenz, D., Blattmann, A., and Rombach, R. Adversarial diffusion distillation. In ECCV , volume 15144 of Lecture Notes in Computer Science, pp.\ 87--103. Springer, 2024

    2024

  32. [40]

    A survey of multimodal-guided image editing with text-to-image diffusion models

    Shuai, X., Ding, H., Ma, X., Tu, R., Jiang, Y., and Tao, D. A survey of multimodal-guided image editing with text-to-image diffusion models. CoRR, abs/2406.14555, 2024

    2024 arXiv

  33. [41]

    A., Maheswaranathan, N., and Ganguli, S

    Sohl - Dickstein, J., Weiss, E. A., Maheswaranathan, N., and Ganguli, S. Deep unsupervised learning using nonequilibrium thermodynamics. In ICML , volume 37 of JMLR Workshop and Conference Proceedings , pp.\ 2256--2265. JMLR.org, 2015

    2015

  34. [42]

    Denoising diffusion implicit models

    Song, J., Meng, C., and Ermon, S. Denoising diffusion implicit models. In ICLR . OpenReview.net, 2021

    2021

  35. [43]

    and Ermon, S

    Song, Y. and Ermon, S. Generative modeling by estimating gradients of the data distribution. In NeurIPS, pp.\ 11895--11907, 2019

    2019

  36. [44]

    Consistency models

    Song, Y., Dhariwal, P., Chen, M., and Sutskever, I. Consistency models. CoRR, abs/2303.01469, 2023

    2023 arXiv

  37. [45]

    Su, J., Ahmed, M. H. M., Lu, Y., Pan, S., Bo, W., and Liu, Y. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568: 0 127063, 2024

    2024

  38. [46]

    Diffusion model-based video editing: A survey

    Sun, W., Tu, R., Liao, J., and Tao, D. Diffusion model-based video editing: A survey. CoRR, abs/2407.07111, 2024

    2024 arXiv

  39. [47]

    Fastvideo: a lightweight framework for accelerating large video diffusion models

    Team, F. Fastvideo: a lightweight framework for accelerating large video diffusion models. https://github.com/hao-ai-lab/FastVideo, 2024 a . Accessed: 2024-12-30

    2024

  40. [48]

    Mochi 1: A new sota in open-source video generation models

    Team, G. Mochi 1: A new sota in open-source video generation models. https://www.genmo.ai/blog, 2024 b . Accessed: 2024-11-20

    2024

  41. [49]

    Team, H. F. M. Hunyuanvideo: A systematic framework for large video generative models. CoRR, abs/2412.03603, 2024 c

    2024 arXiv

  42. [50]

    Team, M. G. Movie gen: A cast of media foundation models. CoRR, abs/2410.13720, 2024 d

    2024 arXiv

  43. [51]

    Global and local semantic completion learning for vision-language pre-training

    Tu, R., Ji, Y., Jiang, J., Kong, W., Cai, C., Zhao, W., Wang, H., Yang, Y., and Liu, W. Global and local semantic completion learning for vision-language pre-training. CoRR, abs/2306.07096, 2023

    2023 arXiv

  44. [52]

    Spagent: Adaptive task decomposition and model selection for general video generation and editing

    Tu, R.-C., Sun, W., Jin, Z., Liao, J., Huang, J., and Tao, D. Spagent: Adaptive task decomposition and model selection for general video generation and editing. CoRR, abs/2411.18983, 2024

    2024 arXiv

  45. [53]

    W., Shen, D., Gao, P., Lingelbach, M., Sun, K., Bian, W., Song, G., Liu, Y., Wang, X., and Li, H

    Wang, F., Huang, Z., Bergman, A. W., Shen, D., Gao, P., Lingelbach, M., Sun, K., Bian, W., Song, G., Liu, Y., Wang, X., and Li, H. Phased consistency models. In NeurIPS, 2024

    2024

  46. [54]

    R., and Verd \' u , S

    Wang, Q., Kulkarni, S. R., and Verd \' u , S. Divergence estimation for multidimensional densities via k-nearest-neighbor distances. IEEE Trans. Inf. Theory , 55 0 (5): 0 2392--2405, 2009

    2009

  47. [55]

    Z., Khabsa, M., Fang, H., and Ma, H

    Wang, S., Li, B. Z., Khabsa, M., Fang, H., and Ma, H. Linformer: Self-attention with linear complexity. CoRR, abs/2006.04768, 2020

    2006 arXiv

  48. [56]

    S., Kohler, J., Rupprecht, C., Cremers, D., Vajda, P., and Wang, J

    Wimbauer, F., Wu, B., Sch \" o nfeld, E., Dai, X., Hou, J., He, Z., Sanakoyeu, A., Zhang, P., Tsai, S. S., Kohler, J., Rupprecht, C., Cremers, D., Vajda, P., and Wang, J. Cache me if you can: Accelerating diffusion models through block caching. In CVPR , pp.\ 6211--6220. IEEE , 2024

    2024

  49. [57]

    Duoattention: Efficient long-context LLM inference with retrieval and streaming heads

    Xiao, G., Tang, J., Zuo, J., Guo, J., Yang, S., Tang, H., Fu, Y., and Han, S. Duoattention: Efficient long-context LLM inference with retrieval and streaming heads. CoRR, abs/2410.10819, 2024

    2024 arXiv

  50. [58]

    A survey on video diffusion models

    Xing, Z., Feng, Q., Chen, H., Dai, Q., Hu, H., Xu, H., Wu, Z., and Jiang, Y. A survey on video diffusion models. ACM Comput. Surv. , 57 0 (2): 0 41:1--41:42, 2025

    2025

  51. [59]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Yang, Z., Teng, J., Zheng, W., Ding, M., Huang, S., Xu, J., Yang, Y., Hong, W., Zhang, X., Feng, G., Yin, D., Gu, X., Zhang, Y., Wang, W., Cheng, Y., Liu, T., Xu, B., Dong, Y., and Tang, J. Cogvideox: Text-to-video diffusion models with an expert transformer. CoRR, abs/2408.06...

    2024 arXiv

  52. [60]

    M., Mallya, A., Kautz, J., and Molchanov, P

    Yin, H., Vahdat, A., \' A lvarez, J. M., Mallya, A., Kautz, J., and Molchanov, P. A-vit: Adaptive tokens for efficient vision transformer. In CVPR , pp.\ 10799--10808. IEEE , 2022

    2022

  53. [61]

    Zhang, C., Zhang, C., Zhang, M., and Kweon, I. S. Text-to-image diffusion models in generative AI: A survey. CoRR, abs/2303.07909, 2023

    2023 arXiv

  54. [62]

    A., Shechtman, E., and Wang, O

    Zhang, R., Isola, P., Efros, A. A., Shechtman, E., and Wang, O. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR , pp.\ 586--595. IEEE, 2018

    2018

  55. [63]

    Z., and Schmidhuber, J

    Zhang, W., Liu, H., Xie, J., Faccio, F., Shou, M. Z., and Schmidhuber, J. Cross-attention makes inference cumbersome in text-to-image diffusion models. CoRR, abs/2404.02747, 2024

    2024 arXiv

  56. [64]

    Real-time video generation with pyramid attention broadcast

    Zhao, X., Jin, X., Wang, K., and You, Y. Real-time video generation with pyramid attention broadcast. CoRR, abs/2408.12588, 2024

    2024 arXiv

  57. [65]

    Open-sora: Democratizing efficient video production for all

    Zheng, Z., Peng, X., Yang, T., Shen, C., Li, S., Liu, H., Zhou, Y., Li, T., and You, Y. Open-sora: Democratizing efficient video production for all. https://github.com/hpcaitech/Open-Sora, 2024. Accessed: 2024-11-20

    2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.