Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

CascadeV: An Implementation of Wurstchen Architecture for Video Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A cascaded latent diffusion decoder can upscale existing text-to-video outputs 4× without fine-tuning.

desk verdict A clever resize-based cascade trick that is currently only validated by reconstruction experiments; the no-fine-tuning 4x claim needs a direct test on real T2V outputs before it is taken seriously. read the letter →

arxiv 2501.16612 v1 pith:TNVG2XMX submitted 2025-01-28 cs.CV

Wenfeng Lin , Jiangchuan Wei , Boyuan Liu , Yichen Zhang , Shiyue Yan , Mingyu Guo This is my paper · ORCID
classification cs.CV
keywords videogenerationlatentdiffusionmodelcascadedWürstchenarchitecturesuper-resolutionframeinterpolation3Dattentioncompressionratio
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that high-resolution video can be generated far more cheaply by making the video autoencoder itself a diffusion model. The authors build CascadeV, a two-stage pipeline in which an existing text-to-video model produces small latents and a latent-diffusion VAE decoder adds back high-frequency detail, reaching a 32:1 spatial compression ratio and 2K output. The central claim is that because the decoder shares the same standard VAE latent space as many existing video models, it can be dropped onto their outputs, raising resolution or frame rate 4×, with no fine-tuning. If this holds, video generation becomes modular: any base model can be upgraded in post-processing, and the compute cost of high-resolution synthesis drops sharply.

What carries the argument

The central object is the LDM-VAE decoder: a diffusion transformer (DiT) cascaded with a standard VAE. It takes as conditional input the upsampled latent of the base model, concatenated with fresh noise, and produces a representation the standard VAE can decode at much higher resolution. Two ideas carry the argument. First, the spatiotemporal alternating grid 3D attention partitions the feature map into blocks whose temporal dimension is the square of the spatial dimension, so every attention computation mixes space and time at uniform cost $O\left(\frac{F^2H^2W^2}{P^2}D\right)$, cutting global-attention complexity by $1/P^2$ while keeping frames consistent. Second, aligning the latent spaces by simple resizing, rather than a learned semantic compressor, makes the decoder a latent-space super-resolution model that can be inserted after any model sharing the same VAE.

What would settle it

Take a text-to-video model that was trained with a different VAE or a different latent normalization, cascade CascadeV onto its outputs without fine-tuning, and measure whether the decoded 2K video preserves the base model's semantic content and temporal coherence. A clear failure, such as severe artifacts, semantic drift, or flicker, would show the natural alignment is not universal.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that a VAE built from a latent diffusion model can decode latents from an independently trained base model if both live in the same standard VAE latent space. The authors achieve this by resizing the input video before encoding, so the base model and the DiT decoder see latents that are naturally aligned, rather than by training a special semantic compressor tied to one base model. In experiments on 4K 60 fps footage, the cascaded model reaches a 1 × 32 × 32 = 1024 compression ratio while keeping temporal consistency metrics, such as subject consistency, background consistency, and temporal flickering, competitive with or better than lower-compression VAEs. The result is presented as a general recipe: using the same VAE during training gives the LDM-VAE super-resolution and frame-interpolation abilities, letting it enhance outputs of models like Open-Sora-Plan v1.1.0 and Stable Video Diffusion without retraining.

Load-bearing premise

The load-bearing premise is that upsampling a video and encoding it with the standard VAE puts it in the same latent space as the base model's own latents, so the decoder can be grafted on without retraining; if that alignment breaks for some models, the 4× no-fine-tuning claim collapses.

Editorial extensions

If this is right

  • Any existing latent video model that shares the standard VAE latent space can be upgraded to 2K output by appending the LDM-VAE decoder, without retraining or modifying the base model.
  • The 32:1 spatial compression ratio lowers the compute spent on high-resolution video generation, since the expensive diffusion steps run in a much smaller latent space.
  • The grid-based 3D attention keeps spatial and temporal information interacting in every block, translating into improved subject consistency, background consistency, and temporal flicker scores at equal or higher compression.
  • Using VAE variants with different temporal compression rates during training gives the same decoder frame-interpolation ability, as demonstrated by raising Stable Video Diffusion outputs to 28 fps.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 'naturally aligned' argument implies a universal adapter: any pixel-space video encoded with the standard VAE becomes a valid input, turning CascadeV into a video super-resolution service rather than a module tied to one generator.
  • A stress test the paper does not run is cascading onto a model trained with a different VAE or different latent scaling; if alignment fails there, the no-fine-tuning claim is conditional on the VAE family rather than universal.
  • The traditional-metric deficit (lower PSNR and SSIM) indicates the decoder is generative rather than reconstructive; for tasks where ground-truth fidelity matters, users should expect hallucinated high-frequency detail, while for creative generation this may be an acceptable trade.
  • The complexity argument suggests the grid partition $P$ is a free knob, and larger $P$ may scale to even higher resolutions, but the paper does not explore that trade-off curve.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes CascadeV, a two-stage cascaded latent video model in the style of Würstchen: a base T2V model produces latents, and an LDM-based VAE (a DiT followed by a standard VAE decoder) decodes them at a claimed 32:1 spatial compression ratio. The main architectural novelty is a spatiotemporal alternating grid 3D attention mechanism intended to reduce the cost of full 3D attention while preserving spatiotemporal interaction. The paper further claims that, by training the DiT on standard-VAE latents of downsampled real videos, the cascade can be attached to arbitrary existing T2V models (e.g., Open-Sora-Plan, SVD) to obtain a 4x resolution or FPS increase without fine-tuning. Quantitative evaluation is limited to reconstruction of Intern4k videos compared against the VAEs of Open-Sora-Plan, EasyAnimate, and StableCascade; the cascading claims are supported only by qualitative examples in Figure 1.

Significance. If the central cascade claim were validated, the work would offer a practical way to boost the resolution and frame rate of existing video generators without retraining them, and the 32:1 compression idea could reduce the compute cost of high-resolution video synthesis. The reconstruction experiments are cleanly framed and the Intern4k results at 1024:1 compression are non-trivial; the release of code is a strength. The grid-attention mechanism is a plausible efficiency idea, although its complexity analysis is under-specified. However, the headline claims -- state-of-the-art 2K generation and training-free 4x resolution/FPS cascading -- go beyond what the experiments demonstrate. The quantitative study is reconstruction-only, and no controlled experiment measures whether the DiT transfers to the latent distribution of arbitrary base-model outputs. There is no internal circularity concern: the complexity claim is a derivation and the reconstruction numbers are direct measurements, not fitted predictions.

major comments (4)
  1. [Abstract, §3.4, Figure 1] The central claim that CascadeV can be cascaded with arbitrary existing T2V models to achieve a 4x resolution or FPS increase without fine-tuning is not tested. Tables 1 and 2 evaluate reconstruction of ground-truth Intern4k videos after downsampling and encoding, not the outputs of Open-Sora-Plan or SVD. The DiT was trained on standard-VAE latents of real videos, and applying it to latents of generated base-model outputs is an unquantified domain-transfer assumption. Figure 1 provides only qualitative examples with no comparisons to standard upsampling or frame-interpolation baselines and no metrics. Please add a controlled experiment that takes actual base-model outputs, runs the proposed cascade, and reports quantitative resolution/FPS enhancement results (e.g., PSNR/SSIM/LPIPS against high-resolution references, or VBench/FVD on the enhanced videos), together with simple baselines such as bicubic upscaling and optical-flow interpolation.
  2. [§4.4] The FPS-increase variant is neither specified nor evaluated. Section 4.4 says only that 'by employing VAEs with different temporal compression rates during training' the model can increase FPS, and Figure 1 shows one qualitative row for SVD. There is no description of how the temporal compression rate is varied, what conditioning the DiT receives to perform frame interpolation, or any quantitative result for the FPS variant. Since a 4x FPS increase is part of the headline claim, this is a load-bearing gap rather than a minor omission.
  3. [Abstract, §4.2] The claim of 'state-of-the-art 2K resolution video generation' is not supported by the reported experiments. Tables 1 and 2 measure reconstruction quality on a fixed set of Intern4k source videos; they do not assess text-to-video generation, do not compare against state-of-the-art T2V models at 2K, and do not include human evaluation. At most, the evidence supports a claim about reconstruction fidelity at a high compression ratio, not a claim about end-to-end generation quality.
  4. [§3.3] The complexity analysis of the spatiotemporal alternating grid 3D attention is under-specified and cannot be checked as written. The text states that 'we set the temporal dimension partition to be the square of the spatial dimension' and then immediately gives a per-attention complexity of O(F^2 H^2 W^2 / P^2 D), but it does not define the number of partitions in each dimension or show how this expression is derived. Since the efficiency benefit is a stated contribution, please specify the exact partition schedule, derive the per-block and total complexity, and state clearly what the adjustable parameter P controls.
minor comments (5)
  1. [Table 1] The LPIPS column header uses an up arrow, but lower LPIPS values are better; the arrow should point down.
  2. [Throughout] There are multiple typos and spacing inconsistencies, including 'latnet' (Section 3.1), 'unsampled' should likely be 'upsampled' (Section 3.1), 'Open-Spra-Plan' (Section 3.2), 'conducte' and 'compression ration' (Section 4.2), and inconsistent spacing in 'V AE'.
  3. [References] Reference [42] (EasyAnimate) lacks a venue or arXiv identifier, and reference [1] (Open-Sora-Plan) lists only an institutional author; both should be completed for reproducibility.
  4. [Figure 3] The caption 'Our model significantly reduce the computational complexity' contains a subject-verb agreement error; it should read 'reduces'.
  5. [§4.1] The experimental section does not report the number of evaluation videos used, error bars, or standard deviations for the metrics in Tables 1 and 2, making it difficult to assess the significance of the reported differences.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the compression ratio, attention-complexity reduction, and cascade claims are transparent arithmetic, an algebraic consequence of the stated partition schedule, or an externally referenced design hypothesis.

full rationale

Reviewing the derivation chain: (1) The 32:1 compression ratio is presented transparently in Section 3.4 and Figure 4b as a 4× resizing followed by an 8:1 standard VAE; it is simple arithmetic (4×8=32), not a fitted or predicted quantity, and the paper does not conceal the resizing step. (2) The grid-attention complexity reduction is a direct algebraic consequence of the stated partition schedule (temporal partition equal to the square of the spatial partition), giving total complexity O(F^2H^2W^2/P^2 D); it is not an empirical claim fitted to a target result. (3) The headline no-fine-tuning cascade property is an asserted domain-transfer hypothesis ('since the latent spaces are naturally aligned, our DiT can be cascaded into existing T2V model decoders without fine-tuning') and is explicitly hedged as 'theoretically' in both the Abstract and Section 3.4. The lack of quantitative validation on real base-model outputs is a correctness or empirical gap, not a circular reduction, because the claim is not obtained by renaming a fitted parameter or by importing a self-citation as proof. (4) All load-bearing references (Würstchen [13], Open-Sora-Plan [1], SVD [2], PixArt-σ [40]) are external; there are no author self-citations and no imported uniqueness theorem. No equation in the paper reduces to its own input, and no prediction is statistically forced by construction. Therefore the paper is not circular.

Assumptions & free parameters 1 free parameters · 3 assumptions · 1 invented entities

The central claims rest on the Würstchen cascade idea, the resizing-based latent alignment trick, and a tunable attention partition size. No external validation of the no-fine-tuning cascading claim is provided, and the evaluation relies on a single small dataset.

free parameters (1)
  • grid partition size P = not specified
    The spatiotemporal grid attention partitions the feature map into blocks of size P (Section 3.3). P is a tunable hyperparameter that trades quality against compute; the paper gives no fixed value or search strategy, and results may depend on it.
assumptions (3)
  • domain assumption The Würstchen semantic compressor can be replaced by existing T2V latent models, and latent spaces can be aligned via simple resizing.
    The decoupling property rests on an assumption that resizing preserves the semantic content needed for decoding, stated in Section 3.4 and Figure 4b.
  • domain assumption The standard VAE and DiT components (from LDM [12] and PixArt-σ [40]) perform as described in their source papers.
    The paper adopts these components without re-evaluation or adaptation details, so correctness relies on the source papers.
  • domain assumption The Intern4k dataset of 1000 high-quality 4K videos is sufficient to evaluate reconstruction and generalization.
    Evaluation uses only this single dataset (Section 4.1); no train/test split is specified, raising potential overfitting and selection-bias concerns.
invented entities (1)
  • Spatiotemporal alternating grid 3D attention
    purpose: Jointly processes spatial and temporal information to improve frame consistency while reducing attention complexity to O(F^2 H^2 W^2 / P^2 D).
    The mechanism is introduced in this paper; its benefit is only shown in the provided qualitative figures and VBench scores, with no external validation or ablation isolating its effect.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CascadeV: An Implementation of Wurstchen Architecture for Video Generation." pith.science (2026). https://pith.science/paper/TNVG2XMX

@misc{pith2026250116612,
  author       = {Pith},
  title        = {Pith review of: CascadeV: An Implementation of Wurstchen Architecture for Video Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TNVG2XMX}},
  note         = {Machine review of arXiv:2501.16612}
}
abstract

Recently, with the tremendous success of diffusion models in the field of text-to-image (T2I) generation, increasing attention has been directed toward their potential in text-to-video (T2V) applications. However, the computational demands of diffusion models pose significant challenges, particularly in generating high-resolution videos with high frame rates. In this paper, we propose CascadeV, a cascaded latent diffusion model (LDM), that is capable of producing state-of-the-art 2K resolution videos. Experiments demonstrate that our cascaded model achieves a higher compression ratio, substantially reducing the computational challenges associated with high-quality video generation. We also implement a spatiotemporal alternating grid 3D attention mechanism, which effectively integrates spatial and temporal information, ensuring superior consistency across the generated video frames. Furthermore, our model can be cascaded with existing T2V models, theoretically enabling a 4$\times$ increase in resolution or frames per second without any fine-tuning. Our code is available at https://github.com/bytedance/CascadeV.

Figures

Figures reproduced from arXiv: 2501.16612 by the authors.

Figure 1
Figure 1. Samples of CascadeV. Top: video reconstruction (of samples from Open-Sora-Plan v1.1.0 [1]) with high compression ratio. Even at 64 : 1 compression ratios, our model is still able to reconstruct high-frequency details. Middle: 4× resolution enhancement of Open-Sora-Plan v1.1.0 [1] results. Bottom: 4× FPS improvement of SVD [2] results. By considering the output of existing T2V models as intermediate results, our mode… view at source ↗
Figure 2
Figure 2. Overall architecture. By cascading the Latent Diffusion Model (LDM), the LDM-VAE can decode the output [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. 3D attention with grid. Our model significantly reduce the computational complexity of 3D attention while [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Two approaches for generating conditioning in DiT. (a) Utilizing high-compression Semantic compressor, [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Qualitative results. Despite the higher compression ratio of our model, the reconstructed results still exhibit [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Reward Lightning: Fast Video Generation via Homologous Preference Distillation

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Homologous preference distillation evaluates adversarial distillation and latent reward alignment on identical latent features, yielding 1–4-step video generators that improve VBench by 2.1% while leading text, motion...

Reference graph

Works this paper leans on

43 extracted references · 16 canonical work pages · cited by 1 Pith paper

  1. [1]

    Open-sora-plan, April 2024

    PKU-Yuan Lab and Tuzhan AI etc. Open-sora-plan, April 2024

    2024

  2. [2]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023

    arXiv 2023

  3. [3]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  4. [4]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020

    arXiv 2011

  5. [5]

    Diffsound: Discrete diffusion model for text-to-sound generation

    Dongchao Yang, Jianwei Yu, Helin Wang, Wen Wang, Chao Weng, Yuexian Zou, and Dong Yu. Diffsound: Discrete diffusion model for text-to-sound generation. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023

    work page 2023

  6. [6]

    Make-an-audio: Text-to-audio generation with prompt-enhanced diffusion models

    Rongjie Huang, Jiawei Huang, Dongchao Yang, Yi Ren, Luping Liu, Mingze Li, Zhenhui Ye, Jinglin Liu, Xiang Yin, and Zhou Zhao. Make-an-audio: Text-to-audio generation with prompt-enhanced diffusion models. In International Conference on Machine Learning, pages 13916–13932. PMLR, 2023

    work page 2023

  7. [7]

    simple diffusion: End-to-end diffusion for high resolution images

    Emiel Hoogeboom, Jonathan Heek, and Tim Salimans. simple diffusion: End-to-end diffusion for high resolution images. In International Conference on Machine Learning, pages 13213–13232. PMLR, 2023

    work page 2023

  8. [8]

    Inf-dit: Upsampling any-resolution image with memory-efficient diffusion transformer

    Zhuoyi Yang, Heyang Jiang, Wenyi Hong, Jiayan Teng, Wendi Zheng, Yuxiao Dong, Ming Ding, and Jie Tang. Inf-dit: Upsampling any-resolution image with memory-efficient diffusion transformer. arXiv preprint arXiv:2405.04312, 2024

    arXiv 2024

Show all 43 references
  1. [9]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machine ...

    2024

  2. [10]

    Improving image generation with better captions

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8, 2023

    2023

  3. [11]

    Photorealistic video generation with diffusion models

    Agrim Gupta, Lijun Yu, Kihyuk Sohn, Xiuye Gu, Meera Hahn, Li Fei-Fei, Irfan Essa, Lu Jiang, and José Lezama. Photorealistic video generation with diffusion models. arXiv preprint arXiv:2312.06662, 2023

    2023 arXiv

  4. [12]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  5. [13]

    Würstchen: An efficient architecture for large-scale text-to-image diffusion models

    Pablo Pernias, Dominic Rampas, Mats Leon Richter, Christopher Pal, and Marc Aubreville. Würstchen: An efficient architecture for large-scale text-to-image diffusion models. In The Twelfth International Conference on Learning Representations, 2023

    2023

  6. [14]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4195–4205, 2023

    2023

  7. [15]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020

    2010 arXiv

  8. [16]

    Fast sampling of diffusion models with exponential integrator

    Qinsheng Zhang and Yongxin Chen. Fast sampling of diffusion models with exponential integrator. In NeurIPS 2022 Workshop on Score-Based Methods, 2022. 7 A PREPRINT

    2022

  9. [17]

    Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775–5787, 2022

    2022

  10. [18]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2021

    2021

  11. [19]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023

    2023

  12. [20]

    Cascaded diffusion models for high fidelity image generation

    Jonathan Ho, Chitwan Saharia, William Chan, David J Fleet, Mohammad Norouzi, and Tim Salimans. Cascaded diffusion models for high fidelity image generation. Journal of Machine Learning Research, 23(47):1–33, 2022

    2022

  13. [21]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023

    2023 arXiv

  14. [22]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part ...

    2015

  15. [23]

    Videogpt: Video generation using vq-vae and transformers

    Wilson Yan, Yunzhi Zhang, Pieter Abbeel, and Aravind Srinivas. Videogpt: Video generation using vq-vae and transformers. arXiv preprint arXiv:2104.10157, 2021

    2021 arXiv

  16. [24]

    Nüwa: Visual synthesis pre-training for neural visual world creation

    Chenfei Wu, Jian Liang, Lei Ji, Fan Yang, Yuejian Fang, Daxin Jiang, and Nan Duan. Nüwa: Visual synthesis pre-training for neural visual world creation. InEuropean conference on computer vision, pages 720–736. Springer, 2022

    2022

  17. [25]

    Scaling autoregressive video models

    Dirk Weissenborn, Oscar Täckström, and Jakob Uszkoreit. Scaling autoregressive video models. In International Conference on Learning Representations, 2019

    2019

  18. [26]

    Generating videos with dynamics-aware implicit generative adversarial networks

    Sihyun Yu, Jihoon Tack, Sangwoo Mo, Hyunsu Kim, Junho Kim, Jung-Woo Ha, and Jinwoo Shin. Generating videos with dynamics-aware implicit generative adversarial networks. In 10th International Conference on Learning Representations, ICLR 2022. International Conference on Learnin...

    2022

  19. [27]

    G3an: Disentangling appearance and motion for video generation

    Yaohui Wang, Piotr Bilinski, Francois Bremond, and Antitza Dantcheva. G3an: Disentangling appearance and motion for video generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5264–5273, 2020

    2020

  20. [28]

    A good image generator is what you need for high-resolution video synthesis

    Yu Tian, Jian Ren, Menglei Chai, Kyle Olszewski, Xi Peng, Dimitris N Metaxas, and Sergey Tulyakov. A good image generator is what you need for high-resolution video synthesis. In International Conference on Learning Representations, 2020

    2020

  21. [29]

    Long video generation with time-agnostic vqgan and time-sensitive transformer

    Songwei Ge, Thomas Hayes, Harry Yang, Xi Yin, Guan Pang, David Jacobs, Jia-Bin Huang, and Devi Parikh. Long video generation with time-agnostic vqgan and time-sensitive transformer. In European Conference on Computer Vision, pages 102–118. Springer, 2022

    2022

  22. [30]

    Cogvideo: Large-scale pretraining for text-to-video generation via transformers

    Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers. In The Eleventh International Conference on Learning Representations, 2022

    2022

  23. [31]

    Videopoet: A large language model for zero-shot video generation

    Dan Kondratyuk, Lijun Yu, Xiuye Gu, José Lezama, Jonathan Huang, Rachel Hornung, Hartwig Adam, Hassan Akbari, Yair Alon, Vighnesh Birodkar, et al. Videopoet: A large language model for zero-shot video generation. arXiv preprint arXiv:2312.14125, 2023

    2023 arXiv

  24. [32]

    Imagen video: High definition video generation with diffusion models

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303, 2022

    arXiv

  25. [33]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792, 2022

    2022 arXiv

  26. [34]

    Align your latents: High-resolution video synthesis with latent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p...

    2023

  27. [35]

    Preserve your own correlation: A noise prior for video diffusion models

    Songwei Ge, Seungjun Nah, Guilin Liu, Tyler Poon, Andrew Tao, Bryan Catanzaro, David Jacobs, Jia-Bin Huang, Ming-Yu Liu, and Yogesh Balaji. Preserve your own correlation: A noise prior for video diffusion models. In Proceedings of the IEEE/CVF International Conference on Compu...

    2023

  28. [36]

    Latent video diffusion models for high-fidelity long video generation

    Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for high-fidelity long video generation. arXiv preprint arXiv:2211.13221, 2022

    2022 arXiv

  29. [37]

    Video probabilistic diffusion models in projected latent space

    Sihyun Yu, Kihyuk Sohn, Subin Kim, and Jinwoo Shin. Video probabilistic diffusion models in projected latent space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18456–18466, 2023

    2023

  30. [38]

    Video diffusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. Advances in Neural Information Processing Systems, 35:8633–8646, 2022

    2022

  31. [39]

    I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models

    Shiwei Zhang, Jiayu Wang, Yingya Zhang, Kang Zhao, Hangjie Yuan, Zhiwu Qin, Xiang Wang, Deli Zhao, and Jingren Zhou. I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models. arXiv preprint arXiv:2311.04145, 2023

    2023 arXiv

  32. [40]

    Pixart- \sigma: Weak-to-strong training of diffusion transformer for 4k text-to-image generation

    Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart- \sigma: Weak-to-strong training of diffusion transformer for 4k text-to-image generation. arXiv preprint arXiv:2403.04692, 2024

    2024 arXiv

  33. [41]

    Adapool: Exponential adaptive pooling for information-retaining downsampling

    Alexandros Stergiou and Ronald Poppe. Adapool: Exponential adaptive pooling for information-retaining downsampling. IEEE Transactions on Image Processing, 32:251–266, 2022

    2022

  34. [42]

    Easyani- mate: A high-performance long video generation method based on transformer architecture, 2024

    Jiaqi Xu, Xinyi Zou, Kunzhe Huang, Yunkuo Chen, Bo Liu, MengLi Cheng, Xing Shi, and Jun Huang. Easyani- mate: A high-performance long video generation method based on transformer architecture, 2024

    2024

  35. [43]

    VBench: Comprehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Comprehensive benchmark suite for video generative models. I...

    2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.