Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

ContentV: Efficient Training of Video Generation Models with Limited Compute

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ContentV is an 8B-parameter text-to-video model that the authors report reaching 85.14 on VBench after four weeks of training on 256 NPUs, by minimally adapting Stable Diffusion 3.5 Large.

desk verdict Useful NPU training recipe, but the abstract's SOTA claim is contradicted by the paper's own Table 3. read the letter →

arxiv 2506.05343 v2 pith:QXWAYTG6 submitted 2025-06-05 cs.CV

Wenfeng Lin , Renjie Chen , Boyuan Liu , Shiyue Yan , Ruoyu Feng , Jiangchuan Wei , Yichen Zhang , Yimeng Zhou
show 5 more authors
Chao Feng Jiao Ran Qi Wu Zuotao Liu Mingyu Guo
This is my paper
classification cs.CV
keywords text-to-videogenerationdiffusiontransformerflowmatching3DvariationalautoencoderreinforcementlearningfromhumanfeedbackefficienttrainingNPUVBench
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ContentV is an 8-billion-parameter text-to-video model trained on 256 NPUs for about a month, with the authors reporting a 85.14 VBench overall score. The paper's central claim is that a nearly untouched image-generation model, Stable Diffusion 3.5 Large, can be turned into a competitive video generator by swapping its two-dimensional autoencoder for a causal three-dimensional autoencoder, adding temporal position information, and training with staged flow matching plus a light RLHF stage. The efficiency comes from reusing the open-source image model's learned knowledge instead of training video capabilities from scratch. If the claim holds, it lowers the compute barrier for video generation research and shows that NPUs can plausibly substitute for high-end GPUs on this workload.

What carries the argument

The load-bearing change is the 3D VAE swap: a causal 3D autoencoder with temporal downsample 4 and spatial downsample 8 compresses images and videos into a shared latent space, and the existing SD3.5L image DiT is retrained on those latents for only a few hundred steps. Around that sits a staged flow-matching recipe, moving from short low-resolution clips to long high-resolution clips, with dynamic bucketing by aspect ratio and duration. The third mechanism is a compute-light RLHF loop that makes the flow-matching sampler differentiable by enabling gradients only on a random subset of denoising steps, then rewards the decoded first frame using a CLIP-based reward model, keeping memory within 64 GB per NPU.

What would settle it

Run ContentV through VBench under a documented prompt split, sampling configuration, and aggregation rule; if the resulting overall score does not reproduce 85.14 and does not stay above HunyuanVideo's reported 83.24, the paper's central ranking and efficiency claim are not established.

Watch

Extended reading notes

Core claim

The central discovery is that minimal adaptation of a pretrained image diffusion transformer is sufficient for strong text-to-video generation. ContentV keeps the SD3.5L MMDiT backbone and its text encoder, replaces the 2D VAE with the causal 3D VAE from Wan2.1, and adds temporal position embeddings; the model adapts to the new latent space within roughly 500 steps, and FID at 512 resolution recovers from 294.3 with the unadapted VAE to 13.05 after 1600 steps, close to the original 12.8. Training then moves through stages of increasing duration and resolution under a flow-matching objective, with flow shift 1 during training and 17 during sampling, followed by SFT on a filtered one-million-clip subset and RLHF that backpropagates through randomly selected denoising steps. The authors report an 85.14 VBench overall score for long-prompt evaluation and human-preference win rates against CogVideoX-5B, HunyuanVideo-13B, and Wan2.1-14B.

Load-bearing premise

The main load-bearing premise is that ContentV's reported VBench score was computed under the same evaluation setup, meaning the same prompts, sampling settings, and averaging rule, as the comparison models' official scores; the paper does not report those settings.

Editorial extensions

If this is right

  • An existing open-source image diffusion transformer can be reused for video with a VAE swap and a few hundred adaptation steps, so video training inherits image knowledge rather than starting from random weights.
  • A 256-NPU, four-week budget is claimed to be enough to reach the reported VBench range, a much smaller compute envelope than the hundreds or thousands of GPUs reported for comparable video models.
  • The staged schedule of image adaptation, short video, long video, and high-resolution training, followed by SFT and RLHF, should transfer as a recipe to other base image models or other 3D VAEs.
  • Backpropagating through only some denoising steps lets RLHF optimize final video quality without full differentiability, and decoding only the first frame keeps the post-training step within a 64 GB memory limit.
  • If the evaluation protocol is held fixed, an 8B model built this way can stay competitive with 13B and 14B open models, suggesting raw parameter count is not the dominant factor in VBench scores in this range.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test would be to apply the same VAE-swap recipe to a different pretrained image diffusion backbone; the reported FID recovery curve predicts that a few hundred adaptation steps should restore image quality before video training begins.
  • Because the RLHF reward is computed on the first frame, the reported motion-quality gains are likely indirect, mediated by the shared denoiser rather than by direct temporal supervision; comparing first-frame rewards against full-video rewards would isolate that effect.
  • The human-preference edge over Wan2.1 despite a lower VBench score suggests VBench's aggregate weights dimensions differently than human raters do, which could be checked by publishing per-dimension VBench scores for the same videos used in the user study.
  • Comparing against official leaderboard numbers rather than running every model under one harness is the main source of uncertainty, so a public re-evaluation with fixed prompts, sampling steps, and CFG scale would settle how much of the efficiency claim depends on the evaluation protocol.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The manuscript presents ContentV, an 8B-parameter text-to-video model initialized from Stable Diffusion 3.5 Large and adapted to video generation by replacing the 2D VAE with the Wan2.1 3D VAE and adding 3D position embeddings. The training recipe consists of multi-stage flow-matching pre-training at increasing resolutions and durations, followed by SFT on a high-quality subset and an RLHF stage that uses VideoAlign and MPS as reward models. The training is performed on 256 NPUs over about four weeks. The authors report a VBench score of 85.14 and claim state-of-the-art performance, plus a user study showing preference over CogVideoX-5B, HunyuanVideo-13B, and Wan2.1-14B. The paper also describes a large-scale data curation pipeline, asynchronous encoding infrastructure, and 3D parallelism strategies.

Significance. If the central claims are substantiated, the paper would make a meaningful contribution: it demonstrates that a minimal adaptation of an open image-diffusion model, combined with a swapped 3D VAE, staged flow-matching training, and RLHF, can produce a competitive video model with limited compute on NPUs. The paper's strengths include detailed training configurations (Tables 1 and 2), an explicit description of the data filtering pipeline, quantitative VAE-adaptation curves (FID progression in Section 5.2), and the public release of code and models. However, the headline comparative claims are currently undermined by an internally inconsistent use of VBench scores and by evaluation-protocol ambiguity, so the significance cannot be fully assessed until those issues are resolved.

major comments (4)
  1. [Abstract and §8.2, Table 3] The abstract's claim of "state-of-the-art performance (85.14 on VBench)" is contradicted by Table 3. ContentV(Long) = 85.14 is below Vidu-Q1 (87.41) and Wan2.1-14B (86.22), and ContentV(Short) = 84.11 is below several additional entries. The paper never states whether the official leaderboard rows are short-prompt, long-prompt, or a weighted average, nor does it report the number of prompts per dimension, sampling steps, CFG, resolution, or the aggregation rule used for ContentV. Because the Long/Short difference is about one point, comparable to or larger than the gaps between adjacent rows, selecting the higher Long score for the abstract is not a defensible SOTA claim. The authors should either match the official VBench evaluation protocol exactly and report all settings, or restrict the claim to "competitive with leading open-source models" and clearly report both Long and Short scores.
  2. [§6.2 and Table 4] The RLHF stage is trained with VideoAlign and MPS as reward models, and Table 4 then uses VideoAlign win rates as evidence of improvement from RLHF. This is partially circular: the reported gains may reflect optimization of the evaluation metric itself rather than generalizable quality improvements. Please provide an independent evaluation (for example, a larger human study or evaluation with a different VLM not used in training) and clarify whether the VideoAlign scores in Table 4 are computed on held-out prompts that were not seen during RLHF training.
  3. [§8.3] The user study reports GSB ratios of 1.57, 1.68, and 1.30 against CogVideoX, HunyuanVideo, and Wan2.1, respectively, but the manuscript states only that five human annotators were invited. It does not report the number of prompts or videos evaluated, the total number of ratings, confidence intervals, or significance tests. With five annotators, the claimed "slight edge over Wan2.1" is not statistically established. Please report the full protocol and uncertainty estimates.
  4. [§6.2, Eq. (2)] The stated RLHF objective in Eq. (2) includes a KL-regularization term βD_KL, but the implementation "ignore[s] the KL constraint for the sake of computational efficiency." This means the algorithm optimizes a different objective and removes the standard protection against reward overoptimization. Please state the modified objective explicitly and provide evidence that reward overoptimization is not occurring, for example by plotting reward scores against an independent quality metric over the course of RLHF training.
minor comments (6)
  1. [§3.1] The sentence about watermark detection ends with "black borders,." — the punctuation should be corrected to "black borders.".
  2. [§4.1] The sentence "As shown in the Figure 1, the number of attention layers and dimensions of ContentV are consistent with those of SD3.5L" appears to refer to Table 1 rather than Figure 1; please fix the cross-reference.
  3. [§5.1, Figure 5] The caption describing flow shift sampling is garbled: "using small-step updates with more steps in the high-noise stage and small-step updates with fewer large-step updates in the low-noise stage" should be clarified.
  4. [§7.1] The hardware description says "The NPU feature 64 GB of High Bandwidth Memory" — "feature" should be "features".
  5. [§5.3] The bullet list for the three progressive training stages would be easier to read if each stage name ended with a colon consistently (e.g., "Stage 3: high-resolution, long-duration training.").
  6. [References] For VBench leaderboard comparisons, please specify the leaderboard version and the date the official scores were retrieved, since leaderboard values can change over time.

Circularity Check

1 steps flagged · score 4.0 of 10

No derivation-level circularity, but Table 4's RLHF gains are partly trained into the VideoAlign metric that was also used as an RLHF reward; the headline VBench claim and user study remain independent evidence.

  1. fitted input called prediction [Section 6.2 (Reward Models) and Section 8.2 (Table 4)]
    "Initially, we employed VideoAlign [28], a reward model based on the visual language model (VLM), to evaluate generated videos across three key dimensions: visual quality (VQ), motion quality (MQ), and text alignment (TA). ... The RLHF stage produces more comprehensive improvements across all evaluation dimensions. Most remarkably, this stage yields an 89.38% win rate enhancement in VQ (from -0.2160 to 0.3034) and an 85.62% win rate improvement in overall score (from 0.9071 to 1.8972)."

    Equation (2) defines RLHF as maximizing a reward r(c,x1). VideoAlign is explicitly introduced as a reward model used during RLHF, and Table 4 then reports VideoAlign VQ/MQ/TA win-rate gains as evidence of RLHF improvement. The measurement instrument is therefore, at least initially, identical to the training objective: the reported 'enhancement' is in part the reward function being optimized, not an independent evaluation. The paper's own observation that VideoAlign scores 'improved steadily' while qualitative performance did not improve confirms that VideoAlign gains can be decoupled from genuine quality. Because the final reward was switched to MPS and because VBench and the user study are separate, the circularity is partial and does not by itself force the headline SOTA number.

full rationale

The paper contains no mathematical derivation chain whose conclusion equals its assumptions: flow matching, the RLHF objective, and the architecture choices are standard and are not defined in terms of the claimed results. The load-bearing VBench claim (85.14) is not circular in the derivation sense, though it is vulnerable to an evaluation-protocol mismatch: Table 3 reports ContentV(Long) and ContentV(Short) separately while competitor rows are unlabeled official leaderboard values, and the Abstract selects the higher Long score; that is a correctness and benchmarking concern, not circularity of the kind defined here. No self-citations are load-bearing: the reference list contains no prior work by the present authors, and all key components (SD3.5L, Wan2.1 VAE, VideoAlign, MPS, VBench) are external. The only identifiable circular step is the RLHF/VideoAlign linkage described above, which affects the stage-gain table but not the central independent benchmark comparisons. Overall circularity score 4.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

This is an empirical systems report, not a derivation. The ledger lists hand-tuned training choices and background assumptions on which the reported performance depends. The model introduces no new entities.

free parameters (5)
  • flow_shift_sampling = 17
    Selected after qualitative experiments in Section 5.1; at shift 1 sampling produces indistinguishable results, so the final VBench score depends on this choice.
  • flow_shift_training = 1
    Chosen in Section 5.1 for faster convergence; affects training dynamics and final weights.
  • high_quality_subset_rank_threshold = top 10% in aesthetic score and motion dynamics
    Section 3.2 defines the 1M SFT subset by keeping top 10% in two criteria; changes the post-training data and therefore reported gains.
  • RLHF_KL_regularization_beta = 0 (KL constraint ignored)
    Section 6.2 drops the KL term in Eq. (2) for memory; this alters the RLHF objective and the resulting quality/alignment trade-off.
  • RLHF_gradient_steps_k = not specified
    Section 6.2 randomly samples k gradient-carrying steps; k is never reported though it controls the RLHF update strength.
assumptions (5)
  • standard math Flow matching objective (Eq. 1) and first-order Euler sampler correctly approximate the generative process.
    Used throughout Section 5.1; standard prior result.
  • domain assumption SD3.5L MMDiT can be transferred to video by swapping in Wan2.1's 3D VAE and adding 3D position encodings.
    Core architectural assumption in Section 4; supported only by the paper's own FID adaptation curves.
  • domain assumption Wan2.1 3D VAE latents are compatible with SD3.5L DiT without retraining the VAE.
    Section 4.2 reuses Wan2.1's VAE as-is; no analysis of latent distribution shift is provided.
  • domain assumption VBench leaderboard scores for other models are comparable to ContentV's self-run evaluation.
    Section 8.2 compares against official leaderboard values; the paper never defines the prompt aggregation for its own two rows.
  • ad hoc to paper Dropping the KL regularization in Eq. (2) still yields a valid RLHF procedure.
    Section 6.2 removes the KL term solely for computational reasons, changing the objective from Eq. (2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of ContentV: Efficient Training of Video Generation Models with Limited Compute." pith.science (2026). https://pith.science/paper/QXWAYTG6

@misc{pith2026250605343,
  author       = {Pith},
  title        = {Pith review of: ContentV: Efficient Training of Video Generation Models with Limited Compute},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QXWAYTG6}},
  note         = {Machine review of arXiv:2506.05343}
}
read the original abstract

Recent advances in video generation demand increasingly efficient training recipes to mitigate escalating computational costs. In this report, we present ContentV, an 8B-parameter text-to-video model that achieves state-of-the-art performance (85.14 on VBench) after training on 256 x 64GB Neural Processing Units (NPUs) for merely four weeks. ContentV generates diverse, high-quality videos across multiple resolutions and durations from text prompts, enabled by three key innovations: (1) A minimalist architecture that maximizes reuse of pre-trained image generation models for video generation; (2) A systematic multi-stage training strategy leveraging flow matching for enhanced efficiency; and (3) A cost-effective reinforcement learning with human feedback framework that improves generation quality without requiring additional human annotations. All the code and models are available at: https://contentv.github.io.

Figures

Figures reproduced from arXiv: 2506.05343 by the authors.

Figure 1
Figure 1. Video Generation Samples. ContentV can generate diverse, creative, and high-quality multi-resolution videos based on textual prompts. Abstract Recent advances in video generation demand increasingly efficient training recipes to mitigate escalating computational costs. In this report, we present ContentV, an 8B-parameter text-to-video model that achieves state-of-the-art performance (85.14 on VBench) after training … view at source ↗
Figure 2
Figure 2. Overall of data filtering pipeline. For the filtered pre-training data, data from different [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Architecture of ContentV. Simply replace the VAE in SD3.5L with a causal 3D-VAE and incorporate 3D position embedding to unlock the model’s video generation capabilities. As shown in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: RoPE v.s. APE 3D Position Embedding. In SD3.5L [12], Absolute Posi￾tion Embedding (APE) is used to capture spatial relationships. When extending the DiT of SD3.5L from image input to video input, we add additional time position embeddings to appropri￾ately represent se…
Figure 5
Figure 5. Figure 5: Illustration of Flow shift [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 7
Figure 7. Figure 7: Generated images during 3D-VAE adaptation. The rapid recovery of image generation [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Illustration of diffusion RLHF. An iteration in the flow matching sampling can be written [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Asynchronous Encoding Server Architecture. The feature extraction and model training [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: Qualitative comparison between ContentV and other T2V models. Prompt: A lively group of friends, diverse in appearance and style, gather in a cozy, warmly lit living room, filled with laughter and camaraderie. They sit on a plush, colorful rug, surrounded by soft cush…
Figure 11
Figure 11. Figure 11: Qualitative comparison of ContentV at different stages. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: User studies of ContentV against open-source SOTA T2V models. Compared to CogVideoX, HunyuanVideo, and Wan2.1, ContentV achieves a GSB ratio of 1.57, 1.68, and 1.30, respectively. 9 Conclusion We present ContentV, the first video generation model fully trained on NPUs…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Reward Lightning: Fast Video Generation via Homologous Preference Distillation

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Homologous preference distillation evaluates adversarial distillation and latent reward alignment on identical latent features, yielding 1–4-step video generators that improve VBench by 2.1% while leading text, motion...

Reference graph

Works this paper leans on

56 extracted references · 18 canonical work pages · cited by 1 Pith paper

  1. [1]

    Pyscenedetect: Video scene cut detection tool.https://www.scenedetect

    Breakthrough AI. Pyscenedetect: Video scene cut detection tool.https://www.scenedetect. com/, 2023. Version 0.6.2

    work page 2023

  2. [2]

    Paddleocr, awesome multilingual ocr toolkits based on paddlepaddle

    PaddlePaddle Authors. Paddleocr, awesome multilingual ocr toolkits based on paddlepaddle. https://github.com/PaddlePaddle/PaddleOCR, 2020

    2020

  3. [3]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025

    arXiv 2025

  4. [4]

    Improving image generation with better captions

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8, 2023

    2023

  5. [5]

    Align your latents: High-resolution video synthesis with latent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22563–22575, 2023

    2023

  6. [6]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, et al. Video generation models as world simulators. OpenAI Blog, 1:8, 2024

    2024

  7. [7]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, et al. Video generation models as world simulators. 2024. URL https://openai. com/research/video-generation-models-as-world-simulators, 3:1, 2024

    2024

  8. [8]

    ByteDance. Jimeng. https://jimeng.jianying.com/, 2024

    work page 2024

Show all 56 references
  1. [9]

    Panda-70m: Captioning 70m videos with multiple cross-modality teachers

    Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Ekaterina Deyneka, Hsiang-wei Chao, Byung Eun Jeon, Yuwei Fang, Hsin-Ying Lee, Jian Ren, Ming-Hsuan Yang, et al. Panda-70m: Captioning 70m videos with multiple cross-modality teachers. In Proceedings of the IEEE/CVF Confere...

    2024

  2. [10]

    Scaling vision transformers to 22 billion parameters

    Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Piotr Padlewski, Jonathan Heek, Justin Gilmer, Andreas Peter Steiner, Mathilde Caron, Robert Geirhos, Ibrahim Alabdulmohsin, et al. Scaling vision transformers to 22 billion parameters. In International Conference on Machine Lea...

    2023

  3. [11]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780–8794, 2021

    2021

  4. [12]

    Scaling rectified flow transform- ers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transform- ers for high-resolution image synthesis. In Forty-first International Conference on Machin...

    2024

  5. [13]

    Google. Veo-2. https://deepmind.google/technologies/veo/veo-2/, 2024

    2024

  6. [14]

    Animatediff: Animate your personalized text-to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. In The Twelfth International Conference on Learning Representations, 2024

    2024

  7. [15]

    Photorealistic video generation with diffusion models

    Agrim Gupta, Lijun Yu, Kihyuk Sohn, Xiuye Gu, Meera Hahn, Li Fei-Fei, Irfan Essa, Lu Jiang, and José Lezama. Photorealistic video generation with diffusion models. arXiv preprint arXiv:2312.06662, 2023

    2023 arXiv

  8. [16]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017. 18

    2017

  9. [17]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  10. [18]

    Imagen video: High definition video generation with diffusion models

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303, 2022

    arXiv

  11. [19]

    Vbench: Comprehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat...

    2024

  12. [20]

    Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models

    Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia Zhang, Shuaiwen Leon Song, Samyam Rajbhandari, and Yuxiong He. Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models. arXiv preprint arXiv:2309.14509, 2023

    2023 arXiv

  13. [21]

    Hunyuanvideo: A systematic framework for large video generative models

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603, 2024

    2024 arXiv

  14. [22]

    Reducing activation recomputation in large transformer models

    Vijay Anand Korthikanti, Jared Casper, Sangkug Lym, Lawrence McAfee, Michael Ander- sch, Mohammad Shoeybi, and Bryan Catanzaro. Reducing activation recomputation in large transformer models. Proceedings of Machine Learning and Systems, 5:341–353, 2023

    2023

  15. [23]

    Kuaishou. Kling. https://klingai.kuaishou.com/, 2024

    2024

  16. [24]

    Black Forest Labs. Flux. https://blackforestlabs.ai/, 2024

    2024

  17. [25]

    Sequence paral- lelism: Long sequence training from system perspective

    Shenggui Li, Fuzhao Xue, Chaitanya Baranwal, Yongbin Li, and Yang You. Sequence paral- lelism: Long sequence training from system perspective. arXiv preprint arXiv:2105.13120, 2021

    2021 arXiv

  18. [26]

    Open-sora plan: Open-source large video generation model

    Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang, Xianyi He, Yang Ye, Shenghai Yuan, Liuhan Chen, et al. Open-sora plan: Open-source large video generation model. arXiv preprint arXiv:2412.00131, 2024

    2024 arXiv

  19. [27]

    Flow matching for generative modeling

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022

    2022 arXiv

  20. [28]

    Improving video generation with human feedback

    Jie Liu, Gongye Liu, Jiajun Liang, Ziyang Yuan, Xiaokun Liu, Mingwu Zheng, Xiele Wu, Qiulin Wang, Wenyu Qin, Menghan Xia, et al. Improving video generation with human feedback. arXiv preprint arXiv:2501.13918, 2025

    2025 arXiv

  21. [29]

    Step-video-t2v technical report: The practice, challenges, and future of video foundation model

    Guoqing Ma, Haoyang Huang, Kun Yan, Liangyu Chen, Nan Duan, Shengming Yin, Changyi Wan, Ranchen Ming, Xiaoniu Song, Xing Chen, et al. Step-video-t2v technical report: The practice, challenges, and future of video foundation model. arXiv preprint arXiv:2502.10248, 2025

    2025 arXiv

  22. [30]

    Minimax. Hailuo. https://hailuoai.com/video, 2024

    2024

  23. [31]

    Diatom autofocusing in brightfield microscopy: a comparative study

    José Luis Pech-Pacheco, Gabriel Cristóbal, Jesús Chamorro-Martinez, and Joaquín Fernández- Valdivia. Diatom autofocusing in brightfield microscopy: a comparative study. In Proceedings 15th International Conference on Pattern Recognition. ICPR-2000, volume 3, pages 314–317. IEEE, 2000

    2000

  24. [32]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4195–4205, 2023

    2023

  25. [33]

    Würstchen: An efficient architecture for large-scale text-to-image diffusion models

    Pablo Pernias, Dominic Rampas, Mats Leon Richter, Christopher Pal, and Marc Aubreville. Würstchen: An efficient architecture for large-scale text-to-image diffusion models. In The Twelfth International Conference on Learning Representations, 2024

    2024

  26. [34]

    Pika. Pika. https://pikartai.com/, 2024

    2024

  27. [35]

    Pixelverse

    Pixelverse. Pixelverse. https://www.pixelverse.xyz/, 2025. 19

    2025

  28. [37]

    Sampson, Shikai Li, Simone Parmeggiani, Steve Fine, Tara Fowler, Vladan Petrovic, , and Yuming Du

    Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih-Yao Ma, Ching-Yao Chuang, David Yan, Dhruv Choudhary, Dingkang Wang, Geet Sethi, Guan Pang, Haoyu Ma, Ishan Misra, Ji Hou, Jialiang Wang, Kiran Ja- gadeesh, Kunpeng Li, ...

    arXiv

  29. [38]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67, 2020

    2020

  30. [39]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  31. [40]

    Runway. Gen-3. https://runwayml.com/, 2024

    2024

  32. [41]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...

    2022

  33. [42]

    Laion- 5b: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion- 5b: An open large-scale dataset for training next generation image-text models. Advances in neural inform...

    2022

  34. [43]

    ShengShu-AI. Vidu. https://www.vidu.com/, 2024

    2024

  35. [44]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. In The Eleventh International Conference on Learning Representations, 2022

    2022

  36. [45]

    Genmo Team. Mochi 1. https://github.com/genmoai/models, 2024

    2024

  37. [46]

    Wan: Open and advanced large-scale video generative models

    Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314, 2025

    2025 arXiv

  38. [47]

    Videoclip- xl: Advancing long description understanding for video clip models, 2024

    Jiapeng Wang, Chengyu Wang, Kunzhe Huang, Jun Huang, and Lianwen Jin. Videoclip- xl: Advancing long description understanding for video clip models, 2024. URL https: //arxiv.org/abs/2410.00741

    2024 arXiv

  39. [48]

    Microcinema: A divide-and-conquer approach for text-to-video generation

    Yanhui Wang, Jianmin Bao, Wenming Weng, Ruoyu Feng, Dacheng Yin, Tao Yang, Jingxu Zhang, Qi Dai, Zhiyuan Zhao, Chunyu Wang, et al. Microcinema: A divide-and-conquer approach for text-to-video generation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern R...

    2024

  40. [49]

    Internvid: A large-scale video-text dataset for multimodal understanding and generation

    Yi Wang, Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li, Guo Chen, Xinyuan Chen, Yaohui Wang, et al. Internvid: A large-scale video-text dataset for multimodal understanding and generation. arXiv preprint arXiv:2307.06942, 2023

    2023 arXiv

  41. [50]

    Deep reward supervisions for tuning text-to-image diffusion models

    Xiaoshi Wu, Yiming Hao, Manyuan Zhang, Keqiang Sun, Zhaoyang Huang, Guanglu Song, Yu Liu, and Hongsheng Li. Deep reward supervisions for tuning text-to-image diffusion models. In European Conference on Computer Vision, pages 108–124. Springer, 2024

    2024

  42. [51]

    Gmflow: Learning optical flow via global matching

    Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, and Dacheng Tao. Gmflow: Learning optical flow via global matching. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8121–8130, 2022

    2022

  43. [52]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072, 2024

    2024 arXiv

  44. [53]

    Language model beats diffusion–tokenizer is key to visual generation

    Lijun Yu, José Lezama, Nitesh B Gundavarapu, Luca Versari, Kihyuk Sohn, David Minnen, Yong Cheng, Vighnesh Birodkar, Agrim Gupta, Xiuye Gu, et al. Language model beats diffusion–tokenizer is key to visual generation. arXiv preprint arXiv:2310.05737, 2023

    arXiv

  45. [54]

    Root mean square layer normalization

    Biao Zhang and Rico Sennrich. Root mean square layer normalization. Advances in Neural Information Processing Systems, 32, 2019

    2019

  46. [55]

    Learning multi-dimensional human preference for text-to-image generation

    Sixian Zhang, Bohan Wang, Junqiang Wu, Yan Li, Tingting Gao, Di Zhang, and Zhongyuan Wang. Learning multi-dimensional human preference for text-to-image generation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8018–8027, 2024

    2024

  47. [56]

    Pytorch fsdp: experiences on scaling fully sharded data parallel

    Yanli Zhao, Andrew Gu, Rohan Varma, Liang Luo, Chien-Chin Huang, Min Xu, Less Wright, Hamid Shojanazeri, Myle Ott, Sam Shleifer, et al. Pytorch fsdp: experiences on scaling fully sharded data parallel. arXiv preprint arXiv:2304.11277, 2023

    2023 arXiv

  48. [57]

    Open-sora: Democratizing efficient video production for all

    Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. arXiv preprint arXiv:2412.20404, 2024. 21

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.