Pith. sign in

REVIEW 5 major objections 5 minor 48 references

GigaVideo-1: Advancing Video Generation via Automatic Feedback with 4 GPU-Hours Fine-Tuning

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a pre-trained text-to-video model can be fine-tuned with automatic feedback from a frozen vision-language model in just 4 GPU-hours, raising its average VBench-2.0 score by about 4% across 17 dimensions without human…

desk verdict Solid per-dimension auto-tuning pipeline with a real efficiency win in the ablations; the '4 GPU-hours, almost all dimensions' headline is not supported by any single model. read the letter →

arxiv 2506.10639 v1 pith:3KZQFHUC submitted 2025-06-12 cs.CV

Xiaoyi Bao , Jindi Lv , Xiaofeng Wang , Zheng Zhu , Xinze Chen , YuKun Zhou , Jiancheng Lv , Xingang Wang
show 1 more author
Guan Huang
This is my paper
classification cs.CV
keywords text-to-videogenerationfine-tuningautomaticfeedbackreward-guidedoptimizationvision-languagemodelprompt-drivendataengineVBench-2.0diffusionmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a pre-trained text-to-video model can be fine-tuned to better handle dimensions like motion rationality, physical plausibility, and human interaction without any human annotation, using only 4 GPU-hours and a small amount of synthetic data. The proposed pipeline, GigaVideo-1, builds training samples with a prompt-driven data engine that turns weak-dimension descriptions into diverse synthetic prompts, then trains with a reward-guided loss in which a frozen vision-language model scores each generated video. On VBench-2.0 with Wan2.1 as the base, the method reports an average gain of about 4 percentage points across 17 dimensions, with the largest improvements in Motion Rationality (+12.07), Multi-view Consistency (+8.27), and Human Interaction (+8.34), and no gain on Camera Motion. When all 17 dimensions are optimized jointly, the mean gain shrinks to about 2 points and several dimensions regress below baseline. If the gains hold up on independent evaluation, they would show that large-scale annotation and compute are not prerequisites for meaningfully improving video generators.

What carries the argument

The load-bearing mechanism is a reward-weighted denoising loss. For each synthetic video, a frozen multimodal LLM outputs a scalar score based on yes/no QA pairs that an LLM generates for the target dimension; that score multiplies the flow-matching velocity loss, so samples judged by the vision-language model as high-quality dominate the gradient. A second loss term applies a KL divergence between the predicted latent and the real video's latent for real-caption prompts, preventing the model from drifting into the synthetic distribution. The data engine feeds this loss by having an LLM expand seed phrases into base prompts and stylistic variants, then generating ~9.5k synthetic videos plus ~3.5k real-caption/synthetic-video pairs.

What would settle it

Run the identical 4-GPU-hour pipeline with the reward scores randomly shuffled before weighting the loss; if the average VBench-2.0 gain stays near 4%, the reward signal is not the driver, while a fall to zero or negative would confirm that automatic feedback carries the result.

Watch

Extended reading notes

Core claim

On the paper's own account, the central discovery is that a frozen multimodal LLM can supply sufficiently reliable dimension-specific scores to drive fine-tuning of a video diffusion model. Weighting the standard flow-matching loss by these scores, with a KL constraint pulling the model back toward real-video latents, improves Wan2.1's VBench-2.0 scores on 16 of 17 dimensions by an average of roughly 4 percentage points in 4 GPU-hours. The per-dimension gains range from +0.53 on Thermotics to +12.07 on Motion Rationality, with Camera Motion unchanged. The paper frames this as unlocking latent potential of the pretrained model rather than injecting new information from external high-quality data.

Load-bearing premise

The load-bearing premise is that the frozen vision-language model's per-dimension scores are accurate enough that reweighting samples by them improves real video quality; if those scores are noisy or biased, the weighted loss amplifies the bias and the reported gains could be overfitting to the scorer.

Editorial extensions

If this is right

  • Fine-tuning video generators no longer has to start from human preference annotations; a frozen scorer and synthetic prompts are enough to push a base model up on specific quality axes.
  • The same pipeline transfers across dimensions: re-targeting a new weakness only requires a new dimension description and seed prompts, not new data collection or reward-model training.
  • Offline, precomputed reward weights beat both online reweighting and backpropagating through the reward, so the reward signal is best treated as a fixed reweighting of the denoising loss rather than an end-to-end objective.
  • Jointly optimizing all 17 dimensions yields a smaller average gain (about 2 points) and regresses on several axes, suggesting that the dimensions' objectives conflict and need more careful prompt design before multi-dimension training pays off.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the vision-language scorer is noisy or biased, part of the reported 4% gain could be overfitting to the scorer; the paper's own concession that MLLMs are unreliable for fine-grained perception, and the zero Camera Motion gain, are consistent with that risk.
  • The headline 'consistently improves almost all dimensions' applies to separately fine-tuned per-dimension models; the single all-17 model gains only about 2 points on average and falls below baseline on several axes, so the practical one-model claim is weaker than the abstract suggests.
  • Because the whole loop needs no human labels, it is naturally iterative: the fine-tuned model could generate the next round of synthetic training videos, potentially compounding gains without adding annotation cost.
  • A direct way to extend the method would be to give underrepresented dimensions such as Camera Motion a larger prompt pool and a more sensitive scorer; the paper's own analysis points to sparse valid feedback, not optimizer failure, as the bottleneck.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes GigaVideo-1, a fine-tuning pipeline for pretrained text-to-video diffusion models that requires no human annotations and only small amounts of data and compute. The method has two components: a prompt-driven data engine that uses an LLM to generate synthetic prompts targeting VBench-2.0 dimensions and synthesizes corresponding videos, and a reward-guided optimization strategy that uses a frozen MLLM (plus specialized models such as CoTracker2 and YOLO-World for selected dimensions) to score generated videos and reweight the denoising loss, with a KL realism constraint against real-video captions. Using Wan2.1-T2V-1.3B as the baseline, the paper reports per-dimension VBench-2.0 results in Table 1, ablations of the data engine and reward strategy in Tables 2 and 3, grouped joint training in Table 4, and a joint all-17-dimension model in Table 7. The central claim is that GigaVideo-1 improves almost all VBench-2.0 dimensions by about 4% on average using only 4 GPU-hours.

Significance. If the 4-GPU-hour claim held for a single model, the paper would be a significant practical result: it would show that annotation-free, low-compute fine-tuning can push a pretrained T2V model up on many quality dimensions simultaneously, which is useful for both practitioners and the fine-tuning literature. The paper also makes a credible methodological contribution by combining prompt-driven synthetic data, offline reward reweighting, and a realism constraint, and it explicitly compares against SFT baselines and reports per-dimension user-study preferences. The two strongest points are the efficiency comparison in Table 3 and the recognition that specialized scoring models are needed for fine-grained dimensions. However, as detailed below, the headline claim conflates per-dimension fine-tuned checkpoints with a single jointly fine-tuned model, and the reward-model reliability and benchmark-derived training signal are under-validated. The result remains valuable if reframed as a per-dimension automatic fine-tuning recipe, but the current abstract and conclusions overstate what the experiments establish.

major comments (5)
  1. [Abstract, §4.2, Table 1 vs. Table 7] The headline claim that a single 4-GPU-hour run 'consistently improves performance on almost all the dimensions with an average gain of about 4%' is not supported by the experiments. Table 1 reports results per dimension, and §4.2 explicitly states 'with only 4 GPU-hours of training for single dimension,' indicating that each column of the GigaVideo-1 row is a separately fine-tuned checkpoint. Reproducing all 17 dimensions would therefore cost roughly 17×4 = 68 GPU-hours, not 4. The only evidence for a single jointly trained model is Table 7, which shows a mean improvement of 52.12→54.27 (+2.15 points, about 4.1% relative) but regresses on five dimensions: Mechanics (74.42→70.53), Thermotics (72.66→65.94), Multi-view Consistency (44.60→33.55), Camera Motion (36.11→35.49), and Complex Plot (10.69→10.36). The text should distinguish the per-dimension result from the joint model result, and the abstract and conclusion should be revised to state 'per dimension' or to report the joint model's actual gains and regressions.
  2. [§3.4, §4.2, Appendix C] The training signal depends on the reliability of the frozen MLLM and LLM-generated QA pairs for dimension-specific scoring, but the paper itself concedes in Appendix C that 'current MLLMs exhibit limitations in fine-grained visual perception' and reports in §4.2 that Camera Motion gains are zero because CoTracker2-based scoring reduced the number of samples with valid feedback. Since Eq. (3) directly weights the denoising loss by these scores, noisy or biased rewards can cause the model to overfit to the reward model rather than genuinely improve the target dimension. The paper should provide evidence of reward reliability, for example per-dimension agreement between reward scores and human judgments on a validation subset, the number of training samples retained per dimension after filtering, and score distributions. Without such evidence, the reported per-dimension improvements cannot be cleanly separated from reward-model overfitting.
  3. [§3.3, §3.4, §4.1] The data engine and reward QA prompts are constructed directly from VBench-2.0 dimension definitions, and the method is then evaluated on VBench-2.0. This is not fully circular because the benchmark grader is a different VLM (or a specialized model) than the reward model, but it does create a risk that the training prompts and filtering criteria are tailored to the benchmark's evaluation protocol. The paper should clarify whether any VBench-2.0 prompt templates or evaluation criteria were used to construct the training prompts, and should report at least one held-out evaluation, such as a different video benchmark or a human study outside the training dimensions, to confirm that the gains are not an artifact of benchmark-specific prompt engineering. The user study in Table 6 is helpful but uses the same dimensions and may inherit the same bias.
  4. [Tables 2 and 3] The ablations that validate the data engine and reward strategy report an unnamed metric 'Acc' without specifying which of the 17 VBench-2.0 dimensions this corresponds to. Since the reward-strategy ablation is the central evidence that reward reweighting outperforms SFT on filtered data (80.67 vs. 75.00), the target dimension must be named and the evaluation protocol described. Without this, the reader cannot judge whether the improvement is representative or cherry-picked.
  5. [§4.3, Table 3] The text says that 'Compared to standard SFT, GigaVideo-1 achieves a notable accuracy gain of 5% while reducing training time by more than 6×,' and attributes the efficiency to reward-guided optimization. In Table 3, however, the >6× speedup over standard SFT (5.75h/epoch to 0.90h/epoch) is already achieved by the 'filtered SFT' baseline, which is a data-filtering effect, not an effect of reward reweighting. The unique contribution of reward reweighting in the offline variant is the accuracy gain at the same 0.90h/epoch cost. The efficiency claim should be attributed to the filtered data selection step, and the reward-guided optimization should be credited with the accuracy improvement rather than the speedup.
minor comments (5)
  1. [§4.2] The subsection is titled 'Quantitative Analysis' but the text describes qualitative visualizations; the heading appears to be swapped with the preceding 'Qualitative Comparison' subsection.
  2. [Table 4] The table uses abbreviations such as HAn, HCl, HId, HIn, DSR, MOU, MoR, DAt, Mec, The, Mat, Com, IPr, CMt, MVC, CoL, and CoP without a caption legend; the reader has to map them from Table 1, and this should be stated directly in the caption.
  3. [Eq. (3) and Eq. (4)] The loss is written with a negative sign multiplying the reward-weighted MSE; since the reward is described as a positive score, this is correct reward-weighted regression, but the sign convention should be stated explicitly to avoid confusion with a loss that is minimized by increasing the MSE.
  4. [Appendix F] The appendix says 'In Tab.4' when referring to the reward-strategy ablation, but the relevant table is Table 3; the cross-reference should be corrected.
  5. [§4.3 and Appendix A] The paper does not report the reward filtering threshold or the exact number of samples used per dimension after filtering; since the method's efficiency and accuracy depend on this threshold, its value should be given in the experimental setup.

Circularity Check

1 steps flagged · score 2.0 of 10

Shared VBench-2.0 taxonomy between training targets and evaluation creates a minor circularity burden; no hard definitional equivalence.

  1. other [Sec. 4.1 (Dimension and Evaluations); Sec. 3.4 (Equation 3)]
    "The remaining 17 dimensions are used as our fine-tuning targets and serve as the benchmark for evaluating model performance."

    The optimization in Eq. (3) reweights samples using a reward r_phi built from LLM-generated QA pairs that are conditioned on the target VBench-2.0 dimension, and Sec. 4.1 then uses those same 17 dimensions as the evaluation benchmark. The reported gains therefore partly measure the model's alignment with the benchmark taxonomy that generated the training signal. The circularity is only partial: r_phi is a separate frozen MLLM (LLaVA-Video-7B), not the VBench-2.0 scorer, and Table 7 shows regressions on 5 of 17 dimensions for the joint model, so the improvement is not forced by construction.

full rationale

The paper is an empirical fine-tuning study, not a derivation from first principles. The reward model in Eq. (3) is a frozen MLLM whose QA criteria are generated by an LLM conditioned on dimension names, while the reported success metric is the external VBench-2.0 benchmark; the two are not the same scorer and no parameter is fitted to VBench-2.0 scores. The main circularity burden is that the training data engine and reward criteria are explicitly constructed from the VBench-2.0 dimension taxonomy, so the per-dimension gains partly reflect optimizing toward the same conceptual dimensions used for evaluation. This is mitigated by the separate reward model and by the joint-model regressions in Table 7, which show the improvements are not guaranteed. The abstract's 'using only 4 GPU-hours' and 4% average gain also mix per-dimension and joint-model evidence, but that is a claim-evidence or reporting concern rather than a definitional circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method rests on three kinds of unproven input: the flow-matching objective inherited from the baseline, the assumption that the KL realism term anchors the synthetic distribution, and the assumption that the LLM/MLLM reward signal is a faithful proxy for each VBench-2.0 dimension. The loss weights lambda_ps, lambda_pr, lambda_kl and the reward filtering threshold are hand-set or unspecified, and no sensitivity analysis is given.

free parameters (4)
  • lambda_ps (synthetic loss weight) = 0.5
    Set empirically in Eq. 5 with no sensitivity analysis.
  • lambda_pr (real loss weight) = 0.5
    Set empirically in Eq. 5 with no sensitivity analysis.
  • lambda_kl (KL realism weight) = 0.3
    Set empirically in Appendix A with no sensitivity analysis.
  • reward filtering threshold = unspecified (samples with positive score only)
    Table 3 says 'filtered' uses only samples with a positive score; the exact threshold and criteria are not defined.
assumptions (4)
  • standard math The flow-matching objective in Eq. 2 is the correct training loss for the pretrained model.
    The method inherits the rectified flow formulation from Wan2.1 and ref [23].
  • domain assumption The KL divergence term in Eq. 4 approximates distributional alignment with real videos.
    The paper assumes this term prevents drift from synthetic distributions; no measurement validates the alignment.
  • domain assumption LLM-generated QA pairs and the frozen MLLM reward model accurately capture each VBench-2.0 dimension.
    This is the core reward signal; the paper itself notes MLLMs are unreliable for fine-grained dimensions and uses CoTracker2 or YOLO-World for some.
  • domain assumption VBench-2.0 scores are a valid proxy for video generation quality.
    The evaluation and the training target both rely on VBench-2.0's dimensional decomposition.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GigaVideo-1: Advancing Video Generation via Automatic Feedback with 4 GPU-Hours Fine-Tuning." pith.science (2026). https://pith.science/paper/3KZQFHUC

@misc{pith2026250610639,
  author       = {Pith},
  title        = {Pith review of: GigaVideo-1: Advancing Video Generation via Automatic Feedback with 4 GPU-Hours Fine-Tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3KZQFHUC}},
  note         = {Machine review of arXiv:2506.10639}
}
read the original abstract

Recent progress in diffusion models has greatly enhanced video generation quality, yet these models still require fine-tuning to improve specific dimensions like instance preservation, motion rationality, composition, and physical plausibility. Existing fine-tuning approaches often rely on human annotations and large-scale computational resources, limiting their practicality. In this work, we propose GigaVideo-1, an efficient fine-tuning framework that advances video generation without additional human supervision. Rather than injecting large volumes of high-quality data from external sources, GigaVideo-1 unlocks the latent potential of pre-trained video diffusion models through automatic feedback. Specifically, we focus on two key aspects of the fine-tuning process: data and optimization. To improve fine-tuning data, we design a prompt-driven data engine that constructs diverse, weakness-oriented training samples. On the optimization side, we introduce a reward-guided training strategy, which adaptively weights samples using feedback from pre-trained vision-language models with a realism constraint. We evaluate GigaVideo-1 on the VBench-2.0 benchmark using Wan2.1 as the baseline across 17 evaluation dimensions. Experiments show that GigaVideo-1 consistently improves performance on almost all the dimensions with an average gain of about 4% using only 4 GPU-hours. Requiring no manual annotations and minimal real data, GigaVideo-1 demonstrates both effectiveness and efficiency. Code, model, and data will be publicly available.

Figures

Figures reproduced from arXiv: 2506.10639 by the authors.

Figure 1
Figure 1. Visualization of GigaVideo-1 performance. The left figure compares videos generated by the baseline Wan2.1 [6] and our GigaVideo-1 across two different dimensions. The right figure provides the performance of GigaVideo-1 and other state-of-the-art T2V models on VBench-2.0. With only 4 GPU-hours of training, GigaVideo-1 achieves notable improvements over the baseline, demonstrating both effectiveness and efficiency. … view at source ↗
Figure 2
Figure 2. GigaVideo-1 Training Pipeline. Our pipeline consists of two components: prompt-driven data engine and reward-guided optimization. On the left, we generate synthetic prompts targeting weak dimensions using LLMs, and synthesize training videos via a pre-trained T2V model. These are combined with real-caption–based samples to balance diversity and realism. On the right, a frozen MLLM scores each video on dimension-spec… view at source ↗
Figure 3
Figure 3. Visual comparisons of videos generated by GigaVideo-1 and Wan2.1 [ [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: LLM prompts we use in the first stage for analogy and sentence making, targeting the [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 5 canonical work pages

  1. [1]

    Photorealistic video generation with diffusion models

    Agrim Gupta, Lijun Yu, Kihyuk Sohn, Xiuye Gu, Meera Hahn, Fei-Fei Li, Irfan Essa, Lu Jiang, and José Lezama. Photorealistic video generation with diffusion models. InEuropean Confer- ence on Computer Vision, pages 393–411. Springer, 2024

    2024

  2. [2]

    Venhancer: Generative space-time enhancement for video generation

    Jingwen He, Tianfan Xue, Dongyang Liu, Xinqi Lin, Peng Gao, Dahua Lin, Yu Qiao, Wanli Ouyang, and Ziwei Liu. Venhancer: Generative space-time enhancement for video generation. arXiv preprint arXiv:2407.07667, 2024

    arXiv 2024

  3. [3]

    Modelscope text-to-video technical report.arXiv preprint arXiv:2308.06571, 2023

    Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. Modelscope text-to-video technical report.arXiv preprint arXiv:2308.06571, 2023

    arXiv 2023

  4. [4]

    Videocomposer: Compositional video synthesis with motion controllability.Advances in Neural Information Processing Systems, 36:7594–7611, 2023

    Xiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Jiuniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, and Jingren Zhou. Videocomposer: Compositional video synthesis with motion controllability.Advances in Neural Information Processing Systems, 36:7594–7611, 2023

    2023

  5. [5]

    Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2024

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2024

    arXiv 2024

  6. [6]

    Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    arXiv 2025

  7. [7]

    Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024

    arXiv 2024

  8. [8]

    Improving video generation with human feedback

    Jie Liu, Gongye Liu, Jiajun Liang, Ziyang Yuan, Xiaokun Liu, Mingwu Zheng, Xiele Wu, Qiulin Wang, Wenyu Qin, Menghan Xia, et al. Improving video generation with human feedback. arXiv preprint arXiv:2501.13918, 2025

    arXiv 2025

Show all 48 references
  1. [9]

    Lift: Leveraging human feedback for text-to-video model alignment.arXiv preprint arXiv:2412.04814, 2024

    Yibin Wang, Zhiyu Tan, Junyan Wang, Xiaomeng Yang, Cheng Jin, and Hao Li. Lift: Leveraging human feedback for text-to-video model alignment.arXiv preprint arXiv:2412.04814, 2024

    2024 arXiv

  2. [10]

    Videodpo: Omni-preference alignment for video diffusion generation.arXiv preprint arXiv:2412.14167, 2024

    Runtao Liu, Haoyu Wu, Zheng Ziqiang, Chen Wei, Yingqing He, Renjie Pi, and Qifeng Chen. Videodpo: Omni-preference alignment for video diffusion generation.arXiv preprint arXiv:2412.14167, 2024

    2024 arXiv

  3. [11]

    Visionreward: Fine-grained multi-dimensional human preference learning for image and video generation.arXiv preprint arXiv:2412.21059, 2024

    Jiazheng Xu, Yu Huang, Jiale Cheng, Yuanming Yang, Jiajun Xu, Yuan Wang, Wenbo Duan, Shen Yang, Qunlin Jin, Shurun Li, et al. Visionreward: Fine-grained multi-dimensional human preference learning for image and video generation.arXiv preprint arXiv:2412.21059, 2024

    2024 arXiv

  4. [12]

    Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Do- minik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023

    2023 arXiv

  5. [13]

    Videocrafter1: Open diffusion models for high-quality video generation.arXiv preprint arXiv:2310.19512, 2023

    Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, et al. Videocrafter1: Open diffusion models for high-quality video generation.arXiv preprint arXiv:2310.19512, 2023

    arXiv

  6. [14]

    Flexible diffusion modeling of long videos.Advances in Neural Information Processing Systems, 35:27953–27965, 2022

    William Harvey, Saeid Naderiparizi, Vaden Masrani, Christian Weilbach, and Frank Wood. Flexible diffusion modeling of long videos.Advances in Neural Information Processing Systems, 35:27953–27965, 2022

    2022

  7. [15]

    Latte: Latent diffusion transformer for video generation.arXiv preprint arXiv:2401.03048, 2024

    Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Ziwei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation.arXiv preprint arXiv:2401.03048, 2024

    2024 arXiv

  8. [16]

    Open-sora: Democratizing efficient video production for all

    Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. arXiv preprint arXiv:2412.20404, 2024. 11

    2024 arXiv

  9. [17]

    Vidm: Video implicit diffusion models

    Kangfu Mei and Vishal Patel. Vidm: Video implicit diffusion models. InProceedings of the AAAI conference on artificial intelligence, volume 37, pages 9117–9125, 2023

    2023

  10. [18]

    Make-a-video: Text-to-video generation without text-video data.arXiv preprint arXiv:2209.14792, 2022

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data.arXiv preprint arXiv:2209.14792, 2022

    2022 arXiv

  11. [19]

    Show-1: Marrying pixel and latent diffusion models for text-to-video generation.International Journal of Computer Vision, pages 1–15, 2024

    David Junhao Zhang, Jay Zhangjie Wu, Jia-Wei Liu, Rui Zhao, Lingmin Ran, Yuchao Gu, Difei Gao, and Mike Zheng Shou. Show-1: Marrying pixel and latent diffusion models for text-to-video generation.International Journal of Computer Vision, pages 1–15, 2024

    2024

  12. [20]

    Allegro: Open the black box of commercial-level video generation model.arXiv preprint arXiv:2410.15458, 2024

    Yuan Zhou, Qiuyue Wang, Yuxuan Cai, and Huan Yang. Allegro: Open the black box of commercial-level video generation model.arXiv preprint arXiv:2410.15458, 2024

    2024 arXiv

  13. [21]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF international conference on computer vision, pages 4195–4205, 2023

    2023

  14. [22]

    Elucidating the design space of diffusion-based generative models.Advances in neural information processing systems, 35:26565–26577, 2022

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models.Advances in neural information processing systems, 35:26565–26577, 2022

    2022

  15. [23]

    Scaling rectified flow trans- formers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow trans- formers for high-resolution image synthesis. InForty-first international conference on machine...

    2024

  16. [24]

    Vbench-2.0: Advancing video generation benchmark suite for intrinsic faithfulness.arXiv preprint arXiv:2503.21755, 2025

    Dian Zheng, Ziqi Huang, Hongbo Liu, Kai Zou, Yinan He, Fan Zhang, Yuanhan Zhang, Jingwen He, Wei-Shi Zheng, Yu Qiao, et al. Vbench-2.0: Advancing video generation benchmark suite for intrinsic faithfulness.arXiv preprint arXiv:2503.21755, 2025

    2025 arXiv

  17. [25]

    Worldmodelbench: Judging video generation models as world models.arXiv preprint arXiv:2502.20694, 2025

    Dacheng Li, Yunhao Fang, Yukang Chen, Shuo Yang, Shiyi Cao, Justin Wong, Michael Luo, Xiaolong Wang, Hongxu Yin, Joseph E Gonzalez, et al. Worldmodelbench: Judging video generation models as world models.arXiv preprint arXiv:2502.20694, 2025

    2025 arXiv

  18. [26]

    Advantage-weighted regression: Simple and scalable off-policy reinforcement learning, 2019

    Xue Bin Peng, Aviral Kumar, Grace Zhang, and Sergey Levine. Advantage-weighted regression: Simple and scalable off-policy reinforcement learning, 2019. InURL https://openreview. net/forum, 1910

    2019

  19. [27]

    Aligning text-to-image models using human feedback.arXiv preprint arXiv:2302.12192, 2023

    Kimin Lee, Hao Liu, Moonkyung Ryu, Olivia Watkins, Yuqing Du, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, and Shixiang Shane Gu. Aligning text-to-image models using human feedback.arXiv preprint arXiv:2302.12192, 2023

    2023 arXiv

  20. [28]

    Improving dynamic object interactions in text-to-video generation with ai feedback.arXiv preprint arXiv:2412.02617, 2024

    Hiroki Furuta, Heiga Zen, Dale Schuurmans, Aleksandra Faust, Yutaka Matsuo, Percy Liang, and Sherry Yang. Improving dynamic object interactions in text-to-video generation with ai feedback.arXiv preprint arXiv:2412.02617, 2024

    2024 arXiv

  21. [29]

    Latent video diffusion models for high-fidelity long video generation.arXiv preprint arXiv:2211.13221, 2022

    Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for high-fidelity long video generation.arXiv preprint arXiv:2211.13221, 2022

    2022 arXiv

  22. [30]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728–53741, 2023

    2023

  23. [31]

    Diffusion model alignment using direct preference optimization

    Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. InProceedings of the IEEE/CVF Conference on Computer Vision and ...

    2024

  24. [32]

    Raft: Reward ranked finetuning for generative foundation model alignment.arXiv preprint arXiv:2304.06767, 2023

    Hanze Dong, Wei Xiong, Deepanshu Goyal, Yihan Zhang, Winnie Chow, Rui Pan, Shizhe Diao, Jipeng Zhang, Kashun Shum, and Tong Zhang. Raft: Reward ranked finetuning for generative foundation model alignment.arXiv preprint arXiv:2304.06767, 2023. 12

    2023 arXiv

  25. [33]

    Using human feedback to fine-tune diffusion models without any reward model

    Kai Yang, Jian Tao, Jiafei Lyu, Chunjiang Ge, Jiaxin Chen, Weihan Shen, Xiaolong Zhu, and Xiu Li. Using human feedback to fine-tune diffusion models without any reward model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8941–8951, 2024

    2024

  26. [34]

    Step-aware preference optimization: Aligning preference with denoising performance at each step.arXiv preprint arXiv:2406.04314, 2(3), 2024

    Zhanhao Liang, Yuhui Yuan, Shuyang Gu, Bohan Chen, Tiankai Hang, Ji Li, and Liang Zheng. Step-aware preference optimization: Aligning preference with denoising performance at each step.arXiv preprint arXiv:2406.04314, 2(3), 2024

    2024 arXiv

  27. [35]

    Flow-dpo: Improving llm mathematical reasoning through online multi-agent learning.arXiv preprint arXiv:2410.22304, 2024

    Yihe Deng and Paul Mineiro. Flow-dpo: Improving llm mathematical reasoning through online multi-agent learning.arXiv preprint arXiv:2410.22304, 2024

    2024 arXiv

  28. [36]

    Gapo: Learning preferential prompt through generative adversarial policy optimization.arXiv preprint arXiv:2503.20194, 2025

    Zhouhong Gu, Xingzhou Chen, Xiaoran Shi, Tao Wang, Suhang Zheng, Tianyu Li, Hongwei Feng, and Yanghua Xiao. Gapo: Learning preferential prompt through generative adversarial policy optimization.arXiv preprint arXiv:2503.20194, 2025

    2025 arXiv

  29. [37]

    Proximal policy optimization algorithms

    J Schulman, F Wolski, P Dhariwal, A Radford, and O Klimov. Proximal policy optimization algorithms. arxiv.[preprint]. doi: 10.48550.arXiv preprint arXiv.1707.06347, 2017

    2017 arXiv

  30. [38]

    Training diffusion models with reinforcement learning.arXiv preprint arXiv:2305.13301, 2023

    Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning.arXiv preprint arXiv:2305.13301, 2023

    2023 arXiv

  31. [39]

    Reinforcement learning for fine- tuning text-to-image diffusion models

    Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Reinforcement learning for fine- tuning text-to-image diffusion models. InThirty-seventh Conference on Neural Information Processing ...

    2023

  32. [40]

    Structure and content-guided video synthesis with diffusion models

    Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog, and Anastasis Germanidis. Structure and content-guided video synthesis with diffusion models. InProceedings of the IEEE/CVF international conference on computer vision, pages 7346–7356, 2023

    2023

  33. [41]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, et al. Video generation models as world simulators. OpenAI Blog, 1:8, 2024

    2024

  34. [42]

    Cotracker: It is better to track together

    Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Cotracker: It is better to track together. InEuropean Conference on Computer Vision, pages 18–35. Springer, 2024

    2024

  35. [43]

    Video generation models as world simulators, 2024

    OpenAI. Video generation models as world simulators, 2024

    2024

  36. [44]

    Kling ai.https://klingai.kuaishou.com/, 2024.06

    Kuaishou. Kling ai.https://klingai.kuaishou.com/, 2024.06

    2024

  37. [45]

    Koala-36m: A large-scale video dataset improving con- sistency between fine-grained conditions and video content.arXiv preprint arXiv:2410.08260, 2024

    Qiuheng Wang, Yukai Shi, Jiarong Ou, Rui Chen, Ke Lin, Jiahao Wang, Boyuan Jiang, Haotian Yang, Mingwu Zheng, Xin Tao, et al. Koala-36m: A large-scale video dataset improving con- sistency between fine-grained conditions and video content.arXiv preprint arXiv:2410.08260, 2024

    arXiv

  38. [46]

    Videoscore: Building automatic metrics to simulate fine-grained human feedback for video generation.arXiv preprint arXiv:2406.15252, 2024

    Xuan He, Dongfu Jiang, Ge Zhang, Max Ku, Achint Soni, Sherman Siu, Haonan Chen, Abhranil Chandra, Ziyan Jiang, Aaran Arulraj, et al. Videoscore: Building automatic metrics to simulate fine-grained human feedback for video generation.arXiv preprint arXiv:2406.15252, 2024

    2024 arXiv

  39. [47]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report.arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  40. [48]

    Camera zoom in, Disneyland

    Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. Video instruction tuning with synthetic data.arXiv preprint arXiv:2410.02713, 2024. 13 A Experiments Details Models and Settings.We specify key hyperparameters and architectural choices here for re...

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.